Want to earn 10+ hours of professional development?
Learn more about the OT School House: Back to School Conference Here!
Press play below to listen to the podcast
Or click on your preferred podcast player link!
Welcome to the show notes for the Episode 67 of the OT School House Podcast.
Sometimes we simply forget what average is and need a reminder. In episode 67, we are having a discussion about a recent journal article that looks at the typical handwriting abilities among second graders and compares their scores on the Evaluation Tool of Children's Handwriting - Manuscript to the teacher's subjective observations of the children's work samples.
Links to Show References:
Transcript
Download the Transcript or read the episode below!
SUMMARY KEYWORDS
student, legibility, ETCH, OT, score, handwriting, teachers, tool, compare, assessments, schoolhouse, criterion reference, identified, study, podcast, legible, typical, test, article, evaluation tool
SPEAKERS
Amazing Narrator, Jayson Davies
Jayson Davies 00:00
So tell me, have you ever done this before? You're evaluating a student who's kind of on the fence for maybe, or maybe not needing occupational therapy services, and you're thinking to yourself, Hmm, I wonder how that typical third grade student who doesn't need OT would perform on this evaluation tool, or whatever age they might be. has an occupational therapist without a kid. I don't get to see typical developing kids very often. And so my mind wanders. Where would this kid score on this evaluation if he didn't need OT? Or where would he score if he did need OT ? Well, today we are going to do just that, we're going to dive into an article that looks at the evaluation tool of children's handwriting to try and figure out what some of the expectations for typical developing children are when it comes to legibility. This article also looks at what a teacher considers legible. So we're gonna dive all up into that in just a minute. But first, let's go ahead and cue the intro. I'll be back with you in just a moment.
Amazing Narrator 01:05
Hello, and welcome to the OT schoolhouse podcast, your source for school based occupational therapy, tips, interviews and professional development. Now to get the conversation started. here's your host, Jayson Davies. Class is officially in session.
Jayson Davies 01:22
Alright, welcome back. My name is Jayson Davies and I am your host for the OT school house podcast. This is Episode 67. We are moving right along. And that's the same with this podcast. We're gonna move right into our journal article today. Like I said, as OTs we often see kids with special needs and sometimes we can forget what the expectations are for typical developing children. And this is important to know because we're often almost comparing the kids that we evaluate to their typically developing peers. So it's important to know what does typical developing mean, especially when it comes to handwriting because so many of our referrals do come for handwriting. So today, we are diving into the article titled handwriting performance of typical second grade students as measured by the evaluation tool of children's handwriting manuscript and teacher perceptions of legibility. This article comes from the open Journal of occupational therapy in the fall of 2019. The authors come to us from Ithaca College, they are Diane long and James Conklin. For a full citation of this article, be sure to check out the notes below in your podcast player or head over to OTschoolhouse.com/episode67. There, I will have a link for you to get access to this journal article. All right, so let's go ahead and jump right into it. We are looking at the evaluation tool of children's handwriting, you will hear me reference that as the ETCH,. And we are specifically looking at the manuscript version of the ETCH. There is a manuscript and a cursive version, we are looking at the manuscript version today. So I will use the term ETCH to reference that tool. One aspect of this article that I really appreciated was the detail in the references that they looked into the research that they actually put in before they did this study. Now, I'm not going to go into everything, because that's not what this podcast is about. But a few key highlights that I do want to touch upon is that they really did a good job at establishing the difference between a few different types of assessments, they established the difference between norm referenced criterion reference and therapist design tools. recap on that, in case you're not as familiar with that norm referenced assessment tools are tools that compare a child to their peers by looking at the percentage or standard score that a student receives. A criterion reference tool is different. And that's actually what the ETCH is, where it doesn't compare them to an actual other child. It just looks at did they hit a certain mark. So for instance, when you're talking about legibility, and the ETCH, the ETCH says that the writing is 80% legible or 90% legible. It doesn't say it's 80% legible compared to other peers. It's just giving you that that sentence, those words, those sentences, those letters are 80% legible period. Whether or not that is better than or worse than their typical developing peers or any of their peers is up for your interpretation. Just to give you a few examples, the BOT-2 the test of handwriting skills, revised THS-R and WRAVMA. Those are some norm referenced assessments. They give you scores in comparison to a large group of kids that were studied at whatever time they develop the assessment. On the contrary, the ETCH, the print tool, the SFA, the school function, assessment, those are all criterion referenced assessments. They do not compare student that you are assessing to other students of their age. Instead, these assessments simply provide you a starting point and a guide to where a student is in that particular moment so that in the future, you could potentially compare to where they were three months ago, six months ago, or however long it was ago that you gave that initial assessment. And the third type of tool that they do outline, or mentioned, at least, is a therapist design tool. And that is simply a tool that you might design to use for your own assessments that is personal to you. And you can then keep that actual work that the student did, and compare that to a later date. The author's then went into a pretty detailed review of the research behind the ETCH to look at the inter rater reliability, the test retest reliability, and even to try and establish norms for the tool as it pertains to the students age compared to the legibility and that is what kind of brings us to this specific journal article. The author's have three particular goals that they were looking at with this study,
Jayson Davies 06:05
they wanted to identify the typical performance of second grade students on the edge. That's the first one. The second goal that they had was to describe whether or not gender influences scores on the ETCH. So if you're a female or a male, do you score higher or lower, typically at the same age. And then finally, the third goal that they had was to compare teachers perceptions of handwriting for second grade students with that. So they were wondering if there's a correlation between a teacher rating a student's eligibility as compared to an occupational therapist or someone completing the ETCH and rating their legibility. Real quickly, before we get to those results. Let's talk about the participants. The participants did come from a convenient sample of teachers and second grade students, all from which came from eight classrooms in four different districts in Central New York, a total of five classrooms and a high needs district as they described, and three classes from a faith based schools and other districts, there were a total of 74 kids with an average age of about seven and a half 54% were female 15% were identified as having some sort of delay disorder or disability. 13 of the kids received additional resource or special education services, and a handful received other special education services, such as OT, speech and whatnot. Alright, so I know I already mentioned the ETCH, and that it was used for this evaluation or this research, but I didn't quite mention what it is, for those of you who are not as familiar with it, maybe you use the THS or the print tool, and you haven't used the ETCH. The ETCH is a tool of children's handwriting, as the name suggests, and it consists of several sub tests that look at the legibility of a student's lowercase uppercase and numeral writing. By writing the alphabet or writing several numbers in order without a model, then it looks at near point copying through copying a sentence and grading the words as well as the letters, then you do the same thing for FAR point copying. So you write something up on the board and have them write that. After that you look at dictation, which is asking the child to write some letters that you share with them to almost create fake words. And then after that, you have them write their own sentence, and you grade the words and letters in that sentence. So that is a little rundown of the ETCH that might help the results make a little more sense when we get there. Alright, so there was one more tool and that was the tool that the teachers use to subjectively grade the students handwriting. So teachers were provided a sliding scale from 0.0 to 5.0. To rate each student's overall legibility. Based on work samples in the classroom, alignment, spacing size, they were also obtained using this same scale. The teachers were not blinded to which students handwriting they were scoring, because that would be typical during the referral process, right? A student isn't being referred by a teacher that doesn't know whose handwriting looks bad or good. They're referring a student that they have concerns with. Something I found pretty clever as it relates to this study is that the researchers provided the teachers with 10 additional work samples that were not their students to grade so that they could establish inter-rater agreement between the teachers to see if teachers kind of rated handwriting similarly, so each of the I believe it was eight teachers got the same 10 handwriting samples, just to see if teacher A B, C and D, all rated students 123 similarly, Now this may or may not come as a surprise to you, but they found that teachers did actually rate the handwriting samples pretty similarly across the board, meaning that teachers A, B, and C Did rate student one pretty similar? Alright, so let's go ahead and check out the results earlier, I gave you the three goals that the researchers aim to answer with this study. So let's go over them one by one. We'll start off with the question about gender and scores on the ETCH. I don't think it's a surprise to anyone. But with 95% confidence interval, the boys did consistently score lower than the girls. On average, the boys scored seven points lower on total word legibility, seven points lower on total letter legibility, and five points. On average, when it came to
Jayson Davies 10:39
numeral legibility. Most of the sub tests hovered around those averages. However, far point letter legibility, dictation of words and dictation of letters, there was a surprisingly large gap of 10 to 16 points between the average female score to the average male score. And that brings us to the overall average scores. Remember, we're talking about second grade students with a small percentage that did have some form of additional support the resource and, and identified as needing special education services. But in general, these students scored an 84% on legibility for total letters 88% for legibility of total words, and 89% for total number legibility. And that now leads us to the final goal that the researchers had, and that is trying to predict at what score, a teacher would perceive a student's eligibility as being okay or not. Okay. And so I'm gonna kind of relate this to the scores that we just talked about the mean score for a second grade student, for total letter legibility was 84%. However, the conservative cut off recommended by the researchers is actually 77% for letter legibility, because there were students below that 77% that may still actually qualify or be identified as having illegible writing. The total word score that we had an average for before was 88%. However, the researchers recommended a score of 82% be used as the cut off, again, because some students who may score lower may actually not be identified or, or quote, unquote, fall through the cracks, potentially, if you don't use that lower cutoff. And when they looked at each of the individual sub tests, they found that the upper case legibility scores had the highest indication of predicting a teacher's overall perception of a student's legibility. So basically, that means that if the teacher found the student to have good legibility, then the student was likely to score higher on that one sub test, which is writing the alphabet, all in uppercase letters. And if the teacher identified a student as scoring low, or having limited legibility, or decreased legibility, they would likely score lower on that uppercase Alphabet Writing sub test. So what that means for the researchers is that their mean scores were in line with other studies that came before theirs. Their results were also in line with previous studies that showed that, on average girls performed better in writing than the boys did in school. And this idea of the females performing better than the males actually made them question whether or not there needs to be separate expectations for those populations, should there be different standards of legibility for males and females at the same age level? When it comes to my own personal takeaways from reading this article? I, you know, I can't help but just understand that teachers are pretty good at identifying poor handwriting. I mean, they're seeing poor handwriting as students performing less than 80% legibility on the ETCH. At least, that's what it seems like to me. I think this study also kind of reinforces that idea that I feel like most educators have that 80% is proficient, and less than 80% is not proficient, especially when it comes to handwriting. And so if you're scoring a student's ETCH, and they are hovering around that 80 percentile ranking, then that is cause for potential concern. And it is definitely caused to potentially look into it further and look at you know, visual motor, visual, spatial, and some fine motor skills to determine if there are some underlying deficits, preventing the student from scoring higher. And finally, this study also confirms why my handwriting is such a chicken scratch in comparison to my wife's, just gonna say that I now have an excuse to use when she can't read my writing. Alright, well, that is going to wrap up Episode 67 of the OT schoolhouse podcast the final episode of March 2021. Up next we have April coming up, which you know what that means it is OT month, and I'm looking forward to celebrating that with you all. Take care. Have a great rest of your March, start to April and just enjoy OT month and be sure to advocate for our wonderful profession. All right. I will see you all next time. Take care and have a great April. Bye.
Amazing Narrator 15:29
Thank you for listening to the OT schoolhouse podcast. For more ways to help you and your students succeed right now. Head on over to OTschoolhouse.com until next