Nicholas Ullrich (top right) presents to his dissertation committee and readers.

Congrats to Nicholas Ullrich who successfully defended his dissertation titled “The Influence of Text on Coherence of Story Retells, Reading Comprehension, Vocabulary Acquisition, and Eye Gaze: A Computer-based Storytelling Task with Eye Tracking.”

Committee Members:
Chair: Dr. Patricia Brooks, Professor, Educational Psychology, College of Staten Island/Graduate Center
Dr. Bruce Homer, Associate Professor, Educational Psychology, Graduate Center
Dr. Irina A. Sekerina, Professor, Psychology, College of Staten Island

Readers:
Dr. Linnea Ehri, Professor Emeritus, Educational Psychology, Graduate Center
Dr. Robin O’Leary, Clinical Placement Specialist for Childhood Education, School of Education, Hunter College

Abstract:
With the growing trend of using multimedia platforms such as YouTube to facilitate storytelling, understanding how and when to integrate text with visuals would benefit both the creators of these platforms and the young readers viewing them. The current study examined the effect of orthography on vocabulary acquisition and narrative comprehension in young readers (children in 2nd and 3rd grade, ages 6-9) during a computer-based storytelling task.  We aimed to determine if having text available during storytelling benefits readers as predicted by Perfetti’s Lexical Quality Hypothesis (Perfetti & Hart, 2002), or hampers learning as predicted by Mayer’s Redundancy Effect (Moreno & Mayer, 2002). The dissertation also explored if having access to the text would affect children’s eye gaze patterns while viewing the story, which had been found during the pilot study. It was hypothesized that the group who had access to their native text would perform better in measures of comprehension and vocabulary knowledge and that they would show a different eye gaze pattern during the computer-based storytelling task than the group presented with Greek symbols in place of the English.

Children completed a battery of pretests to assess reading level and working memory ability and then were randomly assigned to conditions where picture books contained either the story text in English (meaningful condition) or in Greek symbols (meaningless text condition).  They then watched a PowerPoint presentation featuring Mercer Mayer’s “Frog, where are you?” (Mayer, 1969) while having their eye gaze monitored using Tobii 4C eye tracker. After viewing the presentation, the children retold the story and answered ten comprehension questions, assessing both factual and inferential comprehension.  They then completed a vocabulary knowledge scale that used words from the stories to assess vocabulary comprehension, as well as a cloze task created by the experimenter using target words from the story.

Results found that the presence of text did not affect comprehension or vocabulary acquisition, as both groups had similar scores across measures.  However, the study found significant differences in eye gaze patterns between the groups.  In both conditions participants spent significantly more time viewing the text during the beginning of the story and then reduced their dwell time during the middle and the end.  Participants in both conditions also spent higher percentage of dwell time on the illustrations during the beginning and middle of the story before reducing their focus at the end.  However when compared to participants exposed to the Greek symbols,  participants with access to their native text spent significantly less time viewing the story pictures overall.  Results also found that for each page of the story all participants focused more on the text before and during the playing of the audio narration when compared to after the narration was completed, and an interaction between condition and narration suggests that this finding is more differences in dwell time during narration effects participants differently depending on whether they have access to the meaningful text.

While the study needs to be replicated to confirm results due to limitations caused by sample size and data loss in eye tracking, the present findings could help educators better integrate multimedia story telling within their classrooms by designing the activities to better match their students’ attentional and cognitive needs.

This entry is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license.