Electronic Thesis and Dissertation Repository

Thesis Format

Monograph

Degree

Master of Arts

Program

Education

Supervisor

Kim, Mi Song.

Abstract

Educational neuroscience has become an important role in understanding education and the association with brain development. However, few previous studies have applied neuroimaging techniques to multiliteracies research, which is an important literacy pedagogy addressing multimodal learning and cultural and linguistic diversity. This study used functional near infrared spectroscopy (fNIRS) to investigate the association of multiliteracies learning on adult English Second Language (ESL) students’ performance through multimodal tasks.

Students’ multimodality background was collected through a technology questionnaire. Behavioural and fNIRS data were collected before and after multiliteracies learning. Results showed that there was no significance change in behavioural responses while the model for predicting them changed after multiliteracies learning. The fNIRS data showed that multiliteracies learning is associated with activation of the learning network in the brain including the superior temporal gyrus (STG). This research has found a way for educational researchers to understand multiliteracies from neural perspectives.

Summary for Lay Audience

With the development of interdisciplinary research, educational neuroscience has begun to play an important role in understanding the association of education. However, few studies have applied educational neuroscience methods to multiliteracies research, which is an important pedagogy for literacy learning, and emphasizes the use of multimodal technologies. To address this gap, this study used fNIRS, which is a neuroimaging technique that can estimate brain activity through hemoglobin concentrations to investigate the effect of multiliteracies learning on adult ESL students’ brain development and multiliteracies performance.

Students were ESL learners aged 17-25 who were enrolled in an ESL course in Canada, referred to as multiliteracies learning. Three data sources were collected and analyzed by designing an experimentation using emotional videos in English. Students were asked to distinguish the actors’ emotions in the videos after each video was presented. The emotions being expressed by the actors were either congruent (e.g. happy face, happy voice) or incongruent (e.g. happy face, sad voice). The technology background questionnaires were collected to understand students’ multimodality background before they started the ESL course that involved multiliteracies learning. During the experiment, behavioural data including reaction time and correct responses were collected twice, before and after multiliteracies learning over a 3-month period. fNIRS data were also collected when students engaged in the multimodal experimental tasks before and after the multiliteracies learning.

Multiple linear regression (MLR) and binary logistic regression (BLR) models were used to understand the relationship between reaction time, correct responses and six independent variables including multimodal background; gender; intensity, congruency and emotion types of stimuli and question types. Results showed that there was no significant change on behavioural data while the models for predicting the behavioural data changed after multiliteracies learning. The fNIRS data showed that brain regions related to emotion, language, and multi-sensory processing were activated as well as motor regions when students were viewing and listening to the stimuli before and after multiliteracies learning. Despite the limitations of the small sample size, this research has broadened neuroscientific findings on the association of multiliteracy learning’s impact on brain development and has helped educational researchers to understand multiliteracies from a behavioural and a neural perspective.

Download

Share

COinS