What is your understanding of 'multimodality'? - Macquarie ICT Innovations Centre

(Part 2 of a six-part series on 21st century Literacy)

Multimodal text has always been a part of the English curriculum in NSW schools, but since the implementation of the new national Curriculum began in 2014, teachers are hearing a lot more about multimodality.

So, what is it? And why does it matter?

At its most basic definition, multimodal text refers to a combination of two (or more) modes – for example, text and images. In previous syllabus iterations it was interpreted simply as picture books.

But things have changed. And to gain a more comprehensive understanding of multimodality, and what constitutes multimodal text, we need to dive a bit deeper to understand multimodal literacies in a broader context.

According to Bull & Anstey (2010), a multimodal text combines two or more of the five semiotic (meaning-making) systems. They are LINGUISTIC (vocab, generic structure and the grammar of oral and written language), VISUAL (colour, vectors and viewpoints in still and moving images), AUDIO (volume, pitch and rhythm of music and sound effects), GESTURAL (movement, speed and stillness in facial expression and body language) and SPATIAL (proximity, direction, position of layout and organisation of objects in space).

Traditionally, a multimodal text would organise these modes into hierarchical structures with linear formats – think picture book or comic book with sequential framing integrated via an editing process, or a movie with a clearly defined series of events, or a play presented in Acts or chapters, a song in verses. They’re all stories with a beginning, middle, and end.

But technology tools and a highly networked internet context is changing the way students engage with text. Nowadays they are engaging with text in a non-linear fashion, often across different platforms in different modes, sometimes at the same time. They find and exploit multiple entry and exit points in their reading. They think about and respond to the world in the same way, and often the same time, as they experience it. Ken Robinson explains this in a more general context when he talks about creativity. He says that people “think visually, we think in sound, we think kinaesthetically, we think in abstract terms, we think in movement…” We don’t always think in linearity.

Viewing, responding, and representing text – whether imaginatively, persuasively, or informationally – across multiple platforms at different times are all part of the same conversation. It’s a world of transmedia (or cross-platform) text where story no longer necessarily has a beginning, a middle, and an end. In A Remaking pedagogy: adaptation and archetypes in the child’s multimodal reading and writing, education researchers Burker & Zezulkova (2016) call this a ‘user-sphere,’ a place where previously accepted lines between media and text no longer apply and because of this the traditional practices and processes we use to teach children to read and write should no longer apply either. They say that it’s time for us, as teachers, to challenge our own thinking around multimodal text and that “education policy and pedagogy needs to be ‘remade’ for a digitally synchronous world.” It’s understandable when you think about it. Literacy pedagogy was developed in an “analogue asynchronous medium and subject-specific world,” in response to industrialisation, and as we move from an industrial society to a much more technology rich society , it isn’t really very relevant anymore.

Ken Robinson is right when he says we need to “rethink the fundamental principles upon which we are education our children.” So too, are Burker & Zezulkova (2016) when they say that we, as teachers, need to redefine what we mean when use the terms ‘reading’, ‘writing’ and ‘literacy.’

How are your students viewing, responding to, and representing multimodal text in your classroom? How are you?