(from my dissertation: Visual Poetics: Meaning Space from Mallarmé to Metalheart, 2004)
This paper describes a simple empirical methodology for applying attention tracking to the literary study of visual texts. In particular I apply it to making sense of the notion “visual poetry”, as a tool to substantiate and explain “poetic” effects on the intimate cognitive grounds of readerly reception. The paper starts by identifying reading order and sequential processing as gateway questions for addressing visual language and visual poetry as concepts. It then takes up the more established methodology of eye tracking, reviewing the literature on reading and scene perception for insights relevant to an intermedial theory of visual reading. Describing also the limitations of eye tracking as a methodology, in particular the difficulty of concluding anything about mental attention and reading from the dry data on eye movements, I describe my attention tracking methodology as a radically simplified alternative, and argue for its superiority on this key point. I then report on an extended study conducted with Dr. Barbara Tversky (Psych) of Stanford University, and demonstrate how the analysis of attention can be used to reveal textual structures, ground interpretations, and explain literary effects in a visual reading experience. The theory of reading that emerges from this analysis is one that can easily embrace word and image in a coherent account, laying the foundation for a mature reception poetics of visual poetry and intermedial textuality in general.
After conducting a pilot study using very high-cost, head-mounted eye-tracking technology, I built a simple tool in Powerpoint, where subjects would simply place prepared colored dots on the image for each slide, to mark where they remember looking when they first viewed the visual poem or other visual sample.
The Flash-based online tool below from 2010 (development Lauri Aaltonen, graphic design Redouane Oumahi) was built as a prototype to display the functionalities necessary for easily carrying out Attention Tracking research along the lines of this technique. An argument for the full empirical seriousness of this method is made in the paper above, and supported by my research and test development with Prof. Dr. Barbara Tversky (D.Psych) Stanford, Psychology.
ONLINE TOOL: Attention Tracking Test – original Flash and Flex-based application prototype 2010 – with developer Lauri Aaltonen, screens Redouane Oumahi
Single Instruction: Mouse-click to mark the spots where you remember looking when you first viewed the image, in the order you now remember them; complete for max 9 spots.
Original research with Prof. Dr. Barbara Tversky, Stanford Pysch, on a research grant in the first year of Stanford’s Media X initiative 2000-2001.
Web content first published at http://visualpoetics.net/attentiontracking – (discontinued url)