Serious Play , Serious Problems : issues with eBook applications

Maureen Henninger University of Technology, Sydney Abstract This paper explores the extent to which a digital technology (Apple i-Pad) supports the development of literacy skills. Literacy was traditionally deemed to include reading and writing and numeracy, but now encompasses digital literacies, the capacity to understand and even create the multi-media, multi-modal texts of digital technologies. Whereas once, basic literacy was considered essential to active citizenship, now there is a greater emphasis on this broader view of literacies. Along with the acknowledgement that individuals learn in different ways, there has been an emphasis on the features of digital technologies which facilitate the notion of “universal access”, making text accessible in a variety of forms to people with disabilities. Multi-modal technologies, many supported by applications designed by third party developers, were seen, therefore, to offer significant opportunities for helping people, especially children, to develop literacy skills. Using the metaphor of ‘serious play’, this study has found that people with disabilities may still be unable to develop literacy through their ‘play’ in the same way as others. The study concludes that a set of heuristic principles needs to be adopted by the developers of applications for digital technologies, so that the digital playground is accessible to all.


Introduction
The concept of a civil society as the basis of a functioning society in which citizens can participate in democracy is a phenomenon of the 20 th century (Habermas1989; Cohen & Arato 1992).For the past several decades there have been national and international efforts in development cooperation in which it is recognised that education in general and literacy in particular are necessary for developing a civil society."The essential feature of development is that it represents a transformation of society . . .[which] entails a transformation of the way individual persons think and behave . . .[and] education is at the centre of efforts at individual development".(Stiglitz 2003, pp. 77, 92).The role of literacy in the development of democratic societies is widely acknowledged (Sen & Alderice 2007;Nussbaum 2010;GCE 2001).One of UNESCO's goals in its Education for All program is "ensuring that the learning needs of all young people and adults are met through equitable access to appropriate learning" (UNESCO 2000, p. 8).The purpose of this study is to explore the extent to which a digital technology, the Apple i-Pad supports the development of literacy skills.

Literacy
The concept of literacy is multi-faceted ranging from a relatively narrow view of literacy as competence in reading, writing and numeracy to a broader view of the literate individual as "the flexible and sustainable master of a repertoire of practices with the texts of traditional and new communications technologies via spoken language, print and multimedia" (State of Queensland 2000, p. 6), to Gee's definition of literacy "as the mastery of or fluent control over a secondary discourse" (Gee 1989 p. 9).This study is concerned in the first instance with a narrow definition of literacy, the ability to read English, to comprehend what is being read and to build vocabulary.
It is what is described by Wells as the third stage of language development, "the freeing of language from its context of immediate experience" (Wells 1981, p. 241).Reading is an "information processing skill comprising a number of cognitive sub-skills that enable us to acquire purely visual information from a page and convert it into meaning" (Underwood 2000, p. 137). 1his study is also concerned with visual literacy and multimodal literacy -"the many modes involved in learning: the semiotic resources of image, colour, animated movement, writing, sound-effect, speech and so on" (Jewett 2006, p. 16).Kress et al. argue that communication and learning, two elements that are basic to literacy, are multimodal (Kress et al. 2001), involving image, speech, sound-effect, movement and so on.Jewett's research into the use of CD-ROMs for teaching school English challenged the traditional assumption that learning happens primarily via language (Jewett 2002).
The question of digital literacy cannot be ignored since "access to and proficiency in the manipulation of information, especially through electronic technologies, define in part the citizen of the modern democracy" (Martinez 1994, p. 395).Moreover, as new technologies develop, new literacies emerge, and thus new ways of communication -"literacy as technological deixis" (Leu & Kinzer 2000, p. 112).Katherine Au (2006) suggests that technology can be used to close the literacy achievement gap between students of diverse backgrounds and their mainstream peers, if employed effectively.

Serious Play and Digital Technologies
The past three decades have seen an increasing use of digital technologies in schools, from basic computers through to ever-increasingly sophisticated and portable digital devices: CD-ROMs, electronic games, table-top augmented reality (AR) environments, and electronic books including interactive books, and e-storybooks.These technologies are being used for many types of tasks such as finding information, self-paced learning and teaching and learning numeracy, reading, science and social studies.Increasingly, they are found in the home environment.
It has been recognized that the concept of 'playing' with technologies can have very positive outcomes in the development of literacies.Sara Lawrence-Lightfoot the Harvard sociologist stated in an interview with Bill Moyers in 1988(Moyers 1988), that "learning is at its very best when it's deadly serious and very playful at the same time."Thus the concept of 'serious play' has developed, defined by Rieber & Matzko (2001, p. 18) as "purposeful, or goal oriented, with the person able to modify goals as desired or needed".This concept has been well-documented in the education literature (Fromberg 1987;Mann, 1996;Ceglowski 1997;De Castell & Jenson 2003;Cassell 2004;Lillemyr 2009).Fromberg (1987) in her review of the literature notes that when children engage in any type of educational play, they are developing skills, including literacy development, and knowledge that contribute to future school success.Dünser & Hornecker's work (2007) on AR (augumented reality) books developed tangible elements to provide innovative ways for children to play and learn and to bring playfulness back into learning.In fact, Rogers and Sawyer state that "although play is not a necessary condition for learning language and literacy skills, play is probably the best environment for these abilities to thrive" (Rogers & Sawyer 1988, p. 64).
Early electronic books were delivered on CD-ROM and their use has generated a large number of studies (for example Chang & Osguthorpe 1990;Miller et al. 1994;Davidson et al. 1996;Dejean et al. 1997;Adam & Wild 1997;Lewin 1997;Blok, et al. 2002).Sixteen years and enormous advances in technology have passed since Miller et al. in their study of CD-ROM storybooks (1994) asked the question "what would happen if the children were given more choice?Students not only can click on the words to activate the help features, but they can also double-click or press and drag" (Miller et al. 1994 p. 203).CD-ROM books were limited in their ability to support 'serious play' because they did not support a two-way interaction, rather the flow of information is predominantly one-way, i.e. mono-logical, mediated by technology (Burrell & Trushell 1997).
The newer dialogical technology of interactive books is now widely available and has evolved to incorporate all of the features suggested by Miller et al. (1994).An interactive storybook is multimodal in nature providing "the reader with a predetermined storyline but places diversions under the control of the user" (Trushell et al. 2001 p. 390).Diversions, in this sense, are features diverting the reader away from the traditional linear reading path which Kress contends consists of "relatively empty things occur[ring] in a strict ordering which forces me to follow, in reading, precisely the order in which they appear" (Kress 2003, p. 3).
These diversions include elements such as non-linear progress through the text, cued animations, touch sensitive material, voice, animated speech (natural or synthetic), and speech recognition.
"Electronic books that incorporate these features [provision of narration, accompanied by animated pictures and sound effects that relate directly to the storyline ] have the capacity to increase children's comprehension and enjoyment of storybooks" (Grimshaw et al. 2007, p. 598).
There have been many studies of e-storybooks and emergent literacy skills (for example De Jong & Bus 2004;Doty et al. 2001Doty et al. , 2003;;Fisch et.al. 2002;Verhallen & Bus 2009;Moody 2010) and the general conclusion is that e-storybooks "may serve as a valuable tool for increasing children's exposure to text, creating opportunities for independent reading, and facilitating interest in text through use of animated digital features" (Moody 2010 p. 296).
There has also been criticism that some electronic books are not concerned with serious play but lean more toward 'edutainment'.'Eye-candy' -visual images that are pleasing to see but are intellectually undemanding (WordNet 3.0) -has been found to be distracting and "appears to correspond with poor comprehension" (Trushell et al. 2001, p. 400).While Underwood (2000) also reported that pupils' recall of the story of an 'interactive storybook' was poor, Miller in his earlier study had noted that "on-demand mediation features available in CD-ROM books played an important role in promoting reading improvements in terms of a diminution of 'search for meaning' miscues" (Miller et al. 1994).

Serious play for all
If we accept the concept of literacy as a basic building block for active citizenship, then it follows that the opportunities for developing the skills of literacy need to be available.And there needs to be standards of accessibility for all.This includes standards for tools and technologies by which all persons are able to access information and to understand it.The notion of a digital divide is one which has been discussed widely, particularly in regard to the availability of the Internet and other information technologies (for example, Castells 1999Castells , 2001;;DiMaggio et al. 2001;Milner 2006).
Lloyd's final report on interactive books (Lloyd et al. 2003) pointed out that in the mainstream classroom there are often three types of students: 'ordinary students' who have few, if any, problems reading; ESL (English as a second language) students, who may or may not be able to read in their native language, and students with special needs who have fundamental difficulties learning to read due to a variety of physical, neurological, or psychological problems they face.
In the case of the special needs students the challenge of learning literacy skills in an increasingly multimodal digital world has been demonstrated in studies such as those by Calabrò et al. (2009) and Savidis, et al. (2007) Among these emerging technologies are reading technologies such as CD-ROMs, electronic and interactive books.Furthermore, schools, under the digital education program are purchasing Apple iPad tablets for use in the curriculum, including reading.It follows that any reading device must be accessible to all students (Nielsen 1994;Lloyd et al., 2003;Donker & Reitsma 2004).
Lloyd et al. in their early evaluation of interactive books (Lloyd et al. 2003) tested an interactive book using a set of heuristics based on those developed by Jacob Nielsen (1994).They found that such a device needed flexibility to account for the different styles and problem-solving strategies of children who cannot read well.They also noted the importance for the system to "just work" (Lloyd et al. 2003, p.5), otherwise children lose interest in what they are trying to achieve, which is to learn to read, and they focus on the technology.
To ensure that learning materials are available to all and individuals with special needs are catered for, it is important that applications and software programs that are available in technologically-mediated learning environments, including the home, adhere to a set of usability and accessibility principles and guidelines.Designers of technologically mediated learning environments have a responsibility to be aware of inaccessibility issues and their implications in the use of digital technologies for developing literacy.At present there is no substantial set of guidelines or heuristics for universal accessibility which can be applied and the emergence of new technologies such as Apple's multi-touch iPad is bringing many to question the effectiveness of the ideal notion of ubiquitous and pervasive computing.To support the principle that there should be many approaches to solving problems and diverse ways to access information, Apple, like other producers of multi-modal technologies, has made its software development kit available to third party developers and many of the 'apps' which enhance the interactivity of digital technologies are created by these third party developers.This code is available, for a price, to anyone who wants to develop applications for the iPhone, iPad, and iPod touch devices.
This study sets out to explore the extent to which the features of interactive e-books used on the Apple i-Pad facilitate 'serious play' in the development of literacy, with a focus on accessibility for the visually impaired who would normally use the VoiceOver function.The eBooks in this study were developed by third-party developers, using Apple's proprietary programming code called Objective-C.

Method
The study involved a qualitative analysis of interactive eBook applications developed for the Apple iTunes database and therefore readily available for use in the home.The analysis was undertaken in September and October 2010.Ten interactive eBook applications were selected via the Apple iTune database.The selection was based on three criteria (see Table 1): • user star rating of 3.5 and above; • iTune's ranking within the top 100 downloads (based on popularity); • the possibility that the 'text' would be used in primary school classrooms in Sydney, NSW (established through personal knowledge and informal interviews with primary school educators).To determine their capacity for facilitating 'serious play' and developing literacy skills, these ten interactive eBook applications were analysed using the WCAG 2.0 Accessibility Guidelines (W3C 2008).This technical analysis was further informed by the heuristic methodologies used by researchers such as Bernsen & Dybkjaer (2010), Bertini et al. (2006), Billi et al. (2010), and Nielsen (1994Nielsen ( , 2010)).These guidelines were initially developed for Web applications but have since been applied to the evaluation of mobile devices and applications (see Billi et al. 2010).
The fundamental principles of these guidelines are perceivability, operability, understandability, and robustness (POUR).
Cosmopolitan Civil Societies Journal, Vol.3, No.2, 2011 Perceivability refers to the information and the onscreen interface; the ways the design and presentation allow perception through the use of senses.To measure perceivability this study asked questions concerning perception from the point of view of a visually impaired user.With the VoiceOver tool enabled, how is a user able to navigate and access the interface and the information of the interactive eBook application?Do the applications themselves offer internal customisation options, for example text enlargement, internal voice over, or font style variation.
Operability is the functionality of the interface, how the user actually operates and navigates through it.This was measured through answers to a series of questions concerning the users' engagement with and navigation of the interface when the VoiceOver tool was enabled.Was the reading experience enhanced or hindered?Were users able to navigate the interface successfully?
Understandability is about the content and whether or not it is readable and therefore understandable.It takes into account both textual and semiotic meaning-making principles.This was measured by asking the single question is the information and content within the eBook applications easy to read and therefore easy to understand.
Robustness is taken to mean "the content must be robust enough so that it can be interpreted reliably by a wide variety of user agents, including assistive technologies" (WCAG 2.0) and was measured through answers to the following questions: can the content of the eBook application adapt to personalised abilities and skills requirements when accessibility tools such as the VoiceOver are enabled on the Apple iPad device?

Perceivability
Human beings use the five senses (sight sound, touch, taste and smell) to "perceive" the world around us.In a computer-mediated environment, it can be argued that only three of these senses are actively engaged with, the sense of smell and taste being limited to a cognitive memory and/or imaginative process (for example, when we see a cupcake on the screen we are likely to imagine or remember what a cupcake might taste and smell like).We often use touch to engage with an interface, we use sight to see information and sound to listen and we sometimes speak within the computer-mediated environment.All of these senses, combined with cognitive processes such as memory, imagination, thought and learning, create a perceivable environment capable of engaging people.
Many digital devices offer accessibility features for individuals who are visually, hearing and cognitively impaired, so that they can also experience a perceivable environment when they use software, applications and files through the device.However, many applications limit the effectiveness of these features or even cause them to be disabled.For example, someone who is visually impaired and requires text on the page to be of a certain size can go to the system settings of the device and modify the settings to suit his or her need.However, if the tool is disabled when a new application is opened, then the information will not be perceivable and the person will be disadvantaged.
The focus in this study is the Voiceover tool, which allows the text of the story to be read aloud and which also vocalises instructions to the user.In the tests, we found that use of the VoiceOver tool, so important in the development of literacy and particularly important for the visually impaired, did not always work in its normal way and it also had a detrimental effect on the functionality of eBook applications.
The following examples demonstrate how users' perception of the limitations of the eBook applications when the VoiceOver tool is enabled.The purpose of an interactive eBook is to allow readers to interact with images on the screen and to engage sight and sound in the process of learning to read.If you touch an image of a cow, you can expect not only to see the word 'cow' appear on the screen, you would also expect to hear the cow make a mooing sound.Our tests showed that this use of the senses was disabled when the VoiceOver tool is turned on.Many eBooks have two reading options 'read it myself' and 'read it to me'; of the ones we tested some had both.In either case, when the VoiceOver tool was enabled, the text could not be read aloud, again leading to a limitation of perceivability.In How to Train Your Dragon, the only eBook with background music throughout, the music conflicts with the VoiceOver.When the VoiceOver tool was required to read navigation options such as turning the page, the music did not quieten, leaving the user unable to distinguish the VoiceOver instruction.

Operability
The iPad is a multimodal device, emphasising interactivity through touch, hearing and visionsensing.It is heavily reliant on visuals to convey information; therefore all the applications developed for this device have graphical interfaces.Furthermore users are required to perform many touch actions such as 'swipe up', 'swipe right to left', 'tap once', 'tap with three fingers' etc.In our evaluation of the interactive eBook applications human perceptual functionality (see Bernsen & Dybkjaer, 2010) did not become an issue until it was tested against interactive design elements of consistency and accessibility.
The operability of the application on the iPad was one of the major areas of concern.Consistency of the interface functions is always important, but especially so for users who require accessibility tools.Our study showed that three of the eBooks (Alice for the iPad, The Tale of Peter Rabbit and Toy Story 2 Read Along) ceased to function entirely when the VoiceOver accessibility tool was enabled.The remaining seven eBooks tested all presented inconsistent functionality errors including disabling of page turning, navigation inconsistencies and 'alt text' confusion (for full results of this testing see Table 2 in the Appendix).
One of the major conventions of the Accessible Web (WCAG) is to provide a textual description (alt text) for navigation buttons and links.In practice when a screen reader program 'reads' the button or link, it vocalises something meaningful to the user.The majority of eBooks tested produced confusing button text VoiceOvers.For example when the arrow button (for turning the page) is pressed the user should hear 'click here to turn to the next page'; instead for example, in Mickey's Spooky Night Puzzle the user hears 'arrowrightapp2x', which is not a very helpful instruction.
In May 2010 usability pioneer, Jakob Nielsen, conducted an initial test of the iPad's usability and one of the major areas of concern for him was the inconsistency of the interaction design.He stated in his report that, "once they [the users] do figure out how something works, users can't transfer their skills from one app to the next.Each application has a completely different user interface for similar features."This was also evident in our study; a case in point is the page turning.Six of the eBooks required an interaction with arrow buttons (left and right) to turn the pages however two of these (Mickey's Spooky Night Puzzle and Shrek Forever After) presented the buttons at the top of the screen instead of the normal bottom position.The other four applications used a 'swipe' action from the right-to-left of the screen.While such inconsistencies are annoying, most readers are able to adapt.However, other, potentially more serious inconsistencies in the interaction design were apparent when conducting accessibility tests on the eBook applications.
The VoiceOver tool caused changes to navigation.For example, the simple action of opening a link or selecting a navigation option required a double tap gesture on the screen; to go to the next page required a three-finger swipe instead of one-finger swipe.In most of the eBooks tested (for example The Tale of Peter Rabbit, The Cat in the Hat, and Green Eggs and Ham) the testers were not initially aware of these changes and there was nothing to indicate how to proceed, so that the eBook application became potentially unusable.These functionality issues are detrimental to successful operation of the eBooks and could have a significant impact on the potential learning capabilities these tools and applications offer users with disabilities.

Understandability
Children learning to read soon learn to associate a written word with an image and similarly to associate a spoken word with a written word.The Tale of Peter Rabbit offers multiple ways in which a child can engage with and learn from the text.The text is presented clearly on the righthand side of the "book" and as the narrator speaks the words on the page, each individual word is highlighted in order to demonstrate both audibly and visually how the word is spelt and pronounced.A child, carer or teacher can then select an individual word again by touching it on the screen and the narrator will pronounce that word again, exactly how it appears.The text is then accompanied by illustrations, with which you can actively engage, that produce a visual, illustrative meaning to accompany the words further enhancing the overall understanding of the content within the book.However, there is little in the way of instructions throughout and a lot of the navigation is visually bounded, making it difficult for children with visual impairment to be engaged in this process.
This convention that links a written word with an image and a spoken word is applied inconsistently in the eBooks, so that there is sometimes no correlation between text, sound and image.For example, in The Cat in the Hat, users have the ability to 'tap' on an image and the narrator speaks the name of the object, such as 'ball' and the word 'ball' appears on the screen.
However, when the user touches the bicycle the user is presented with the word and narration of the word "play" and when users tap on the chair image, they are given three different responses (sit, sat, jumped) before the word 'chair' appears.This example indicates inconsistencies that lead to verbal and semiotic confusion, thus weakening the application as an aid to the development of literacy skills.

Robustness
For an individual with no special needs and who is comfortable working with technology, the interactive eBook applications provide reasonable opportunities to develop skills in literacy and in digital literacy.However, as the findings above have shown, those individuals who need to use the iPad's VoiceOver tool find their access to the multi-modal capacities of the interactive eBooks limited in a number of ways.Applications and devices need to be designed and developed with the user in mind and developers must acknowledge that users have diverse abilities, needs and skills and therefore may have to adapt and personalize the technology (Bernsen & Dybkjaer, 2010;Bertini et al., 2006;Billi et al., 2010).While most of the eBooks we tested had options such as turning sound on/off and enabling narration, (Alice for the iPad and Animalia did not offer narration) none offered customisation options such as setting brightness/contrast, enabling VoiceOver buttons, enlarging text, and changing font type.Lack of such options ignore the principles of accessibility and affects the robustness of an application.
As can be seen from the above analysis of the impact of the VoiceOver tool on the functionality of the ten interactive eBooks in this study, there are a number of factors which limit the promise of these multi-modal technologies in facilitating the development of literacy skills and which further disadvantage those with visual impairment wishing to use the VoiceOver tool to access the content of the application.The eBooks are applications which have been created by third party developers, who as noted above may be amateurs or may be employed by major design studios.The fragmentation of approaches to the development of interactive applications is both a great strength and a significant weakness.It brings creativity and flair to the presentation of information and the design of applications, encouraging playful approaches to diverse problems.
At the same time, it leads to inconsistencies in design and incompatibilities with what are taken as basic functionalities.The demands of accessibility add a significant level of complexity to the development of interactive applications.The existence of a code and heuristics for designers and developers would lead to applications with higher levels of functionality and better accessibility.
The foundations for literacy, including digital literacy, are laid at an early age.Interactive eBooks, used in schools and found at home, are significant tools in the development of this foundation.Guidelines for designers and developers are an essential part of their toolkit, broadening the development of their skill set beyond that required for basic interactive functionality.These, in turn, would support the development of the literacies which are significant for life in the 21 st century.

Conclusion
The issue of the accessibility of interactive eBooks for children is a significant one for contemporary society.The findings of this study are timely as multi-touch digital technologies are relatively new and the identification of the weaknesses in current applications could go some way to ensuring that future generations of technologies can support the development of literacies rather than add to the confusion of young learners.The standards for universal accessibility to the content of information and communication technologies are well recognised and their consistent application by all designers and developers would foster the development of digital literacies, and consequently enhance the development of active citizens.
This study is attempting to open a conversation on this important topic.We seek to shift the debate from technical questions of functionality which tend to preoccupy application developers to the broader implications for civil society of widely used technologies which do not (yet) provide strong support for the literacies that will underpin active citizenship for all.

.
Under the current Australian government's digital literacy program there are many initiatives "to ensure that students across Australia have the necessary skills to engage in a digital environment.Programs are . . .focused on building and development of ICT technologies and developing competence with new and emerging technologies" (Commonwealth of Australia 2009, p. 2).

Table 1
Selection of interactive eBooks for evaluation