Challenging a Statistic : Why should we accept that 60 percent of adult Australians have low health literacy ?

This paper briefly considers Australia’s only national health survey published by the Australian Bureau of Statistics (ABS) in 2008 which has been widely referenced within the health sector. The main issue discussed is the use of a criterion level (level 3) to determine the point below which nearly 60 percent of Australian adults can be considered to have inadequate health literacy. The argument is made that this criterion level is arbitrary and statistically unjustified, yet it serves the purpose of presenting health literacy as a ‘crisis’ demanding action, which in turn represents the interests of dominant groups in this globalised, neo-liberal era. Introduction – a critical dimension to the Health Literacy, Australia 2006 survey When health literacy is discussed in Australian health forums it is commonplace for leading health researchers and organisations to cite the statistic that 60 percent of adult Australians have low or inadequate health literacy (e.g. Nutbeam 2009, Australian Commission on Safety and Quality in Health Care [ACSQHC] 2014a). To do so serves to answer an obvious question – what is the extent of low health literacy in Australia? And, moreover, it serves to indicate the severity of the problem, indicating that health literacy represents in some way a ‘crisis’ in Australia. The 60 percent statistic derives from the publication Health Literacy, Australia, 2006, to date R E F R A C T I O N S B L A C K 67 the only national survey of health literacy in Australia, undertaken and published by the Australian Bureau of Statistics (ABS 2008), the leading authority on national statistics. And possibly in view of such authority, the health research and professional community appear to accept this statistic uncritically. The point of this paper however, is to be critical – to question the 60 percent statistic; where does it come from, how is it determined and defined, and whose interests are served by it? In so doing the paper argues that despite its common and expedient use within the Australian health sector in recent years, this statistic is unjustified, and at the very least, requires qualification. The research and policy take-up of the survey The literature on health literacy in Australia since the release of the ABS survey in 2008 features a wide range of references to the survey statistics, including articles: promoting health literacy and its general application (Nutbeam 2009, Adams, Stocks, Wilson & Hill 2009), conceptualising health literacy (Pearson & Saunders 2009), and indicating its significance to primary health care (Harris et al 2010) and specific population sub groups (Ethnic Communities Council of Victoria 2012, Lambert et al. 2014, Lê, Terry & Woodroffe 2013, Velardo & Drummond 2013). A prime example of the unqualified acceptance of the 60 percent statistic is provided in the recent National Statement on Health Literacy which states: ‘Only about 40 per cent of adults have the level of individual health literacy needed to meet the complex demands of everyday life’ (ACSQHC 2014a:2). The fact that 60 percent of adult Australian do not have this level appears to comprise the rationale for the need to take action on health literacy. The report accompanying the National Statement is replete with the term ‘adequate’ and its implied corollary ‘inadequate’, to describe those who fall either side of this 40/60 percent divide, and how these percentages vary according to demographic factors such as educational levels, culture, language or gender (ACSQHC 2014b). To explain how the ABS arrived at the 60 percent statistic and why it should be challenged, it is necessary firstly to outline some details about the survey. The ALLS and the level 3 criterion The publication Health Literacy, Australia, 2006 uses information obtained from the Adult Literacy and Life Skills Survey (ALLS) conducted in Australia in 2006 (ABS 2006) which measured the knowledge and skills of 15 to 74 year olds in the following four R E F R A C T I O N S 68 L I T E R A C Y & N U M E R A C Y S T U D I E S domains: prose literacy, document literacy, numeracy and problem solving. The fifth domain was health literacy, produced ‘as a byproduct’ of the other four domains, and published by the ABS as a separate report two years later. The ALLS followed on from a number of international adult literacy surveys coordinated by the OECD and Statistics Canada since the mid-1990s which have enabled comparisons to be made of the literacy levels of adult populations across a range of Western nations. The simulated health literacy items for the ALLS were drawn from a comparable Canadian health literacy scale (Canadian Council on Learning 2007) which covered five health-related activities health promotion, health protection, disease prevention, health care maintenance and systems navigation. Using item response theory, skills in health literacy in the ALLS were measured according to scores for individuals across five levels, with level one being the lowest and level five the highest. While the survey methodology is internationally well established and has featured in several previous international surveys coordinated by the OECD and Statistics Canada, the critique in this paper relates to the decision by the ABS to present their findings using level 3 (of the five skill levels) as the criterion to determine those who have adequate health literacy and those who do not. The ABS health literacy survey (p 7) defines skill level 3 as the: minimum required for individuals to meet the complex demands of everyday life and work in the emerging knowledge-based economy (Statistics Canada 2005). Necessarily, any measurement tool which determines a cut-off point indicating those who do or do not have a particular level of skills for effective functioning in society is likely to be problematic, and the case of health literacy is no exception (see Barber et al 2009). But the argument in this paper is that the level 3 criterion represents a particularly misleading example. The above ‘minimum required’ quotation is the key to the ABS survey because primarily the survey’s findings relate to it. The ABS (2008:9) states in consequence that ‘Approximately 59% of Australians aged 15 to 74 years achieved scores below Level 3 for the health domain’. Thus, based solely on the above ABS quotation regarding level 3 as the ‘minimum required for individuals ...’, extensive references have been made in the Australian health literacy literature to the rounded up 60 percent of adult Australians who have low or inadequate health literacy. Level 3 in the survey has become for many researchers the accepted R E F R A C T I O N S B L A C K 69 benchmark for determining the percentage of those who have or do not have adequate health literacy in Australia. Critiquing the level 3 criterion To date the 60 percent health literacy statistic based on level 3 has not been examined in the health literature. However, in the related adult literacy sector, a study by the author of this current paper asks some key questions about the level 3 criterion in the ALLS, including: where did it come from, and on what basis was it determined, and by whom? (Black & Yasukawa 2014a). It was found for example, that the ‘minimum required’ quotation about level 3 that the ABS (2006) attributed in their report to ‘Statistics Canada, 2005’ did not in fact feature verbatim in that publication (though two references were similarly worded, see Statistics Canada & OECD 2005). Further, the history of the significance of level 3 in international adult literacy survey reports was found to be both minimal and obscure. Tracing back the reference links to the earliest mention of level 3, it was found that OECD/Statistics Canada publications referenced it to ‘focus groups and experts’ responsible for a 1992 literacy survey in the United States that actually preceded the first of the OECD/Statistics Canada coordinated international adult literacy surveys (Black & Yasukawa 2014a). The Canadian health literacy survey published a year before the ABS health literacy survey makes very similar claims to the ABS publication regarding the health literacy levels of Canadian adults, that is, the same overall 60 percent of adults are found to be lacking health literacy based on level 3 (Canadian Council on Learning 2007). However, unlike the ABS survey, the authors of the Canadian survey attempt to justify their use of the level 3 criterion with references to three organisational sources the US National Governor’s Association, the OECD and a Canadian skills organisation, all which deemed it to be the level required for labour market success. But judgements about literacy levels for labour market success are highly problematic and are not necessarily related to health literacy, and further, there was no explanation of how these organisations justified the level 3 criterion. Thus the key argument in this paper is that the use of level 3 as a criterion or cut-off point that determines those who have adequate health literacy and those who do not, is unjustified due to the absence of evidence to support it, a view shared recently by other commentators on OECD international surveys (St. Clair 2012). It is primarily with reference to this criterion level that claims about a R E F R A C T I O N S 70 L I T E R A C Y & N U M E R A C Y S T U D I E S particular percentage of adult Australians lacking health literacy can be made. Compounding the issue is the irony of the OECD/Statistics Canada and the ABS using level 3 as a minimum for functioning in modern society, whilst also promoting a concept of literacy that runs counter to it. From their earliest international adult literacy surveys the OECD/Statistics Canada and the ABS (1997) have seen literacy as a ‘complex and multidimensional set of traits, dispositions and competencies’, and that thinking in terms of a single literacy cut off ‘is neither appropriate nor conducive to formulating so


Introduction -a critical dimension to the Health Literacy, Australia 2006 survey
When health literacy is discussed in Australian health forums it is commonplace for leading health researchers and organisations to cite the statistic that 60 percent of adult Australians have low or inadequate health literacy (e.g.Nutbeam 2009, Australian Commission on Safety and Quality in Health Care [ACSQHC] 2014a).To do so serves to answer an obvious question -what is the extent of low health literacy in Australia?And, moreover, it serves to indicate the severity of the problem, indicating that health literacy represents in some way a 'crisis' in Australia.The 60 percent statistic derives from the publication Health Literacy, Australia, 2006, to date the only national survey of health literacy in Australia, undertaken and published by the Australian Bureau of Statistics (ABS 2008), the leading authority on national statistics.And possibly in view of such authority, the health research and professional community appear to accept this statistic uncritically.The point of this paper however, is to be critical -to question the 60 percent statistic; where does it come from, how is it determined and defined, and whose interests are served by it?In so doing the paper argues that despite its common and expedient use within the Australian health sector in recent years, this statistic is unjustified, and at the very least, requires qualification.

The research and policy take-up of the survey
The literature on health literacy in Australia since the release of the ABS survey in 2008 features a wide range of references to the survey statistics, including articles: promoting health literacy and its general application (Nutbeam 2009, Adams, Stocks, Wilson & Hill 2009), conceptualising health literacy (Pearson & Saunders 2009), and indicating its significance to primary health care (Harris et al 2010) and specific population sub groups (Ethnic Communities Council of Victoria 2012, Lambert et al. 2014, Lê, Terry & Woodroffe 2013, Velardo & Drummond 2013).A prime example of the unqualified acceptance of the 60 percent statistic is provided in the recent National Statement on Health Literacy which states: 'Only about 40 per cent of adults have the level of individual health literacy needed to meet the complex demands of everyday life' (ACSQHC 2014a:2).The fact that 60 percent of adult Australian do not have this level appears to comprise the rationale for the need to take action on health literacy.The report accompanying the National Statement is replete with the term 'adequate' and its implied corollary 'inadequate', to describe those who fall either side of this 40/60 percent divide, and how these percentages vary according to demographic factors such as educational levels, culture, language or gender (ACSQHC 2014b).To explain how the ABS arrived at the 60 percent statistic and why it should be challenged, it is necessary firstly to outline some details about the survey.

The ALLS and the level 3 criterion
The publication Health Literacy, Australia, 2006 uses information obtained from the Adult Literacy and Life Skills Survey (ALLS) conducted in Australia in 2006 (ABS 2006) which measured the knowledge and skills of 15 to 74 year olds in the following four domains: prose literacy, document literacy, numeracy and problem solving.The fifth domain was health literacy, produced 'as a byproduct' of the other four domains, and published by the ABS as a separate report two years later.The ALLS followed on from a number of international adult literacy surveys coordinated by the OECD and Statistics Canada since the mid-1990s which have enabled comparisons to be made of the literacy levels of adult populations across a range of Western nations.The simulated health literacy items for the ALLS were drawn from a comparable Canadian health literacy scale (Canadian Council on Learning 2007) which covered five health-related activities -health promotion, health protection, disease prevention, health care maintenance and systems navigation.Using item response theory, skills in health literacy in the ALLS were measured according to scores for individuals across five levels, with level one being the lowest and level five the highest.While the survey methodology is internationally well established and has featured in several previous international surveys coordinated by the OECD and Statistics Canada, the critique in this paper relates to the decision by the ABS to present their findings using level 3 (of the five skill levels) as the criterion to determine those who have adequate health literacy and those who do not.
The ABS health literacy survey (p 7) defines skill level 3 as the: minimum required for individuals to meet the complex demands of everyday life and work in the emerging knowledge-based economy (Statistics Canada 2005).Necessarily, any measurement tool which determines a cut-off point indicating those who do or do not have a particular level of skills for effective functioning in society is likely to be problematic, and the case of health literacy is no exception (see Barber et al 2009).But the argument in this paper is that the level 3 criterion represents a particularly misleading example.The above 'minimum required' quotation is the key to the ABS survey because primarily the survey's findings relate to it.The ABS (2008:9) states in consequence that 'Approximately 59% of Australians aged 15 to 74 years achieved scores below Level 3 for the health domain'.Thus, based solely on the above ABS quotation regarding level 3 as the 'minimum required for individuals ...', extensive references have been made in the Australian health literacy literature to the rounded up 60 percent of adult Australians who have low or inadequate health literacy.Level 3 in the survey has become for many researchers the accepted benchmark for determining the percentage of those who have or do not have adequate health literacy in Australia.

Critiquing the level 3 criterion
To date the 60 percent health literacy statistic based on level 3 has not been examined in the health literature.However, in the related adult literacy sector, a study by the author of this current paper asks some key questions about the level 3 criterion in the ALLS, including: where did it come from, and on what basis was it determined, and by whom?(Black & Yasukawa 2014a).It was found for example, that the 'minimum required' quotation about level 3 that the ABS ( 2006) attributed in their report to 'Statistics Canada, 2005' did not in fact feature verbatim in that publication (though two references were similarly worded, see Statistics Canada & OECD 2005).Further, the history of the significance of level 3 in international adult literacy survey reports was found to be both minimal and obscure.Tracing back the reference links to the earliest mention of level 3, it was found that OECD/Statistics Canada publications referenced it to 'focus groups and experts' responsible for a 1992 literacy survey in the United States that actually preceded the first of the OECD/Statistics Canada coordinated international adult literacy surveys (Black & Yasukawa 2014a).The Canadian health literacy survey published a year before the ABS health literacy survey makes very similar claims to the ABS publication regarding the health literacy levels of Canadian adults, that is, the same overall 60 percent of adults are found to be lacking health literacy based on level 3 (Canadian Council on Learning 2007).However, unlike the ABS survey, the authors of the Canadian survey attempt to justify their use of the level 3 criterion with references to three organisational sources -the US National Governor's Association, the OECD and a Canadian skills organisation, all which deemed it to be the level required for labour market success.But judgements about literacy levels for labour market success are highly problematic and are not necessarily related to health literacy, and further, there was no explanation of how these organisations justified the level 3 criterion.
Thus the key argument in this paper is that the use of level 3 as a criterion or cut-off point that determines those who have adequate health literacy and those who do not, is unjustified due to the absence of evidence to support it, a view shared recently by other commentators on OECD international surveys (St.Clair 2012).It is primarily with reference to this criterion level that claims about a particular percentage of adult Australians lacking health literacy can be made.Compounding the issue is the irony of the OECD/Statistics Canada and the ABS using level 3 as a minimum for functioning in modern society, whilst also promoting a concept of literacy that runs counter to it.From their earliest international adult literacy surveys the OECD/Statistics Canada and the ABS (1997) have seen literacy as a 'complex and multidimensional set of traits, dispositions and competencies', and that thinking in terms of a single literacy cut off 'is neither appropriate nor conducive to formulating sound policy strategies' (OECD & Statistics Canada 1995:24).More recently their publications claim that skills are defined along a continuum of proficiency and 'there is no arbitrary standard distinguishing adults who have or do not have these skills' (Statistics Canada & OECD 2005:15, 2011:14).And yet, the level 3 criterion adopted by the ABS does precisely this -it provides such a cut-off point, an arbitrary standard.

Promoting a health literacy 'crisis'
To be clear, the ABS health literacy survey provides some much needed population data for the health sector, especially on the relationship between health literacy scores and a wide range of demographic variables, including age, gender, educational attainment, parental education, labour force status, income and migrant characteristics.The critique in this paper, however, relates primarily to the use of level 3 as a criterion level for functioning in society, and the implications this has for health literacy in Australia.The 60 percent statistic for low or inadequate health literacy in Australia is powerful, as reflected in the recent National Statement on Health Literacy.More than any other statistic, it promotes health literacy as a 'crisis' that must be addressed because a large percentage of Australia's adult population, the majority in fact, is found to be lacking.Had the ABS instead reported their findings in terms of the percentage of the population (59 percent) falling below level 3 on the OECD's 5 levels of health literacy proficiency without any reference to level 3 as the 'minimum required' for functioning in society, then the main critique element in this paper would be invalidated.The decision by the ABS to report its survey findings using this criterion level can be viewed as a deliberate strategy designed to elevate the survey findings into national prominence as a 'crisis', a strategy which so far seems to have been successful as the ABS survey findings underpin both the health sector's National Statement on Health Literacy, and the adult literacy sector's National Foundations Skills Strategy for Adults (Standing Council on Tertiary Education, Skills and Employment 2012 -see critiques in Black & Yasukawa 2014a, 2014b) The OECD and the neo-liberal agenda The level 3 criterion level adopted by the ABS in its health literacy survey (and the ALLS), while largely unexplained in the OECD's own literature, needs to be seen as part of the apparatus of the OECD's international literacy surveys.These surveys are not politically neutral -they necessarily reflect the values and the worldview of the OECD, the leading international economic thinktank committed to the promotion of human capital in its current neoliberal form.The OECD, in major part through its educational efforts, and in particular through its international literacy surveys, promotes a neo-liberal version of worthy citizenry -individual consumers who are knowledgeable and autonomous (Sellar & Lingard 2013, Walker 2009).In relation to health, it is the ideal of people acting as selfmanaging individuals assuming personal responsibility for their health (Brown & Baker 2012).
In an editorial on health literacy in the Australia & New Zealand Journal of Public Health several years ago, Jamrozik (2010) made reference to the 'upswing of individualism' and a wider neoliberal climate in which identifying gaps in health literacy may encourage victim blaming.Similar arguments have been promoted by critical researchers in literacy education who view the OECDcoordinated international adult literacy surveys, and the level 3 criterion in particular, as 'technologies of neo-liberal governance' which deem people either 'fit' or 'unfit' to participate in society (Atkinson 2012).As explained in this paper, the use of level 3 to identify 60 percent of the Australian adult population as lacking in health literacy effectively helps to create a health crisis that marginalises many and demands a public policy response, finding form, for example, in the recent National Statement on Health Literacy.But we need to recognise and make explicit the politics behind the 60 percent statistic; that it results from the application of an arbitrary level 3 criterion which the ABS adopted from the world's leading neo-liberal think tank, the OECD, whose values and interests it necessarily reflects and promotes.