Assessing the Effectiveness of a Longitudinal Knowledge Dissemination Intervention

Sharing Research Findings in Rural South Africa

Rhian Twine
Kathleen Kahn
University of the Witwatersrand

Gillian Lewando Hundt
University of Warwick

Gateways: International Journal of Community Research and Engagement
Vol 10 (2017)

© 2017 by R Twine, K Kahn & G Lewando Hunt. This is an Open Access article distributed under the terms of the Creative Commons Attribution 4.0 Unported (CC BY 4.0) License (https://creativecommons.org/licenses/by/4.0/), allowing third parties to copy and redistribute the material in any medium or format and to remix, transform, and build upon the material for any purpose, even commercial, provided the original work is properly cited and states its license.

Citation: Twine, R, Kahn, K & Lewando Hunt, G 2017, ‘Assessing the effectiveness of a longitudinal knowledge dissemination intervention: Sharing research findings as part of knowledge brokerage in rural South Africa’, Gateways: International Journal of Community Research and Engagement, vol. 10, pp. 143-163. doi: 10.5130/ijcre.v10i1.5111

Corresponding author: Rhian Twine; Rhian.Twine@wits.ac.za

DOI: http://dx.doi.org/10.5130/ijcre.v10i1.5111

ISSN 1836-3393

Published by UTS ePRESS http://epress.lib.uts.edu.au/journals/index.php/ijcre/index

Health and demographic surveillance systems (HDSSs) carry out longitudinal research and operate in geographically defined areas (Sankoh & Byass 2012). Most HDSSs are located in sub-Saharan Africa and Asia, and are generally situated in rural, resource-poor settings. HDSSs collect population data including births,deaths, in-migrations and out-migrations, as well as health and socio-economic data. Following the baseline census of a defined geographic area, data is collected through regular census rounds during which household and individual characteristics are updated, and thus characteristics of the population living within the HDSS study area are monitored (Ye et al. 2012).

The setting for this article is the South African Medical Research Council/University of the Witwatersrand Rural Public Health and Health Transitions Research Unit (Agincourt) (MRC/Wits-Agincourt Unit) that has run a HDSS in rural northeast South Africa since 1992 (Kahn et al. 2012). The HDSS was established towards the end of the apartheid era in order to gather annual health and population data to inform the future development of a post-apartheid district health system (Tollman 1999). Despite progressive health/other policies in the post-apartheid era, inequalities persist (Naidoo 2012). Two decades after democratic change was introduced in 1994, findings from the annual census updates and nested health and social studies in the study area continue to contribute to health policy and planning in South Africa (Tollman 2008). These findings indicate rapid health, social and demographic transitions. The objectives of the MRC/Wits-Agincourt Unit have expanded to include reasons for these transitions, cross-site collaboration and facilitation of public access to datasets (Kahn et al. 2012).

The longitudinal nature of HDSSs necessitates the fostering of continuing relations between university researchers, participants, policy-makers and service providers. This is particularly important when there are inequities in power and information between the researchers, research participants and those who use the research information (Nuffield Council on Bioethics 2002) – as is the case in most HDSS settings. The concept of civic science (Bäckstrand 2003) promotes public engagement by research institutions with participants, policy-makers and the wider public as a strategy that addresses these inequities.

One strand of public engagement with research is the dissemination of research findings (Lavery et al. 2010). Knowledge dissemination is part of public engagement programs at some HDSS sites, as in the KEMRI-Wellcome Trust Research Programme (Marsh et al. 2008), the Navrongo Health Research Centre in Ghana (Tindana et al. 2011) and the MRC/Wits-Agincourt Unit in South Africa (Madhavan et al. 2007), but are not always routinely included, as in the Niakhar HDSS in Senegal (Mondain et al. 2016). Through the International Network for the Demographic Evaluation of Populations and their Health (INDEPTH) some HDSS sites work together in various research areas including migration and mortality; however, a common platform for work in knowledge dissemination across INDEPTH HDSSs is yet to emerge. There is increasing interest among funding agencies such as the Wellcome Trust and the Economic and Social Research Council UK for evidence around best practice in public engagement activities and public engagement practitioners are also beginning to form networks such as the online MESH Network supported by the Global Health Network.

This article examines a knowledge dissemination intervention (KDI) of the MRC/Wits-Agincourt Unit focusing on the annual sharing of research results to the population and service providers within it’s study area from 2001 to 2015. It presents a single, longitudinal case study (Yin 1994) of this KDI as part of broader knowledge brokerage activities, using the evaluation framework proposed by Lafrenière et al. (2013) to assess effectiveness. The main objective of this KDI is to share research findings with villagers, village leaders and service providers in the study area in order to: increase knowledge acquisition about research activities and study results; change the attitudes of participants and service providers so that multi-directional, collaborative discussion can occur regarding the relevance of research; and positively influence participants’ and service providers’ practices in individual and public health. We analyse data from annual KDI reports from 2001 to 2015, 762 feedback questionnaires of attendees, and qualitative interviews involving 60 local leaders/service providers undertaken in 2015–2016, and discuss the activities of the public engagement office (PEO), established in 2004 by the MRC/Wits-Agincourt Unit, while undertaking this KDI. In response to the research question, ‘What is the effectiveness of this KDI as measured by knowledge acquisition and changes in attitudes and practices of the residents and service providers in the case study area?’, the data suggests modest impact, and a number of ongoing challenges. In conclusion, the authors suggest ways to improve effectiveness, which would be of interest to other practitioners working in KDIs in similar contexts.

LITERATURE REVIEW

Terms such as ‘knowledge dissemination’, ‘transfer’ and ‘translation’ are often used interchangeably, as shown by Lafrenière et al. (2013) in their systematic review of the effectiveness of KDIs. A KDI can be defined as ‘an active intervention that aims at communicating research data to a target audience via determined channels, using planned strategies for the purpose of creating a positive impact on the acquisition of knowledge, attitudes and practice’ (Lafrenière et al. 2013, p. 2). KDIs can be implemented through for example meetings, debates and other interactive activities, websites, distribution of fact sheets and policy briefs, to a range of audiences ranging from lay persons to policy-makers (Mondain et al. 2016). KDIs have a number of components: a clear message, a specific audience, a particular format, a plan for delivery and an evaluation of effectiveness, which necessitates the articulation of a clear aim. The characteristics of the target audience will determine the wording of the message and the method that is used for its delivery and evaluation (Kothari & Armstrong 2011).

Despite nearly 20 years of calls for greater public engagement in health research (Dickert & Sugarman 2005; Tindana et al. 2007), there is relatively little evaluation of the effectiveness of KDIs. In 2003 it was reported that only one in 10 of 175 applied research organisations in Canada evaluated KDIs for their effectiveness (Lavis et al. 2006). In a systematic review by Lafrenière et al. (2013), 11 of 19 KDIs that had been evaluated for effectiveness focused on the dissemination of results to health professionals, not to research participants, (Bhattacharyya et al. 2011; Mitton et al. 2007; Ward et al. 2009) and generally showed changes in knowledge and attitudes but rarely in practices. Lafrenière et al. (2013) identified a framework for evaluating the effectiveness of KDIs, focussing on knowledge acquisition, changes in attitudes and changes in practices. They suggest that evaluating knowledge acquisition can be achieved by assessing if the KDI has increased participants’ knowledge base, while changes in attitudes can be assessed by determining whether or not participants agree with the information presented and could accept it. Changes in practices can be assessed through examining actions taken after the KDI. Apart from the general paucity of evaluation on the effectiveness of KDIs, there is a specific lack of evaluation on longitudinal KDIs (Madhavan et al. 2007).

Often researchers give less attention to the dissemination – and, by implication, reception – of research findings to participants and beneficiaries than they do to academic peers and policy-makers. The voices of participants and local service providers, especially in poorly resourced areas, are seldom considered, even when they are themselves expected to transfer research findings into practice (Molyneux & Geissler 2008). Knowledge dissemination of research findings, with interactive, multi-directional discussion between researchers, participants and service providers, can help in enhancing benefits (Tindana et al. 2007) and is part of the ethics of practice in research (Guillemin & Gillam 2004). Collaborative discussions about research can help to shift research agendas to be more relevant to the needs of participants and service providers, and this is particularly important in developing countries (CIOMS 2016).

There is currently an increased focus on the role of knowledge brokerage in developing collaborative links between researchers and stakeholders, as a means to increase knowledge transfer and translation, and build users’ capacities to apply relevant findings to policy and practice (Meyer 2010). There is increasing pressure on governments and service providers to develop evidence-based policy and practice (Gilson & McIntyre 2008, Strydom et al. 2010). This is slowly creating a ‘pull’ for the provision of relevant research results through knowledge brokers, moving from unilateral dissemination to multi-directional creation and use of information (Godfrey et al. 2010).

Theoretical approaches to knowledge brokerage include the dissemination model and the systemic model, and both identify interpersonal contact as essential to effective knowledge brokerage (Dagenais et al. 2015). A recent systematic review by Bornbaum et al. (2015) analysed 29 articles on the role of knowledge brokers and identified 10 key domains of knowledge brokerage activity (p. 5):

1. Identify, engage and connect with stakeholders
2. Facilitate collaboration
3. Identify and obtain relevant information
4. Facilitate development of analytic and interpretive skills
5. Create tailored knowledge products
6. Project coordination
7. Support communication and information sharing
8. Network development, maintenance and facilitation
9. Facilitate and evaluate change
10. Support sustainability.

We examine the two domains ‘create tailored knowledge products’ and ‘support communication and information sharing’, as they are the most relevant to this case study.

CASE STUDY: THE MRC/WITS-AGINCOURT UNIT HDSS

Setting and Objectives

The MRC/Wits-Agincourt Unit HDSS study area is located in the Bushbuckridge Municipal sub-district of Ehlanzeni District in rural Mpumalanga Province, South Africa. The area is 500 kilometres northeast of Johannesburg, separated from Mozambique by the Kruger National Park on its eastern boundary (Figure 1). The 1992 baseline census enumerated approximately 57 600 people living in 8900 households in 20 villages (Tollman et al. 1999), which, by 2015, had expanded to 115 000 people in 18 500 households in 27 villages (www.indepth-network.org/member-centres/agincourt-hdss). Some 30 per cent of the sub-district population comprises former Mozambican refugees, the majority of whom are now South African citizens or permanent residents (Twine et al. 2016). The majority of people living in the area are from the Tsonga ethnic group, and speak XiTsonga.

Figure 1: Location of the MRC/Wits-Agincourt Unit HDSS study area in South Africa and details of the study area

Many households practice supplementary farming, but land allocated during apartheid for resettlement is inadequate for total reliance on subsistence agriculture. Unemployment is high with most formal employment being male migrant labour in mining, manufacturing, agriculture and tourism. South Africa’s non-contributory social grant system are a major source of household income, together with remittances from labour migrants. Since 1994, with the dawn of the democratic era in South Africa, there has been infrastructure development with improved provision of electricity, roads, water and schools. Currently, there is one health centre and eight primary health-care clinics within the study area, and three district hospitals 25 to 60 kilometres away (Collinson et al. 2014; Kahn et al. 2012).

The 27 villages in the study area fall under three traditional councils, and three local municipal offices. For the purpose of this article, we define a ‘village’ as a cluster of households in a geographically defined area, which has a name and leadership structure, and is geographically separate from other villages. Each village has a head man (induna), who falls under one of the traditional councils presided over by a chief (hosi); traditional councils meet every week. Civic leadership operates at three levels: village-level community development forums (CDFs), wards with an elected ward councillor, and local municipalities. Each village CDF is made up of two representatives from every community-based organisation in the village, and includes the induna as a representative of the traditional council (www.agincourt.co.za/index.php/activities/linc/).

From 1992, public engagement activities were undertaken in relation to village-level consent and annual village-based dissemination of research results for every study. In 2004, a dedicated PEO was established by the MRC/Wits-Agincourt Unit, with contributions to its activities included in all research project budgets, in order to further develop knowledge brokerage activities in the study area. There are three full-time staff members at the PEO. Rhian Twine, lead author on this article, manages the office. She is a healthcare professional who has worked in the area for nearly 30 years; 15 years for the public health services in the district, and 13 years for the MRC/Wits-Agincourt Unit. The two public engagement officers she manages have extensive and long-term experience as fieldworkers/ supervisors of the census and nested research projects as well as in their public engagement roles (25 and 13 years respectively). Both are residents in the study area. A key activity is the KDI, the objective of which is to disseminate research findings to residents and service providers living within the study area. Below, we outline the KDI activities, grouped according to two of the domains of knowledge brokerage defined by Bornbaum (2015).

Creating Tailored Knowledge Products

From 1993 to 2002, only village-specific demographic data were presented. From 2004, three changes were made: GPS village maps with no research household identifiers were distributed to village leaders and service providers; oral and written summaries of HDSS modules on various topics were given, including food security, socio-economic status and uptake of social grants; and dissemination meetings included results from nested research studies.

Over time, village-specific fact sheets increased from two to 14 pages. From 2011, key take-home messages were highlighted at the end of every section and all the information provided was translated into the local language (XiTsonga). Since 2012, village and research project fact sheets have been available on the MRC/Wits-Agincourt Unit website (www.agincourt.co.za/index.php/activities/linc/#Village fact sheets). From 2015, content was simplified to ensure that people with no more than eight years of education could understand the information, using the ‘readability index’ in Microsoft Office Word.

Supporting Communication and Information Sharing

Village-based meetings: Since 1993, unit staff has presented aggregated village-specific demographic data through open village-based meetings in each village annually. The practice has continued and since the establishment of the PEO team in 2004, the KDI has been extended in its format, duration and breadth (see Figure 2).

Figure 2: Changes in KDI over time, 2001-2015

Since 2006, village-based meetings to disseminate research findings have taken place over one month (one or two village meetings daily) at the start of an annual six-month period. Three fieldworkers supplement the PEO for this month. On the prior evening, a public engagement officer drives around the village announcing the meeting using a roof-mounted megaphone. Before each meeting starts, the village leadership decides if there are enough people in attendance to proceed; most meetings are held under a tree or in a school. After each presentation, in which various topics are presented by different fieldworkers, audience questions are answered by the fieldworkers. At the conclusion, 50 copies of printed fact sheets on each topic presented are made available to the attendees, and the village leaders are presented with a folder containing the fact sheets as well as a map of their village. Generally, these meetings last for two hours. At the suggestion of local leaders, relevant service providers have been invited to attend since 2002.

The number of villages included in the meetings has increased, with the number of possible village-based meetings going up from 18 in 2001 to 30 in 2015; five new villages were built within the original study area as part of a government housing development program, and eight villages were added to expand the area population, largely to meet the needs of intervention trials. Actual meetings held were always fewer than those planned: out of 289 possible meetings over 2001–2015, 215 took place (74 per cent). Reasons for meeting failure include cancellations or postponements by village leaders if too few people attended, or other village activities that arose and took precedence.

Meetings with village leaders and service providers: During the following five months, PEO staff conduct face-to-face briefings with village leaders, community organisations and service providers, again giving each group a folder containing research data aggregated across the study area to assure confidentiality, and village-specific demographic data, in fact sheets.

Figure 3: A village-based meeting in 2009

Measuring the Effectiveness of a KDI

This longitudinal, mixed-methods case study of a KDI used multiple sources of data (Yin 1994), as shown in Table 1. Quantitative data were from 14 annual village meeting reports (2001–2015) that contained information on attendees, questions asked and requests for more information, as well as 762 feedback questionnaires that were collected from attendees over 10 years (2005–2015). Attendees were asked at the beginning of the meeting to volunteer to fill in feedback questionnaires with the assistance of a fieldworker after the meeting. The feedback questionnaires, completed after obtaining verbal consent, were largely (50–80 per cent in any one year) completed by younger adults, aged 18–34 years. The number of forms filled in varied depending on whether there was a general village-based meeting immediately following the KDI, the weather, individual willingness, and the meeting’s length. Owing to computer crashes, the 2003 annual report and the 2009 feedback questionnaires are missing.

Table 1: KDI dataset 2001–2015

The qualitative data are from 15 individual semi-structured and five focus group interviews with local leaders and service providers (60 participants in total) carried out in 2015–2016. The interviews were conducted in a mixture of English and XiTsonga and explored the participants’ views and experiences of the annual dissemination of research findings. The lead author (Rhian Twine) conducted the interviews with a local fieldworker, taped recordings of which were translated and transcribed by the fieldworker.

The 15 individual interviewees were service providers and traditional leaders within the study area: two traditional council secretaries from two councils (the third covered only one village), who suggested also interviewing one induna from each of their traditional councils; three ward councillors, who represented the greatest number of villages; both regional municipal managers; clinic managers from the three busiest clinics; and the two education circuit managers responsible for the majority of schools in the site. Four focus group interviews were held with the CDF chair and/or the health desk representative of each of the 20 villages that had been in the study area since its inception, and 3 added in 2007, and one focus group interview was held with the managers of the eight home-based care organisations in the area. Participants were aged between 25 and 70 years. Only four service providers were not resident in the study area, and there was equal gender representation. Quantitative data were analysed using summary statistics in Excel, as well as descriptive analysis. Qualitative data was analysed thematically using NVivo 10 (QSR 2012).

Ethical clearance was obtained from the University of the Witwatersrand’s Human Research Ethics Committee (HREC) (Medical) (certificate no. M140737). All research reported on within the KDI obtained separate ethical clearance from Wits HREC (Medical), the relevant Mpumalanga Province Research and Ethics Committee and, if undertaken with international collaborators, their institutional ethics committees.

Findings

The findings are organised according to the three outcomes for the measurement of effectiveness of KDIs: knowledge acquisition, changes in attitudes and changes in practices (Lafrenière et al. 2013).

Knowledge acquisition

Service providers and village leaders were asked what information was presented and discussed through the KDI. The responses show an understanding of the relevance of the findings to their villages. Census findings were always mentioned first, showing that these were the data with which they were most familiar. Village leaders and service providers found demographics and maps more useful for planning than other results.

I’m glad we have Wits in our community because they are giving us the figures of the people living in the specific villages, and it helps us to know how many people have died each year. It also helps us to know the figures of the children who were born. We are also able to know the people who migrate outside and those [who] immigrated into our village. (CDF member, woman)

The findings from nested studies, added from 2004 onward, were also found to be useful.

When Wits came and gave feedback, there were youth and elders in that meeting. When Wits gave them the results about what is happening to the youth about HIV and TB [tuberculosis], they learnt something, they were asking questions. Even the elders were interested in knowing something. (CDF member, woman)

Respondents, who filled in feedback questionnaires after the village-based meetings, reported that the most useful information was about HIV and tuberculosis (34 per cent), causes of death (19 per cent) and village demographics (16 per cent). A few appreciated learning more about the work of the MRC/Wits-Agincourt Unit (4 per cent) and how to apply for a job within the unit (1 per cent). Information about research results would appear to be more important than other aspects of the unit’s work.

Changes in attitudes

The types of questions asked at village meetings and targeted briefings illustrate changing attitudes in relation to both the research activities and research results over time.

Figure 4 illustrates a steady increase in the proportion of questions related to research results, and a concurrent decrease in requests for government services, and for services from the unit, until 2014. The latter coincided with a lower presence of service providers at the meetings (discussed later). From 2014, enrolment started in the first major randomised control trial in the study area (Pettifor et al. 2016). Due to the scale of the trial, four new villages were added to the study area; data showed these villages had a higher proportion of requests for services than did villages that had been part of the study area since 2001 (Figure 5). This suggests that villagers with a longer exposure to research activities and the KDI had a clearer understanding of the university’s role, and were engaged in discussions about research rather than service provision. This provides evidence of changes in attitudes (and knowledge) about the work of the unit.

Figure 4: Proportions of questions asked at village-based meetings 2002-2015

Figure 5: Questions asked in 2014 - comparison of 23 older villages and four new villages included since 2013

Examples of questions asked at village-based meetings are included below. The research topics presented, as well as which service provider attended the meetings, directly influenced questions asked. For example, in 2006, when data on access to child support grants were presented, with child support grant extension officers present, 45 of the 129 questions related to this issue. From our analysis, questions can be grouped into four main categories:

1. Research results: How do you include people in the census who out migrated from the village? (2014); How do you recruit participants for studies? (2015)
2. Requests for services from the research unit: Can you assist people with epilepsy to get a wheelchair? (2011); Can Wits do something about bilharzia, because if we go to the clinic they don’t help us and our children keep urinating blood? (2011)
3. Health: If I’m HIV-positive and sleep with someone who is also HIV-positive, what’s going to happen? (2014); How do I know I have heart disease? (2015)
4. Requests for government services: We are drinking water from wells and dams and the water is not healthy. Where can we get water for the vegetables we have planted? (2008); How can you help an older person who does not have a pension, but who also doesn’t have an identity document, carer [or] relatives? (2008)

This suggests that the results were accepted as relevant and applicable to both individuals and service providers in their villages and the surrounding area.

Changes in practices

At meetings, villagers directly questioned service providers, when available, using research results as proof to request further services. There is evidence that service provision was sometimes modified in line with such concerns; for example, after hearing requests for the mobile health clinic service to resume, a clinic manager reinstated it. In another village, pit latrines were supplied soon after presentation of data on households with no latrines.

Over 2001–2015, a total of 762 people completed feedback questionnaires (see Table 2). Of those, 397 had attended village-based meetings the year before; 54 per cent of this group reported that the information motivated them to work or volunteer, while 14 per cent lobbied for services and 3 per cent took no action at all (Figure 6). The ‘other’ category (25 per cent) included activities such as using the information to teach the youth, starting a vegetable garden at home, and encouraging other villagers to participate in research studies.

Figure 6: Reported activities undertaken subsequent to the previous year’s KDI

Further, out of the total 762 respondents who completed feedback questionnaires, 117 (15.4 per cent) had attended a previous meeting where handouts (tailored knowledge products) were distributed: over 50 per cent of this group reported not using the information, 25 per cent said they had shared the information, and only 8 per cent said the information had led to an improvement in their health behaviour, such as having their blood pressure measured.

The interviews with village leaders and service providers revealed that, for this group, the information was seen as useful for planning services, student assignments and reports.

We checked how many people were in our villages, and then worked with home affairs. We have 14 000 people in total but only 8000 people have IDs, so we started a campaign with home affairs. (Ward councillor, man)

In our village, we looked at the results and found that our village is too small. We went to the chief to request to extend our village by 500 new stands [plots to build houses on]. (CDF member, man)

It helps us when we do reports because we can quote that, according to Wits, in village X we have got 700 households and 5000 people. (Traditional council secretary, man)

Sometimes they use information in the folders in our schools to set exams, like HIV/AIDS or census information. (CDF member, man)

The questions below were asked at village leader and service provider briefings, and show a commitment to translating research into policy and practice:

I see that there are not so many 11 to 14 year olds having babies, but there are some. Could you let us know how relevant household economic circumstances are, or if there are any other things we can learn about the households of these girls? Maybe then we can do something. (Ward councillor, man, after a presentation on fertility rates.)

According to the statistics presented, cardiac disease is a serious problem in people aged 50 and above. Why is it like that? What can we do in order to solve this problem? (Clinic manager, woman, in response to a presentation on causes of death.)

In the last five years there have been requests after KDI meetings for further information from villagers, students, leaders, service providers and political organisations for data for use in changes of practices. Women who were starting crèches in the villages requested information on numbers of children under five years in/out of preschool, and village maps indicating sites of preschools, to support their funding requests. Social work students have needed information for assignments, and high school teachers have requested statistics on HIV for teaching purposes.

Ward councillors have also requested information, such as the socioeconomic and refugee status of households with teenage mothers. The South African Police Service asked for data on suicides and deaths due to violence in the area as evidence to support an application to set up a satellite police station, which was subsequently established. Managers of nearby private game lodges have requested village fact sheets annually for their social responsibility offices. Municipal representatives of the African National Congress, the governing party, ask annually for all village fact sheets for service provision planning.

Challenges and Limitations

To get a better understanding of the extent of the effectiveness of the KDI, coverage is a consideration. The village-based KDI meetings have attracted 2 to 4 per cent of the adult population over 15 years. There is some variation, ranging from 1.5 to 4.3 per cent, as shown in Figure 7, with a significant (p<0.05, R2=0.272) but weak decline of 0.17 per cent in attendance. This may be due to many of the meetings being held during the week, thus excluding those employed, inadvertent scheduling during cold weather, funerals, and political disputes between village leaders.

Figure 7: Percentage of total population over 18 years attending village-based KDI meetings, 2001-2015

Village leaders gave various explanations for the stable but low attendance rates and for sometimes having to cancel meetings. These were mainly villagers not understanding the relevance of the research feedback, as well as internal village politics. Since 2011, a local government election year, leaders have used the KDI meetings to raise other issues such as elections, water crises and employment with villagers, indicating that these meetings have become a platform for airing critical issues. One village leader suggested that the dissemination should be added onto existing meetings, such as those held by Department of Agriculture as, in his opinion, more people attended these.

The data from the feedback questionnaires sheds a different light on why people don’t attend meetings. Of those who completed questionnaires, 47 per cent were new attendees. As shown in Figure 8, reasons given for not previously attending fell into two major categories: inability to attend (not living in village, visiting/studying elsewhere, not available) and organisational (did not know about it, meeting too early, venue too far, meeting did not happen). Only 1 per cent expressed ‘lack of interest’ as a reason for non-attendance, indicating an acceptance of the data and its possible usefulness.

Figure 8: Why people did not attend previous year’s meeting

Additionally, despite being invited annually and transport provided, some service providers such as social workers, local youth development NGOs, child support grant social securityofficers, home-based carers and municipal workers have not attended regularly. Health-care providers, ward councillors and community development workers have attended more often. In 2004, all but one of the 17 meetings were attended by service providers, compared to 2014, when service providers attended only seven meetings. The reasons for non-attendance were mainly related to heavy workload. This means that service providers seldom gain knowledge regarding research results, and participants do not benefit from information from service providers during village-based meetings.

Wits invites us, but most of the time I fail to take part in those events because I have to attend to some other community issues. (Induna, man)

Service providers reported that handouts were often left in a folder, and some admitted not reading the information. While appreciation was expressed for the translations into XiTsonga, some felt these were not always correct, and some found the font too small. People preferred attending meetings to reading information.

We get the results in writing but I personally enjoy when we sit down together so that if I have questions then I can ask and you clarify those points that I could not understand as I was reading the report. (Education circuit manager, man)

DISCUSSION

The systematic review of KDIs (Lafrenière et al. 2013) identified specific outcomes for the assessment of effectiveness of a KDI: knowledge acquisition and changes in attitudes and practices. This single, mixed-methods longitudinal case study of the dissemination of research results from an HDSS in a rural setting has focused on the effectiveness of the annual KDI using this framework.

This case study does show some evidence of change in knowledge acquisition, but only to a limited extent. Village-based meetings attracted only 2 to 4 per cent of the population, and even if 25 per cent of attendees shared the information gained, knowledge acquisition through village meetings was modest across the study area. Although the audience is not growing, it does not remain static, with different people attending every year. More innovative methods of alerting community members of the upcoming KDI, such as via local radio and strategically placed posters, may increase audience numbers. Even though the number of attendees has been consistently small as a proportion of the village population, village leaders and political representatives have utilised the meetings since 2011 as a platform for discussion of topical village issues. This shows that the meetings have become embedded and routinised, and are considered a useful forum for debate. There have been instances when the amount of time given to research feedback has been compromised owing to the need for village leaders to address the audience on village matters, but, generally, feedback has been allowed to continue as planned with the audience remaining once the PEO staff has left. Since the feedback sessions are already two hours long, the PEO does not make use of village meetings called by other organisations for this KDI, in line with the principle of respect for villagers and service providers underpinning all PEO activities.

Although village-based meetings are limited in terms of coverage, face-to-face briefings with service providers and village leaders show some effectiveness in knowledge acquisition as evidenced by the types of information requested by service providers and village leaders after the KDI activities. This is important, as service providers seldom attended the village meetings, and often missed targeted service provider briefings owing to workload. Given that participants seldom miss meetings owing to lack of interest, and that face-to-face briefings are preferred while hand-outs are seldom read, it would be important for different strategies to be used so as to enable participation. Results clearly show that the KDI had limited effectiveness when solely based on a linear approach and was more effective when multi-pronged. The importance of face-to-face interaction over time has been noted by recent studies and also that varied strategies can be used with different stakeholders (Conklin et al. 2013; Dagenais et al. 2015). This could mean that more frequent meetings are needed, with smaller audiences, which would require concurrent increased human resources in knowledge broker offices.

In regard to changes in attitudes, in villages that had recently been added to the study area, more service-related questions were asked compared to villages that had been in the study area longer, where more questions on research results were asked. These trends suggest a change in attitudes concerning the role and work of the unit, with growing understanding and acceptance that the role of a research unit is not to deliver services, but to undertake research. This is evidence of increased interactive dialogue (Lavis et al. 2003). KDIs, such as this one, do contribute towards changing the attitudes of participants and enhance the possibility of collaborative discussion regarding the relevance of research and research results.

Lastly, there was some evidence of changes in practices, with a few attendees reporting that their health behaviour had altered subsequent to attending a meeting, and a few reporting that they had been motivated to volunteer/work in community projects. Service providers and village leaders had used demographic data for planning at the village level. There was also some evidence of public health service delivery improving after data highlighting these issues were presented.

Implications for Knowledge Dissemination Interventions

We would argue that the process of organising and delivering this KDI is central to knowledge brokerage and supports other domains such as networking, developing collaboration with stakeholders, supporting the sustainability of the HDSS, and building local capacities through the interpretation of research data (Bornbaum et al. 2015).

The MRC/Wits-Agincourt Unit has committed resources for the establishment of a dedicated PEO, which had a clear brief to manage the KDI as knowledge brokers (Bornbaum et al. 2015), and reflects a growing partnership with stakeholders contributing to an increased understanding of the role of research unit and its data by the villagers, leadership and service providers. This has occurred in the social context of the evolving democracy of post-apartheid South Africa, which has involved everyone in a growing awareness of both rights and responsibilities as well as the planning of increased service provision in health, housing and education.

Considerable time and effort was spent writing and translating fact sheets, which proved of limited use. Other methods of dissemination such as theatre, which has been used to effect in this setting (Stuttaford et al. 2006), postal drops of small, focused A5 pictograms, community radio, TV and social media may be useful in communicating results. Radio-based soap operas or talk shows (edutainment) have been effective in engaging the public with health research in Malawi (Nyirenda et al. 2016) and South Africa (Jana et al. 2015). Longitudinal HDSS sites have an opportunity to develop strategies for regular information sharing through community advisory groups (Reddy et al. 2010) and wider village-based dissemination. While difficult to do, it would be important to clearly articulate the expected outcomes of these different strategies in order to evaluate their effectiveness if undertaken in HDSS sites.

The effectiveness of the KDI in this case study has been assessed in relation to three outcomes: knowledge acquisition, changes in attitudes and changes in practices (Lafrenière et al. 2013). There is evidence of changes in all three outcomes over time; it is doubtful if changes would have been evident without a longitudinal approach. In future KDI activities, clearer, measurable objectives will be needed in order to measure effectiveness more rigorously and information disseminated and methods used need to be adapted further to be more specific, useful and contextual (Legaspi & Orr 2007).

CONCLUSION

Sharing research results with study participants and stakeholders is part of the ethics of practice (Guillemin & Gillam 2004). This links to civic science, which frames research as a public good (Ward et al. 2009). Results from this case study of a KDI to communicate research results across the population of the MRC/Wits-Agincourt Unit study area can be used to inform knowledge brokerage and KDIs in other areas with longitudinal studies (Bornbaum et al. 2015). The results show how this KDI developed from linear presentations with little engagement to multi-pronged, diverse activities (Ward et al. 2009), with some impact on knowledge acquisition, attitudes and practices (Lafrenière et al. 2013). The process of evaluating this KDI has been valuable to the PEO and the results have led to change in knowledge and practice within the office itself. For example, upon realisation of the limited reach of the village-based dissemination meetings, a simple infographic is now distributed annually to each household, alongside the continuing village-based and service provider meetings. Fact sheets are clearer, with shorter messages, and more serious thought is given to both the content of the message and how it is conveyed. A limitation of this study was that the evaluation tools were designed for routine use and not for rigorous analysis. Nevertheless, the findings have led to changes in practice, and more effective evaluation tools are being developed. This KDI of disseminating research findings to research participants, village residents and other stakeholders is a knowledge brokerage activity that, in addition to supporting communication, and sharing information with tailored products, involves other components of knowledge brokerage such as networking, building capacity and sustainability. A holistic approach to knowledge brokerage rather than a focus on one domain captures the interrelatedness and complexities of these activities and allows for the development of nuanced understandings of the processes involved.

Acknowledgments

This project was supported by the University of the Witwatersrand, the South African Medical Research Council and the Wellcome Trust (Grants 058893/Z/99/A; 069683/Z/02/Z; 085477/Z/08/Z; 085477/B/08/Z). We thank Audrey Khoza and Simon Khosa of the PEO; Jeffrey Tibane, as PEO consultant; Sulaiman Afolabi for HDSS data extraction; and interns Daniel Hughes and Sydney Walker, who assisted with collating data. None of the work would be possible without the participation of the population, service providers and local leaders in the study area.

REFERENCES

Council for International Organizations of Medical Sciences (CIOMS) 2016. International Ethical Guidelines for Health-related Research Involving Humans. Fourth Edition ed. Geneva: WHO

Bäckstrand, K 2003, ‘Civic science for sustainability: Reframing the role of experts, policy-makers and citizens in environmental governance,’ Global Environmental Politics, vol. 3, no. 4, pp. 24–41, doi: https://doi.org/10.1162/152638003322757916

Bhattacharyya, OK, Estey, EA & Zwarenstein, M 2011, ‘Methodologies to evaluate the effectiveness of knowledge translation interventions: A primer for researchers and health care managers,’ Journal of Clinical Epidemiology, vol. 64, pp. 32–40, doi: https://doi.org/10.1016/j.jclinepi.2010.02.022

Bornbaum, CC, Kornas, K, Peirson, L & Rosella, LC 2015, ‘Exploring the function and effectiveness of knowledge brokers as facilitators of knowledge translation in health-related settings: A systematic review and thematic analysis,’ Implementation Science, vol. 10, doi: https://doi.org/10.1186/s13012-015-0351-9 and https://doi.org/10.1186/s13012-015-0358-2

Collinson, MA, White, MJ, Bocquier, P, McGarvey, ST, Afolabi, SA, Clark, SJ, Kahn, K & Tollman, SM 2014, ‘Migration and the epidemiological transition: Insights from the Agincourt sub-district of northeast South Africa,’ Global Health Action, vol. 7, https://doi.org/10.3402/gha.v7.23514

Conklin, J, Lusk, E, Harris, M & Stolee, P 2013, ‘Knowledge brokers in a knowledge network: The case of Seniors Health Research Transfer Network knowledge brokers knowledge network seniors,’ Implementation Science, 8:7 doi: https://doi.org.10.1186/1748-5908-8-7

Dagenais, C, Laurendeau, M-C & Briand-Lamarche, M 2015, ‘Knowledge brokering in public health: A critical analysis of the results of a qualitative evaluation,’ Evaluation and Program Planning, vol. 53, pp. 10–17, doi: https://doi.org/10.1016/j.evalprogplan.2015.07.003

Dickert, N & Sugarman, J 2005, ‘Ethical goals of community consultation in research,’ American Journal of Public Health, vol. 95, pp. 1123–27, doi: https://doi.org/10.2105/AJPH.2004.058933

Gilson, L & McIntyre, D 2008, ‘The interface between research and policy: Experience from South Africa,’ Social Science and Medicine, vol. 67, pp. 748–59, doi: https://doi.org/10.1016/j.socscimed.2008.02.005

Godfrey, L, Funke, N & Mbizvo, GF 2010, ‘Bridging the science-policy interface: A new era for South African research and the role of knowledge brokering,’ South African Journal of Science, vol. 106, doi: https://doi.org/10.4102/sajs.v106i5/6.247

Guillemin, M & Gillam, L 2004, ‘Ethics, reflexivity, and “ethically important moments” in research,’ Qualitative Inquiry, vol. 10, pp. 261–80, doi: https://doi.org/10.1177/1077800403262360

Jana, M, Letsela, L, Scheepers, E & Weiner, R 2015, ‘Understanding the role of the OneLove campaign in facilitating drivers of social and behavioral change in southern Africa: A qualitative evaluation,’ Journal of Health Communication, vol. 20, doi: https://doi.org/10.1080/10810730.2014.925014

Kahn, K, Collinson, MA, Gómez-Olivé, FX, Mokoena, O, Twine, R, Mee, P, Afolabi, SA, Clark, BD, Kabudula, CW, Khosa, A, Khoza, S, Shabangu, MG, Silaule, B, Tibane, JB, Wagner, RG, Garenne, ML, Clark, SJ & Tollman, SM 2012, ‘Profile: Agincourt health and socio-demographic surveillance system,’ International Journal Epidemiology, vol. 41, pp. 988–1001, doi: https://doi.org/10.1093/ije/dys115

Kothari, A & Armstrong, R 2011, ‘Community-based knowledge translation: Unexplored opportunities,’ Implementation Science, vol. 6, pp. 59–64, doi: https://doi.org/10.1186/1748-5908-6-59

Lafrenière, D, Menuz, V, Hurlimann, T & Godard, B 2013, ‘Knowledge dissemination interventions: A literature review,’ SAGE Open, vol. 3, doi: https://doi.org/10.1177/2158244013498242

Lavery, JV, Tinadana, PO, Scott, TW, Harrington, LC, Ramsey, JM, Ytuarte-Nuñez, C & James, AA 2010, ‘Towards a framework for community engagement in global health research,’ Trends Parasitol, vol. 26, pp. 279–83, doi: https://doi.org/10.1016/j.pt.2010.02.009

Lavis, JN, Lomas, J, Hamid, M & Sewankambo, NK 2006, ‘Assessing country-level efforts to link research to action,’ Bulletin of the World Health Organization, vol. 84, pp. 620–28, doi: https://doi.org/10.2471/BLT.06.030312

Lavis, JN, Robertson, D, Woodside, JM, Mcleod, CB, Abelson, J & the Knowledge Transfer Study Group 2003, ‘How can research organizations more effectively transfer research knowledge to decision makers?’, The Milbank Quarterly, vol. 81

Legaspi, A & Orr, E 2007, ‘Disseminating research on community health and well-being: A collaboration between Alaska native villages and the academe,’ American Indian and Alaska Native Mental Health Research: The Journal of the National Center, vol. 14, pp. 24–43

Madhavan, S, Collinson, M, Townsend, NW, Kahn, K & Tollman, SM 2007, ‘The implications of long term community involvement for the production and circulation of population knowledge,’ Demographic Research, vol. 17, pp. 369–88, doi: https://doi.org/10.4054/DemRes.2007.17.13

Marsh, V, Kamuya, D, Rowa, Y, Gikonyo, C & Molyneux, S 2008, ‘Beginning community engagement at a busy biomedical research programme: Experiences from the KEMRI CGMRC-Wellcome Trust Research Programme, Kilifi, Kenya,’ Social Science & Medicine, vol. 67, pp. 721–33, doi: https://doi.org/10.1016/j.socscimed.2008.02.007

Meyer, M 2010, ‘The rise of the knowledge broker,’ Science Communication, vol. 32, pp. 118–127, doi: https://doi.org/10.1177/1075547009359797

Mitton, C, Adair, CE, McKenzie, E, Patten, SB & Perry, BW 2007, ‘Knowledge transfer and exchange: Review and synthesis of the literature,’ The Milbank Quarterly, vol. 85, pp. 729–68, doi: https://doi.org/10.1111/j.1468-0009.2007.00506.x

Molyneux, S & Geissler, PW 2008, ‘Ethics and the ethnography of medical research in Africa,’ Social Science and Medicine, vol. 67, pp. 685–95, doi: https://doi.org/10.1016/j.socscimed.2008.02.023

Mondain, N, Delauney, V & Ouédraogo, V 2016, ‘Reporting results back in health and demographic surveillance systems (HDSS): An ethical requirement and a strategy for improving health behaviours,’ African Population Studies, vol. 30, pp. 2355–68, doi: https://doi.org/10.11564/30-2-840

Naidoo, S 2012, ‘The South African national health insurance: A revolution in health-care delivery!’ Journal of Public Health, vol. 34, pp. 149–50, doi: https://doi.org/10.1093/pubmed/fds008

Nuffield Council on Bioethics 2002, ‘The ethics of research related to healthcare in developing countries,’ viewed 18 May 2017, http://nuffieldbioethics.org/wp-content/uploads/2014/07/Ethics-of-research-related-to-healthcare-in-developing-countries-I.pdf

Nyirenda, D, Makawa, TC, Chapati, G, Mdalla, C, Nkolokosa, M, O’byrne, T, Heyderman, R & Desmond, N 2016, ‘Public engagement in Malawi through a health-talk radio programme “Umoyo nkukambirana”: A mixed-methods evaluation,’ Public Understanding of Science, viewed 18 May 2017, http://journals.sagepub.com/doi/abs/10.1177/0963662516656110

Pettifor, A, MacPhail, C, Hughes, JP, Selin, A, Wang, J, Gómez-Olivé, FX, Eshleman, SH, Wagner, RG, Mabuza, W, Khoza, N, Suchindran, C, Mokoena, I, Twine, R, Andrew, P, Townley, E, Laeyendecker, O, Agyei, Y, Tollman, S & Kahn, K 2016, ‘The effect of a conditional cash transfer on HIV incidence in young women in rural South Africa (HPTN 068): A phase 3, randomised controlled trial,’ The Lancet Global Health, vol. 4, DOI: doi: http://dx.doi.org/10.1016/S2214-109X(16)30253-4

QSR International 2012, NVivo qualitative data analysis software; Version 10

Reddy, P, Buchanan, D, Sifunda, S, James, S & Naidoo, N 2010, ‘The role of community advisory boards in health research: Divergent views in the South African experience,’ SAHARA-J: Journal of Social Aspects of HIV/AIDS, vol. 7, pp. 2–8

Sankoh, O & Byass, P 2012, ‘The INDEPTH Network: Filling vital gaps in global epidemiology,’ International Journal of Epidemiology, vol. 41, pp. 579–88, doi: https://doi.org/10.1093/ije/dys081

Strydom, WF, Funke, N, Nienaber, S, Nortje, K & Steyn, M 2010, ‘Evidence-based policymaking: A review,’ South African Journal of Science, vol. 106, doi: https://doi.org/10.4102/sajs.v106i5/6.249

Stuttaford, M, Bryanston, C, Hundt, GL, Connor, M, Thorogood, M & Tollman, S 2006, ‘Use of applied theatre in health research dissemination and data validation: A pilot study from South Africa,’ Health: An Interdisciplinary Journal for the Social Study of Health, Illness and Medicine, vol. 10, pp. 31–45, doi: https://doi.org/10.1177/1363459306058985

Tindana, PO, Rozmovits, L, Boulanger, RF, Bandewar, SV, Aborigo, RA, Hodgson, AV, Kolopack, P & Lavery, JV 2011, ‘Aligning community engagement with traditional authority structures in global health research: A case study from northern Ghana,’ American Journal of Public Health, vol. 101, pp. 1857–67, doi: https://doi.org/10.2105/AJPH.2011.300203

Tindana, PO, Singh, JA, Shawn, TC, Upshur, REG, Daar, AS, Singer, PA, Frolich, J & Lavery, JV 2007, ‘Grand challenges in global health: Community engagement in research in developing countries,’ PLOS Medicine, vol. 4, pp. 1–5, doi: https://doi.org/10.1371/journal.pmed.0040273

Tollman, S 2008, Closing the gap: Applying health and socio-demographic surveillance to complex heatlh transitions in South and sub-Saharan Africa, PhD, Umeå University

Tollman, S, Herbst, K, Garenne, M, Gear, JS & Kahn, K 1999, ‘The Agincourt demographic and health study - site description, baseline findings and implications,’ South African Medical Journal, vol. 89, pp. 858–64

Tollman, SM 1999, ‘The Agincourt field site - evolution and current status,’ South African Medical Journal, vol. 89, pp. 853–58

Twine, R, Kahn, K, Scholtz, A & Norris, SA 2016, ‘Involvement of stakeholders in determining health priorities of adolescents in rural South Africa,’ Global Health Action, vol. 9, doi: https://doi.org/10.3402/gha.v9.29162

Ward, V, House, A & Hamer, S 2009, ‘Developing a framework for transferring knowledge into action: A thematic analysis of the literature,’ Journal of Health Services Research & Policy, vol. 14, pp. 156–64, doi: https://doi.org/10.1258/jhsrp.2009.008120

Ye, Y, Wamukoya, M, Ezeh, A, Emina, JBO & Sankoh, O 2012, ‘Health and demographic surveillance systems: A step towards full civil registration and vital statistics system in sub-Sahara Africa?’ BMC Public Health, vol. 12, doi: https://doi.org/10.1186/1471-2458-12-741

Yin, RK 1994, Case study research: Design and methods, Thousand Oaks, California, Sage Publications Inc.