Combining feminist intersectional and community-engaged research commitments : Adaptations for scoping reviews and secondary analyses of national data sets

As Hankivsky & Cormier (2011) and Denis (2008) note, the theoretical evolution of intersectionality has outpaced its methodological development. While past work has contributed to our understanding of how to apply intersectionality in research (CRIAWICREF & DAWN-RAFH 2014; Morris & Bunjun 2007; Simpson 2009), gaps persist. Drawing on a four-year community-university research collaboration called ‘Changing public services: Women and intersectional analysis’, we explore the incorporation of feminist intersectional and community-engaged research commitments into secondary data analyses, specifically a scoping review and secondary analyses of two Statistics Canada data sets. We discuss our application of these commitments across all stages of designing and undertaking these analyses, in particular drawing into focus the importance of dialogue and deliberation throughout our process. Our application of feminist intersectional and community-engaged commitments – including prioritising community benefit and practising self-reflexivity – revealed gaps and silences in the data, in turn improving our understanding of differences in people’s experiences, our critiques of policies and our identification of new research questions. DECLARATION OF CONFLICTING INTEREST The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article. FUNDING The authors gratefully acknowledge the following financial support for the research discussed in the article: the Canadian Social Sciences and Humanities Research Council, the Canadian Union of Public Employees, the Public Service Alliance of Canada, the Canadian Union of Postal Workers, the Canadian Labour Congress, and the University of Guelph, Canada. 1 PAGE NUMBER NOT FOR CITATION PURPOSES The lessons learned, we conclude, are valuable for scholars, whether or not community engagement is central to their scholarly commitment.


Introduction
This article presents the research design and methodological adaptations of a scoping review and a statistical analysis of large-scale survey data, two components of a community-engaged feminist intersectional, interdisciplinary research project about the impacts of changes to Canadian public services on diverse women as service users and providers.Focusing on the community-engaged and feminist intersectional adaptation of secondary data analyses within large community-engaged research projects reveals important gaps in policy-focused research.As Hankivsky & Cormier (2011) and Denis (2008) note, the theoretical evolution of intersectionality has outpaced its methodological development.While past work, for example by the bilingual Canadian Research Institute for the Advancement of Women, CRIAW-ICREF (e.g.CRIAW-ICREF & DAWN-RAFH 2014; Morris & Bunjun 2007;Simpson 2009), has contributed to our understanding of how to apply intersectionality in research, gaps persist.Scholars have also focused on the application of intersectionality in policy analysis (Hankivsky & Cormier 2011) and on the role of diverse communities in policy development and analysis (e.g.Christensen, Krogman & Parlee 2010; Murray 2015), but these practices remain elusive and are not necessarily driven by community-identified needs.Concurrently, as universities increasingly espouse the benefits of working collaboratively with research partners outside the university to advance scholarship and increase impact (Canadian Federation of Humanities and Social Sciences 2017), and as recognition of the importance of communityidentified research questions grows, the related methodological protocols and institutional mechanisms sometimes lag (Hall, Tandon & Tremblay 2015).
We begin with a brief discussion of our understanding of community-engaged research and feminist intersectionality and an overview of the research project, Changing Public Services (CPS), on which we are focusing.We then describe how we incorporated community engagement and feminist intersectionality into the research design and, more specifically, into its two secondary analyses -a scoping review and a statistical analysis using Statistics Canada data.While recognising the limitations of referring generically to 'diverse women', we use the term to signal not only our attention to women's heterogeneity (on such bases as their race, socioeconomic status, gender identity and sexual orientation, primary language and Indigeneity) which informs our intersectional analysis, but also our recognition that women's diverse identities and positions differently structure their social and economic experiences, including their access to, and provision of, public services.Our main argument is twofold.First, in large community-engaged research partnerships, upholding commitments to feminist intersectionality and community-engaged scholarship within secondary analyses that are part of the project design is important.Second, the insights and challenges emerging from the reflexivity required by both feminist intersectionality and community-engaged research helps us identify gaps -and silences -in the available literature and data.These gaps and silences help us better understand differences in people's experiences and raise new research questions.Thus, we assert that the lessons that follow are valuable for scholars, whether or not community engagement is central to their scholarly commitments.

Community-engaged research
Changing Public Services (CPS), described in more detail below, was developed with a unique commitment to principles of community-engaged research and feminist intersectionality.'Community engaged research' (CER) is an umbrella term recognising that 'knowledge acquired in the academic setting is strengthened and enhanced by the real world experience found in communities … [and that] a mutual, reciprocal, and respectful exchange of ideas, practices, and applications among the engaged partners' (Whiteford & Strom 2013, p. 72) provides invaluable opportunities for identifying important research questions and answers and, ultimately, achieving social change.While there is not space here to detail the rich history and development of CER, it is important to note that CER is not a new phenomenon.Its roots go back decades.It is often connected to instrumental action research (the 'northern tradition', frequently associated with the work of Kurt Lewin (Adelman 1993) and/or to popular emancipatory education (the 'southern tradition', which frequently cites Freire's (1970) Pedagogy of the oppressed as foundational).Generally, the former emphasises individual development and change, and the latter social justice (Wallerstein & Duran 2008, pp. 28-29).CER has also been examined from an institutional perspective, particularly in an effort to understand how meaningful community-university partnerships can be created and supported (e.g.Hall, Tandon & Tremblay 2015).
Across disciplines and over time, various terms (e.g.participatory action research, community-based participatory research, emancipatory research) imply similar -though not identical -research approaches (see Etmanski, Hall & Dawson 2014 for a comprehensive overview of terminology) that fit within CER.The principles and protocols informing these approaches have evolved across disciplines, but all emphasise the importance of reciprocity, responding to community-identified needs, ensuring high-quality investigation (Beaulieu, Breton & Brousselle 2018;Mikesell, Bromley & Khodyakov 2013), and self-reflexivity (Wallerstein & Duran 2008), or understanding one's own positionality in the research process.The attention paid to the purpose, principles and institutionalisation of CER has not been matched by its application in secondary analyses, including scoping reviews and statistical analyses.
Feminist scholars across social science, health, education and planning disciplines have a longstanding involvement with participatory, partnership-based or collaborative research with a variety of communities (Creese & Frisby 2011).Much feminist research was and is 'tied to the feminist movement and … [is] critical of how traditional social and scientific theories obscured women's experiences, while portraying men's behaviours as the human norm' (Frisby, Maguire & Reid 2009, pp. 17-18).Further, CRIAW-ICREF and Relais Femmes are examples of feminist organisations involved in partnerships between community groups and academics; both have conducted community-engaged research for many years (Morris 2002;Lacharité & Pasquier 2014).As well, Indigenous responses to colonial research paradigms insist on reciprocity and relationship building (Tobias, Richmond & Luginaah 2013) and on community ownership and control of data (First Nations Information Governance Centre 2014).Across critical, feminist and Indigenous scholarship, a common refrain has been to emphasise working with, rather than on or for communities (Creese & Frisby 2011;Smith 2012).Thus, where diverse women's experiences are the focus, there are compelling reasons to couple the theoretical and methodological orientations of CER and feminist intersectionality (FI).

Feminist intersectionality
Intersectionality is a theoretical and methodological orientation to research and an analytic tool constantly under construction (Collins & Bilge 2016).Without necessarily using the term 'intersectionality', feminists in various countries (for example, in the UK (Anthias & Yuval-Davis 1983), Canada ( Juteau-Lee & Roberts 1981) and the USA (Collins 2015) had begun independently during the late 1970s and early 1980s to critique the assumption that all women are white, middle-class and heterosexual (Denis 2008).The term 'intersectionality', introduced as a metaphor by Kimberlé Crenshaw in 1989, has been widely adopted.Crenshaw used it initially to discuss race and social class in relation to sex/gender, especially in the context of legal rights.Dimensions of the concept found across contemporary literature are synthesised as inequality, social context, complexity (Collins & Bilge 2016;Scott & Siltanen 2017), with the former adding relationality, power and social justice.Intersectionality involves the concurrent examination of multiple sources of subordination that cut across each other, and understands that the impact of a particular source of subordination (such as gender or social class) may vary, depending on its combination with other potential sources of subordination (or of relative privilege).Intersectionality is concerned with the ways in which systems, institutions and social structures of power intersect with individuals' identities and/or social locations to create temporary or sustained experiences of privilege and exclusion.
Contemporary intersectionality is relational and does not automatically prioritise a particular criterion of inequality or assume additivity.It may consider multiple levels of analysis, from the individual to the institutional to the societal and the global.It refers variously to identity and social position/location, with some authors combining the two (Collins & Bilge 2016).Intersectionality acknowledges the possibility of contradictory social positioning (for an individual, a group or a society) and that it is important to consider concurrently both the disadvantages (subordination) and advantages (privilege) of relations of inequality.It also highlights the importance of context, in both time and space, as this can result in differing and changing salience of aspects of specific social attributes, including for power relations.McCall (2005) distinguishes three types of intersectional analysis.Whereas anticategorical analysis emphasises the unique configuration of a specific case, intracategorical and intercategorical both highlight comparisons within and between groups respectively.Our statistical analysis, discussed in more detail below, illustrates both type of comparisons.
If one rejects the assumption that one source of difference is automatically key, what makes an intersectional analysis feminist?We use the term feminist intersectionality (FI) to suggest that gender is a dimension of inequality that must be examined in any intersectional analysis (rather than being either ignored or assumed), but that a priori assumptions about which dimensions of inequality and power will be most relevant in a particular context should be avoided.FI can also involve considering the patriarchal (and heterosexist) societal context of most, if not all, contemporary societies -including Canada -and the possible influences of this context not only on data analyses, but also on human lives.For example, Scott and Siltanen (2017) suggest using context as a higher order level of analysis to assess regressions from a feminist (and intersectional) perspective.

Changing Public Services: Women and intersectional analysis
The Changing Public Services (CPS) project was a four-year research partnership (2013)(2014)(2015)(2016)(2017) between national community organisations, national unions that organise public sector workers, university researchers and universities.CPS was initiated and coordinated by CRIAW-ICREF.The partnership used both primary and secondary data collection and analyses to develop a more comprehensive and systematic understanding of how changes to public services across Canada are being variously and disproportionately experienced by women.To ground this work, we drew on Ross and Savage's (2013) broad definition of public services; they see the public sector as being concerned with the provision, production, distribution and allocation of public goods and services, and thus public services as including those services that are both publicly provided and publicly funded.
There is considerable evidence of different and disproportionate impacts on women as a result of changes to public services.For example, women make up more than half of the federal public service (Treasury Board Secretariat 2011), so job cuts will likely affect women disproportionately.Women with disabilities, women who are recent immigrants, women who are lone mothers and women living in poverty are disproportionately reliant on public services (Dobrowolsky 2009) and are thus more severely affected by service cuts.Further, francophone women may be affected by changes to requirements for the provision of bilingual services, and Indigenous women may see services eliminated (Orsini & Papillon 2012) or be caught in the quagmire of jurisdictional conflicts.At the initiation of the CPS project, while there was widespread concern about these and related issues, there was no research network, guided by both community-engaged commitments and a feminist intersectional perspective, dedicated to tracking and analysing ongoing public service changes and their consequences.While this overarching goal ultimately proved too ambitious, we nevertheless achieved a number of outcomes, including developing a better understanding of the consequences of changes to municipal public transportation, by holding focus groups with both bus users and bus drivers (Ahmed & Pollack 2017) and analysing the causes and consequences of precarious public sector work based on a scoping review and ongoing discussions with union leaders (Levac & Cowper-Smith 2016).Moreover, work continues to emerge from this network, contributing to our growing knowledge about the intersectional consequences of public service changes.The project's leadership team was comprised of two academic partners and two community organisation partners, who were also retired labour organisers and thus able to provide an ongoing connection with other union members involved in the project.The leadership team was supported and directed by a guiding group, a larger circle of community, union and academic partners who worked in four distinct regional clusters across the country (Halifax, NS; the National Capital Region, ON/QC; Saskatoon, SK; and Vancouver, BC) and met annually to draw together insights from across the project's sites.This design, which was intentionally meant to create a more accessible and equitable project leadership structure, eventually needed to be complemented by additional discussions with union partners to try and further ensure equitable participation by community-based members.Each regional cluster identified a priority area of focus for their cluster's research, based on the needs and preferences of collaborators in each region.For example, the Saskatoon team focused on the impacts of Saskatchewan's adoption of private sector management principles in health care, whereas the National Capital Region focused more generally on diverse women's access to public services and changes that both service users and providers had noticed over the previous seven years.In addition, the two types of secondary analyses discussed in this article were developed through ongoing dialogue and deliberation with the guiding group, with the results being available for use by all the partners in the network.
Overall, CPS was initiated, designed and executed in collaboration with all partners, in keeping with commitments common to CER and FI.Our efforts to uphold these commitments led to this article's main contribution: guidance on incorporating CER and FI in the process of conducting secondary data analyses as part of large-scale community-engaged research projects.In so doing, we also offer guidance on using secondary analyses to track and analyse impacts of changes to public services in ways that consider diverse women's experiences, particularly by highlighting data gaps and silences.

PROJECT DESIGN
Our commitment to being guided by FI and CER principles builds on the work of others who have pointed out that incorporating feminist theory into participatory action research (PAR), a form of CER, allows us 'to be transparent about our interest in understanding, through participatory processes, how gender inequalities intersect with other axes of oppression and how they can be transcended' (Frisby, Maguire & Reid 2009, p. 20).First, as noted, the leadership circle and guiding group for the project were comprised of both community and academic partners, who raised critical intellectual questions that underpinned our overall approach and thus some of our research decisions.For example, as discussed elsewhere (Levac & Cowper-Smith 2016), established definitions of the public sector ranged from being narrowly focused on the central government to incorporating publicly funded and publicly provided services.Based on concerns -raised particularly by our union partners -about the steady move towards privatisation of both national and sub-national public services, our partnership approach enabled ongoing discussion about how best to define public services for our purposes.This ultimately led us not only to use a broader definition of public services, but also to include services that advocates across Canada and its provinces have long recognised as warranting public funding (e.g. for child care), and/or that are under significant threat of privatisation (e.g.transportation, health care).
A second general project design feature that reflected our commitment to FI and CER principles was the internal review process for the project's publications.Before release, each product (e.g.community reports, fact sheets, technical reports) under the Changing Public Services banner underwent a review by both an academic and a community team member who were not involved in its development.Internal peer reviewers used a framework, adapted from a past CRIAW-ICREF project, to consider, among other things, the document's: (a) inclusiveness (e.g.'Does the output include often-marginalised groups of women in ways that reject and work to transform relationships of oppression?');(b) commitment to intersectionality (e.g.'Does the analysis explicitly address the multiple and varied situations of women as well as the silences and absences of groups of women?'); and (c) participatory nature (e.g.'In what ways did the communities involved in the research participate in its writing or production?').Not only did this process help to ensure the scholarly rigour and community validity of outputs, it also ensured that we learnt from each other as we strove to uphold our commitment to recognising and revealing diverse voices and experiences.
Beyond the general effects of our CER and FI commitments on our work, they also informed two core secondary analyses that were part of our research -a scoping review and a statistical analysis using large-scale survey data.Our scoping review aimed to uncover empirically reported impacts of changes to public services on service users across Canadian jurisdictions, particularly leading up to and following the 2008 recession.Our emphasis on service users complemented other dimensions of the CPS project, which focused on service providers, including a report on precarious public sector employment (Levac & Cowper-Smith 2016) and statistical analysis.Our statistical analysis drew on two Statistics Canada data sets, the Survey of Labour and Income Dynamics (SLID) and the Public Service Employees Survey (PSES).We used these to explore the characteristics of women working in the federal public service relative to the labour force as a whole during this period, complemented by a more detailed comparative analysis of their employment in the federal public sector.No data sources allowed comparable analyses of provincial or municipal public sector workers.

SCOPING REVIEW ADAPTATIONS
Scoping reviews 'commonly refer to "mapping", a process of summarizing a range of evidence … [in] a field' (Levac, Colquhoun & O'Brien 2010, p. 1).Comparing them to systematic reviews, Arksey and O'Malley (2005) 1).Specifically, they offer complementary recommendations for each of the six stages (five required and one optional) outlined by Arksey and O'Malley (2005), and argue that all six stages should be required.Incorporating FI and CER commitments into our scoping review necessitated adaptations to four of the six stages proposed by Levac, Colquhoun and O'Brien (2010), the last of which, consultation, we argue, should be described instead as dialogue and deliberation, and should cut across all the other stages.An overview of these adaptations is presented in Table 1.Columns 1 and 2 are based on the original framework by Arksey and O'Malley (2005) and adaptations from Levac, Colquhoun and O'Brien (2010) respectively.Column 3 highlights our additions based on our CER and FI commitments.Stages 1 and 2: As noted, the goals of our scoping review were to establish a broad understanding of the extent to which recent public service changes and their impacts on diverse women users had been studied across the country, and to identify key gaps.'Identifying the research question' and 'identifying relevant studies' included embracing the expertise of community collaborators.Discussions and deliberations -which included partners' personal and professional experiences with public service changes -helped to clarify the scope of the review and build a comprehensive set of search terms.Through this process, and combined with an initial scan of the literature, we created a catalogue of four categories of search terms: (a) terms related to gender; (b) terms related to gender plus socio-demographic considerations (to reveal particular intersections); (c) terms related to changing public services; and (d) terms defining the jurisdictions of focus.In addition to relevant English and French academic databases, we identified a range of community literature sources, including research conducted by think tanks, community organisations and union partners themselves.This process, highlighting similar commitments to those discussed by Bassett and McGibbon (2013) in their scoping of literature, helped ensure that our search terms and the locations in which we looked for materials were reflective of social identities/social locations, power structures, and collaborators' knowledge of, and experiences with, the impacts of changing public services on service users.
While these efforts successfully extended the reach of our search, its breadth created a challenge in terms of managing the large number of possible search combinations.We also struggled with, on the one hand, wanting to identify empirical impacts on women of changes to public services, and on the other hand, wanting to avoid implying that the impact(s) of policy changes should only be taken seriously if they could be or had been studied empirically.This created a particular challenge in discussions with our community partners who had extensive direct knowledge and experience of the consequences of changing public services, even when those changes were not easily identifiable in the literature.Without undertaking individual policy analyses, it is difficult to control enough variables to determine causality between a policy change and the situation of individuals.Nevertheless, particularly given ongoing commitments to undertaking gender-based analyses across Canada (Hankivsky 2012; Status of Women Canada 2016), trying to take stock of the gendered impacts of policy changes, specifically the impacts on women, insofar as possible, was important.Finally, our community-engaged commitments, which included treating students as meaningful members of a research team and training them to take up the complexities of CER, inspired us to ensure extensive student participation in the process.This demanded breaking the search into small pieces so that students, especially those on time-limited research placements, could take on manageable chunks of the work.It also necessitated extensive attention to students' inclusion/ exclusion decisions about the potential sources identified.
Stages 3 and 4: Our methodological innovations do not suggest changes to stages 3 and 4 as outlined and elaborated by Arksey and O'Malley (2005) and Levac, Colquhoun and O'Brien (2010).We do note that Levac, Colquhoun and O'Brien (2010) suggest at least two reviewers for both abstracts and full articles; since this was not possible given the range of articles reviewed, we instead tested inter-rater reliability with a sample of abstracts at the beginning of each reviewer's work on the project and, partly due to our community partners' interests, adopted the practice of 'erring on the side of inclusion' to help guard against missing important details.Also, and elaborated further below in our discussion about Stage 6 (consultation), these stages included ongoing invitations to community partners to help uncover sources for inclusion, and to reflect on preliminary findings and offer analyses based on their own experiences.
Stage 5: The fifth stage, 'collating, summarising, and reporting results', was a key point at which feminist intersectionality was incorporated since it pushed us to consider literature gaps in particular ways; it also led us to ask who was invisible in the reported impacts, as opposed to focusing solely on the general lack of attention to empirical impacts.For example, across jurisdictions, we uncovered very little literature focused on the experiences of young women.The literature we did uncover pointed to the importance of provincially legislated policy areas in young women's lives.This led us to examine the extent to which young women's experiences and needs are considered or reflected on in public policies (Levac & Worts 2018), and revealed a persistent failure to address their unique experiences.This finding, with its clear policy implications, revealed the importance of simultaneously considering several dimensions of identity/social position as called for in an FI analysis.Another observation was the frequent silence about precisely who had been studied and the lack of gender disaggregation in the presentation of the results.
Stage 6: The 'consultation' stage is described as an optional step for 'including the perspectives of others with knowledge of, and a vested interest in, the area under examination [to give] an important additional dimension to the reviewing process' (Arksey & O'Malley 2005, p. 30).Levac, Colquhoun & O'Brien (2010) suggest that this stage should be mandatory.We suggest that, based especially on the commitments of CER, consultation should be approached as an underlying practice throughout the scoping review, rather than as a stage in the process, and should be understood as dialogic and deliberative rather than as consultative.Consultation implies that power remains in the hands of the (typically university-based) 'consultant', whereas deliberation invokes a commitment to dialogue and to arriving at mutually agreeable decisions (Gutmann & Thompson 2009).For example, our ongoing deliberations with community partners led us to create another category of literature during our review that we called 'exclude-keep'.This category addressed our partners' interests in gathering details about the context in which changes were, and are, taking place.In essence, we opted to gather -for future use -relevant research about the broader context of changing public services.Deliberation throughout the process should also, we argue, include discussions about relevant outputs, particularly given the limited utility of academic articles for most community partners and organisations providing frontline service and doing advocacy work.To this end, we imported and tagged all the identified literature into a literature repository that we could share with project team members and built a searchable Excel file providing a summary of many of the key results of the review.

STATISTICAL ANALYSES
Statistical analyses using secondary data take many forms, but are generally useful for understanding broad trends across a population or over time, and for trying to establish causal relationships or correlations.In designing its surveys, Statistics Canada undertakes consultations -but largely with academics, other professionals and other government departments which have commissioned the survey and/or will be important users of the results.While there is also community consultation, marginalised groups within communities are not necessarily those included in these nationwide consultations.Furthermore, consultations are not always accompanied by a commitment to seriously considering the implementation of people's feedback.
While much feminist research is qualitative, several feminists, such as Elson (1998), O'Neill (1995) and Reinharz (1992), acknowledge the relevance of quantitative methods within feminist research (see Scott & Siltanen 2017 for an overview of this literature).The survey questions asked, the categories offered for responses, the statistical analysis of the data and the interpretation of the results are all components of statistical analysis in which the use of a critical feminist lens is recommended (Bowleg 2008;Harnois 2013;Luxton 1997).For example, while regression analysis has been identified as a helpful statistical technique for intersectional analysis, the limitations of some of its applications have also been highlighted, together with suggestions about how to at least partially overcome them (Bowleg & Bauer 2016;McCall 2001;Scott & Siltanen 2017;Winker & Degele 2011).
Bearing these debates in mind, a secondary analysis of two Statistics Canada data was conducted as part of the CPS project, to offer our team a national overview of diverse women's experiences with paid work, primarily in the federal public sector.We are not familiar with a discussion about the design stages of statistical analyses comparable to those just discussed about scoping reviews (Arksey & O'Malley 2005;Levac, Colquhoun & O'Brien 2010).In the absence of such a framework, the description of the analytical stages for the statistical analysis that follows is informed by those frameworks (see Table 2).
Stage 1: In 'establishing the research rationale', we determined that the goal of this examination of women's (primarily public sector) employment experiences was to concurrently consider various employment outcomes or experiences, such as the likelihood of being employed or being in precarious employment, along several comparative (and intersecting) axes, including time (in relation to the 2008 recession); gender (female or male); another socio-demographic variable (specifically Aboriginal status, visible minority status, disability); and employment sector (private versus public).These parameters were the result of early and ongoing deliberations within the research team.Stages 2 and 3: The dimensions of marginalisation, identified by both the experience and research of community partners and by academic research, in combination with various measures of the 'employment' variable, helped us 'identify relevant data sets' and variables.Only Statistics Canada data sets available through Canadian Research Data Centres (RDCs) were considered.Our primary Research Assistant's familiarity with the data sets, including their publicly available documentation, was invaluable: necessary considerations included the population studied, sampling design and sample size, the frequency of the survey's administration and the topics examined, both of those related to employment and to sociodemographic characteristics.Three and four cycles respectively of two national data sets -the SLID (Survey of Labour and Income Dynamics) and the PSES (Public Service Employees Survey) -were selected, so that we had data for three salient points in time: 2005 (before the recession of 2008), 2008 and 2011 (the most recent year since the recession for which the results of both surveys were available).
When using the PSES, from which 2014 data were also used, only comparisons within the federal public sector could be made, but it was possible to examine more nuanced aspects of employment, including those about experience with and knowledge about unions.On the other hand, with the SLID, public-private comparisons were possible, but less detailed social identity/position data were available.Our feminist intersectional analyses and commitments were upheld by using two data sets, as each offered a different opportunity for comparative analyses of several historically marginalised groups of workers.
Once the data sets were determined, there was a great deal of further dialogue and deliberation about what variables to include and how answer categories should be grouped.Still, omissions remained.For example, despite team members' reflections on the potential effects of sexual orientation on workplace experiences, sexual orientation was not analysed because none of the data sets considered included appropriate variables.As well, gender was only available as a dichotomised variable.Variables to being a member of a visible minority, being Aboriginal, and having a disability were all dichotomised due to a combination of considerations, which included the type of information available in the data set, the requirements of available options for statistical analysis, and the number of cases in the cells when two or more variables were examined concurrently.The terms 'visible minority' and 'Aboriginal' are problematic.'Visible minority' is used by the Canadian governmentand thus in its surveys.We prefer the term 'racialised', which connotes situations in which human biological (and cultural) characteristics have been used (usually by a dominant group) to construct differentiated social collectivities (Castagna & Dei 2000, p. 21), while acknowledging that not everyone who is 'racialised' has the same experiences.'Aboriginal' is also problematic.It is homogenising, may be equated with the State's definition of who is an 'Indian' according to the Indian Act, and is rejected by many Indigenous peoples.We therefore use the terms 'visible minority' and 'Aboriginal' when referring to specific details of the surveys, but 'racialised' and 'Indigenous' in our broader discussions.
Stage 4: The retained employment variables intentionally allowed for nuanced analyses.For instance, the various groups were compared according to their status as 'employed at all' and as 'employed all year'.The nuance offered by using both of these employment variables is illustrated by Aboriginal women's experiences.Women who identified as Aboriginal were significantly less likely than other women or Aboriginal men to be 'employed all year', but not significantly less likely to be 'employed at all'.
Priorities for data analysis were determined through deliberation with our collaborators, while continuously referencing the project's original research questions.As part of the analysis, logistic regression models were used.By creating new variables which incorporated two dimensions of potential marginalisation at a time (i.e.gender and Indigeneity, gender and racialisation, gender and disability), it was possible, using logistic regression analyses, to compare, for example, women with disabilities to women without disabilities, and men with disabilities to men without disabilities.This allowed us to incorporate simultaneously two dimensions of marginalisation and thus avoid the pitfalls of ignoring the interaction effects that Bowleg and Bauer (2016) illustrate.By doing regressions for each year, it was possible to discern whether time, as a contextual variable (see Scott & Siltanen 2017 on levels of analysis), affected the outcomes.Unfortunately, in a few cases the sample size of those marginalised in respect to two variables was too small to permit release of the results by the RDC (and therefore use by the CPS team).Had we analysed concurrently three axes of marginalisation -such as being a woman, who was Indigenous and who had a disability -considerably more cells would likely have been too small for release and use.
Using the RDC facilities, however, enabled the implementation of both our FI and CER commitments because this allowed those team members with RDC access to work out, in dialogue with the broader team, the parameters of the desired analyses through multiple iterations, and then to conduct the analyses under Statistics Canada's secure conditions.This would not have been possible had we had to contract Statistics Canada to conduct the analyses for us.Despite the benefit of RDC access, those team members with access were exclusively academic partners, which inevitably resulted in a few small decisions being made without the inclusion of community partners.
Stage 5: This stage emphasises using multiple fora and formats for presenting findings of relevance to community and academic audiences.Once the analyses were completed and released by the RDC, the results were summarised in language which was understandable by non-statisticians and presented in a range of formats so that all partners could make use of them.One illustration of this diffusion was an initial selective overview presented as of a UN-NGO session that CRIAW-ICREF organised at the 61st meeting of the Commission on the Status of Women (Denis 2017).As predicted, the public sector has been a better -and more equitable -employer for marginalised women than has the private sector.This was true when comparing their experiences with those of their female and male counterparts in the private sector, and with all other women, and all other men in the private sector.Moreover, marginalised women (whether in the public or private sector) were often less disadvantaged in comparison with other women than they were in comparison with men who were marginalised in the same way as they were.Not surprisingly, non-marginalised men seemed to have the greatest advantage in terms of the various measures we used.The results, which were almost all statistically significant (p < .001),revealed not only the benefits of undertaking a feminist intersectional analysis, but also the critical policy implications of such work.Another illustration of dissemination was a 2017 roundtable discussion by community and academic partners, held at the annual Congress of the Social Sciences and Humanities in Canada and hosted by the Canadian Political Science Association.A final example was a clear language summary of findings focused on how knowledge about their collective agreement and comfort with submitting grievances varied in complex ways among unionised public servants.These variations were independent of whether or not the unionised public servants were marginalised in terms of gender, (dis)ability, Indigeneity or racialisation (Pollack, Stinson & Levac 2018).
Stage 6: As with the methodological innovations in our scoping review, dialogue and deliberation with our research collaborators underpinned the entire research process.These discussions were more challenging in this component of our research than for the scoping review, since many people had limited first-hand experience with undertaking or interpreting the relatively complex statistical analyses that an intersectional analysis requires.Our efforts to collaborate throughout this process included inviting the Research Assistant, who was both familiar with the database and able to articulately explain the statistical results in nontechnical language, to speak directly with the research team at one of our annual gatherings, and hiring additional Research Assistants to present the findings in a (non-technical) summary report in order to try to increase the accessibility of the results.

Challenges of implementing community-engaged and feminist intersectional commitments
There were of course challenges in incorporating FI and CER principles into our work: the innovations described above were both time-consuming and difficult.Overall, this community engagement required considerable commitment from both community partners, some of whom were volunteering their time, and academic partners, whose work is still commonly evaluated more for its traditional scholarly contribution than for its community contribution(s).As discussed by others (e.g.Ross et al. 2010), this point demands ongoing attention, particularly when developing community-university research partnerships such as the one described in this article, and when considering how academic institutions and funding bodies can better resource such complex partnership research.
Trying to incorporate feminist intersectionality into our scoping review and statistical analyses was also challenging.As mentioned earlier, data disaggregation by gender is often omitted from published research results, particularly its intersection with such other variables as age, Indigeneity, (dis)ability and racialisation.We thus uncovered large gaps at these intersections in the literature during our scoping review.In the statistical analyses, we were constrained by the questions asked in the surveys by the answer categories offered.For instance, the SLID did not provide information about union membership, experiences of discrimination and inequity in the workplace, or experiences with seeking redress in such circumstances.And, as noted earlier, none of the surveys reviewed included questions about sexual orientation or offered the option of non-binary gender responses, a shortcoming that is finally being addressed by Statistics Canada (2018).
Another limitation in the survey data was the small sample size at the intersections of marginalised groups, such as women who are also Indigenous and disabled.Small sample size may also preclude differentiating experiences of marginalised groups in particular contexts, such as at the provincial level.Where a survey is about a particular dimension of marginalisation, such as Statistics Canada's (2014) Canadian Survey on Disability 2012, although finer comparisons can be made among those experiencing that aspect of marginalisation, it is typically difficult or impossible to compare the results with those for the population at large, primarily because of the use of different questions across surveys.Thus, while we welcome the return of the Canadian long-form Census as a signal that data matters to government policy-makers, we argue that there is nevertheless a continuing need for vigilance regarding sampling and content in government surveys.

Concluding remarks
This article argues that upholding FI and CER commitments -such as prioritising community benefit and practising self-reflexivity -includes considering their application in secondary data analyses within a project and highlighting several ways that these commitments can be incorporated into scoping reviews and secondary analyses of statistical data.A common theme across this account is our argument that, although described as a stage in the scoping review process (Arksey & O'Malley 2005), 'consultation' should instead be considered 'dialogue and deliberation' and should be an underlying component of the entire research process, rather than a discrete stage.In our scoping review, ongoing dialogue and deliberation informed our choice of search terms, our inclusion of important non-academic sources of knowledge, our inclusion/exclusion criteria, and our decisions about how to organise and make available resulting data.In our statistical analyses, our ongoing collaboration informed our research questions, our selection of data sets, our construction and use of variables in our analyses, and the type of statistical analyses done.The development of a range of clear language outputs for a variety of external audiences, but also for internal team use, was a result of thinking carefully about how power differentials can be exaggerated through inaccessible data outputs.Our efforts to make the results of complex analyses more accessible are consistent with established CER principles, as outlined by Whiteford and Strom (2013).
Especially important to our commitment to incorporating FI was the use of strategies that helped make visible historically overlooked populations.In the fifth stage of our scoping review (collate, summarise and report results), this included organising our findings in several different ways (e.g. by social identity/position group, by public service area) to expose different gaps in the literature.The third and fourth stages of our statistical analyses (identify relevant variables and prioritise analytical questions) included the decision to undertake three types of regression analyses, comparing, for example, racialised women's experiences to those of racialised men, all other women, and all other men.
Our dual commitments to CER and FI challenged us to critically examine established ways of conducting secondary analyses.The principle of self-reflexivity was especially helpful in identifying gaps and silences in the available literature and data.For example, as noted above, included careful thinking about how to reveal often invisible experiences of which we were aware, in many cases because of our own experiences and expertise.Our commitments also led us to explore new ways of asking research questions, of evaluating existing data, and of interpreting the results of the two components of secondary data analysis discussed earlier -scoping reviews and the use of large government-generated data sets.Especially because of the focus of this project on changes to public services and on the experiences of diverse public sector women workers, our adaptations offer guidance not only for researchers interested in advancing a feminist intersectional and community-engaged approach to research, but also for those interested in informing public policy in line with these commitments.
Combining feminist intersectional and community-engaged research commitments:Adaptations for scoping reviews and secondary analyses of national data sets Gateways: International Journal of Community Research and Engagement, Vol. 12, No.
Levac, Colquhoun and O'Brien (2010)g]to address broader topics where many different study designs might be applicable … [and as] less likely to ... address very specific research questions …' (p.20).Building onArksey and O'Malley's (2005)work,Levac, Colquhoun and O'Brien (2010)offer additional methodological guidance for undertaking scoping reviews (see Table

Table 1
Feminist intersectional community-engaged scoping reviews

Table 1 continued
Levac and Denis Gateways: International Journal of Community Research and Engagement, Vol. 12, No. 1, January 2019

Table 2 continued
Levac and Denis Gateways: International Journal of Community Research and Engagement, Vol. 12, No. 1, January 2019