EvaluationMethodology

The 2-year evaluation of the Plan employed a convergent mixed methods design. This involved the concurrent collection of quantitative and qualitative data, which were analysed separately and then integrated to provide a comprehensive understanding of early implementation progress.

This approach enabled the External Evaluation Team to capture stakeholder perspectives, document evidence of activity, and identify areas for further action that will support progress towards the Plan’s 5-year goals and 10-year ambitions, while also considering resource implications and equity, including for Aboriginal and Torres Strait Islander peoples and other priority populations.

Hybrid evaluation model

A hybrid evaluation model, where internal and external staff work together to deliver the evaluation, was adopted, led by Cancer Australia with independent external evaluation expertise. The External Evaluation Team provided methodological and analytical evaluation expertise, led stakeholder consultation and data analysis, while Cancer Australia provided contextual expertise, supported stakeholder engagement, and contributed to joint interpretation and reporting. This strengthened the credibility and policy relevance of the findings, and is consistent with Australian Government evaluation good practice.

Culturally appropriate evaluation

The evaluation was conducted in a culturally appropriate manner, with respect for Aboriginal and Torres Strait Islander peoples and communities. Aboriginal and Torres Strait Islander leadership was embedded throughout. An Aboriginal and Torres Strait Islander leader supported recruitment and led engagement with Aboriginal and Torres Strait Islander sector stakeholders, and Cancer Australia’s Leadership Group on Aboriginal and Torres Strait Islander Cancer Control provided governance and oversight.

Ethical considerations

The evaluation was conducted in accordance with the National Statement on Ethical Conduct in Human Research and established ethical principles, including informed consent. A formal Human Research Ethics Committee review was not undertaken, as the evaluation constituted program evaluation and quality improvement activity involving low-risk stakeholder engagement.

Each dataset was analysed separately using systematic and transparent procedures appropriate to its characteristics. Results were then integrated using joint display methods, side-by-side comparison and sense-making sessions among the External Evaluation Team to identify areas of convergence, divergence, and complementarity.

Qualitative analysis

Qualitative data from consultations and open-text survey responses were analysed using a general inductive approach, which supports identification of recurrent themes while allowing for emergent insights.

The analysis process involved:

• Progressive inductive coding during data collection so emerging themes could be explored in subsequent consultations where relevant
• Development of a coding framework aligned with the semi-structured consultation guide and evaluation questions
• Coding conducted in Microsoft Excel. The team collaboratively refined the coding framework after testing on a sample of transcripts to ensure shared understanding and consistency
• Consolidation of responses to similar questions across stakeholder groups to enable comparisons of perspectives.

The qualitative data were not intended to be representative of the entire cancer sector. Rather, they provided depth and insight into early implementation experiences and survey findings. Quotations are used throughout this report to enhance authenticity and explanatory value. Transcripts were lightly edited for clarity and readability without altering the original meaning, and specific examples and quotes were anonymised to maintain confidentiality.

Implementation activities reported by implementation partners, jurisdictions and the Commonwealth were reviewed, alongside survey responses. Reported activities were assessed for quality, clarity and internal consistency, with duplication and discrepancies identified and resolved where possible. Activities were then mapped against the Plan’s 2- and 5-year actions to assess alignment. This information was consolidated to determine the overall level and spread of implementation activity.

Document analysis examined the completeness, coverage, and consistency of implementation evidence contained in Plan monitoring materials. This helped confirm stakeholder perspectives, helped corroborate reported progress, provided a clearer description of implementation activity undertaken to date, and identify opportunities for additional or enhanced activity to further support system strengthening.

Quantitative analysis

Survey data were cleaned, coded, and prepared for analysis in Microsoft Excel, including extracting outputs from the survey platform, classifying variables, coding categorical responses, and checking data quality for missing or inconsistent values.

Quantitative analysis included:

• Descriptive statistics (e.g. frequencies, proportions, means, measures of central tendency and variation)
• Cross-tabulations to explore potential differences in views between stakeholder groups where sample sizes permitted
• Assessment of whether subgroup analysis was appropriate based on data characteristics and sample size
• Visualisation of selected Likert-scale items using stacked bar charts in Microsoft Excel.

These analytical procedures aligned with the evaluation objective of understanding early implementation progress and variations in experience across stakeholder groups.

Strategies to integrate results

Following separate qualitative and quantitative analyses, results were integrated using several techniques typical of convergent mixed methods designs. Integration strategies included:

• Identifying thematic areas represented across both datasets and comparing the extent to which results converged or diverged
• Creating joint displays that aligned qualitative themes with quantitative patterns to support interpretation
• Exploring similarities and differences between stakeholder groups using integrated data
• Conducting a sense-making session with the External Evaluation Team to synthesise insights and determine the degree of alignment between findings
• Producing integrated summaries combining survey results with explanatory qualitative insights to provide a more complete understanding of early implementation progress.

Strengths and limitations

Each data source used in this evaluation offered distinct advantages while also presenting some constraints that shaped the depth and breadth of insights obtained. Considering these strengths and limitations is important for interpreting the findings and understanding the degree of confidence that can be placed in different types of evidence. Key strengths and limitations of each data source are summarised in the table below.

Attribution

In a complex policy environment characterised by multiple concurrent strategies, funding streams, and initiatives (some pre-dating the Plan) attribution cannot be established with confidence at the 2-year mark. Findings are therefore framed in terms of Plan’s influence and early effects, understood in terms of acceleration, alignment, and system-level change rather than direct causation.

Key Evaluation Questions

For the purposes of this 2-year evaluation report, the eight Key Evaluation Questions (KEQs) in the Framework have been grouped into five thematic categories to support synthesis and reporting of evaluation findings, as outlined in the table below.

The table below summarises participants engaged through consultations and the survey component of the evaluation. These data reflect direct stakeholder engagement only. Participation is reported using the following conventions:

• For organisational stakeholders (cancer sector, Aboriginal and Torres Strait Islander, jurisdictional), one consultation session is counted as one stakeholder.
• Consumers are counted as individual participants rather than by session.
• Invitee totals (response rate denominators): cancer sector stakeholders (non-Aboriginal and Torres Strait Islander) (n=27), Aboriginal and Torres Strait Islander stakeholders (n=11), jurisdictional stakeholders (n=8), and consumers (people affected by cancer) (n=12).
• Total unique evaluation participants are counted once across consultation and survey components.

Summary of participant engagement

• Strong engagement was achieved across the cancer control sector, with 82 unique stakeholders participating across consultation and survey activities.
• Participants included 71 organisations and 11 consumers, with representation from all state and territory governments. This included 55 cancer sector stakeholders (non-Aboriginal and Torres Strait Islander) and 8 Aboriginal and Torres Strait Islander stakeholders.
• Participation patterns varied by stakeholder group: Aboriginal and Torres Strait Islander sector stakeholders, jurisdictional stakeholders, and consumers engaged primarily through consultations, while cancer sector stakeholders (non-Aboriginal and Torres Strait Islander) engaged more through the survey.
• Fourteen organisations (19%) participated in both consultation and survey activities, providing complementary perspectives across both data sources.