Skip to main content

Aboriginal and Torres Strait Islander people are advised that this website may contain images and voices of deceased people.

Back to Resource Centre
Wednesday, 28 Feb 2018
Image
Indigenous Advancement Strategy 2017-18 Annual Evaluation Work Plan

Indigenous Advancement Strategy 2017-18 Annual Evaluation Work Plan

Publication author(s):

Department of the Prime Minister and Cabinet

Publication abstract:

The Annual Evaluation Work Plan for 2017-18 is a key component of the Indigenous Advancement Strategy (IAS) Evaluation Framework. This is part of a multi-year plan.

The IAS Evaluation Framework guides evaluations conducted by the Indigenous Affairs Group of the Department of the Prime Minister and Cabinet (PMC).

The Work Plan covers evaluations and reviews planned over the current year (including those already commenced). Some projects continue across multiple years, in order to examine significant questions about impact.

Attachment
Indigenous Advancement Strategy 2017-18 Annual Evaluation Work Plan - PDF PDF 746.87 KB
Read full resource

Introduction

The Annual Evaluation Work Plan for 2017-18 is a key component of the Indigenous Advancement Strategy (IAS) Evaluation Framework. This is part of a multi-year plan.

The IAS Evaluation Framework guides evaluations conducted by the Indigenous Affairs Group of the Department of the Prime Minister and Cabinet (PMC).

The Work Plan covers evaluations and reviews planned over the current year (including those already commenced). Some projects continue across multiple years, in order to examine significant questions about impact.

The plan also covers future priorities for evaluation activities over four years (2017-18 to 2020-21).

All evaluation reports or summaries (if ethical confidentiality concerns or commercial in confidence requirements trigger a restricted release) will be made available on the PMC website.

Evaluation capability is a prerequisite for conducting robust evaluation. Guided by the IAS Evaluation Framework, some effort under this Work Plan is also directed towards increasing capability. 

This Work Plan has been endorsed by the PMC Evaluation Committee and was approved by the PMC Executive Board. [1]

Principles for identifying evaluation priorities

To support strategic planning of evaluation activity across the Indigenous Affairs Group, a list of priority areas for evaluation is identified each year. This list includes anticipated evaluation priorities for the current year and a list of future priority areas (in line with requirements under the Public Governance, Performance and Accountability Act 2013).

Identifying specific evaluation priorities over a short horizon and broad priority areas over the longer term allows the Indigenous Affairs Group to provide greater oversight around the allocation of resources required for evaluation activity across the Group.

The principles for identifying evaluation priorities are outlined in the IAS Evaluation Framework, and cover issues of significance, contribution and risk. Broadly, prioritisation of evaluation effort and resourcing is based on:

  • size and reach of the program or activity, emphasising what matters to Indigenous Australians as well as policy makers
  • strategic need, potential contribution and importance of evidence in relation to the policy cycle, and
  • the level of ‘policy risk’ of the program or activity, indicated by gaps in the evidence base, and assessments from past evaluations or reviews of the program or activity.

The Work Plan is structured as follows:

Group 1. Strategic projects

Potential for substantial impact or cross-cutting evaluations, relatively higher risk and limited evidence base, or timely/time-sensitive (to inform decisions). These are higher priority evaluations, many of which are undertaken by the Indigenous Affairs Group’s Information and Evaluation Branch, (or in some cases, conducted at arms-length).

Group 2. Program evaluations

Activities that are considered relatively lower risk, or less time sensitive. The Indigenous Affairs Group undertakes a number of program evaluations and reviews each year. These projects are typically commissioned and led by program areas, with support or guidance from the Information and Evaluation Branch.

Group 3. Future topics

Potential priorities over 2017-18 to 2020-21. While evaluation priorities are identified over the next four years, priorities remain flexible and subject to updating. This flexibility recognises that circumstances change and assessments can vary as new information becomes available (including, information from ongoing monitoring and review, changes in government policy-settings, and information from audit activity).

Group 1. Strategic Projects

Strategic evaluations that are underway or expected to commence in 2017-18.

Cross-cutting

Description

Evaluation capability development

Strengthening Indigenous community, service provider and government capability to effectively conduct and utilise high quality evaluations, and undertake performance management.

Data improvement projects

Build the availability of data that can inform evaluations. Significant subjects will be identified, such as, adult literacy and language interpreting.

Mental Health First Aid Training

Implementation review and participatory outcome study of training which aims to provide the critical knowledge and skills required to help with mental health issues.

Empowered Communities

Early implementation review to be co‑designed with Empowered Communities.

Strengthening Organisational Governance Policy

Under the IAS, all organisations receiving grant funding of $500,000 (GST exclusive) or more in any single financial year are required to incorporate under Commonwealth legislation. Evaluation will aim to assess the effectiveness of this policy, including any benefits to organisations and management of Commonwealth funding, and implications of any application beyond the Indigenous affairs portfolio.

Strengthening the system effectiveness of early childhood investments

Using an evidence informed evaluation framework to identify priorities, including examining significant impacts and cross-cutting, multi-sectoral elements to support systems’ improvement.

National Partnership Agreement on Remote Indigenous Housing and Remote Housing Strategy

Development and implementation of an approach to cross-cutting impact evaluation, with a focus on examining the range of potential effects on communities, overcrowding and non-shelter outcomes in the areas of employment, health and education.

Closing the Gap

Qualitative analysis of attitudes and reflections on 10 years of the Closing the Gap initiative.

 

Priority areas by sector

Description

Remote School Attendance Strategy

Measuring the impact of the Strategy on school attendance rates through an analysis of quantitative data.
A qualitative impact study will also explore how the Strategy may influence parental behaviour and choices that generate patterns of school attendance.

School Enrolment and Attendance Measure

A randomised controlled trial to examine the impact of the School Enrolment and Attendance Measure on school attendance rates.

Community Development Program

A multi-year impact evaluation using mixed methods to examine outcomes in the first two years, and explore how and why the program works in various contexts.

Group 2. Program Evaluations and Reviews

Program evaluations and reviews that are underway or expected to commence in 2017-18.

Jobs, Land and Economy Program

Topics

Project and description

 

Economic
and social participation

Indigenous Cadetship Support Program

This impact evaluation will examine the effectiveness of the Indigenous Cadetship Support program in supporting Indigenous Australians to complete their degree and move to ongoing employment.

Aboriginal employment and training activity

The review will examine employment outcomes for those participating in a tailored Aboriginal employment strategy and how these compare to outcomes for similar job seekers under other employment programs.

Indigenous Procurement Policy

A review of the first year of the implementation of the Indigenous Procurement Policy.

Review of Regional Employment Targets under the Community Development Program

The Regional Employment Target methodology will be reviewed and a tool for setting future targets will be developed.

Review of Indigenous Entrepreneurs Fund – Business Advisors

Undertake an operational review to assess the effectiveness of the business advisor model and consider options beyond the initial 12-month pilot period.

Environment

Indigenous Rangers and Protected Areas

The development of a shared measurement and evaluation framework for the Indigenous Rangers and Indigenous Protected Areas programs, developed in collaboration with stakeholders, will support regular impact assessment and on-going adaptive management of the programs into the future.

Indigenous Rangers Cost Analysis

The cost analysis informs a consistent approach to funding future projects and informs funding guidelines to ensure consistency and fairness.

Indigenous Rangers and Protected Areas Fee for Service

An impact analysis study assesses the impact of the Indigenous Rangers and Protected Areas programs as a catalyst for economic and employment outcomes through
fee-for-service commercial activities. The project shares lessons and success factors, and identifies opportunities for government to support the growing interest in developing fee-for-service opportunities.

Land

Our North, Our Future: White Paper on Developing Northern Australia

Development of evaluation strategies for three measures under the White Paper: Township leasing in the Northern Territory, Prescribed Bodies Corporate capacity building, and Pilot land reform projects in the north. This work will guide measurement, data collection and periodic evaluation over the long term, up to 20 years.

Native Title Representative Bodies and Service Providers

Review of five Native Title Representative Bodies and Service Providers covering performance, relative efficiency, and recent complaints made in relation to bodies being reviewed.

Role of Land Councils and Native Title Representative Bodies – post determination

Review of the potential future role of land councils and Native Title Representative Bodies following the determination of land claims and native title claims.

Children and Schooling Program

Topics

Project and description

 

Early childhood

 

Stronger Communities for Children Program

Theory-based evaluation and implementation review, assessing progress towards a range of outcomes.

Aboriginal Families as First Educators Program

A theory-based evaluation to examine what works, for whom, in what circumstances and why. Includes remote and non-remote communities.

Schooling

Girls Academies

Impact evaluation to assess what’s working, what isn’t, any unintended consequences and what could be improved. A review of the costs and benefit of the program to participants, schools and communities.

Indigenous Scholarships

An initial review of IAS funded secondary school scholarships aims to profile delivery models, demand, costs and outcomes. This will inform data improvement to support ongoing monitoring and impact evaluation.

Models of Studying Away from Home

To assess the costs, impacts and benefits of different models for students studying away from home with a view to understanding what works for different students and whether there are models that, in the longer term, we could have a greater emphasis on supporting.

 

Safety and Wellbeing Program

Topics

Project and description

 

Social and emotional wellbeing

Link Up Services Program

An evaluation will consider need and the impact of healing, wellbeing and resilience-building programs and connections to family, community and culture. The evaluation will also test approaches to assessing the longer-term individual and community impact of healing services and activities.

National Indigenous Critical Response Service

An implementation review will assess the effectiveness of the initiative in providing culturally appropriate support to Aboriginal and Torres Strait Islander people affected by suicide-related or other trauma. An efficiency review and impact study will also be considered, including for community development initiatives, which seek to strengthen community capacity to prevent and respond to suicide and other critical incidents.

Safety

Testing the impact of different program models for improving community safety and justice outcomes

This set of trial design projects includes: design and testing of a program model to enhance existing prisoner through-care services for Indigenous prisoners; and design and trial of a youth through-care program model for young Indigenous people exiting detention.

Testing community safety and justice outcomes using Randomised Controlled Trials

The evaluations will test the impact of activities to reducing offending, violence and improving community safety outcomes for Indigenous Australians. Projects include: prisoner through-care services in the NT, WA and Qld for Indigenous prisoners exiting the prison system; and activities aimed at improving the connection to therapeutic supports for children in the child protection system in VIC who have experienced trauma.

Community night patrols

A difference-in-difference analysis is being conducted to test the impact of community night patrols in the Anangu Pitjantjatjara Yankunytjatjara Lands communities of Amata, Pukatja and Mimili on safety outcomes.

Family Violence Prevention Legal Services

This outcome evaluation will use mixed methods and include a robust literature review. Where possible, it will include quantitative analysis of service data and tracking client outcomes.

Third Action Plan to reduce family violence – Indigenous specific measures

An impact evaluation of this significant activity includes a program review and assessment using in-depth interviews and existing service provider data. Evaluation plans will be developed covering a minimum of five sites, using either randomised controlled trials or quasi-experimental evaluation designs.

Substance
misuse

 

Alcohol and Other Drug Treatment Services

This impact evaluation is working with treatment services in NSW to prepare for impact evaluation, implement standardised data collection to support outcome measurement, and conduct impact evaluation using adaptive learning.

Petrol Sniffing

Longitudinal study examining the prevalence of petrol sniffing and other substance misuse in selected remote communities.

Nutrition

School Nutrition Program

An impact evaluation examining whether the School Nutrition Program provides the dietary requirements of students in remote Indigenous schools, and as the delivery and sustainability of the program.

Community Stores

An examination of the capacity of a real-time store food sales monitoring system to provide information to decision-makers at all levels, including participating store boards and other relevant policy makers, covering key food and nutrient performance indicators.

Sugar Reduction Strategy for Remote Community Stores

The evaluation will examine the impact of the Strategy overall and test targeted interventions being trialled in stores which aim to reduce sales of sugared products through remote community stores.

Disability

National Disability Insurance Scheme - East Arnhem Land Co-design project

A participatory explanatory evaluation will study the first 12 months of the implementation of the East Arnhem Land Co-design project using participatory action research.

Culture and Capability Program/Remote Australia Strategies

Topics

Project and description

 

Culture

Indigenous Broadcasting

This evaluation will review Indigenous Broadcasting using Social Return on Investment (SROI) methodology. The SROI will provide insight into the impact Indigenous Broadcasting is having on stakeholders through stakeholder surveys and indicators, and will use financial proxies to value outcomes.

Infrastructure

Army Aboriginal Community Assistance Program

This operational review initially examines effectiveness, sustainability and governance. It will also look at ways to leverage broader benefits to achieve better training, employment and economic development outcomes in communities where the program is implemented.

Leadership

National Aboriginal and Torres Strait Islander Leaders Program Pilot

An operational review of the pilot program covering implementation and delivery and an assessment of outcomes for building leadership aspirations.

Group 3. Future Topics

To assist with forward planning, possible future topics are identified each year. These future topics, which are outlined below, will be considered for possible evaluation activity over the next four years (forward estimates period).

Cross-cutting

Project and description

Building evaluation capability

Improve the capacity across the sector to do and use monitoring and evaluation, and access available evidence, including among service providers, government staff and in communities.

System effectiveness

Cross-cutting impact evaluation of the Indigenous Advancement Strategy (multi-year project).

Gender

Gender-sensitive monitoring and evaluation, considering economic security, leadership and safety across IAS activities.

Drivers of change

Examination of shared outcomes and the underlying drivers of change, for example by evaluating the impact of adult literacy programs in relation to inter-connected outcomes beyond the program (e.g. early childhood).

Youth participation

Examining youth engagement and achievement, building on existing evidence.

Prison to work

Evaluation of the Commonwealth Prison to Work Action Plan, or components therein, to determine the effectiveness of action to support Aboriginal and Torres Strait Islander people as they transition from incarceration into employment.

Place-based approaches

Evaluation of place-based or ‘community designed’ approaches and arrangements to support these approaches. This may include testing approaches to place-based evaluation, as well as conducting process and impact evaluations of the Regional Network led place-based design and practice project.

Regional investment

Regional Program Investment Reviews of Safety and Wellbeing, and Children and Schooling Programs.

Jobs, Land and Economy Program

Project and description

Employment services

Evaluation of the new employment and participation model for remote Australia.

Economic participation

Evaluation of youth transition to work programs focusing on pre-employment training and mentoring for Indigenous participants and the Transition to Work Program.

Business development

Evaluation of the Indigenous Business Sector Strategy.

 

Children and Schooling Program

Project and description

Economic and social participation

Evaluation of the impact of mentoring on youth engagement in high school education.

Tertiary

Evaluation of the Away from Base Program – supporting Indigenous VET and Higher Education distance students.

Evaluation of higher education focusing on the Indigenous Student Success Program.

Safety and Wellbeing Program

Project and description

Community wellbeing

Evaluation of the contribution of IAS sport and recreation activities to Indigenous community wellbeing.

Social and emotional wellbeing

Evaluation of the impact of social and emotional wellbeing support services and service delivery models to improve the lives of Indigenous people.

Social and cultural determinants of health

Investigation of the social and cultural determinants of health outcomes for Indigenous people.

Safety

Test and trial innovations in service models to divert high risk, youth and adult offenders from re-offending.

Adult corrective services data collection, development and analysis to inform prisoner to work strategies.


[1]  Future Work Plans will be endorsed by an independent Indigenous Evaluation Committee.

Feedback

Did you find this page useful?