9. Recommendations
On this page:
This section makes recommendations for the Drum Atweme program on how to improve outcomes measurement for the future and other actions that build on the insights from the analysis.
9.1 Recommendations to improve outcomes measurement
At the commencement of this project, there had been little data collected over time to indicate the changes experienced by stakeholders and quantities of stakeholders experiencing those changes. Through stakeholder consultation, and working with the Drum Atweme Coordinator, we developed an understanding of the changes experienced by stakeholders and constructed a dataset of changes experienced by stakeholders that had been involved with the program in the past.
To indicate the quantity of young people that will experience the outcomes we:
- Mapped the experience of performance group members - starting year, current age range, and current situation of all past members
- Modelled the age profile of the current performance group, and the number of members expected to enter the group each year and the number of years members would be expected to stay
- Grouped members into number of years expected to stay in the program.
We have made the following professional judgements in our modelling of the changes young people will experience:
- All young people in the program for over a year experience each outcome
- The extent of the outcome they experience is based on the number of years they are in the program for
- Five new young people join the group per annum, and five leave
- The age distribution of the future performance group will match the current age distribution.
This approach is described in detail in Section 5.1.
The analysis included in this report is reliant on estimates made by the Drum Atweme Coordinator, based on his extensive knowledge of the young people involved. We cross-checked the estimates and assumptions with other stakeholders, including our notes from interviews with young people themselves, and tested them against population data (where relevant).
To improve the rigour of future analyses, Drum Atweme should collect data on the activity delivered (outputs) and the changes experienced by stakeholders as a result of this activity (outcomes) on an ongoing basis. Drum Atweme should focus on answering these five questions:
- Who is changed?
- How do they change?
- How do you know that they have changed?
- How important are the changes?
- How much is as a result of you?
As part of this project SVA Consulting has worked with the Department of the Prime Minister and Cabinet and IJP service delivery organisations to develop a tool to collect answers to these questions. This has been informed through the stakeholder consultation and the SROI analysis.
Drum Atweme should use the tool to track a sample of the young people that they work with:
- Date commenced with the program (start date) and date exited the program (end date)
- Program engagement (e.g. active, inactive)
- Activities that Drum provides (e.g. recreation, material goods)
- Progress towards the achievement of each of the outcomes (increased self-esteem; increased engagement in meaningful activity; more positive connections to others; avoidance of unlawful behaviour)
- Any unintended or negative outcomes that arise
- Number of years that the outcome is experienced for
- Background (e.g. education level; employment history; mental health issues; housing situation; alcohol and drug use; family situation; offending history) (this could be done through the risk profile surveys)
- Other support services accessed (i.e. to indicate who else is contributing to change, which will be used to calculate attribution in the future).
A Social Outcomes Measurement and Evaluation Tool has been developed as a part of this project.
Due to the nature of the program and the number of young people in the program, the Social Outcomes Measurement and Evaluation Tool was developed with the following objectives in mind:
- Simple - not over-burdensome on program staff or undermine their ability to deliver the project, including the critical relationship development aspect of the program;
- Meaningful - helping the program staff to deliver optimally against their objectives;
- Timely - allowing for regular collection and monitoring of data to allow for course corrections;
- Transparent - negative findings are identified, acknowledged and addressed;
- Context-aware - clear about how changes in the environment of the project impact on project outcomes (e.g. changes to youth justice approaches in different states and territories); and Provide value - to young people, to program staff, as well as funders.
The Social Outcomes Measurement and Evaluation Tool is designed so that the young people self-select whether they are experiencing change as a result of the program. For Outcome 1.1 (Increased self-esteem), the young people are asked to answer a questionnaire based on Rosenberg's Self-Esteem scale, which has been proven to provide accurate representation of an outcome of self-esteem. Based on these responses, the program manager can record whether the young person's self-esteem is extremely low, low, or normal. Similarly, for Outcome 1.3 (More positive connections to others) the young person is asked to answer questions about their connection to the people around them which were designed with reference to The Household, Income and Labour Dynamics in Australia survey conducted by the University of Melbourne, which is widely used by Australian and International researchers and by the Australian Government. The Social Outcomes Measurement and Evaluation Tool will enable objective and subjective indicators to be used.
It is recommended that data be collected on intake of the young person, at six monthly intervals and at exit from the program. Regular monitoring of data will be useful for demonstrating to the young people what progress that they have made and to engage them in their journey through the program. It will also help the Drum Atweme Coordinator to demonstrate the importance of these changes to stakeholders and to determine how much is as a result of the project, as the tool allows for information to be recorded about other services that the young person is using. The tool should also assist the Coordinator to better plan how each young person is supported and how to balance competing needs of different young people with limited resources.
The Social Outcomes Measurement and Evaluation Tool has been developed to be integrated into other monitoring and reporting that the Coordinator already undertakes. Figure 9.1 indicates the proposed timing for using the (the green crosses) and how it can be used as a resource for the program manager to assist with other information collection.
Figure 9.1: Proposed timing of capturing data through the Social Outcomes Measurement and Evaluation Tool
The Drum Atweme Coordinator should also capture the number of families and community mentors that it interacts with and periodically assess the changes and the value they derive, as a result of the program. This could help to identify ways how these stakeholders should be involved in the program in the future, to generate the most value for both the young people and these stakeholder groups. This could be achieved through a short face-to-face or telephone survey conducted on an annual basis. These steps will aid in communicating the full impact that the program is having across the different stakeholder groups.
In future it may be possible to compare the outcomes achieved by the program with the outcomes achieved by similar programs, and with changes in the population. This will help to understand the relative effectiveness of the program.
9.2 Other recommended actions
As a result of the insights from the analysis, the following actions are also recommended for the Drum Atweme program.
Funding the program
1. Secure funding for the full costs of the program for more than one year in recognition of the time taken to generate changes for the young people involved
The Department of the Prime Minister and Cabinet contributes the majority of the funding to Drum Atweme. This investment supports 1 FTE Coordinator to work a standard 37.5 hour working week. On average, the Coordinator works an additional 10 hours per week. This extra time equates to $140k in in-kind investment into the program. The program funding is also supplemented by performance fees, which fund performance trips and purchases of essential items for performers. This is a substantial resource pool that could be invested in expanding the group.
At this stage, the program is still reliant on funding from the Department. This reliance is expected to decline over time as the program operates more as a social enterprise. For the next period, however, it is recommended that Drum Atweme seek funding that reflects the full cash and in-kind investment required for the program to ensure that it can be delivered on a sustainable basis.
There is a range of risk factors that may make young people more likely to engage in anti-social behaviours. The young people in the Drum Atweme performance group are exposed to many of these risk factors. As a result they require intense, long-term support to avoid anti-social behaviour and to develop the sense of identity necessary to maintain a positive path in life. Some young people in the performance group are now 16 and have been involved since they were seven or eight. To ensure continuity of support for these young people, it is recommended that the program is funded for periods greater than one year, ideally for five years or more.
2. Seek funding from the Northern Territory Government in recognition of the significant justice system cost savings forecast to be generated by the program
The Drum Atweme program is expected to generate significant cost savings for the justice system. The justice system is administered by the Northern Territory Government. It is recommended that Drum Atweme share the findings of the analysis with the Northern Territory Government and seek funding to support its activities. As the current primary funder, the Department of the Prime Minister and Cabinet could play a role in communicating the credentials of the program to the relevant section of the Northern Territory Government.
3. Resource the program with more than one person to ensure sustainability and increase impact
The success of the program is heavily reliant on one key figure, the Coordinator. The strength of the relationship between the Coordinator and the young people is critical to the success of the program. However, it poses a risk to the sustainability of the program. Proactive succession planning is required and could be achieved by bringing on-board a trainee to be skilled up to support the work of the Coordinator. The Drum Atweme Coordinator has considered a number of young people as potential trainee drumming instructors; however, these young people have faced different challenges that have not allowed them to take up the role as yet.
Additional resourcing to support another drum instructor would allow Drum Atweme to undertake a number of activities to increase impact, including:
- Meet demand from other local schools that have indicated they would like to have drumming classes
- Expand the size of the performance group to support more young people at risk
- Undertake more performances to increase social enterprise revenue
- Support the young people who have had long-term involvement with the group to build skills and have possible employment as drumming instructors
- Start a 'Young Mums' drumming group for the young women drummers who have had children
- Start a boys group to encourage boys to be involved
- Offer formal Drum Beat sessions to young people with therapy needs
If a significant amount of additional funding could be sourced, the Drum Atweme Coordinator could take forward plans to establish an Indigenous theatre that creates employment opportunities and attracts tourists to Alice Springs.
Scaling the program
4. Engage business mentoring support to enhance the social enterprise element of the program
Drum Atweme focuses on the social impact it can create, with the income from performances a secondary consideration. There appears to be a lot of potential, however, to increase the commercial revenue of the performance group and to operate more like a social enterprise. Stakeholder consultation indicated that the current performance fees are less than the value or market rate that customers would be willing to pay. Drum Atweme currently relies on word of mouth referrals for performance bookings.
Initially, Drum Atweme needs to document the unique aspects of its model and develop its web presence and promotional material. A volunteer position or project could be created to develop a stand-alone website to collate media appearances, photographs, videos, testimonials and case studies on performers, and advertises the group for performance bookings.
Then, seeking business mentoring support from the local community could allow Drum Atweme to conduct an assessment of the local tourism market and trial different price points for different performance activities and audiences. Tourism NT and the NT Convention Bureau could also be approached to provide support in this area. In addition, this support could be used to develop a corporate sponsorship strategy to seek additional funding during the transition to social enterprise.
5. Investigate the feasibility of expanding the model to other areas
With additional government funding, commercial revenue and corporate support, Drum Atweme could consider expanding the model to other geographic areas such as Adelaide or Darwin. The Clontarf Foundation could be used as a model of a program that has scaled its impact significantly.
6. Share knowledge of the program with other organisations focused on youth justice early intervention approaches
The Drum Atweme program has many of the characteristics recognised as best practice in programs for Indigenous youth at risk.16 These include:
- Focusing on early intervention by the age of 8-10 years, before anti-social behaviour starts
- Working with young people who are experiencing multiple risk factors for anti-social behaviour
- Long-term mentoring relationship between the Coordinator and the young people, that continues as young people consolidate positive changes
- Consistent, regular contact between the Coordinator and the young people
- Integration of the mentoring relationship into a broader activity
- Strong partnerships between the program and other community, education, health and welfare services
- Involving Elders in supporting the young people
Drum Atweme should seek out opportunities to share its program design, implementation and impact with other organisations focused on supporting young people at risk, particularly Aboriginal young people. This could take the form of conference presentations, seminars, or research studies.