Appendix 3. Interview guides

The following is an interview guide used to guide conversations with the young people.

Interview guide - Young people

Disclaimer

Our intent is to get as much background on the young people we will be interviewing from the case managers, this will allow us to focus on a smaller number of questions with the young people themselves.

The list of questions is not necessarily in the order that they will be asked. The aim of the interview is to keep the conversations fluid and as open as possible and the questions below serve merely as a guide.

Before each interview with a young person, we will check explicitly with their case manager if there are any sensitive areas that we should avoid such as asking about their past or the future, or their relationships with their family. If such areas exist, the interviewer will not touch upon those areas in conversations with the young people.

Introduction

My name is [ ], and I'm from Melbourne. I work at Social Ventures Australia (SVA), which is an independent non-profit organisation. We work with organisations that are doing good things to help them increase the impact of their activities.

We are working with Drum Atweme to understand and measure the impact of the program. I would like to interview you about your experiences with the program. I will ask you some questions about yourself, and what has changed for you because of being part of Drum Atweme.

This is not a test and your answers will be confidential. No information will be used in any way that reveals your identity. If you feel uncomfortable, you can stop the interview at any point.

Background

1. Tell me about yourself
2. How long have you been involved with Drum Atweme?

Inputs, outputs and outcomes

3. What made you want to join the program? What were you hoping to change in your life?
4. What do you do as part of the program?
5. What were some of the things that changed for you soon after you started the program?
1. Specifically, describe what Drum Atweme helped you do once you started?
2. How important was this support? Do you think you could have gotten support with these things somewhere else?
6. What are some of the good and bad things that have happened in your life because of Drum Atweme?
3. [Prompts for areas to explore: Offending; Health (mental & physical wellbeing); Relationships / Friendships; Education; Employment; Community Engagement?]
7. What things do you do differently now that you didn't do before the program?

Measuring and valuing change

8. Based on what you told me as well as what I have heard about the experiences young people report about the program, I would like you to help me understand how much these different things happen to you. I will read these out to you and I would like you to tell me if you have experienced this not at all, a little bit, quite a bit, or a lot? (Interviewer to tick the correct response).
   [Options for measuring change: Ask young people to pick cards with different sized shapes to represent how much change they experience; ask them to work alone to tick boxes; ask them to move to a point in a line; if speaking with groups of young people, could ask them to ask each other in pairs or put their hands up]
9. How important were these changes to you? (interviewer to tick the correct response)
   [Rating: Not important, a little important, Important, critical, N/A]
10. We are trying to understand how valuable the program is to you. As the world works on dollars and cents, we're trying to put this in dollar terms. We can do this by comparing what's changed for you to the value of other ways that could have happened, or by understanding how the program ranks compared to other things you like.
    
    [Options for testing value:
    
    Stated preferences technique
• Value game: So, I'd like to ask you to order these 5 things in order of the most important thing to you to the least important thing to you. Where would you put the program in this order? [Need to develop a list of 5 things or activities that young people like and that have a market value]
• Willingness to pay: If you could have the money that it costs to deliver the program in your pocket instead ($x), what would you do with it? How much, if any, would you spend on attending the program? [Need to determine cost per participant]

Revealed preferences technique

• Replacement valuation: What are some things you could do that would come closest to getting you the same changes you have experienced through the program? [Need to develop a list of 5 things or activities that young people do and that have a market value]

11. What do you think would be different in your life if you weren't involved in this program?
    [Another way to ask:] If the program did not exist, how much of these things would have happened to you anyway?
12. What do you think the changes you have seen in your life will mean for your future?
1. What are some of your plans for the future? Have you always had these plans or you have changed them since starting the program?
13. How long will / did the changes you spoke about last for?
1. If you were to leave the program today, how long would you continue to experience the changes you described?
2. You told me that some things are different for you now since joining the program, is that just because of being part of this program or because of some other things or people helping too? [For example, are there any other organisations involved? What has been the impact of your teachers or employer?]
14. Has anything changed for your family as a result of your involvement with the program?
15. Are the any other changes you would like to share with me that have happened since joining the program?

The following is an interview guide used to guide conversations with the Drum Atweme Coordinator.

Interview guide - Staff

Disclaimer

The list of questions is not necessarily in the order that they will be asked. The aim of the interview is to keep the conversations fluid and as open as possible and the questions below serve merely as a guide.

Introduction

My name is [ ], and I'm from Melbourne. I work at Social Ventures Australia (SVA), which is an independent non-profit organisation. We work with organisations that are doing good things to help them increase the impact of their activities.

We are working with Drum Atweme to understand and measure the impact of the program. This conversation is to understand the changes experienced by the young people taking part in the program, and also your personal experience of working with Drum Atweme.

No information will be used in any way that reveals your identity. If you feel uncomfortable, you can stop the interview at any point.

Background

1. How long have you been working with the organisation?
2. Please describe your key duties, roles and responsibilities
3. How many young people have you worked with in the past and how many are you working with now?
4. Who are the other stakeholders in the program?

Program activities and inputs/investment

I would like to understand more about the program.

5. Have there been any changes to the program in the past, including level of investment (financial or otherwise) and type of support provided to young people?
6. If so, why did the program change?
7. For the program to run the way it does, what additional un-paid time or community resources are required? How much un-paid time or resources is needed per week/month?
8. Is the program likely to continue in the future?
9. Are you expecting any changes to the program, including level of investment and type of support provided to young people?
10. Would these changes be likely to change the outcomes experienced by young people and other stakeholders?

About the changes experienced by the young people

I would like you to consider some of the changes in the lives and behaviour which you have observed in the young people under your care or more broadly participating in the program.

If you would find it helpful, use specific examples of stories or experiences you have shared with the young people, though it is not necessary to reveal their names.

Background

Please tell me a little bit about the young people who you currently work with

11. What are the situations / circumstances that bring them into the program?
12. Are there key categories of young people who participate (e.g. genders, ages, level of involvement, length of involvement, family circumstances, past involvement in justice system)?
13. How do you support these young people?
14. How often do you see them?
15. How long do you work with each young person?
16. How do you work with other organisations? How significant is the impact of other organisations in being able to successfully do your job?
17. Is there anything else that we need to know about the young people or the program which would influence our understanding of the changes they might experience and the impact of the program on their lives?

Measuring and Valuing Impact

Thinking about young people you worked with both past and present ...

18. What specifically do the young people hope to change in their lives by being part of the program?
19. What are some of the most important changes, good and bad, that these young people experience as a result of the program?
1. [Prompts for areas to explore: Offending; Health (mental & physical wellbeing); Relationships / Friendships; Education; Employment; Community Engagement?]
2. Do these changes affect their families in any way? How?
20. Specifically, what things do they do differently as a result of the program that they didn't do before?
21. Which of the changes that you have described are:
1. The most important / significant to the young people? Why?
2. Important to other stakeholders? Why?
22. What activity is linked to what outcomes?
23. What do these changes mean for the future of these young people?
24. How would young people value the changes?
1. What are some other things young people could have done that would have led them to experience the same changes?
2. What are some things or activities that young people like to do (and that have a market value)?
25. From your experience, before these young people come into the program what sort of support or governmental services would have they been using?
1. For example: welfare services/Centrelink, doctors/hospitals, child protection, police...
2. Do they continue using or contacting these services more or less during the time at the program and after case management ceases? How much less e.g. one less police call out per person, 6 months less working with employment service provider? Is there data available on service use?
3. Are there cost savings associated with decreased service use?
26. How would their lives have been different if this program did not exist?
1. What sort of services and support would have they received?
2. To what extent do you think these young people would have been able to achieve the things you have seen them do, if the program did not exist?
27. The changes in the lives of the young people, how long each of them is likely to last for?
28. What percentage or number of young people have experienced these outcomes, and to what extent? How important are these changes to the young people? Who else contributed to these changes? What would have happened if the program wasn't there? How long is each of these changes likely to last for?

About your experience of working with the organisation

Measuring and Valuing Impact

29. How does having this job make you feel?
30. Were there any immediate changes to your life/wellbeing that you noticed after starting work here?
1. How are these changes different to those experienced in previous workplaces?
31. What are some of the new competencies and skills that you have developed from working here?
1. How important are they to you?
2. Do you think you would have acquired similar skills and competencies in other jobs that were available to you? Why or why not? How big is the difference?
32. What has changed in your life as a result of working for the program?
1. Is there anyone else who has been affected by these changes besides yourself as a result of your working for the program? (for example, members of your family, friends etc)
33. Is there anything that you do differently now that you did not do before you started working here?
34. Are there any negative changes as a result of having a job here?
35. What do you believe would be different now in your life if you had not got this job?
36. Were there any other factors / organisations / people which contributed to the changes you told me about?
37. How long would the impacts you spoke about last?
38. Are the any other feelings you can share with me that you have experienced since having this job?

The following is an interview guide used in conversations with other agencies and organisations.

Interview guide - Other organisations

Disclaimer

The list of questions is not necessarily in the order that they will be asked. The aim of the interview is to keep the conversations fluid and as open as possible and the questions below serve merely as a guide.

Introduction

My name is [ ], and I'm from Melbourne. I work at Social Ventures Australia (SVA), which is an independent non-profit organisation. We work with organisations that are doing good things to help them increase the impact of their activities.

We are working with Drum Atweme to understand and measure the impact of the program. This conversation is to understand the changes experienced by the young people taking part in the program, and also your personal experience of working with the program.

No information will be used in any way that reveals your identity. If you feel uncomfortable, you can stop the interview at any point.

Background

1. How long have you been working for your organisation?
2. Please describe your key duties, roles and responsibilities
3. How is your organisation involved with the program, or the young people who participate in the program??
4. How long has your organisation been involved with program?
5. What made you want to get involved with the program?
6. How many young people have you worked with in the past and how many are you working with now? How many of them have been involved in the program?
7. What do you put into the program? Time? Effort? Money?

About the changes experienced by the young people

I would like you to consider some of the changes in the lives and behaviour which you have observed in the young people participating in the program.

If you would find it helpful, use specific examples of stories or experiences you have shared with the young people, though it is not necessary to reveal their names.

Measuring and Valuing Impact

Thinking about young people you worked with both past and present .

8. What specifically do the young people hope to change in their lives by being part of the program?
9. What are some of the most important changes, good and bad, that these young people experience as a result of the program?
1. [Prompts for areas to explore: Offending; Health (mental & physical wellbeing); Relationships / Friendships; Education; Employment; Community Engagement?]
2. Do these changes affect their families in any way? How?
10. Specifically, what things do they do differently as a result of the program that they didn't do before?
11. Which of the changes that you have described are
1. The most important / significant to the young people? Why?
2. Which of these changes are important to other stakeholders? Why?
12. What activity is linked to what outcomes?
13. What do these changes mean for the future of these young people?
14. How would young people value the changes?
1. What are some other things young people could have done that would have led them to experience the same changes?
2. What are some things or activities that young people like to do (and that have a market value)?
15. How would their lives have been different if this program did not exist?
1. What sort of services and support would have they received?
2. To what extent do you think these young people would have been able to achieve the things you have seen them do, if the program did not exist?
16. For the changes described in the lives of the young people, how long is each of them is likely to last for?

About the changes experienced by you and your organisation

Measuring and Valuing Impact

17. What has changed for you and your organisation as a result of being involved with the program? Which changes are most important?
18. What do you hope your involvement with the program will lead to in the future?
19. Are there any negative factors arising from your involvement with the program?
20. How do you measure the changes (outcomes) you have described above?
21. How would you value these outcomes?
22. Is there a financial proxy you can use to value that outcome?
23. From your experience, before these young people come into the program what sort of support or governmental services would have they been using?
1. For example: welfare services/Centrelink, doctors/hospitals, child protection, police...
2. Do they continue using or contacting these services more or less during the time at the program and after case management ceases? How much less e.g. one less police call out per person, 6 months less working with employment service provider? Is there data available on service use?
3. Are there cost savings associated with decreased service use?
24. What would have happened for you and your organisation without the program?
25. What other organisations or people, if any, played a role in helping you achieve the changes you described?
26. How long would you continue to experience the outcomes you described if your organisation was no longer involved in the program?
27. Has being involved in the program displaced other activities or outcomes you would have done / achieved?