As someone who has been designing, implementing, and examining treatment outcomes for people with AOD issues since the last century (no exaggeration!), I was delighted that my team at Turning Point was able to work on this exciting project with First Step, evaluating the impact of services including co-designing their ongoing outcome monitoring framework.
Research tells us that psychological and pharmacological treatments can be effective in reducing AOD harms, but this evidence typically comes from rigorous, tightly controlled clinical trials with treatment intensities and incentivised compliance rates that are near impossible to achieve in real-world settings and within current funding structures. More importantly, trial participation eligibility criteria often exclude people with co-occurring disorders or complex issues, meaning we fail to understand how treatment works for those who stand to benefit the most.
Routine outcome monitoring ensures that anyone who is willing to tell us about their treatment experience can do so. It is an expected and routine part of medical care for most chronic disorders, yet in the AOD sector, it occurs on an ad-hoc basis, using inconsistent measures and approaches. Furthermore, the use of standardised measures to assess treatment effectiveness and satisfaction allows data to be aggregated to indicate service performance, inform quality improvement activities, and inspires new clients to come forward for treatment.
Effective AOD treatment means more than minimising harm and risk, and more than ceasing or reducing use of a psychoactive substance. It may also include improved mental health and quality of life, stable housing or employment, better quality relationships, or greater community connectedness. Since treatment goals differ from person to person, any assessment of treatment impact may capture change that is meaningful to the individual.
A common challenge in using Patient Reported Outcome Measures (PROMs) and Patient Reported Experience Measures (PREMs), is the time it takes to complete them. Thus, in pursuit of both comprehensiveness and brevity, it is critical that we ask questions that are directly relevant to what clients hope to achieve. To do this, Turning Point researchers have been running focus groups and surveys with staff and clients at First Step to co-design and pilot a tailored outcome measure.
With its unique co-location of GP and legal support services on site, we expect the findings to illustrate improvements across a broad range of life domains. We are also engaging other key stakeholders to ensure the resulting measures enable First Step to demonstrate the true impact of its integrated care. We look forward to sharing the results of the evaluation with First Step and other AOD services at the end of 2025.
Victoria Manning
Professor in Addiction Studies, Monash University and Head of Research and Workforce Development, Turning Point
Professor Victoria Manning and Associate Professor Annette Peart from Monash University and Turning Point have been engaged to conduct an independent 24-month impact evaluation of First Step's services, a project generously funded by the John T Reid Charitable Trusts. The work has several elements, including developing tools for ongoing assessment of the impact of services. We are currently trialling the tool developed in this project, and the final report of the impact evaluation is due in October 2025.
