Get 20M+ Full-Text Papers For Less Than $1.50/day. Start a 14-Day Trial for You or Your Team.

Learn More →

Staff Perceptions on the Implementation of Structured Risk Assessment with the START:AV: Identifying Barriers and Facilitators in a Residential Youth Care Setting

Staff Perceptions on the Implementation of Structured Risk Assessment with the START:AV:... INTERNATIONAL JOURNAL OF FORENSIC MENTAL HEALTH 2020, VOL. 19, NO. 3, 297–314 https://doi.org/10.1080/14999013.2020.1756994 Staff Perceptions on the Implementation of Structured Risk Assessment with the START:AV: Identifying Barriers and Facilitators in a Residential Youth Care Setting a,b b c Tamara L. F. De Beuf , Corine de Ruiter , and Vivienne de Vogel a b Research Department, OG Heldring Institution, Zetten, the Netherlands; Department of Clinical Psychological Science, Maastricht University, Maastricht, the Netherlands; Research Department, The Forensic Care Specialists, Utrecht, the Netherlands KEYWORDS ABSTRACT START:AV; implementation An effective implementation approach is crucial for successful integration of structured risk determinants; risk assessment instruments into practice. This qualitative study explored barriers and facilitators assessment; consolidated to the implementation of the Short-Term Assessment of Risk and Treatability: Adolescent framework for Version (START:AV) in a Dutch residential youth care service. Perceptions of staff members implementation research; from various disciplines were gathered through focus group interviews at three consecutive focus groups occasions. After inductive coding of the interview extracts using thematic analysis, the identi- fied codes were linked to the consolidated framework for implementation research. Through this framework, factors that influence an implementation project can be organized into mul- tiple domains and constructs. In the present study, staff members described implementation barriers related to characteristics of the risk assessment instrument, staff, and the implementa- tion process. In addition, features of the setting were frequently mentioned as hindering the implementation, such as hierarchy, culture, communication, as well as implementation climate and readiness for change. Staff members also identified multiple facilitators, such as experi- enced advantages of the START:AV compared to the previous risk assessment practice and positive beliefs about the instrument. The article concludes with recommendations for suc- cessful implementation of structured risk assessment instruments in forensic-clinical practice. Over the past decade, researchers have flagged the “positive outcomes are achieved only when both the need for more research and policy on the implemen- implementation process and the practice are effective” tation of risk assessment instruments (Desmarais, (p. 465). Other researchers have expressed concerns 2017; Nonstad & Webster, 2011). Implementation that even psychometrically sound instruments will fail refers to the processes that bridge the gap between the to improve outcomes for clients if not implemented decision to adopt a new practice and the committed properly (Desmarais, 2017; Schlager, 2009). For use of this practice (Damschroder et al., 2009). Once example, studies have found that services with better risk assessment instruments with sufficient predictive implementation quality (e.g., adherence to the admin- validity became available, successful implementation istration procedure) had significantly better results in was considered ‘the new challenge’ in risk assessment terms of risk management and service allocation practice (Nonstad & Webster, 2011, p. 94). How can (Vincent et al., 2012; 2016). In turn, improved risk we effectively integrate risk assessment instruments in management, by matching identified needs with practice and ensure fidelity of application? This can appropriate service provision, is associated with be a challenging undertaking, for example, due to staff reduced reoffending (Peterson-Badali et al., 2015). resistance and insufficient awareness of potential pit- To ensure implementation quality, a greater aware- falls (Schlager, 2009; Webster et al., 2006). Yet, the ness and understanding of the barriers to successful quality of implementation is crucial for the effective- implementation of risk assessment instruments in ness of risk assessment instruments in reducing recid- forensic-clinical practice is needed (Haque, 2016). ivism. Muller-Isberner € et al. (2017) stated that Webster and colleagues (2006) underscored the CONTACT Tamara L. F. De Beuf t.debeuf@ogheldring.nl Ottho Gerhard Heldring Institution, P.O. Box 1, Zetten 6670 AA, the Netherlands This article has been republished with minor changes. These changes do not impact the academic content of the article. 2020 The Author(s). Published with license by Taylor & Francis Group, LLC. This is an Open Access article distributed under the terms of the Creative Commons Attribution-NonCommercial-NoDerivatives License (http://creativecommons.org/licenses/by- nc-nd/4.0/), which permits non-commercial re-use, distribution, and reproduction in any medium, provided the original work is properly cited, and is not altered, transformed, or built upon in any way. 298 T. L. F. DE BEUF ET AL. importance of developing an implementation plan setting. Although the external context (e.g., govern- that reflects upon potential obstacles, such as limited mental guidelines and legislation) was mentioned in resources or poor communication. Factors that hinder most studies, it was not reported as a factor affecting effective implementation are referred to as implemen- the implementation. Determinants related to the inner tation barriers (Damschroder et al., 2009). Reflecting setting and the implementation process were most upon factors that promote successful implementation commonly reported. For example, all studies (e.g., management support, staff buy-in) should also addressed the importance of ensuring staff engage- be part of the implementation plan. These factors are ment from an early stage as well as providing infor- referred to as implementation facilitators. Insight into mation and training to facilitate the implementation. the barriers and facilitators is crucial when deciding Characteristics of the risk assessment instruments on which implementation strategies to adopt. were also cited as determinants, mainly as barriers. In 2016, Levin and colleagues published a system- For example, some risk assessment instruments were atic review of studies that documented determinants (initially) considered time-consuming or difficult to of successful risk assessment implementation in adult use. On the other hand, the potential to adjust an and adolescent (forensic) psychiatric and correctional instrument to fit the local routines was repeatedly settings. They included 11 studies, published between mentioned as a facilitator. Beliefs and concerns of the 2000 and 2013, in which the authors discussed factors users were reported in all but one study as influencing they perceived as hindering or facilitating the imple- the implementation. For example, the perception of mentation. Levin et al. (2016) organized these deter- clinical usefulness among users was identified as a minants according to the consolidated framework for facilitator, whereas a previous negative experience implementation research (CFIR; Damschroder et al., with structured risk assessment was considered a bar- 2009). The CFIR is a typology of implementation rier to the implementation. determinants compiled from 19 theories, and consists Central to the present study is the Short-Term of five domains and 39 constructs. The first domain Assessment of Risk and Treatability: Adolescent ‘intervention characteristics’ (eight constructs) con- Version (START:AV; Viljoen et al., 2016). The cerns features of the method that is being imple- START:AV is an evidence-based structured risk mented, such as cost, complexity, and adaptability of assessment instrument for use with boys and girls the new method. The second domain ‘outer setting’ between 12 and 18 years old. It was developed, piloted (four constructs) refers to external economic, political and evaluated in North America, mainly within juven- and social influences that can impact an implementa- ile justice populations. The START:AV was deemed tion process within an organization. In addition to the the most appropriate instrument for the present political context and the professional network of an secure youth care facility, because of its emphasis on organization, the needs of service users are also con- dynamic risk factors and its balanced approach that sidered as determinants within the outer setting. The includes both risk and protective factors. The dynamic domain ‘inner setting’ (14 constructs) concerns the nature of the instrument, with a recommended organization’s internal context, such as culture and reassessment interval of three to six months matched climate, communication structures, staff involvement, the setting’s four-month cycle of care. Additionally, and available resources. The fourth domain ‘user char- compared to other risk assessment instruments for acteristics’ (five constructs) pertains to the characteris- adolescents, the START:AV evaluates a wider spec- tics of the users of the implemented method. Users, trum of adverse outcomes. Short-term risk is eval- usually staff members, are considered active recipients uated for violence to others, nonviolent reoffending, with beliefs, attitudes, and ambitions that affect their substance abuse, unauthorized absence, suicide, self- behavior in relation to the implementation. The last harm, victimization, and self-neglect. All of these domain concerns the ‘process’ (eight constructs). An adverse outcomes are highly prevalent among adoles- implementation process consists of various steps that cents admitted to residential youth care settings in the require action, from planning and engaging, to exe- Netherlands (Vermaes et al., 2014). cuting and evaluating. Each step includes activities We are aware of one implementation study with that can enable or hinder the implementation. the START:AV conducted by Sher and Gralton (2014) Levin et al. (2016) used these domains and con- in a British medium secure adolescent service. This structs to organize the hindering and enabling factors study aimed to establish gaps in training and involve documented in the reviewed studies. They identified staff in the implementation process by inquiring their determinants linked to all CFIR domains, except outer views and experiences with the instrument. Although INTERNATIONAL JOURNAL OF FORENSIC MENTAL HEALTH 299 the authors did not explore barriers and facilitators to of the implementation process and initial experiences the implementation, implementation determinants can with the START:AV, using a focus group interview be derived from their findings and the CFIR domains method. The present article focuses on one particular can be applied. With respect to intervention character- issue that was addressed by the qualitative approach, istics, staff members indicated that the START:AV that is, implementation determinants (i.e., barriers and was easy to use, with some difficulties in distinguish- facilitators) as perceived by staff members. ing between the item ratings (low, moderate, high)as well as differentiating between strengths and vulner- Setting abilities. Some staff members also felt that the The study took place in one of 14 Dutch secure resi- START:AV could be confusing and somewhat repeti- dential youth care facilities for boys and girls who suf- tive. With respect to the ‘inner setting’ domain, issues fer from severe behavioral and mental health with untrained staff and insufficient practice with the problems. Youth are admitted under civil law, with a instrument were reported. In addition, staff experi- child protection order, to improve their safety (e.g., enced not enough time to complete an assessment. suicidal behavior, victimization) and/or the safety of Lastly, determinants related to user characteristics others (e.g., violence toward others). At the time of were noted: staff expressed facilitating beliefs about the study, the service had three high secure (of which the START:AV as valuable and helpful, including the two were observation units) and six medium-secure overall clinical usefulness of the instrument, and the (treatment) units, serving about 240 youths each year. added value of rating vulnerabilities as well as In 2016, adolescents (58% girls) were admitted for strengths. On the other hand, some believed that the 262 days on average, ranging from 4 to 717 days (A. START:AV might not be sensitive enough to measure Baanders, personal communication, January 31, 2019). change within a complex adolescent population. In Prior to the implementation of the START:AV, no addition, some raised the concern that the instrument validated structured risk assessment instrument was was only as useful as the level of insight of the team routinely used within the service. Although the Dutch into a patient’s problems. Nevertheless, most staff version of the Structured Assessment of Violence Risk members felt confident in their ability to complete the in Youth (SAVRY; Borum et al., 2006/2006) was avail- START:AV and contribute meaningfully to the assess- able and some treatment coordinators were trained in ments. Aspects of the outer setting and implementa- its use, the instrument was completed only a few tion process could not be gathered from this study. times a year. Specifically, it was used when boys with severe aggressive and delinquent behavior, transferred The present study from juvenile detention, were admitted to the service. Loosely based on the SAVRY, the service had con- In line with Levin et al.’s review (2016), the present structed a 12-item risk checklist that was used system- study explored which factors influenced the imple- atically in decision making regarding the youths’ leave mentation of the START:AV in a residential youth status (escorted/unescorted). The items of this risk care service. Implementation determinants from the list, such as ‘impulsivity’ and ‘association with deviant perspective of staff members were gathered through peers’, were rated by treatment coordinators as low, focus group sessions and organized according to the moderate or high without any rating criteria. Thus, CFIR. To our knowledge, this is the first primary the START:AV was introduced to replace both this study to apply the CFIR to risk assessment implemen- 12-item risk list and the SAVRY. In addition, the tation research. START:AV substituted another self-constructed list: the ‘dimension list’. This list was used upon admission Method to the observation units to gather information on 15 This study is part of a larger evaluation study on the developmental areas, the so-called dimensions (e.g., implementation of the START:AV, using a mixed- parent-child interaction, autonomy). Hence, in add- method design to assess various aspects of implementa- ition to risk assessment, the START:AV would serve tion within a youth care service. Previously, we used a as the service’s primary instrument to gather and quantitative web survey method to ask staff members structure treatment-relevant information on about multiple implementation outcomes (e.g., feasibil- the adolescent. ity, acceptability; De Beuf et al., 2019). Using a qualita- The START:AV implementation took place in a tive approach, we explored staff members’ perceptions turbulent social and political context. In 2015, a new 300 T. L. F. DE BEUF ET AL. Dutch Youth Act went into effect, resulting in major and an (external) risk assessment expert (second changes in the youth care system. One of the objec- author). The coordinator and the implementation tives of the new act was to decrease the use of costly committee were responsible for the implementation specialized services such as residential youth care procedure and the development of policies and proce- dures that detailed the incorporation of the (Hilverdink et al., 2015). This affected the present ser- vice in terms of a reduction of beds, staff reorganiza- START:AV into the service’s workflow. In the months tion and lay-offs, and an increased workload for the between September 2014 and March 2015, the imple- mentation coordinator trained treatment coordinators remaining staff. in using the instrument. The training included a one- day workshop in which they learned to rate the START:AV START:AV, followed by additional practice cases that The START:AV consists of 26 dynamic risk factors were discussed during a two-hour workshop. Later, a addressing characteristics of the adolescent (e.g., coping, third half-day workshop was organized in which treat- social skills), their relationships and environment (e.g., ment coordinators received guidance on how to trans- peers, parental functioning), and their response to treat- late the risk assessment findings into a risk ment (e.g., insight, treatability). The items are rated on management plan according to the Risk-Need- a3-point scale (low, moderate, high) based on function- Responsivity (RNR) principles (Bonta & Andrews, ing in the last three months: once as a protective factor 2017). All treatment coordinators that were hired after (strength) and once as a risk factor (vulnerability). After the initial training, received one-on-one training weighing and integrating all available information, the shortly after recruitment, by the same trainer. From evaluator formulates a final risk estimate (low, moder- June 2015 until November 2015, a pilot implementa- ate, high) for eight adverse outcomes, relevant for the tion was carried out on two units to test the new next three months. The START:AV is a so-called fourth workflow and accompanying documents (e.g., treat- generation risk assessment instrument which means ment plan). The coordinator and the implementation that it explicitly links the assessment process to risk for- committee evaluated the pilot and subsequent recom- mulation and risk management (Haque, 2016). In terms mendations were implemented by the coordinator. of psychometric properties, there is evidence for fair to The official, service-wide implementation started in excellent interrater reliability with intraclass correlation February 2016 on all units simultaneously, for new admissions only. Due to this gradual implementation, coefficients (ICC ) ranging from .52 to .88 for the risk it was not until fall 2016 that all treatment coordina- estimates, .86 for the vulnerabilities total score, and .92 for the strengths total score (Viljoen et al., 2012). tors had completed at least one START:AV assess- Significant predictive validity was found for the majority ment in their caseload. The assessments were initially a task of treatment coordinators and included com- of adverse outcomes, with ‘area under the curve’-values pleting the START:AV comprehensive rating form (in ranging from .63 to .83 for vulnerabilities total score, .63 to .80 for strengths total score, and .71 to .91 for MS Word) as a ‘master file’ in which all available the risk estimates. Thus far, the START:AV has been information about the youth was gathered (e.g., file information, interviews, psychological testing, and found to significantly predict all adverse outcomes, observations). The estimated completion time within except unauthorized absences and health neglect this approach ranged from one hour to four hours. (Bhanwer et al., 2016). The Dutch START:AV (De Beuf et al., 2019) was implemented at the current setting and However, starting from March 2017, all frontline staff (i.e., teachers, group care workers, therapists, family research on the psychometric properties is cur- social workers, occupational therapists, drug counse- rently conducted. lors) were instructed to report their evaluation of the past months directly into the START:AV form. The Implementation process implementation coordinator organized information The START:AV implementation project was based on sessions to educate frontline staff on the instrument’s the eight steps presented in Risk Assessment in items. Although the information was now entered via Juvenile Justice: A Guidebook for Implementation by a multidisciplinary approach, treatment coordinators Vincent and colleagues (2012). The implementation remained responsible for rating the items and formu- was led by an implementation coordinator (first lating a final risk judgment. Over the course of 2017, author) who was hired for the project, in collabor- a computerized version of the START:AV was devel- ation with an (internal) implementation committee oped and tested. INTERNATIONAL JOURNAL OF FORENSIC MENTAL HEALTH 301 START:AV observed during team meetings. We con- � FG treatment coordinators (n = 3) � FG group care workers (n = 5) sidered them as those who could best voice the imple- � FG mixed professions (n = 7; 1 treatment coordinator, 3 group mentation challenges experienced by the teams. At the care workers, 2 policy advisors, 1 therapist) March 2016 follow-up time points, we started with personally T1 � FG treatment coordinators (n=7) inviting the participants from earlier sessions. � FG group care workers (n = 3) Individuals in a purely managerial position (i.e., top- � FG mixed professions (n = 6; 1 treatment coordinator, 2 group care workers, 1 policy advisor, 1 social worker, 1 test assistant) Sept 2016 management, senior management, and operational T2 middle management) were excluded from the study. � FG treatment coordinators (n=5) Although treatment coordinators were considered � Interview group care workers unit 1 (n = 3) � Interview group care workers unit 2 (n = 2) middle management, they were invited because they June - � Interview group care workers unit 3 (n = 2) July 2017 supervised the treatment approach for the individual � Interview group care workers unit 4 (n = 2) � Interview group care workers unit 5 (n = 1) T3 adolescent rather than the workforce. Higher manage- � Interview group care workers unit 6 (n = 3) ment was not invited to the focus groups to prevent Note. FG = focus group; T = time point. power differentials during the discussions. Moreover, Figure 1. Focus Group Participants Throughout the because the authors did not focus on the perceptions Three Sessions. of higher management, they were not considered as a Note. FG¼ focus group; T¼ time point. separate user group in the design. Focus group size Focus group design We aimed for five to ten participants per focus group. Although the literature is mixed about the ideal focus The initial design was to organize nine focus group group size, recommendations typically range between discussions; three focus groups at three time points. 5 or 6 to 10 participants for noncommercial research The first time point was at the end of March 2016. (Masadeh, 2012). Small groups of 4 to 6 participants Although this was almost two months into the imple- are recognized to be productive because they encour- mentation, most participants had not yet used the age participants to partake in the discussion, and they instrument due to the gradual process. The second are increasingly popular because they are easier to time point took place in September 2016, and the last organize (Krueger & Casey, 2014; Masadeh, 2012). one was in June 2017. Each time point was scheduled Figure 1 shows that we reached this goal for five focus to have one focus group exclusively with treatment groups. Especially at time point three, it proved diffi- coordinators (TC), one exclusively with group care cult to gather enough participants: only treatment workers (GCW), and one interdisciplinary group coordinators responded. As a result, the study design (Mix). The choice for two homogeneous groups and was adjusted: the third mixed focus group was can- one heterogeneous group was guided by the assump- celed as staff had expressed concerns that they could tion that homogeneous groups might generate more not meaningfully contribute to the group discussions in-depth information about a group’s experience with due to a lack of experience with the START:AV. The the START:AV, while a heterogeneous group might focus group with group care workers was replaced by generate a wider range of information (Schutt, 2004). interviews for which the moderator visited frontline workers on their unit with a slightly adjusted inter- Sampling view guide. The group interviews were held with one All staff members who worked directly with adoles- up to three participants depending on how many cents were invited by a general invitation shared via group care workers were available on the unit (see email, newsletters, and announcements on the serv- Figure 1). In total, seven focus groups and six inter- ice’s intranet. In addition, two policy advisors were invited because of their involvement in the service’s views took place. Participants signed an informed consent at the quality assurance procedures for new and existing beginning of each focus group, after being informed methods, as well as their involvement in the integra- about the content, procedure, audio recording, and tion of the START:AV in the electronic patient file. their rights in relation to the research. Their participa- However, this approach generated a poor response tion was voluntary and during work hours. On average, from group care workers and in the next step individ- ual group care workers were invited via a personalized focus groups lasted 64 minutes (range: 40-79 minutes) email. They were selected based on their active and interviews 12 minutes (range: 4-20 minutes). All involvement in task forces and their interest in the interviews were held in Dutch and audiotaped with a 302 T. L. F. DE BEUF ET AL. voice-recorder. Afterwards, participants received a treatment process. In consultation with their team, summary of findings via email and they had the oppor- they decided on the treatment approach in terms of tunity to respond before the internal report type of therapy and treatment goals, and evaluated the was finalized. treatment progress. They also provided treatment-rele- vant supervision and guidance to group care workers. Interview questions Treatment coordinators, all female, were between 30 During the first focus group interview, previous and 40 years old (M¼ 34) and had on average 10 years implementation efforts within the service were dis- of work experience (range¼ 5-15) within the facility. cussed, as well as expectations about the START:AV Seventy percent had prior experience (i.e., use and/or and potential barriers to the implementation. At the training) with a structured risk assessment instrument, second and third time point, focus group interviews mainly the SAVRY. Group care workers were front- elaborated on the previously mentioned challenges line staff who were responsible for implementing the and whether they had been resolved. In addition, treatment approach for the youths’ safety and well- positive and negative effects of the new risk assess- being on the residential units. They supported adoles- ment method, for example, on communication, were cents with their everyday responsibilities. Of the discussed, as well as workflow issues. An interview participating group care workers, 43% were female; guide, modeled after Crocker and colleagues (2008), they were between 25 and 51 years old (M¼ 35), with was used to guide the discussions and is available at 1 to 17 years of experience within the organization the project page on Open Science Framework (OSF; (M¼ 8.5). Among the other participants were profes- https://osf.io/5gztr/). The open-ended questions pro- sionals such as family social workers, psychotherapists vided a framework, and specific questions and and testing assistants, as well as policy advisors. Of prompts by the moderator followed from the discus- these professionals, 80% were female and their average sion. The interview approach was informed by resour- age was 44 years old (range ¼ 36-56), with 7 to ces such as Krueger (2002), Krueger and Casey 29 years of service (M¼ 14). (2014), and Patton and Cochran (2002). Ethical considerations Role of moderator In qualitative research, the investigator is considered The general director of the service gave permission to conduct the research within the facility. The Ethics an integral part of the research process and the final product (Galdas, 2017). Therefore, researcher influ- Review Committee Psychology and Neuroscience ence should be disclosed and reflected upon. In the (ERCPN) of Maastricht University approved the present study, due to resource constraints, the moder- research protocol, procedures, and staff consent forms ator of the focus groups was also the implementation (ERCPN number 174_06_12_2016). All data were coordinator as well as the data analyst. The investiga- analyzed anonymously and stored according to the tor-moderator was not part of the treatment staff and university’s Data Management Code of Conduct and she was hired by the organization to implement and the institution’s Data Protection Guidelines. evaluate the START:AV as a relative ‘outsider’ with a university affiliation. The moderator received one-on- Data analysis one coaching by an experienced focus group facilitator The focus group discussions were transcribed using and discussed the focus group sessions with free transcription software (NCH Software, 2016). The the coauthors. moderator was selected as transcriber because she was familiar with the context of the discussions, recog- Participants nized the recorded voices, and was able to differenti- Across all time points, 36 unique staff members par- ate between relevant and irrelevant (e.g., ticipated: 10 treatment coordinators, 21 group care interruptions) audio material. Furthermore, transcrib- works and 5 other professionals (see Figure 1). This ing increased familiarity with the data, which was reflects 100% of the treatment coordinators, 20% of beneficial for subsequent thematic analysis. The tran- group care workers, and 63% of other disciplines that scription followed an edited approach; filler words, were invited. Treatment coordinators, professionals interruptions, self-corrected words and non-relevant with at least a master’s degree in psychology or special content (e.g., small talk at the beginning of a session) needs education, were responsible for the adolescents’ were omitted while maintaining integrity of the INTERNATIONAL JOURNAL OF FORENSIC MENTAL HEALTH 303 recordings. All transcripts were anonymized: names end, all three raters reached consensus on the allo- were replaced by numbers. All transcriptions were cated codes. Moreover, as a result from the discussion, double-checked against the audio recordings. one new code was added as determinant and other codes were renamed for clarification. Thematic analysis Transcriptions from the focus groups and interviews Consolidated framework for implementation were analyzed using thematic analysis (Vaismoradi research After the inductive approach to data gathering and data et al., 2013). This is a widely used qualitative method analysis, we applied a deductive approach to the classifi- for identifying, analyzing, and reporting patterns cation and interpretation of the ‘Implementation within transcribed data (Braun & Clarke, 2006). It Determinants’ codes. The CFIR codebook (Consolidated relies on a step-by-step process of coding and re-cod- Framework for Implementation Research, n.d.) was con- ing, and codes are collated into themes. The present sulted to connect our codes to the CFIR constructs, as it study followed an inductive (data-driven) approach, provides definitions and inclusion/exclusion criteria for which means that codes were identified independently the majority of the constructs. of a theoretical framework. Applying a manual coding procedure, the first author started the thematic ana- lysis by rereading the full transcript and entering the Results text in Excel with one sentence, or data extract, per Descriptive information on the distribution of the 211 row. Each data extract received one or more codes codes accumulated by the thematic analysis can be that summarized its content in one or two words. accessed via the OSF project page (https://osf.io/5gztr/). Examples of codes included ‘provide training’ or We focused on the codes related to the implementation ‘resistance to change’. After working through all data determinants and their link with the CFIR domains and extracts, initial codes with their definitions were listed constructs (see Table 1). All five domains and 21 of the in one file and examined for similarity. Similar codes 39 constructs were addressed. In the following, we pre- were collated into a more manageable number of sent the identified constructs per domain, illustrated codes. Collated codes or newly identified codes were with quotes. For each construct, it is also specified compared with the original statements of the partici- whether it was experienced as a facilitator, a barrier, pants to ensure they matched. Internal homogeneity or both. of a code was examined by collating all data extracts labeled with this particular code and assessing coher- ence, while external heterogeneity was ensured by Intervention characteristics identifying clear distinctions between codes. This pro- Features of the START:AV were mostly discussed by cess was repeated for each focus group. The codes of the treatment coordinators. Contrary to other staff, the interviews at T3 were combined into one group. they had considerable hands-on experience with the In the next step, all codes were sorted into poten- instrument and insight into its procedures, especially tial themes (Braun & Clarke, 2006). Five overarching at the second and third time point. themes were identified: ‘Implementation Determinants’, ‘Implementation Strategies’, Complexity (facilitator and barrier) ‘Implementation Outcomes’, ‘Feedback on Practice’, The majority of group care workers found it easy to and ‘Suggestions for Practice’. The first three themes provide information on the items because the item applied to the implementation process, whereas the indicators on the form were felt to be self-evident. latter two concerned the practicalities of the Although several group care workers reported difficul- START:AV workflow. Because of the present article’s ties separating strengths from vulnerabilities, which focus on determinants, only codes from the was corroborated by treatment coordinators, they sup- ‘Implementation Determinants’ theme were retained ported the inclusion of strengths. for further examination. For these codes, an inter- You have to be very aware of what is considered coder check was performed. The second and third strengths and what is considered vulnerabilities, and author independently coded 27% (i.e., 113 of 417) of without realizing you are documenting vulnerabilities the extracts related to ‘Implementation Determinants’ among the strengths: “Oh no, that doesn’t belong here” and vice versa. I find the strengths very using a codebook compiled by the first author (see valuable because otherwise they wouldn’t be noted https://osf.io/5gztr/). They compared coding sheets very often. (T3 GCW) and discrepancies were resolved by discussion. In the 304 T. L. F. DE BEUF ET AL. Table 1 Overview of the Implementation Determinants Codes Categorized according to the CFIR Implementation Determinant CFIR Domains and Constructs Barrier Facilitator Intervention Characteristics Relative Advantage Less detailed information (T3 TC) Value: more straightforward (T2 TC) Lost narrative (T3 TC) Value: multiple adverse outcomes (T2 TC) Limited innovation (T3 TC) More complete risk assessment (T2 Mix) Missing information (T2 TC) Adaptability Rating Differentiation (T1 TC) Compatibility (T2 TC) Complexity Strength vs Vulnerability (T3 TC) Understandable (T3 GCW) Similarity items (T3 GCW) Strength vs Vulnerability (T3 GCW) Design Quality & Packaging Software support (T3 TC) Cost Cost-benefit ratio (T1 GCW) Outer Setting External Policy & Incentives – Bigger picture (T3 TC) Inner Setting Structural Characteristics Layers in organization (T1 TC) – Layers in organization (T1 GCW) Organization islands (T1 GCW) Networks & Communications Communication (T1 GCW) – Communication (T1 Mix) Culture Fear of negative effects (T1 TC) – Organizational history (T1 GCW) Organizational Inflexibility (T1 TC) Overall Resistance (T1 TC) Overall Resistance (T1 GCW) Top-down decisions (T1 GCW) Compatibility – Setting’s aim (T3 TC) Relative Priority No priority (T1 TC) – Learning Climate – Staff involvement (T1 GCW) Leadership Engagement Management Involvement (T1 TC) – Organizational Inconsistency (T1 Mix) Available Resources Increased workload (T1 TC) – Investment (T1 GCW) Lack of time (T2 GCW) Access to Knowledge & Information Availability START (T2 GCW) Inform Staff (T2 Mix) New Staff (T3 GCW) Characteristics of Individuals Knowledge & Beliefs about the Intervention Confidentiality Concern (T2 Mix) Experienced Advantage (T3 TC) Improved Explanation (T2 TC) Improved Explanation (T3 TC) Increased focus on Facts (T2 Mix) Increased focus on Strengths (T2 Mix) Increased focus on Strengths (T2 TC) Population match (T3 TC) Present Observations (T3 GCW) Provides Overview (T2 TC) Staff Buy-in (T2 TC) Individual Stages of Change Individual response to change (T1 Mix) Enthusiasm (T1 TC) START:AV Resistance (T2 TC) START:AV Resistance (T2 Mix) Other Personal Attributes Experienced (role) Conflict (T2 Mix) – Process Planning Endless Preparation (T1 TC) – External Change Agents – External Expertise (T1 TC) Executing Timing Information-sharing (T1 TC) – Note.T¼ time point; TC¼ treatment coordinators; GCW¼ group care workers; Mix¼ mixed group of staff members Relative advantage (facilitator and barrier) documentation of the observed risks and the wide When comparing the START:AV with the self-con- scope of potential adverse outcomes. structed risk checklist, treatment coordinators I notice that I reflect much more on how often the described benefits, such as more explicit child has had such an aggressive outburst or hasn’t. INTERNATIONAL JOURNAL OF FORENSIC MENTAL HEALTH 305 While in the past, you just might have said “this is a moderate or high?” (T2 TC). Treatment coordinators very aggressive boy”, this time you are more looking also commented on the utility of the START:AV for for uh … That you make the connection: “Because of less restricted settings, such as community-based serv- this lack in skills, the risk is high”. I think I … It ices, arguing that it could be useful to detect unsafe provides more guidance to do this in a good way. (T2 TC) conditions. Yet, they hypothesized that in such set- tings only a small group of juveniles would benefit The focus group interview with mixed staff mem- from the risk assessment and therefore wondered bers added that the START:AV provides a more com- whether it would be worth the (implementa- prehensive account compared to the setting’s self- tion) effort. constructed checklist. However, when comparing the START:AV with the dimension list, treatment coordi- Design (barrier) nators felt that the START:AV did not add much in The computerized version of the START:AV compre- terms of information gathering and structuring. They hensive form, piloted prior to the last focus group, felt that the items were not novel compared to the was mentioned as a barrier. According to the treat- themes that were included in the dimension list: “I ment coordinators, the digital form “made it even also think that it’s a good summary of current think- ing in terms of the socio-emotional domain, the cog- more difficult” for group care workers to report the nitive domain, social network, … but we had that required information, because it did not include the already, it’s not like the START:AV added value to items’ anchors (i.e., item descriptors) to rely on. this” (T3). Neither did they experience benefits from the structured approach, “… because we were already Cost (barrier) working fairly structured prior to this. It is not like it Group care workers expressed that, in the past, their was completely blank before and that we did not have work on the unit had not benefited from completing any–how do you call it–structure or so”. (T3 TC). questionnaires and forms. This made some staff Overall, they indicated that the START:AV did not apprehensive about the present implementation: produce new insights or different treatment goals. Completing many lists, completing many forms, to One treatment coordinator seemed to carry the ultimately make one report. I think that half of those expectation that the START:AV would lead to consid- forms are not even being used to reach a proper erably different conclusions: “In the end, you actually conclusion. But because it’s obligated, we have to want it to result in making completely different complete them. That brings along a lot of work that choices and if that’s not the case, then you just I’d rather spend on the ward instead of in the office. (T1 GCW) think…” (T2 TC). Others countered this argument by indicating that this, in fact, validated their prior deci- sion making. Furthermore, on the observation units where teams had been using the dimension list (see Outer setting Setting), treatment coordinators regretted the change The outer setting refers to the larger political, eco- in information they now received from frontline staff nomic and social context in which the organization is on the START:AV items. They argued that although embedded. It includes those who are served by the they obtained information on more domains, this setting and the external network of the organization. information was less detailed than before. Likewise, This domain was mentioned least often. treatment coordinators commented on the loss of nar- rative due to reporting per item. Other treatment External policy and incentives (facilitator) coordinators added that they were missing informa- Treatment coordinators indicated that the introduc- tion on domains that were not included in the tion of the START:AV within the service corre- START:AV, such as developmentally-appropriate sponded with national reforms in youth care policy. knowledge of sexuality. Legal mandates for residential treatment were reduced in length, and evidence such as risk assessment infor- Adaptability (barrier) mation was increasingly required when imposing or Although treatment coordinators understood the prolonging mandated treatment. According to treat- importance of adhering to the item ratings, there ment coordinators, it “certainly goes nicely hand-in- seemed to be some negligence when allocating scores. A treatment coordinator asked rhetorically: “Do I hand because with that [risk assessment] you can bet- really have to worry five minutes about whether it is ter indicate ‘I really find that this individual needs 306 T. L. F. DE BEUF ET AL. more time in mandated residential youth care’,or ‘I This reluctance impacted treatment coordinators, really find that we should continue because…’.” (T3). because the service had not yet followed through on their promise to relieve them from other tasks. A treatment coordinator noticed: “In this case [of the Inner setting START:AV], everyone thinks that we should do it, but there is still, for example, nothing else removed Contrary to the outer setting, characteristics of the from our workload” (T1 TC). inner setting (i.e., the service itself) were often noted Not only were those in charge thought to resist as factors that influenced the implementation. It change, a group care worker voiced that it was quite includes both tangible aspects, such as size and struc- common for frontline staff to respond reluctantly to ture of the service, as well as immaterial features such new initiatives: as work culture and climate. Aspects of the inner set- ting were primarily discussed at the first time point. For some, it is more difficult than for others. And, it Statements on the inner setting in relation to previous is precisely those who experience difficulty, who deserve attention, and it is precisely those who need implementation efforts were also coded as they were to be involved in the process and perhaps those who thought to be relevant to the START:AV you need to talk to more frequently. (T1 GCW) implementation. Furthermore, group care workers felt new initia- tives were often imposed upon them, in a top-down Compatibility (facilitator) fashion, without sufficiently accounting for the team’s Treatment coordinators agreed that the START:AV possibilities. This generated frustration among front- was compatible with the objective of mandated resi- line staff: “When it comes out of the blue and some- dential youth care to reduce risks without trying to one says ‘You have to do that’, with no room for resolve all developmental challenges of the admitted discussion, that causes irritability”. (T1 GCW) youth: “Because you actually have to reduce the risks and should not want to change issues in all develop- Learning climate (facilitator) mental areas” (T3). However, one treatment coordin- Nevertheless, group care workers were positive about ator shared concerns that focusing on the risks and past implementation efforts in which they had been needs would not sufficiently address the actively involved. For example, one group care worker youths’ problems. enthusiastically described active involvement in the implementation of a group-oriented intervention: Culture (barrier) “What I really liked about implementing [interven- Staff members shared that, 10 years ago, the facility tion], what I found positive about it, is that we were was a juvenile detention center. They believed that allowed to contribute: when are we going to provide this background was still affecting today’s practice, the training? How are we going to do it?” (T1 GCW). which was perceived as strict and inflexible by some. With regard to the START:AV implementation, par- A group care worker used the metaphor of a “slowly ticipants did not express that they felt involved. turning heavy tanker ship” (T1) to describe the organ- ization. Overall, staff members believed that the ser- Structural characteristics (barrier) vice had difficulty adapting to change and preferred The facility’s hierarchical management structure was maintaining routine practice. Participants recognized reported as a barrier due to its many administrative both an aspiration to innovate and a rigidity to pre- levels. Staff explained that a decision had to pass mul- serve the status quo among top-management. tiple levels before it could lead to actual change. Treatment coordinators speculated that this rigidity This is also part of our culture. Here, everyone wants was fueled by worries about losing a good standing to have their say about it and everyone is allowed to reputation within the field. have their opinion. At a certain point, because of this, we just stagnate: “Okay, when will we finally get It’s ambiguous because we see that many authorities, started? When will we act? It still has to pass this such as youth care offices, local authorities, refer committee and then that one has to give their [youths] to us because we deliver very good service, opinion”. At times, this works against us because it in their opinion. So it is somewhat ambiguous takes three quarters of a year before anything finally because on the other hand, this reluctance – sticking seeps through in the organization. (T1 GCW) to what goes well– also makes us steady, well- functioning and therefore delivering a good job. To A material barrier to the implementation was the me, that is … uh well, difficult. (T1 TC) physical scattering of units across the grounds, INTERNATIONAL JOURNAL OF FORENSIC MENTAL HEALTH 307 The challenging part is that our direct supervisor has weakening the sense of unity within the service. Staff insufficient insight in how much time it takes. What typically referred to the treatment units as ‘islets’. he does, is personalize the problem: those who complain more, are considered to have a difficult Networks & communications (facilitator and barrier) personality, and those who don’t complain, are just The barriers were found to affect communication. not that busy. This is his strategy to ignore the Group care workers did not always feel sufficiently problem and that is frustrating. (T1 TC) informed on past implementations and other staff members also shared experiences of not being Available resources (barrier) included in the information exchange. Concerning the Concerning the facility’s readiness to implement the START:AV implementation, a therapist explained that START:AV, all disciplines addressed the lack of the communication was clear, but that the expecta- resources. The START:AV placed a burden on the tions for her particular position were unknown. treatment coordinators’ workload and this was, at Overall, participants found that the introduction of least for some, a major barrier to adopt the instru- the START:AV was well-announced. A group care ment. They realized that becoming adept in using the worker commented: START:AV required a substantial (time) investment: I do think, the communication … of course you “We keep repeating this: time is really the biggest [implementation coordinator] have been working on impeding factor in this.” (T1 TC). Similarly, group it for a while now and you have represented yourself pretty well. Everyone, in my opinion, knows care workers found it difficult to carve out time to something is going to change. ( … ) That makes a familiarize themselves with the completed forms they difference: the new system does not appear out of the had to complete for their caseload. One group care blue. It is already [introduced], step-by-step, in the worker linked this to the occupancy on the unit: newsletter or in an email. Somewhere attention is paid to it, everyone is being reached. (T1 Mix) Usually there is not much time. That’s also complicated, it depends on how many adolescents you have on your unit. We already have a lot of lists Relative priority (barrier) to complete and there is already a lot that we must Treatment coordinators felt that, at the beginning of do. At the moment, we have six youth on the ward and then you have some extra time to do other the implementation, the START:AV was not priori- things. When the unit is full with 11 adolescents, tized enough. On the question “What if you could you’re lucky if you have managed to write the daily change one thing?” a treatment coordin- progress notes by 10: 30 p.m. (T2 GCW) ator responded: ( … ) that our supervisor takes more responsibility as in “this has priority now and that means that we Access to knowledge & information (facilitator don’t do this or this or that for a while”, including and barrier) some task forces or what not … That all of us agree With respect to accessing information, participants as an organization: this has our priority now. (T1 TC) appreciated the visits of the implementation coordin- ator to inform the teams about the START:AV: “Ido Leadership engagement (barrier) think that what you did —discussing it in the team Frontline staff warned against the multitude of superi- meetings—is very useful to make it more concrete.” ors to whom they were accountable. In the past, this (T2 Mix). However, group care workers experienced had led to inconsistencies in approach and deviations difficulties accessing the forms as they were not stored from agreed strategies: “Exactly! One of the managers in a location available to them. Furthermore, new thinks it is really important, while my operational group care workers did not receive a formal introduc- manager suddenly starts questioning the implementa- tion to the START:AV; they learned it on the job, tion, and you are in the middle of it” (T1 Mix). during case conferences and by asking colleagues. A Receiving conflicting messages was perceived as frus- group care worker considered this part of one’s own trating and confusing. Treatment coordinators responsibility: “I mean, children are assigned to a expected their leadership to take on a supportive atti- mentor based on availability at the time, and when tude toward the increasing workload and be engaged you become mentor of a child in an observation tra- in the implementation, for example, by completing jectory, you have to think for yourself what you need START:AV assessments themselves. to do.” (T3 GCW) 308 T. L. F. DE BEUF ET AL. User characteristics Other personal attributes (barrier) Knowledge & beliefs about the intervention (facilita- One participant who works closely with the adoles- tor and barrier) cents’ relatives, expressed concerns about reporting Treatment coordinators were convinced about the sig- incriminating information, without an extended narra- nificance of the START:AV and the usefulness of the tive, in the START:AV. This staff member seemed to adverse outcomes for the setting’s clients. Moreover, experience a conflict between providing information treatment coordinators argued that with the on the START:AV items and his role as counselor: START:AV they were better equipped to substantiate You are working with a human process and their decisions about the treatment approach and the devastating situations, and then you can’t simply state recommended level of supervision. In addition, they in the report “Look, this is what this family has shared”. Then you fall short on these people and that believed in the value of focusing on strengths. This is not okay. (T2 Mix) was illustrated by a treatment coordinator: “I feel that it is fair to the youths to also consider strengths. I think that in our field there is a strong tendency to Process focus on the negative things. Therefore, I feel this really is of added value.” (T2). Frontline staff agreed: External change agent (facilitator) “This way, you are forced to pay attention to the posi- Treatment coordinators considered the service’s choice tive things.” (T2). Another advantage of the to hire an implementation coordinator as a facilitator. START:AV, according to staff members, is its focus In past implementations, they had noticed better on facts and the unambiguous presentation of risks facilitation of external project coordinators by man- and concerns. A treatment coordinator in the mixed agement and staff members had also been more recep- group explained: tive to external agents. They said: “I think it makes a big difference when an external person is brought on What I find very positive, is that you substantiate the risks for an adolescent much more. Often, as a team, board, as now happens with the START:AV, and this you agree that it is a very worrisome case, but if you person takes charge” (T1 TC). put it [the risks] together like that, then “Yes indeed, this is very worrisome”. And you can write it down Planning (barrier) much more clearly and communicate to the child Treatment coordinators regretted the long planning guardian agency or the parents that there are major phase with extensive preparation, which they referred concerns in these areas. It helps me to describe things more objectively. (T2) to as “an endless start-up phase” (T1). According to them, the implementation committee should have Still, one participant expressed concerns about con- started sooner with the full implementation. fidentiality because it was not clear to this staff mem- ber that the START:AV was treated as an internal Executing (barrier) document and would not be shared with third parties. They were especially dissatisfied about the substantial time that passed between the training and the actual Individual stages of change (facilitator and barrier) use of the instrument. One treatment coordinator On the one hand, treatment coordinators reported explained how this impacted her: enthusiasm and willingness among their team mem- bers to use the instrument, on the other hand, much It’s just really too bad that if you look back: when did we complete the [START:AV] practice cases? That resistance was noted. Reported reasons for resistance was over a year ago! Look, if … for me that to the implementation were lack of time and not [knowledge] really is already disappearing. That’s just being able to complete a START:AV assessment with- a waste of energy. (T1 TC) out being interrupted. Furthermore, staff members’ For this reason, treatment coordinators supported response to the new practice appeared to depend on the decision to inform group care workers later in the their openness to change. Some readily expressed process, otherwise “they might become frus- enthusiasm, whereas others were more reluctant. A trated” (T1). group care worker commented: One person might immediately start thinking about it, getting excited, while another, when he does not Discussion see the benefits or when it is not yet clear, This qualitative study explored staff members’ views immediately thinks: “Ah, again another list”. I find it varies a lot, at least if you look at our team. (T1 Mix) on factors that affected the implementation of the INTERNATIONAL JOURNAL OF FORENSIC MENTAL HEALTH 309 START:AV risk assessment tool in a residential youth form, while simultaneously working on building and care service. Using the consolidated framework for sustaining a therapeutic alliance with the youths’ fami- implementation research, we organized a set of imple- lies. This concern is in line with staff’s expected loss mentation determinants, which we derived from focus of discretion described in three of the studies reviewed group interviews using a data driven approach. On by Levin et al. (2016). For example, in an implementa- multiple domains, staff members identified factors tion study by Vincent et al. (2012), 21% of juvenile that they perceived as impeding or facilitating imple- probation officers “feared that their years of experi- mentation of the instrument. Aspects of the inner set- ence would be discounted in favor of a score from a ting were mentioned most frequently, followed by tool” (p. 573). However, this anticipation proved user characteristics and features of the risk assessment unwarranted as only four officers (4.7%) reported feel- instrument itself. The implementation process itself ing invalidated by the instrument at 10 months into and the outer setting were rarely mentioned. the implementation. The cultural and structural features of the service With respect to the instrument’s characteristics, were widely discussed as barriers to the implementa- there are several parallels with the findings of Sher tion. Structural barriers included the physical environ- and Gralton (2014). For instance, group care workers ment (e.g., physical scattering of units) and the in our study also found the items to be straightfor- ‘corporate’ environment (e.g., hierarchical levels). ward and easy to complete; yet, some struggled with According to participants, these structural barriers differentiating strengths from vulnerabilities, similar hindered communication, which is an essential com- to the UK study. Most group care workers preferred a ponent of introducing change (Damschroder et al., narrative approach in which they connect, and poten- 2009). Moreover, communication is key in earning tially counterbalance, vulnerabilities and strengths in staff buy-in, creating enthusiasm, and encouraging one paragraph. Strictly separating strengths from vul- staff involvement. Multiple aspects of the implementa- nerabilities required additional effort. In addition, tion relied on communication, such as sharing the group care workers reported that differences between rationale for introducing structured risk assessment, some of the items were quite subtle, making it more providing practical information about the START:AV difficult to allocate information to the appropriate workflow, and updating staff on the implementation item without duplicating, a concern also mentioned progress. In addition, resources were a frequently by Sher and Gralton (2014). It is unclear from the UK mentioned barrier: the START:AV increased staff study, whether staff was already familiar with struc- members’ workload while operating within the same tured information gathering. In the current setting, (time) conditions. Similarly, staff in Sher and Gralton treatment coordinators experienced this as a consider- (2014) study reported a lack of time to effectively able barrier: they seemed disappointed by the limited complete the START:AV. The present approach to novelty of the START:AV in terms of the included treat the START:AV as a master file, and the time items and the provided structure, compared to what investment that came with it, led to resistance among they were already used to. Moreover, replacing the staff members. Overall, resistance was a common dimension list with the START:AV was accompanied theme, from the board of directors to frontline staff, by a sense of loss (e.g., less detail, missing themes, each for their own reasons. This is not surprising, loss of narrative). The participants rarely commented on the planning because implementing change is “fighting against one’s inner desire to maintain the status quo” and the execution phase of the implementation, per- (Tran, 2019). haps because of the timing of the focus groups. The Nevertheless, despite the experienced strain in planning phase had ended and the official implemen- terms of workload, staff members were positive about tation had begun. Nevertheless, staff stated that prep- the value of the START:AV and its usefulness for aration, training, and actual implementation had not their practice, in line with Sher and Gralton’s findings succeeded each other within a reasonable time frame. (2014). In both settings, the focus on strengths was The lengthy intervals between these steps were per- highly valued. In addition, in the present study, the ceived as impeding the implementation. multiple adverse outcomes were particularly appreci- Similar to findings reported by Levin et al. (2016), ated and perceived as relevant for the setting, even and Sher and Gralton (2014), staff members did not more so than the individual items. Yet, one staff consider factors from the outer setting as affecting the member expressed experiencing difficulties with implementation. This was somewhat surprising, reporting sensitive information in the START:AV because the implementation took place during 310 T. L. F. DE BEUF ET AL. turbulent times for youth care organizations, with However, triangulation was applied during data ana- many legal and budgetary changes occurring simultan- lysis: a subsample of the extracts was coded by raters eously. The restraints that were imposed on the ser- who were external to the service, reaching satisfactory vice from the outside (e.g., reduced financial agreement. This procedure improves standardization and accuracy in the coding process and helps control resources) may have hindered the implementation. On the other hand, growing political pressure to use evi- for bias (Boeije, 2010). However, researcher triangula- dence-based practices, such as structured risk assess- tion was not repeated during the deductive phase ment, could also have facilitated the implementation. when the determinant codes were linked to the CFIR constructs. Not having multiple coders leaves the pro- Yet, staff did not allude to this. We contemplate that staff members have a tendency to focus primarily on cess to one researcher’s judgment. For example, the internal organizational factors. This might be espe- code ‘fear of negative effects’ could be considered a cially true for (secure) residential settings that are characteristic of the organizational culture (i.e., ‘Culture’) or a characteristic of the leading CEO (i.e., more closed off from society than community-based ‘Individual Stage of Change’). Nevertheless, both the services. Furthermore, the needs of the assessed ado- lescents themselves, and barriers and facilitators for codes and the constructs were defined in their them to participate in the risk assessment were not respective codebooks prior to the deductive phase, discussed. Such absence was also noted by Levin et al. reducing the opportunity for interpretation. Second, the sampling method might have influ- (2016). In risk assessment practice, patients’ views are enced who participated in the focus groups. It is pos- typically not included and assessments tend to be con- ducted top-down by professionals (Langan, 2010). sible that because of the voluntary nature, staff Similarly, in the present setting, adolescents were not members with strong resistance toward the implemen- tation did not sign up for participation. The opposite involved in the risk assessments. This approach likely could also be true; perhaps dissatisfied staff members limited staff members’ consideration of the adolescent took the focus group discussions as an opportunity to as a stakeholder in the implementation. The majority of determinants reported in the pre- voice their negative opinions. Moreover, participants sent study are recognized in the existing literature as knew prior to accepting the invitation that the imple- mentation coordinator would be moderating the ses- important conditions for successful implementation. sions. It is plausible that individuals who felt In accordance with prior work on risk assessment uncomfortable talking to the implementation coordin- implementation (Levin et al., 2016;; Muller-Isberner € et al., 2017; Nonstad & Webster, 2011; Schlager, 2009; ator refrained from participating in the study. As a result, the breadth of experiences shared in the focus Sher & Gralton, 2014; Webster et al., 2006), our find- groups could have been affected, with less diversity in ings highlight the importance of involvement and reported barriers and facilitators. Nevertheless, this commitment on all levels, dedicated leadership, inclu- issue might have been partially resolved by randomly sive and transparent communication, adequate resource allocation, training, timing, monitoring, and approaching the teams for interviews at the third time point. We used this recruitment strategy as an alterna- integration in existing structures. tive to the focus group discussions that could not be organized at that time. Although interviews with a Limitations and future research maximum of three group care workers likely produced A first limitation is the involvement of the implemen- less discussion and reduced the range of experiences tation coordinator as moderator and data analyst. On that were reported (Krueger & Casey, 2014), on the the one hand, familiarity with the service organization other side, this may have resulted in reaching group helped the moderator to better understand partici- care workers who would otherwise not have attended pants’ comments and to know when to probe for fur- a focus group discussion. ther information. On the other hand, as an ‘insider’, Another way the focus group procedure may have the moderator may have relied on implicit informa- impacted the findings is that familiarity with the mod- tion about the organization that was not checked for erator could have increased the likelihood that some accuracy (Chenail, 2011). Including more moderators participants made statements to please the interviewer into the design, especially moderators without famil- (Chenail, 2011). However, because the topic was not iarity with the service, might have compensated for particularly sensitive or personal and participation did the potential bias stemming from having the imple- not bring personal benefits, there was no obvious mentation coordinator as the single moderator. motive for socially desirable responding. Moreover, INTERNATIONAL JOURNAL OF FORENSIC MENTAL HEALTH 311 the participants in the present study were rather out- approach could be followed for the implementation of spoken about the work environment. Considering the risk management in clinical practice. For example, large number of experienced barriers to implementa- studies have shown that programs that adhere to the tion gathered over the interview sessions, it is fair to Risk-Need-Responsivity (RNR) principles are more assume that the majority of participants felt comfort- effective in reducing recidivism than programs who able voicing criticism during the discussions. do not follow these principles (Koehler et al., 2013). A third limitation is that higher and middle man- Yet, a recent systematic review (Viljoen et al., 2018) agement were not included in the study. This limits found that professionals only showed moderate adher- our understanding of the beliefs and attitudes of this ence to the risk principle and limited adherence to the group about the START:AV and what they perceive need principle when making risk management deci- as implementation determinants. Higher management sions. Thus, it might be equally relevant to extend is typically more involved with external stakeholders, implementation research to risk management strat- and from that perspective, they might have reported egies and deepen our understanding of the barriers more external influences. Thus, not having this per- and facilitators that professionals face in adhering to spective embedded in the findings is a limitation. the RNR principles. Nevertheless, the role of leadership and their potential influence on the START:AV implementation was dis- Practical implications cussed by other staff members. Lastly, the reader should take into consideration that When planning an implementation, every coordinat- the present study reflects perceptions of staff members ing committee could benefit from studying the CFIR. That way, potential barriers can be identified early in from a particular service with a particular client popula- tion, during a particular (political) time. Nevertheless, the implementation process and strategies can be parallels with previous studies (Levin et al., 2016; adopted to increase the odds of a successful imple- Muller-Isberner € et al., 2017; Nonstad & Webster, 2011; mentation. In addition, determinants and strategies Schlager, 2009; Sher & Gralton, 2014; Webster et al., should be reconsidered throughout the process as 2006) suggest that our results might be transferable to their relevance will change depending on the stage of other contexts, at least to residential treatment settings. the implementation (Damschroder et al., 2009). In To enhance applicability of risk assessment implemen- Figure 2, we listed recommendations based on sugges- tation studies, we advocate for the use of implementa- tions from staff and the experiences of the implemen- tion frameworks, such as the CFIR to allow comparison tation coordinator. This list can be complemented between settings and instruments. Moreover, future with strategies from the ‘CFIR-ERIC implementation research could work on identifying the most essential strategy matching tool’, which is freely available online determinants to implementation success and how they (https://cfirguide.org). This matching tool assists in can be facilitated. Ideally, this research would involve allocating strategies from the Expert multiple sites, to allow comparisons and identification Recommendations for Implementing Change (ERIC; of common determinants. Powell et al., 2015) compilation to the determinants Although we purposefully decided on using an of interest. inductive approach to data gathering and data coding, all codes could subsequently be linked to the CFIR. Conclusion Therefore, this comprehensive framework might pro- In the past decade, there has been growing attention vide an adequate starting point for future risk assess- to the study of implementation of risk assessment ment implementation studies, for example, when developing interview questions that explicitly prompt instruments in forensic-clinical practice. The system- for certain domains and constructs, such as the outer atic review of Viljoen et al. (2018) suggested that the use of structured risk assessment instruments does not setting or implementation climate (Kirk et al., 2015). In addition, we would like to encourage future studies yet reliably result in violence reduction. One potential to investigate relationships between determinants and explanation for this finding are the challenges faced between determinants and implementation outcomes. when implementing risk assessment with fidelity into Better understanding of what impedes and facilitates practice. To move risk assessment practice to a higher successful implementation (e.g., integration, adoption, level, it would require mental health services to adopt satisfaction) paves the way for (more) effective risk a systematic, evidence-based approach to its imple- assessment implementation strategies. A similar mentation (Haque, 2016). Increased understanding of 312 T. L. F. DE BEUF ET AL. Preparation of the Implementation - Implementation is more than providing training. Develop a stepwise implementation plan. Get inspired by the literature, for example, Vincent, Guy & Grisso (2012). - Be prepared to make a large investment at the beginning. Invest in planning, prioritizing, and communication. Provide time (i.e., money). - Identify all direct and indirect stakeholders and their influence on the implementation. Get them involved in the process, make them feel heard, make them responsible. - Consider actively involving the service user in the implementation. - Be aware of the organization’s historical and contemporary cultural background. - Have a strong formal mandate from management, as well as their informal support. - Appoint a formal coordinator whose duties continue after the implementation until sufficient integration and sustainability (as defined in the plan) is reached. - Connect with a risk assessment expert/academic to be involved as an advisor. - Be attentive to and eliminate overlapping tasks. During Implementation - Prioritize the implementation for a dedicated period. - Be open to adjust peripheral features of the tool. Be firm about its core characteristics. - Find the right balance between planning and execution: timing is crucial. - Assess and manage expectations of staff concerning the impact of risk assessment. - Accommodate users by investing in user-friendly integrative software. - Keep reiterating the relevance of risk assessment, for example by repeatedly mentioning its value in (political, clinical, managerial) conversations at all levels. - Communicate with all stakeholders at all levels at all times. - Identify influencers among users. Give a positive influencer informal leadership, give a negative influencer individual time and attention. - Look for ways to visualize the risk assessment findings to make it more tangible. - Use the change curve to keep track of where the service, a team and/or an individual staff member is at in the change process. Be prepared to loose staff in the process. - Monitor the implementation. Monitor fidelity to the instrument’s instructions. - Persevere. Give staff time to experience the new practice. - When evaluations clearly indicate that the instrument is a misfit with the setting, be receptive to these signals and be prepared to de-implement. Include thresholds for de- implementation in the implementation plan. Figure 2. Suggestions for Implementing Risk Assessment Instruments. the implementation process and the conditions that the first, probably crucial steps in reducing adverse create a foundation for successful implementation outcomes among adolescents. (e.g., adherence to the risk assessment guidelines) is necessary to optimize risk assessment. The present ORCID study provided insight into the impeding and promot- Tamara L. F. De Beuf http://orcid.org/0000-0001- ing factors of an implementation as perceived by the 5273-8523 users of a risk assessment instrument, in this case the Corine de Ruiter http://orcid.org/0000-0002-0135-9790 START:AV. The CFIR was useful in organizing and Vivienne de Vogel http://orcid.org/0000-0001-7671-1675 synthesizing complex and multi-leveled information gathered via interviews with professional users of the References instrument. The more knowledge about risk assess- ment implementation accumulates, the better the field Bhanwer, A., Shaffer, C., & Viljoen, J. L. (2016). Short-term will understand which approach works for which Assessment of Risk and Treatability: Annotated instruments in which settings. High-quality imple- Bibliography. British Columbia University, https://doi. mentation of risk assessment instruments is one of org/10.13140/RG.2.2.36232.67845 INTERNATIONAL JOURNAL OF FORENSIC MENTAL HEALTH 313 Boeije, H. (2010). Analysis in qualitative research. Sage Research. Implementation Science, 11(1), 72–84. https:// Publications. doi.org/10.1186/s13012-016-0437-z Bonta, J., & Andrews, D. A. (2017). The psychology of crim- Koehler, J. A., L€osel, F., Akoensi, T. D., & Humphreys, D. K. (2013). A systematic review and meta-analysis on inal conduct (6th ed.). Routledge. the effects of young offender treatment programs in Borum, R., Bartel, P., & Forth, A. (2006). Structured Europe. Journal of Experimental Criminology, 9(1), 19–43. Assessment of Violence Risk in Youth. (Lodewijks, H., https://doi.org/10.1007/s11292-012-9159-7 Doreleijers, T., de Ruiter, C., & de Wit-Grouls, H., Krueger, R. A., & Casey, M. A. (2014). Focus Groups. A Trans.). Psychological Assessment Resources. (Original practical guide for applied research (5th ed.). Sage work published 2006) Publications. Braun, V., & Clarke, V. (2006). Using thematic analysis in Krueger, R. A. (2002). Designing and Conducting focus group psychology. Qualitative Research in Psychology, 3(2), interviews. Eastern Illinois University. http://www.eiu. 77–101. https://doi.org/10.1191/1478088706qp063oa edu/ihec/Krueger-FocusGroupInterviews.pdf. Chenail, R. J. (2011). Interviewing the Investigator: Langan, J. (2010). Challenging assumptions about risk fac- Strategies for addressing instrumentation and researcher tors and the role of screening for violence risk in the bias concerns in qualitative research. The Qualitative field of mental health. Health, Risk & Society, 12(2), Report, 16(1), 255–262. https://nsuworks.nova.edu/tqr/ 85–100. https://doi.org/10.1080/13698571003632429 vol16/iss1/16 Levin, S. K., Nilsen, P., Bendtsen, P., & Bulow, P. (2016). Consolidated Framework for Implementation Research Structured risk assessment instruments: A systematic (n.d.). CFIR codebook. https://cfirguide.org/tools/tools- review of implementation determinants. Psychiatry, and-templates/ Psychology and Law, 23(4), 602–628. https://doi.org/10. Crocker, A. G., Garcia, A., Isra€el, M., Hindle, Y., Gagnon, 1080/13218719.2015.1084661 D., Venegas, C., Braithwaite, E., Rot, M.-E. (2008). Masadeh, M. A. (2012). Focus group: reviews and practices. Implementing and using a systematic risk assessment International Journal of Applied Science and Technology, scheme to increase patient safety on a risk management 2(10), 63–69. http://ijastnet.com/journals/Vol_2_No_10_ unit for individuals with severe mental illness: A demon- December_2012/9.pdf stration project. Canadian Safety Patient Institute. http:// Muller-Isberner, € R., Born, P., Eucker, S., & Eusterschulte, B. crocker.mcgill.ca/docpdf/final_report.pdf (2017). Implementation of evidence-based practices in Damschroder, L. J., Aron, D. C., Keith, R. E., Kirsh, S. R., forensic mental health services. In R. Roesch & A. N. Alexander, J. A., & Lowery, J. C. (2009). Fostering imple- Cook (Eds.), Handbook of forensic mental health services. mentation of health services research findings into prac- (pp. 443–469). Routledge. tice: A consolidated framework for advancing Nonstad, K., & Webster, C. D. (2011). How to fail the implementation science. Implementation Science, 4(1), implementation of a risk assessment scheme or any other 50–66. https://doi.org/10.1186/1748-5908-4-50 new procedure in your organization. American Journal of De Beuf, T. L. F., de Vogel, V., & de Ruiter, C. (2019). Orthopsychiatry, 81(1), 94–99. https://doi.org/10.1111/j. Implementing the START:AV in a Dutch residential 1939-0025.2010.01076.x youth facility: Outcomes of success. Translational Issues Patton, M. Q., Cochran, M. (2002). A guide to using qualita- in Psychological Science, 5(2), 193–205. https://doi.org/ tive research methodology. MSF Intersectional Evaluation 10.1037/tps0000193 Group. http://evaluation.msf.org/sites/evaluation/files/a_ Desmarais, S. L. (2017). Commentary: Risk assessment in guide_to_using_qualitative_research_methodology.pdf the age of evidence-based practice and policy. Peterson-Badali, M., Skilling, T., & Haqanee, Z. (2015). International Journal of Forensic Mental Health, 16(1), Implementation of risk assessment in case management 18–22. https://doi.org/10.1080/14999013.2016.1266422 for youth in the justice system. Criminal Justice and Galdas, P. (2017). Revisiting bias in qualitative research: Behavior, 42(3), 304–320. https://doi.org/10.1177/ Reflections on its relationship with funding and impact [Editorial. International Journal of Qualitative Methods, Powell, B. J., Waltz, T. J., Chinman, M. J., Damschroder, 16(1), 160940691774899. ]. https://doi.org/10.1177/ L. J., Smith, J. L., Matthieu, M. M., Proctor, E. K., & Kirchner, J. E. (2015). A refined compilation of imple- Haque, Q. (2016). Implementation of violence risk assess- mentation strategies: results from the Expert ment instruments in mental healthcare settings. In J. P. Recommendations for Implementing Change (ERIC) pro- Singh, S. Bjorkly, & S. Fazel (Eds.), International perspec- ject. Implementation Science, 10(1), 21–34. https://doi. tives on violence risk assessment. (pp. 40–52). Oxford org/10.1186/s13012-015-0209-1 University Press. https://doi.org/10.1093/acprof:oso/ Schlager, M. D. (2009). The organizational politics of imple- 9780199386291.001.0001 menting risk assessment instruments in community cor- Hilverdink, P., Daamen, W., Vink, C. (2015). Children and rections. Journal of Contemporary Criminal Justice, 25(4), youth support and care in the Netherlands. Nederlands 412–423. https://doi.org/10.1177/1043986209344555 Jeugdinstituut. http://www.youthpolicy.nl/en/Download- Schutt, R. K. (2004). Investigating the social world. The pro- NJi/Publicatie-NJi/Children-and-youth-support-and-care- cess and practice of research (7th ed.). Sage Publications. in-The-Netherlands.pdf Sher, M. A., & Gralton, E. (2014). Implementation of the Kirk, M. A., Kelley, C., Yankey, N., Birken, S. A., Abadie, START:AV in a secure adolescent service. The Journal of B., & Damschroder, L. (2015). A systematic review of the Forensic Practice, 16(3), 184–193. https://doi.org/10.1108/ use of the Consolidated Framework for Implementation JFP-04-2013-0021 314 T. L. F. DE BEUF ET AL. Software, N. C. H. (2016). Express Scribe Transcription soft- Viljoen, J. L., Nicholls, T. L., Cruise, K. R., Desmarais, S. L., ware (Version 5.84) [Computer Software]. NCH & Webster, C. D. (2016). Short-Term Assessment of Risk Software. https://www.nch.com.au/scribe/versions.html and Treatability: Adolescent Version (START:AV) – User Tran, N. (2019, September 16). State of the field: Guide. (T. L. F. De Beuf, C. de Ruiter, & V. de Vogel, Implementation research panel. [Scientific Panel]. Global trans.). Mental Health, Law, and Policy Institute, Simon Implementation Conference. Fraser University. (Original work published 2014) Vaismoradi, M., Turunen, H., & Bondas, T. (2013). Content Vincent, G. M., Guy, L. S., & Grisso, T. (2012). Risk analysis and thematic analysis: Implications for conduct- Assessment in juvenile justice: A guidebook for implemen- ing a qualitative descriptive study. Nursing & Health tation. John D. & Catherine T. MacArthur Foundation. Sciences, 15(3), 398–405. https://doi.org/10.1111/nhs. http://modelsforchange.net/publications/346 Vincent, G. M., Guy, L. S., Perrault, R. T., & Gershenson, Vermaes, I., Konijn, C., Jambroes, T., & Nijhof, K. (2014). B. (2016). Risk assessment matters, but only when imple- Statische en dynamische kenmerken van jeugdigen in mented well: A multisite study in juvenile probation. Law JeugdzorgPlus: Een systematische review [Static and dynamic characteristics of youth in secured residential and Human Behavior, 40(6), 683–696. https://doi.org/10. care: A systematic review. Orthopedagogiek: Onderzoek en 1037/lhb0000214 Praktijk, 53(6), 278–292. ]. Vincent, G. A., Paiva-Salisbury, M. L., Cook, N. E., Guy, Viljoen, J. L., Beneteau, J. L., Gulbransen, E., Brodersen, E., L. S., & Perrault, R. T. (2012). Impact of risk/need assess- Desmarais, S. L., Nicholls, T. L., & Cruise, K. R. (2012). ment on juvenile probation officers’ decision-making: Assessment of multiple risk outcomes, strengths, and Importance of implementation. Psychology, Public Policy change with the START:AV: A short-term prospective & the Law, 18(4), 549–576. https://doi.org/10.1037/ study with adolescent offenders. International Journal of a0027186 Forensic Mental Health, 11(3), 165–180. https://doi.org/ Webster, C. D., Nicholls, T. L., Martin, M.-L., Desmarais, 10.1080/14999013.2012.737407 S. L., & Brink, J. H. (2006). Short-Term Assessment of Viljoen, J. L., Cochrane, D. M., & Jonnson, M. R. (2018). Risk and Treatability (START): the case of a new struc- Do risk assessment tools help manage and reduce risk of tured professional judgement scheme. Behavioral Sciences reoffending? A systematic review. Law and Human & the Law, 24(6), 747–766. https://doi.org/10.1002/bsl. Behavior, 42(3), 181–214. https://doi.org/10.1037% 2Flhb0000280 https://doi.org/10.1037/lhb0000280 737 http://www.deepdyve.com/assets/images/DeepDyve-Logo-lg.png International Journal of Forensic Mental Health Taylor & Francis

Staff Perceptions on the Implementation of Structured Risk Assessment with the START:AV: Identifying Barriers and Facilitators in a Residential Youth Care Setting

Staff Perceptions on the Implementation of Structured Risk Assessment with the START:AV: Identifying Barriers and Facilitators in a Residential Youth Care Setting

Abstract

AbstractAn effective implementation approach is crucial for successful integration of structured risk assessment instruments into practice. This qualitative study explored barriers and facilitators to the implementation of the Short-Term Assessment of Risk and Treatability: Adolescent Version (START:AV) in a Dutch residential youth care service. Perceptions of staff members from various disciplines were gathered through focus group interviews at three consecutive occasions. After inductive coding of the interview extracts using thematic analysis, the identified codes were linked to the consolidated framework for implementation research. Through this framework, factors that influence an implementation project can be organized into multiple domains and constructs. In the present study, staff members described implementation barriers related to characteristics of the risk assessment instrument, staff, and the implementation process. In addition, features of the setting were frequently mentioned as hindering the implementation, such as hierarchy, culture, communication, as well as implementation climate and readiness for change. Staff members also identified multiple facilitators, such as experienced advantages of the START:AV compared to the previous risk assessment practice and positive beliefs about the instrument. The article concludes with recommendations for successful implementation of structured risk assessment instruments in forensic-clinical practice.

Loading next page...
 
/lp/taylor-francis/staff-perceptions-on-the-implementation-of-structured-risk-assessment-QLgUUibsx0

References (43)

Publisher
Taylor & Francis
Copyright
© 2020 The Author(s). Published with license by Taylor & Francis Group, LLC.
ISSN
1932-9903
eISSN
1499-9013
DOI
10.1080/14999013.2020.1756994
Publisher site
See Article on Publisher Site

Abstract

INTERNATIONAL JOURNAL OF FORENSIC MENTAL HEALTH 2020, VOL. 19, NO. 3, 297–314 https://doi.org/10.1080/14999013.2020.1756994 Staff Perceptions on the Implementation of Structured Risk Assessment with the START:AV: Identifying Barriers and Facilitators in a Residential Youth Care Setting a,b b c Tamara L. F. De Beuf , Corine de Ruiter , and Vivienne de Vogel a b Research Department, OG Heldring Institution, Zetten, the Netherlands; Department of Clinical Psychological Science, Maastricht University, Maastricht, the Netherlands; Research Department, The Forensic Care Specialists, Utrecht, the Netherlands KEYWORDS ABSTRACT START:AV; implementation An effective implementation approach is crucial for successful integration of structured risk determinants; risk assessment instruments into practice. This qualitative study explored barriers and facilitators assessment; consolidated to the implementation of the Short-Term Assessment of Risk and Treatability: Adolescent framework for Version (START:AV) in a Dutch residential youth care service. Perceptions of staff members implementation research; from various disciplines were gathered through focus group interviews at three consecutive focus groups occasions. After inductive coding of the interview extracts using thematic analysis, the identi- fied codes were linked to the consolidated framework for implementation research. Through this framework, factors that influence an implementation project can be organized into mul- tiple domains and constructs. In the present study, staff members described implementation barriers related to characteristics of the risk assessment instrument, staff, and the implementa- tion process. In addition, features of the setting were frequently mentioned as hindering the implementation, such as hierarchy, culture, communication, as well as implementation climate and readiness for change. Staff members also identified multiple facilitators, such as experi- enced advantages of the START:AV compared to the previous risk assessment practice and positive beliefs about the instrument. The article concludes with recommendations for suc- cessful implementation of structured risk assessment instruments in forensic-clinical practice. Over the past decade, researchers have flagged the “positive outcomes are achieved only when both the need for more research and policy on the implemen- implementation process and the practice are effective” tation of risk assessment instruments (Desmarais, (p. 465). Other researchers have expressed concerns 2017; Nonstad & Webster, 2011). Implementation that even psychometrically sound instruments will fail refers to the processes that bridge the gap between the to improve outcomes for clients if not implemented decision to adopt a new practice and the committed properly (Desmarais, 2017; Schlager, 2009). For use of this practice (Damschroder et al., 2009). Once example, studies have found that services with better risk assessment instruments with sufficient predictive implementation quality (e.g., adherence to the admin- validity became available, successful implementation istration procedure) had significantly better results in was considered ‘the new challenge’ in risk assessment terms of risk management and service allocation practice (Nonstad & Webster, 2011, p. 94). How can (Vincent et al., 2012; 2016). In turn, improved risk we effectively integrate risk assessment instruments in management, by matching identified needs with practice and ensure fidelity of application? This can appropriate service provision, is associated with be a challenging undertaking, for example, due to staff reduced reoffending (Peterson-Badali et al., 2015). resistance and insufficient awareness of potential pit- To ensure implementation quality, a greater aware- falls (Schlager, 2009; Webster et al., 2006). Yet, the ness and understanding of the barriers to successful quality of implementation is crucial for the effective- implementation of risk assessment instruments in ness of risk assessment instruments in reducing recid- forensic-clinical practice is needed (Haque, 2016). ivism. Muller-Isberner € et al. (2017) stated that Webster and colleagues (2006) underscored the CONTACT Tamara L. F. De Beuf t.debeuf@ogheldring.nl Ottho Gerhard Heldring Institution, P.O. Box 1, Zetten 6670 AA, the Netherlands This article has been republished with minor changes. These changes do not impact the academic content of the article. 2020 The Author(s). Published with license by Taylor & Francis Group, LLC. This is an Open Access article distributed under the terms of the Creative Commons Attribution-NonCommercial-NoDerivatives License (http://creativecommons.org/licenses/by- nc-nd/4.0/), which permits non-commercial re-use, distribution, and reproduction in any medium, provided the original work is properly cited, and is not altered, transformed, or built upon in any way. 298 T. L. F. DE BEUF ET AL. importance of developing an implementation plan setting. Although the external context (e.g., govern- that reflects upon potential obstacles, such as limited mental guidelines and legislation) was mentioned in resources or poor communication. Factors that hinder most studies, it was not reported as a factor affecting effective implementation are referred to as implemen- the implementation. Determinants related to the inner tation barriers (Damschroder et al., 2009). Reflecting setting and the implementation process were most upon factors that promote successful implementation commonly reported. For example, all studies (e.g., management support, staff buy-in) should also addressed the importance of ensuring staff engage- be part of the implementation plan. These factors are ment from an early stage as well as providing infor- referred to as implementation facilitators. Insight into mation and training to facilitate the implementation. the barriers and facilitators is crucial when deciding Characteristics of the risk assessment instruments on which implementation strategies to adopt. were also cited as determinants, mainly as barriers. In 2016, Levin and colleagues published a system- For example, some risk assessment instruments were atic review of studies that documented determinants (initially) considered time-consuming or difficult to of successful risk assessment implementation in adult use. On the other hand, the potential to adjust an and adolescent (forensic) psychiatric and correctional instrument to fit the local routines was repeatedly settings. They included 11 studies, published between mentioned as a facilitator. Beliefs and concerns of the 2000 and 2013, in which the authors discussed factors users were reported in all but one study as influencing they perceived as hindering or facilitating the imple- the implementation. For example, the perception of mentation. Levin et al. (2016) organized these deter- clinical usefulness among users was identified as a minants according to the consolidated framework for facilitator, whereas a previous negative experience implementation research (CFIR; Damschroder et al., with structured risk assessment was considered a bar- 2009). The CFIR is a typology of implementation rier to the implementation. determinants compiled from 19 theories, and consists Central to the present study is the Short-Term of five domains and 39 constructs. The first domain Assessment of Risk and Treatability: Adolescent ‘intervention characteristics’ (eight constructs) con- Version (START:AV; Viljoen et al., 2016). The cerns features of the method that is being imple- START:AV is an evidence-based structured risk mented, such as cost, complexity, and adaptability of assessment instrument for use with boys and girls the new method. The second domain ‘outer setting’ between 12 and 18 years old. It was developed, piloted (four constructs) refers to external economic, political and evaluated in North America, mainly within juven- and social influences that can impact an implementa- ile justice populations. The START:AV was deemed tion process within an organization. In addition to the the most appropriate instrument for the present political context and the professional network of an secure youth care facility, because of its emphasis on organization, the needs of service users are also con- dynamic risk factors and its balanced approach that sidered as determinants within the outer setting. The includes both risk and protective factors. The dynamic domain ‘inner setting’ (14 constructs) concerns the nature of the instrument, with a recommended organization’s internal context, such as culture and reassessment interval of three to six months matched climate, communication structures, staff involvement, the setting’s four-month cycle of care. Additionally, and available resources. The fourth domain ‘user char- compared to other risk assessment instruments for acteristics’ (five constructs) pertains to the characteris- adolescents, the START:AV evaluates a wider spec- tics of the users of the implemented method. Users, trum of adverse outcomes. Short-term risk is eval- usually staff members, are considered active recipients uated for violence to others, nonviolent reoffending, with beliefs, attitudes, and ambitions that affect their substance abuse, unauthorized absence, suicide, self- behavior in relation to the implementation. The last harm, victimization, and self-neglect. All of these domain concerns the ‘process’ (eight constructs). An adverse outcomes are highly prevalent among adoles- implementation process consists of various steps that cents admitted to residential youth care settings in the require action, from planning and engaging, to exe- Netherlands (Vermaes et al., 2014). cuting and evaluating. Each step includes activities We are aware of one implementation study with that can enable or hinder the implementation. the START:AV conducted by Sher and Gralton (2014) Levin et al. (2016) used these domains and con- in a British medium secure adolescent service. This structs to organize the hindering and enabling factors study aimed to establish gaps in training and involve documented in the reviewed studies. They identified staff in the implementation process by inquiring their determinants linked to all CFIR domains, except outer views and experiences with the instrument. Although INTERNATIONAL JOURNAL OF FORENSIC MENTAL HEALTH 299 the authors did not explore barriers and facilitators to of the implementation process and initial experiences the implementation, implementation determinants can with the START:AV, using a focus group interview be derived from their findings and the CFIR domains method. The present article focuses on one particular can be applied. With respect to intervention character- issue that was addressed by the qualitative approach, istics, staff members indicated that the START:AV that is, implementation determinants (i.e., barriers and was easy to use, with some difficulties in distinguish- facilitators) as perceived by staff members. ing between the item ratings (low, moderate, high)as well as differentiating between strengths and vulner- Setting abilities. Some staff members also felt that the The study took place in one of 14 Dutch secure resi- START:AV could be confusing and somewhat repeti- dential youth care facilities for boys and girls who suf- tive. With respect to the ‘inner setting’ domain, issues fer from severe behavioral and mental health with untrained staff and insufficient practice with the problems. Youth are admitted under civil law, with a instrument were reported. In addition, staff experi- child protection order, to improve their safety (e.g., enced not enough time to complete an assessment. suicidal behavior, victimization) and/or the safety of Lastly, determinants related to user characteristics others (e.g., violence toward others). At the time of were noted: staff expressed facilitating beliefs about the study, the service had three high secure (of which the START:AV as valuable and helpful, including the two were observation units) and six medium-secure overall clinical usefulness of the instrument, and the (treatment) units, serving about 240 youths each year. added value of rating vulnerabilities as well as In 2016, adolescents (58% girls) were admitted for strengths. On the other hand, some believed that the 262 days on average, ranging from 4 to 717 days (A. START:AV might not be sensitive enough to measure Baanders, personal communication, January 31, 2019). change within a complex adolescent population. In Prior to the implementation of the START:AV, no addition, some raised the concern that the instrument validated structured risk assessment instrument was was only as useful as the level of insight of the team routinely used within the service. Although the Dutch into a patient’s problems. Nevertheless, most staff version of the Structured Assessment of Violence Risk members felt confident in their ability to complete the in Youth (SAVRY; Borum et al., 2006/2006) was avail- START:AV and contribute meaningfully to the assess- able and some treatment coordinators were trained in ments. Aspects of the outer setting and implementa- its use, the instrument was completed only a few tion process could not be gathered from this study. times a year. Specifically, it was used when boys with severe aggressive and delinquent behavior, transferred The present study from juvenile detention, were admitted to the service. Loosely based on the SAVRY, the service had con- In line with Levin et al.’s review (2016), the present structed a 12-item risk checklist that was used system- study explored which factors influenced the imple- atically in decision making regarding the youths’ leave mentation of the START:AV in a residential youth status (escorted/unescorted). The items of this risk care service. Implementation determinants from the list, such as ‘impulsivity’ and ‘association with deviant perspective of staff members were gathered through peers’, were rated by treatment coordinators as low, focus group sessions and organized according to the moderate or high without any rating criteria. Thus, CFIR. To our knowledge, this is the first primary the START:AV was introduced to replace both this study to apply the CFIR to risk assessment implemen- 12-item risk list and the SAVRY. In addition, the tation research. START:AV substituted another self-constructed list: the ‘dimension list’. This list was used upon admission Method to the observation units to gather information on 15 This study is part of a larger evaluation study on the developmental areas, the so-called dimensions (e.g., implementation of the START:AV, using a mixed- parent-child interaction, autonomy). Hence, in add- method design to assess various aspects of implementa- ition to risk assessment, the START:AV would serve tion within a youth care service. Previously, we used a as the service’s primary instrument to gather and quantitative web survey method to ask staff members structure treatment-relevant information on about multiple implementation outcomes (e.g., feasibil- the adolescent. ity, acceptability; De Beuf et al., 2019). Using a qualita- The START:AV implementation took place in a tive approach, we explored staff members’ perceptions turbulent social and political context. In 2015, a new 300 T. L. F. DE BEUF ET AL. Dutch Youth Act went into effect, resulting in major and an (external) risk assessment expert (second changes in the youth care system. One of the objec- author). The coordinator and the implementation tives of the new act was to decrease the use of costly committee were responsible for the implementation specialized services such as residential youth care procedure and the development of policies and proce- dures that detailed the incorporation of the (Hilverdink et al., 2015). This affected the present ser- vice in terms of a reduction of beds, staff reorganiza- START:AV into the service’s workflow. In the months tion and lay-offs, and an increased workload for the between September 2014 and March 2015, the imple- mentation coordinator trained treatment coordinators remaining staff. in using the instrument. The training included a one- day workshop in which they learned to rate the START:AV START:AV, followed by additional practice cases that The START:AV consists of 26 dynamic risk factors were discussed during a two-hour workshop. Later, a addressing characteristics of the adolescent (e.g., coping, third half-day workshop was organized in which treat- social skills), their relationships and environment (e.g., ment coordinators received guidance on how to trans- peers, parental functioning), and their response to treat- late the risk assessment findings into a risk ment (e.g., insight, treatability). The items are rated on management plan according to the Risk-Need- a3-point scale (low, moderate, high) based on function- Responsivity (RNR) principles (Bonta & Andrews, ing in the last three months: once as a protective factor 2017). All treatment coordinators that were hired after (strength) and once as a risk factor (vulnerability). After the initial training, received one-on-one training weighing and integrating all available information, the shortly after recruitment, by the same trainer. From evaluator formulates a final risk estimate (low, moder- June 2015 until November 2015, a pilot implementa- ate, high) for eight adverse outcomes, relevant for the tion was carried out on two units to test the new next three months. The START:AV is a so-called fourth workflow and accompanying documents (e.g., treat- generation risk assessment instrument which means ment plan). The coordinator and the implementation that it explicitly links the assessment process to risk for- committee evaluated the pilot and subsequent recom- mulation and risk management (Haque, 2016). In terms mendations were implemented by the coordinator. of psychometric properties, there is evidence for fair to The official, service-wide implementation started in excellent interrater reliability with intraclass correlation February 2016 on all units simultaneously, for new admissions only. Due to this gradual implementation, coefficients (ICC ) ranging from .52 to .88 for the risk it was not until fall 2016 that all treatment coordina- estimates, .86 for the vulnerabilities total score, and .92 for the strengths total score (Viljoen et al., 2012). tors had completed at least one START:AV assess- Significant predictive validity was found for the majority ment in their caseload. The assessments were initially a task of treatment coordinators and included com- of adverse outcomes, with ‘area under the curve’-values pleting the START:AV comprehensive rating form (in ranging from .63 to .83 for vulnerabilities total score, .63 to .80 for strengths total score, and .71 to .91 for MS Word) as a ‘master file’ in which all available the risk estimates. Thus far, the START:AV has been information about the youth was gathered (e.g., file information, interviews, psychological testing, and found to significantly predict all adverse outcomes, observations). The estimated completion time within except unauthorized absences and health neglect this approach ranged from one hour to four hours. (Bhanwer et al., 2016). The Dutch START:AV (De Beuf et al., 2019) was implemented at the current setting and However, starting from March 2017, all frontline staff (i.e., teachers, group care workers, therapists, family research on the psychometric properties is cur- social workers, occupational therapists, drug counse- rently conducted. lors) were instructed to report their evaluation of the past months directly into the START:AV form. The Implementation process implementation coordinator organized information The START:AV implementation project was based on sessions to educate frontline staff on the instrument’s the eight steps presented in Risk Assessment in items. Although the information was now entered via Juvenile Justice: A Guidebook for Implementation by a multidisciplinary approach, treatment coordinators Vincent and colleagues (2012). The implementation remained responsible for rating the items and formu- was led by an implementation coordinator (first lating a final risk judgment. Over the course of 2017, author) who was hired for the project, in collabor- a computerized version of the START:AV was devel- ation with an (internal) implementation committee oped and tested. INTERNATIONAL JOURNAL OF FORENSIC MENTAL HEALTH 301 START:AV observed during team meetings. We con- � FG treatment coordinators (n = 3) � FG group care workers (n = 5) sidered them as those who could best voice the imple- � FG mixed professions (n = 7; 1 treatment coordinator, 3 group mentation challenges experienced by the teams. At the care workers, 2 policy advisors, 1 therapist) March 2016 follow-up time points, we started with personally T1 � FG treatment coordinators (n=7) inviting the participants from earlier sessions. � FG group care workers (n = 3) Individuals in a purely managerial position (i.e., top- � FG mixed professions (n = 6; 1 treatment coordinator, 2 group care workers, 1 policy advisor, 1 social worker, 1 test assistant) Sept 2016 management, senior management, and operational T2 middle management) were excluded from the study. � FG treatment coordinators (n=5) Although treatment coordinators were considered � Interview group care workers unit 1 (n = 3) � Interview group care workers unit 2 (n = 2) middle management, they were invited because they June - � Interview group care workers unit 3 (n = 2) July 2017 supervised the treatment approach for the individual � Interview group care workers unit 4 (n = 2) � Interview group care workers unit 5 (n = 1) T3 adolescent rather than the workforce. Higher manage- � Interview group care workers unit 6 (n = 3) ment was not invited to the focus groups to prevent Note. FG = focus group; T = time point. power differentials during the discussions. Moreover, Figure 1. Focus Group Participants Throughout the because the authors did not focus on the perceptions Three Sessions. of higher management, they were not considered as a Note. FG¼ focus group; T¼ time point. separate user group in the design. Focus group size Focus group design We aimed for five to ten participants per focus group. Although the literature is mixed about the ideal focus The initial design was to organize nine focus group group size, recommendations typically range between discussions; three focus groups at three time points. 5 or 6 to 10 participants for noncommercial research The first time point was at the end of March 2016. (Masadeh, 2012). Small groups of 4 to 6 participants Although this was almost two months into the imple- are recognized to be productive because they encour- mentation, most participants had not yet used the age participants to partake in the discussion, and they instrument due to the gradual process. The second are increasingly popular because they are easier to time point took place in September 2016, and the last organize (Krueger & Casey, 2014; Masadeh, 2012). one was in June 2017. Each time point was scheduled Figure 1 shows that we reached this goal for five focus to have one focus group exclusively with treatment groups. Especially at time point three, it proved diffi- coordinators (TC), one exclusively with group care cult to gather enough participants: only treatment workers (GCW), and one interdisciplinary group coordinators responded. As a result, the study design (Mix). The choice for two homogeneous groups and was adjusted: the third mixed focus group was can- one heterogeneous group was guided by the assump- celed as staff had expressed concerns that they could tion that homogeneous groups might generate more not meaningfully contribute to the group discussions in-depth information about a group’s experience with due to a lack of experience with the START:AV. The the START:AV, while a heterogeneous group might focus group with group care workers was replaced by generate a wider range of information (Schutt, 2004). interviews for which the moderator visited frontline workers on their unit with a slightly adjusted inter- Sampling view guide. The group interviews were held with one All staff members who worked directly with adoles- up to three participants depending on how many cents were invited by a general invitation shared via group care workers were available on the unit (see email, newsletters, and announcements on the serv- Figure 1). In total, seven focus groups and six inter- ice’s intranet. In addition, two policy advisors were invited because of their involvement in the service’s views took place. Participants signed an informed consent at the quality assurance procedures for new and existing beginning of each focus group, after being informed methods, as well as their involvement in the integra- about the content, procedure, audio recording, and tion of the START:AV in the electronic patient file. their rights in relation to the research. Their participa- However, this approach generated a poor response tion was voluntary and during work hours. On average, from group care workers and in the next step individ- ual group care workers were invited via a personalized focus groups lasted 64 minutes (range: 40-79 minutes) email. They were selected based on their active and interviews 12 minutes (range: 4-20 minutes). All involvement in task forces and their interest in the interviews were held in Dutch and audiotaped with a 302 T. L. F. DE BEUF ET AL. voice-recorder. Afterwards, participants received a treatment process. In consultation with their team, summary of findings via email and they had the oppor- they decided on the treatment approach in terms of tunity to respond before the internal report type of therapy and treatment goals, and evaluated the was finalized. treatment progress. They also provided treatment-rele- vant supervision and guidance to group care workers. Interview questions Treatment coordinators, all female, were between 30 During the first focus group interview, previous and 40 years old (M¼ 34) and had on average 10 years implementation efforts within the service were dis- of work experience (range¼ 5-15) within the facility. cussed, as well as expectations about the START:AV Seventy percent had prior experience (i.e., use and/or and potential barriers to the implementation. At the training) with a structured risk assessment instrument, second and third time point, focus group interviews mainly the SAVRY. Group care workers were front- elaborated on the previously mentioned challenges line staff who were responsible for implementing the and whether they had been resolved. In addition, treatment approach for the youths’ safety and well- positive and negative effects of the new risk assess- being on the residential units. They supported adoles- ment method, for example, on communication, were cents with their everyday responsibilities. Of the discussed, as well as workflow issues. An interview participating group care workers, 43% were female; guide, modeled after Crocker and colleagues (2008), they were between 25 and 51 years old (M¼ 35), with was used to guide the discussions and is available at 1 to 17 years of experience within the organization the project page on Open Science Framework (OSF; (M¼ 8.5). Among the other participants were profes- https://osf.io/5gztr/). The open-ended questions pro- sionals such as family social workers, psychotherapists vided a framework, and specific questions and and testing assistants, as well as policy advisors. Of prompts by the moderator followed from the discus- these professionals, 80% were female and their average sion. The interview approach was informed by resour- age was 44 years old (range ¼ 36-56), with 7 to ces such as Krueger (2002), Krueger and Casey 29 years of service (M¼ 14). (2014), and Patton and Cochran (2002). Ethical considerations Role of moderator In qualitative research, the investigator is considered The general director of the service gave permission to conduct the research within the facility. The Ethics an integral part of the research process and the final product (Galdas, 2017). Therefore, researcher influ- Review Committee Psychology and Neuroscience ence should be disclosed and reflected upon. In the (ERCPN) of Maastricht University approved the present study, due to resource constraints, the moder- research protocol, procedures, and staff consent forms ator of the focus groups was also the implementation (ERCPN number 174_06_12_2016). All data were coordinator as well as the data analyst. The investiga- analyzed anonymously and stored according to the tor-moderator was not part of the treatment staff and university’s Data Management Code of Conduct and she was hired by the organization to implement and the institution’s Data Protection Guidelines. evaluate the START:AV as a relative ‘outsider’ with a university affiliation. The moderator received one-on- Data analysis one coaching by an experienced focus group facilitator The focus group discussions were transcribed using and discussed the focus group sessions with free transcription software (NCH Software, 2016). The the coauthors. moderator was selected as transcriber because she was familiar with the context of the discussions, recog- Participants nized the recorded voices, and was able to differenti- Across all time points, 36 unique staff members par- ate between relevant and irrelevant (e.g., ticipated: 10 treatment coordinators, 21 group care interruptions) audio material. Furthermore, transcrib- works and 5 other professionals (see Figure 1). This ing increased familiarity with the data, which was reflects 100% of the treatment coordinators, 20% of beneficial for subsequent thematic analysis. The tran- group care workers, and 63% of other disciplines that scription followed an edited approach; filler words, were invited. Treatment coordinators, professionals interruptions, self-corrected words and non-relevant with at least a master’s degree in psychology or special content (e.g., small talk at the beginning of a session) needs education, were responsible for the adolescents’ were omitted while maintaining integrity of the INTERNATIONAL JOURNAL OF FORENSIC MENTAL HEALTH 303 recordings. All transcripts were anonymized: names end, all three raters reached consensus on the allo- were replaced by numbers. All transcriptions were cated codes. Moreover, as a result from the discussion, double-checked against the audio recordings. one new code was added as determinant and other codes were renamed for clarification. Thematic analysis Transcriptions from the focus groups and interviews Consolidated framework for implementation were analyzed using thematic analysis (Vaismoradi research After the inductive approach to data gathering and data et al., 2013). This is a widely used qualitative method analysis, we applied a deductive approach to the classifi- for identifying, analyzing, and reporting patterns cation and interpretation of the ‘Implementation within transcribed data (Braun & Clarke, 2006). It Determinants’ codes. The CFIR codebook (Consolidated relies on a step-by-step process of coding and re-cod- Framework for Implementation Research, n.d.) was con- ing, and codes are collated into themes. The present sulted to connect our codes to the CFIR constructs, as it study followed an inductive (data-driven) approach, provides definitions and inclusion/exclusion criteria for which means that codes were identified independently the majority of the constructs. of a theoretical framework. Applying a manual coding procedure, the first author started the thematic ana- lysis by rereading the full transcript and entering the Results text in Excel with one sentence, or data extract, per Descriptive information on the distribution of the 211 row. Each data extract received one or more codes codes accumulated by the thematic analysis can be that summarized its content in one or two words. accessed via the OSF project page (https://osf.io/5gztr/). Examples of codes included ‘provide training’ or We focused on the codes related to the implementation ‘resistance to change’. After working through all data determinants and their link with the CFIR domains and extracts, initial codes with their definitions were listed constructs (see Table 1). All five domains and 21 of the in one file and examined for similarity. Similar codes 39 constructs were addressed. In the following, we pre- were collated into a more manageable number of sent the identified constructs per domain, illustrated codes. Collated codes or newly identified codes were with quotes. For each construct, it is also specified compared with the original statements of the partici- whether it was experienced as a facilitator, a barrier, pants to ensure they matched. Internal homogeneity or both. of a code was examined by collating all data extracts labeled with this particular code and assessing coher- ence, while external heterogeneity was ensured by Intervention characteristics identifying clear distinctions between codes. This pro- Features of the START:AV were mostly discussed by cess was repeated for each focus group. The codes of the treatment coordinators. Contrary to other staff, the interviews at T3 were combined into one group. they had considerable hands-on experience with the In the next step, all codes were sorted into poten- instrument and insight into its procedures, especially tial themes (Braun & Clarke, 2006). Five overarching at the second and third time point. themes were identified: ‘Implementation Determinants’, ‘Implementation Strategies’, Complexity (facilitator and barrier) ‘Implementation Outcomes’, ‘Feedback on Practice’, The majority of group care workers found it easy to and ‘Suggestions for Practice’. The first three themes provide information on the items because the item applied to the implementation process, whereas the indicators on the form were felt to be self-evident. latter two concerned the practicalities of the Although several group care workers reported difficul- START:AV workflow. Because of the present article’s ties separating strengths from vulnerabilities, which focus on determinants, only codes from the was corroborated by treatment coordinators, they sup- ‘Implementation Determinants’ theme were retained ported the inclusion of strengths. for further examination. For these codes, an inter- You have to be very aware of what is considered coder check was performed. The second and third strengths and what is considered vulnerabilities, and author independently coded 27% (i.e., 113 of 417) of without realizing you are documenting vulnerabilities the extracts related to ‘Implementation Determinants’ among the strengths: “Oh no, that doesn’t belong here” and vice versa. I find the strengths very using a codebook compiled by the first author (see valuable because otherwise they wouldn’t be noted https://osf.io/5gztr/). They compared coding sheets very often. (T3 GCW) and discrepancies were resolved by discussion. In the 304 T. L. F. DE BEUF ET AL. Table 1 Overview of the Implementation Determinants Codes Categorized according to the CFIR Implementation Determinant CFIR Domains and Constructs Barrier Facilitator Intervention Characteristics Relative Advantage Less detailed information (T3 TC) Value: more straightforward (T2 TC) Lost narrative (T3 TC) Value: multiple adverse outcomes (T2 TC) Limited innovation (T3 TC) More complete risk assessment (T2 Mix) Missing information (T2 TC) Adaptability Rating Differentiation (T1 TC) Compatibility (T2 TC) Complexity Strength vs Vulnerability (T3 TC) Understandable (T3 GCW) Similarity items (T3 GCW) Strength vs Vulnerability (T3 GCW) Design Quality & Packaging Software support (T3 TC) Cost Cost-benefit ratio (T1 GCW) Outer Setting External Policy & Incentives – Bigger picture (T3 TC) Inner Setting Structural Characteristics Layers in organization (T1 TC) – Layers in organization (T1 GCW) Organization islands (T1 GCW) Networks & Communications Communication (T1 GCW) – Communication (T1 Mix) Culture Fear of negative effects (T1 TC) – Organizational history (T1 GCW) Organizational Inflexibility (T1 TC) Overall Resistance (T1 TC) Overall Resistance (T1 GCW) Top-down decisions (T1 GCW) Compatibility – Setting’s aim (T3 TC) Relative Priority No priority (T1 TC) – Learning Climate – Staff involvement (T1 GCW) Leadership Engagement Management Involvement (T1 TC) – Organizational Inconsistency (T1 Mix) Available Resources Increased workload (T1 TC) – Investment (T1 GCW) Lack of time (T2 GCW) Access to Knowledge & Information Availability START (T2 GCW) Inform Staff (T2 Mix) New Staff (T3 GCW) Characteristics of Individuals Knowledge & Beliefs about the Intervention Confidentiality Concern (T2 Mix) Experienced Advantage (T3 TC) Improved Explanation (T2 TC) Improved Explanation (T3 TC) Increased focus on Facts (T2 Mix) Increased focus on Strengths (T2 Mix) Increased focus on Strengths (T2 TC) Population match (T3 TC) Present Observations (T3 GCW) Provides Overview (T2 TC) Staff Buy-in (T2 TC) Individual Stages of Change Individual response to change (T1 Mix) Enthusiasm (T1 TC) START:AV Resistance (T2 TC) START:AV Resistance (T2 Mix) Other Personal Attributes Experienced (role) Conflict (T2 Mix) – Process Planning Endless Preparation (T1 TC) – External Change Agents – External Expertise (T1 TC) Executing Timing Information-sharing (T1 TC) – Note.T¼ time point; TC¼ treatment coordinators; GCW¼ group care workers; Mix¼ mixed group of staff members Relative advantage (facilitator and barrier) documentation of the observed risks and the wide When comparing the START:AV with the self-con- scope of potential adverse outcomes. structed risk checklist, treatment coordinators I notice that I reflect much more on how often the described benefits, such as more explicit child has had such an aggressive outburst or hasn’t. INTERNATIONAL JOURNAL OF FORENSIC MENTAL HEALTH 305 While in the past, you just might have said “this is a moderate or high?” (T2 TC). Treatment coordinators very aggressive boy”, this time you are more looking also commented on the utility of the START:AV for for uh … That you make the connection: “Because of less restricted settings, such as community-based serv- this lack in skills, the risk is high”. I think I … It ices, arguing that it could be useful to detect unsafe provides more guidance to do this in a good way. (T2 TC) conditions. Yet, they hypothesized that in such set- tings only a small group of juveniles would benefit The focus group interview with mixed staff mem- from the risk assessment and therefore wondered bers added that the START:AV provides a more com- whether it would be worth the (implementa- prehensive account compared to the setting’s self- tion) effort. constructed checklist. However, when comparing the START:AV with the dimension list, treatment coordi- Design (barrier) nators felt that the START:AV did not add much in The computerized version of the START:AV compre- terms of information gathering and structuring. They hensive form, piloted prior to the last focus group, felt that the items were not novel compared to the was mentioned as a barrier. According to the treat- themes that were included in the dimension list: “I ment coordinators, the digital form “made it even also think that it’s a good summary of current think- ing in terms of the socio-emotional domain, the cog- more difficult” for group care workers to report the nitive domain, social network, … but we had that required information, because it did not include the already, it’s not like the START:AV added value to items’ anchors (i.e., item descriptors) to rely on. this” (T3). Neither did they experience benefits from the structured approach, “… because we were already Cost (barrier) working fairly structured prior to this. It is not like it Group care workers expressed that, in the past, their was completely blank before and that we did not have work on the unit had not benefited from completing any–how do you call it–structure or so”. (T3 TC). questionnaires and forms. This made some staff Overall, they indicated that the START:AV did not apprehensive about the present implementation: produce new insights or different treatment goals. Completing many lists, completing many forms, to One treatment coordinator seemed to carry the ultimately make one report. I think that half of those expectation that the START:AV would lead to consid- forms are not even being used to reach a proper erably different conclusions: “In the end, you actually conclusion. But because it’s obligated, we have to want it to result in making completely different complete them. That brings along a lot of work that choices and if that’s not the case, then you just I’d rather spend on the ward instead of in the office. (T1 GCW) think…” (T2 TC). Others countered this argument by indicating that this, in fact, validated their prior deci- sion making. Furthermore, on the observation units where teams had been using the dimension list (see Outer setting Setting), treatment coordinators regretted the change The outer setting refers to the larger political, eco- in information they now received from frontline staff nomic and social context in which the organization is on the START:AV items. They argued that although embedded. It includes those who are served by the they obtained information on more domains, this setting and the external network of the organization. information was less detailed than before. Likewise, This domain was mentioned least often. treatment coordinators commented on the loss of nar- rative due to reporting per item. Other treatment External policy and incentives (facilitator) coordinators added that they were missing informa- Treatment coordinators indicated that the introduc- tion on domains that were not included in the tion of the START:AV within the service corre- START:AV, such as developmentally-appropriate sponded with national reforms in youth care policy. knowledge of sexuality. Legal mandates for residential treatment were reduced in length, and evidence such as risk assessment infor- Adaptability (barrier) mation was increasingly required when imposing or Although treatment coordinators understood the prolonging mandated treatment. According to treat- importance of adhering to the item ratings, there ment coordinators, it “certainly goes nicely hand-in- seemed to be some negligence when allocating scores. A treatment coordinator asked rhetorically: “Do I hand because with that [risk assessment] you can bet- really have to worry five minutes about whether it is ter indicate ‘I really find that this individual needs 306 T. L. F. DE BEUF ET AL. more time in mandated residential youth care’,or ‘I This reluctance impacted treatment coordinators, really find that we should continue because…’.” (T3). because the service had not yet followed through on their promise to relieve them from other tasks. A treatment coordinator noticed: “In this case [of the Inner setting START:AV], everyone thinks that we should do it, but there is still, for example, nothing else removed Contrary to the outer setting, characteristics of the from our workload” (T1 TC). inner setting (i.e., the service itself) were often noted Not only were those in charge thought to resist as factors that influenced the implementation. It change, a group care worker voiced that it was quite includes both tangible aspects, such as size and struc- common for frontline staff to respond reluctantly to ture of the service, as well as immaterial features such new initiatives: as work culture and climate. Aspects of the inner set- ting were primarily discussed at the first time point. For some, it is more difficult than for others. And, it Statements on the inner setting in relation to previous is precisely those who experience difficulty, who deserve attention, and it is precisely those who need implementation efforts were also coded as they were to be involved in the process and perhaps those who thought to be relevant to the START:AV you need to talk to more frequently. (T1 GCW) implementation. Furthermore, group care workers felt new initia- tives were often imposed upon them, in a top-down Compatibility (facilitator) fashion, without sufficiently accounting for the team’s Treatment coordinators agreed that the START:AV possibilities. This generated frustration among front- was compatible with the objective of mandated resi- line staff: “When it comes out of the blue and some- dential youth care to reduce risks without trying to one says ‘You have to do that’, with no room for resolve all developmental challenges of the admitted discussion, that causes irritability”. (T1 GCW) youth: “Because you actually have to reduce the risks and should not want to change issues in all develop- Learning climate (facilitator) mental areas” (T3). However, one treatment coordin- Nevertheless, group care workers were positive about ator shared concerns that focusing on the risks and past implementation efforts in which they had been needs would not sufficiently address the actively involved. For example, one group care worker youths’ problems. enthusiastically described active involvement in the implementation of a group-oriented intervention: Culture (barrier) “What I really liked about implementing [interven- Staff members shared that, 10 years ago, the facility tion], what I found positive about it, is that we were was a juvenile detention center. They believed that allowed to contribute: when are we going to provide this background was still affecting today’s practice, the training? How are we going to do it?” (T1 GCW). which was perceived as strict and inflexible by some. With regard to the START:AV implementation, par- A group care worker used the metaphor of a “slowly ticipants did not express that they felt involved. turning heavy tanker ship” (T1) to describe the organ- ization. Overall, staff members believed that the ser- Structural characteristics (barrier) vice had difficulty adapting to change and preferred The facility’s hierarchical management structure was maintaining routine practice. Participants recognized reported as a barrier due to its many administrative both an aspiration to innovate and a rigidity to pre- levels. Staff explained that a decision had to pass mul- serve the status quo among top-management. tiple levels before it could lead to actual change. Treatment coordinators speculated that this rigidity This is also part of our culture. Here, everyone wants was fueled by worries about losing a good standing to have their say about it and everyone is allowed to reputation within the field. have their opinion. At a certain point, because of this, we just stagnate: “Okay, when will we finally get It’s ambiguous because we see that many authorities, started? When will we act? It still has to pass this such as youth care offices, local authorities, refer committee and then that one has to give their [youths] to us because we deliver very good service, opinion”. At times, this works against us because it in their opinion. So it is somewhat ambiguous takes three quarters of a year before anything finally because on the other hand, this reluctance – sticking seeps through in the organization. (T1 GCW) to what goes well– also makes us steady, well- functioning and therefore delivering a good job. To A material barrier to the implementation was the me, that is … uh well, difficult. (T1 TC) physical scattering of units across the grounds, INTERNATIONAL JOURNAL OF FORENSIC MENTAL HEALTH 307 The challenging part is that our direct supervisor has weakening the sense of unity within the service. Staff insufficient insight in how much time it takes. What typically referred to the treatment units as ‘islets’. he does, is personalize the problem: those who complain more, are considered to have a difficult Networks & communications (facilitator and barrier) personality, and those who don’t complain, are just The barriers were found to affect communication. not that busy. This is his strategy to ignore the Group care workers did not always feel sufficiently problem and that is frustrating. (T1 TC) informed on past implementations and other staff members also shared experiences of not being Available resources (barrier) included in the information exchange. Concerning the Concerning the facility’s readiness to implement the START:AV implementation, a therapist explained that START:AV, all disciplines addressed the lack of the communication was clear, but that the expecta- resources. The START:AV placed a burden on the tions for her particular position were unknown. treatment coordinators’ workload and this was, at Overall, participants found that the introduction of least for some, a major barrier to adopt the instru- the START:AV was well-announced. A group care ment. They realized that becoming adept in using the worker commented: START:AV required a substantial (time) investment: I do think, the communication … of course you “We keep repeating this: time is really the biggest [implementation coordinator] have been working on impeding factor in this.” (T1 TC). Similarly, group it for a while now and you have represented yourself pretty well. Everyone, in my opinion, knows care workers found it difficult to carve out time to something is going to change. ( … ) That makes a familiarize themselves with the completed forms they difference: the new system does not appear out of the had to complete for their caseload. One group care blue. It is already [introduced], step-by-step, in the worker linked this to the occupancy on the unit: newsletter or in an email. Somewhere attention is paid to it, everyone is being reached. (T1 Mix) Usually there is not much time. That’s also complicated, it depends on how many adolescents you have on your unit. We already have a lot of lists Relative priority (barrier) to complete and there is already a lot that we must Treatment coordinators felt that, at the beginning of do. At the moment, we have six youth on the ward and then you have some extra time to do other the implementation, the START:AV was not priori- things. When the unit is full with 11 adolescents, tized enough. On the question “What if you could you’re lucky if you have managed to write the daily change one thing?” a treatment coordin- progress notes by 10: 30 p.m. (T2 GCW) ator responded: ( … ) that our supervisor takes more responsibility as in “this has priority now and that means that we Access to knowledge & information (facilitator don’t do this or this or that for a while”, including and barrier) some task forces or what not … That all of us agree With respect to accessing information, participants as an organization: this has our priority now. (T1 TC) appreciated the visits of the implementation coordin- ator to inform the teams about the START:AV: “Ido Leadership engagement (barrier) think that what you did —discussing it in the team Frontline staff warned against the multitude of superi- meetings—is very useful to make it more concrete.” ors to whom they were accountable. In the past, this (T2 Mix). However, group care workers experienced had led to inconsistencies in approach and deviations difficulties accessing the forms as they were not stored from agreed strategies: “Exactly! One of the managers in a location available to them. Furthermore, new thinks it is really important, while my operational group care workers did not receive a formal introduc- manager suddenly starts questioning the implementa- tion to the START:AV; they learned it on the job, tion, and you are in the middle of it” (T1 Mix). during case conferences and by asking colleagues. A Receiving conflicting messages was perceived as frus- group care worker considered this part of one’s own trating and confusing. Treatment coordinators responsibility: “I mean, children are assigned to a expected their leadership to take on a supportive atti- mentor based on availability at the time, and when tude toward the increasing workload and be engaged you become mentor of a child in an observation tra- in the implementation, for example, by completing jectory, you have to think for yourself what you need START:AV assessments themselves. to do.” (T3 GCW) 308 T. L. F. DE BEUF ET AL. User characteristics Other personal attributes (barrier) Knowledge & beliefs about the intervention (facilita- One participant who works closely with the adoles- tor and barrier) cents’ relatives, expressed concerns about reporting Treatment coordinators were convinced about the sig- incriminating information, without an extended narra- nificance of the START:AV and the usefulness of the tive, in the START:AV. This staff member seemed to adverse outcomes for the setting’s clients. Moreover, experience a conflict between providing information treatment coordinators argued that with the on the START:AV items and his role as counselor: START:AV they were better equipped to substantiate You are working with a human process and their decisions about the treatment approach and the devastating situations, and then you can’t simply state recommended level of supervision. In addition, they in the report “Look, this is what this family has shared”. Then you fall short on these people and that believed in the value of focusing on strengths. This is not okay. (T2 Mix) was illustrated by a treatment coordinator: “I feel that it is fair to the youths to also consider strengths. I think that in our field there is a strong tendency to Process focus on the negative things. Therefore, I feel this really is of added value.” (T2). Frontline staff agreed: External change agent (facilitator) “This way, you are forced to pay attention to the posi- Treatment coordinators considered the service’s choice tive things.” (T2). Another advantage of the to hire an implementation coordinator as a facilitator. START:AV, according to staff members, is its focus In past implementations, they had noticed better on facts and the unambiguous presentation of risks facilitation of external project coordinators by man- and concerns. A treatment coordinator in the mixed agement and staff members had also been more recep- group explained: tive to external agents. They said: “I think it makes a big difference when an external person is brought on What I find very positive, is that you substantiate the risks for an adolescent much more. Often, as a team, board, as now happens with the START:AV, and this you agree that it is a very worrisome case, but if you person takes charge” (T1 TC). put it [the risks] together like that, then “Yes indeed, this is very worrisome”. And you can write it down Planning (barrier) much more clearly and communicate to the child Treatment coordinators regretted the long planning guardian agency or the parents that there are major phase with extensive preparation, which they referred concerns in these areas. It helps me to describe things more objectively. (T2) to as “an endless start-up phase” (T1). According to them, the implementation committee should have Still, one participant expressed concerns about con- started sooner with the full implementation. fidentiality because it was not clear to this staff mem- ber that the START:AV was treated as an internal Executing (barrier) document and would not be shared with third parties. They were especially dissatisfied about the substantial time that passed between the training and the actual Individual stages of change (facilitator and barrier) use of the instrument. One treatment coordinator On the one hand, treatment coordinators reported explained how this impacted her: enthusiasm and willingness among their team mem- bers to use the instrument, on the other hand, much It’s just really too bad that if you look back: when did we complete the [START:AV] practice cases? That resistance was noted. Reported reasons for resistance was over a year ago! Look, if … for me that to the implementation were lack of time and not [knowledge] really is already disappearing. That’s just being able to complete a START:AV assessment with- a waste of energy. (T1 TC) out being interrupted. Furthermore, staff members’ For this reason, treatment coordinators supported response to the new practice appeared to depend on the decision to inform group care workers later in the their openness to change. Some readily expressed process, otherwise “they might become frus- enthusiasm, whereas others were more reluctant. A trated” (T1). group care worker commented: One person might immediately start thinking about it, getting excited, while another, when he does not Discussion see the benefits or when it is not yet clear, This qualitative study explored staff members’ views immediately thinks: “Ah, again another list”. I find it varies a lot, at least if you look at our team. (T1 Mix) on factors that affected the implementation of the INTERNATIONAL JOURNAL OF FORENSIC MENTAL HEALTH 309 START:AV risk assessment tool in a residential youth form, while simultaneously working on building and care service. Using the consolidated framework for sustaining a therapeutic alliance with the youths’ fami- implementation research, we organized a set of imple- lies. This concern is in line with staff’s expected loss mentation determinants, which we derived from focus of discretion described in three of the studies reviewed group interviews using a data driven approach. On by Levin et al. (2016). For example, in an implementa- multiple domains, staff members identified factors tion study by Vincent et al. (2012), 21% of juvenile that they perceived as impeding or facilitating imple- probation officers “feared that their years of experi- mentation of the instrument. Aspects of the inner set- ence would be discounted in favor of a score from a ting were mentioned most frequently, followed by tool” (p. 573). However, this anticipation proved user characteristics and features of the risk assessment unwarranted as only four officers (4.7%) reported feel- instrument itself. The implementation process itself ing invalidated by the instrument at 10 months into and the outer setting were rarely mentioned. the implementation. The cultural and structural features of the service With respect to the instrument’s characteristics, were widely discussed as barriers to the implementa- there are several parallels with the findings of Sher tion. Structural barriers included the physical environ- and Gralton (2014). For instance, group care workers ment (e.g., physical scattering of units) and the in our study also found the items to be straightfor- ‘corporate’ environment (e.g., hierarchical levels). ward and easy to complete; yet, some struggled with According to participants, these structural barriers differentiating strengths from vulnerabilities, similar hindered communication, which is an essential com- to the UK study. Most group care workers preferred a ponent of introducing change (Damschroder et al., narrative approach in which they connect, and poten- 2009). Moreover, communication is key in earning tially counterbalance, vulnerabilities and strengths in staff buy-in, creating enthusiasm, and encouraging one paragraph. Strictly separating strengths from vul- staff involvement. Multiple aspects of the implementa- nerabilities required additional effort. In addition, tion relied on communication, such as sharing the group care workers reported that differences between rationale for introducing structured risk assessment, some of the items were quite subtle, making it more providing practical information about the START:AV difficult to allocate information to the appropriate workflow, and updating staff on the implementation item without duplicating, a concern also mentioned progress. In addition, resources were a frequently by Sher and Gralton (2014). It is unclear from the UK mentioned barrier: the START:AV increased staff study, whether staff was already familiar with struc- members’ workload while operating within the same tured information gathering. In the current setting, (time) conditions. Similarly, staff in Sher and Gralton treatment coordinators experienced this as a consider- (2014) study reported a lack of time to effectively able barrier: they seemed disappointed by the limited complete the START:AV. The present approach to novelty of the START:AV in terms of the included treat the START:AV as a master file, and the time items and the provided structure, compared to what investment that came with it, led to resistance among they were already used to. Moreover, replacing the staff members. Overall, resistance was a common dimension list with the START:AV was accompanied theme, from the board of directors to frontline staff, by a sense of loss (e.g., less detail, missing themes, each for their own reasons. This is not surprising, loss of narrative). The participants rarely commented on the planning because implementing change is “fighting against one’s inner desire to maintain the status quo” and the execution phase of the implementation, per- (Tran, 2019). haps because of the timing of the focus groups. The Nevertheless, despite the experienced strain in planning phase had ended and the official implemen- terms of workload, staff members were positive about tation had begun. Nevertheless, staff stated that prep- the value of the START:AV and its usefulness for aration, training, and actual implementation had not their practice, in line with Sher and Gralton’s findings succeeded each other within a reasonable time frame. (2014). In both settings, the focus on strengths was The lengthy intervals between these steps were per- highly valued. In addition, in the present study, the ceived as impeding the implementation. multiple adverse outcomes were particularly appreci- Similar to findings reported by Levin et al. (2016), ated and perceived as relevant for the setting, even and Sher and Gralton (2014), staff members did not more so than the individual items. Yet, one staff consider factors from the outer setting as affecting the member expressed experiencing difficulties with implementation. This was somewhat surprising, reporting sensitive information in the START:AV because the implementation took place during 310 T. L. F. DE BEUF ET AL. turbulent times for youth care organizations, with However, triangulation was applied during data ana- many legal and budgetary changes occurring simultan- lysis: a subsample of the extracts was coded by raters eously. The restraints that were imposed on the ser- who were external to the service, reaching satisfactory vice from the outside (e.g., reduced financial agreement. This procedure improves standardization and accuracy in the coding process and helps control resources) may have hindered the implementation. On the other hand, growing political pressure to use evi- for bias (Boeije, 2010). However, researcher triangula- dence-based practices, such as structured risk assess- tion was not repeated during the deductive phase ment, could also have facilitated the implementation. when the determinant codes were linked to the CFIR constructs. Not having multiple coders leaves the pro- Yet, staff did not allude to this. We contemplate that staff members have a tendency to focus primarily on cess to one researcher’s judgment. For example, the internal organizational factors. This might be espe- code ‘fear of negative effects’ could be considered a cially true for (secure) residential settings that are characteristic of the organizational culture (i.e., ‘Culture’) or a characteristic of the leading CEO (i.e., more closed off from society than community-based ‘Individual Stage of Change’). Nevertheless, both the services. Furthermore, the needs of the assessed ado- lescents themselves, and barriers and facilitators for codes and the constructs were defined in their them to participate in the risk assessment were not respective codebooks prior to the deductive phase, discussed. Such absence was also noted by Levin et al. reducing the opportunity for interpretation. Second, the sampling method might have influ- (2016). In risk assessment practice, patients’ views are enced who participated in the focus groups. It is pos- typically not included and assessments tend to be con- ducted top-down by professionals (Langan, 2010). sible that because of the voluntary nature, staff Similarly, in the present setting, adolescents were not members with strong resistance toward the implemen- tation did not sign up for participation. The opposite involved in the risk assessments. This approach likely could also be true; perhaps dissatisfied staff members limited staff members’ consideration of the adolescent took the focus group discussions as an opportunity to as a stakeholder in the implementation. The majority of determinants reported in the pre- voice their negative opinions. Moreover, participants sent study are recognized in the existing literature as knew prior to accepting the invitation that the imple- mentation coordinator would be moderating the ses- important conditions for successful implementation. sions. It is plausible that individuals who felt In accordance with prior work on risk assessment uncomfortable talking to the implementation coordin- implementation (Levin et al., 2016;; Muller-Isberner € et al., 2017; Nonstad & Webster, 2011; Schlager, 2009; ator refrained from participating in the study. As a result, the breadth of experiences shared in the focus Sher & Gralton, 2014; Webster et al., 2006), our find- groups could have been affected, with less diversity in ings highlight the importance of involvement and reported barriers and facilitators. Nevertheless, this commitment on all levels, dedicated leadership, inclu- issue might have been partially resolved by randomly sive and transparent communication, adequate resource allocation, training, timing, monitoring, and approaching the teams for interviews at the third time point. We used this recruitment strategy as an alterna- integration in existing structures. tive to the focus group discussions that could not be organized at that time. Although interviews with a Limitations and future research maximum of three group care workers likely produced A first limitation is the involvement of the implemen- less discussion and reduced the range of experiences tation coordinator as moderator and data analyst. On that were reported (Krueger & Casey, 2014), on the the one hand, familiarity with the service organization other side, this may have resulted in reaching group helped the moderator to better understand partici- care workers who would otherwise not have attended pants’ comments and to know when to probe for fur- a focus group discussion. ther information. On the other hand, as an ‘insider’, Another way the focus group procedure may have the moderator may have relied on implicit informa- impacted the findings is that familiarity with the mod- tion about the organization that was not checked for erator could have increased the likelihood that some accuracy (Chenail, 2011). Including more moderators participants made statements to please the interviewer into the design, especially moderators without famil- (Chenail, 2011). However, because the topic was not iarity with the service, might have compensated for particularly sensitive or personal and participation did the potential bias stemming from having the imple- not bring personal benefits, there was no obvious mentation coordinator as the single moderator. motive for socially desirable responding. Moreover, INTERNATIONAL JOURNAL OF FORENSIC MENTAL HEALTH 311 the participants in the present study were rather out- approach could be followed for the implementation of spoken about the work environment. Considering the risk management in clinical practice. For example, large number of experienced barriers to implementa- studies have shown that programs that adhere to the tion gathered over the interview sessions, it is fair to Risk-Need-Responsivity (RNR) principles are more assume that the majority of participants felt comfort- effective in reducing recidivism than programs who able voicing criticism during the discussions. do not follow these principles (Koehler et al., 2013). A third limitation is that higher and middle man- Yet, a recent systematic review (Viljoen et al., 2018) agement were not included in the study. This limits found that professionals only showed moderate adher- our understanding of the beliefs and attitudes of this ence to the risk principle and limited adherence to the group about the START:AV and what they perceive need principle when making risk management deci- as implementation determinants. Higher management sions. Thus, it might be equally relevant to extend is typically more involved with external stakeholders, implementation research to risk management strat- and from that perspective, they might have reported egies and deepen our understanding of the barriers more external influences. Thus, not having this per- and facilitators that professionals face in adhering to spective embedded in the findings is a limitation. the RNR principles. Nevertheless, the role of leadership and their potential influence on the START:AV implementation was dis- Practical implications cussed by other staff members. Lastly, the reader should take into consideration that When planning an implementation, every coordinat- the present study reflects perceptions of staff members ing committee could benefit from studying the CFIR. That way, potential barriers can be identified early in from a particular service with a particular client popula- tion, during a particular (political) time. Nevertheless, the implementation process and strategies can be parallels with previous studies (Levin et al., 2016; adopted to increase the odds of a successful imple- Muller-Isberner € et al., 2017; Nonstad & Webster, 2011; mentation. In addition, determinants and strategies Schlager, 2009; Sher & Gralton, 2014; Webster et al., should be reconsidered throughout the process as 2006) suggest that our results might be transferable to their relevance will change depending on the stage of other contexts, at least to residential treatment settings. the implementation (Damschroder et al., 2009). In To enhance applicability of risk assessment implemen- Figure 2, we listed recommendations based on sugges- tation studies, we advocate for the use of implementa- tions from staff and the experiences of the implemen- tion frameworks, such as the CFIR to allow comparison tation coordinator. This list can be complemented between settings and instruments. Moreover, future with strategies from the ‘CFIR-ERIC implementation research could work on identifying the most essential strategy matching tool’, which is freely available online determinants to implementation success and how they (https://cfirguide.org). This matching tool assists in can be facilitated. Ideally, this research would involve allocating strategies from the Expert multiple sites, to allow comparisons and identification Recommendations for Implementing Change (ERIC; of common determinants. Powell et al., 2015) compilation to the determinants Although we purposefully decided on using an of interest. inductive approach to data gathering and data coding, all codes could subsequently be linked to the CFIR. Conclusion Therefore, this comprehensive framework might pro- In the past decade, there has been growing attention vide an adequate starting point for future risk assess- to the study of implementation of risk assessment ment implementation studies, for example, when developing interview questions that explicitly prompt instruments in forensic-clinical practice. The system- for certain domains and constructs, such as the outer atic review of Viljoen et al. (2018) suggested that the use of structured risk assessment instruments does not setting or implementation climate (Kirk et al., 2015). In addition, we would like to encourage future studies yet reliably result in violence reduction. One potential to investigate relationships between determinants and explanation for this finding are the challenges faced between determinants and implementation outcomes. when implementing risk assessment with fidelity into Better understanding of what impedes and facilitates practice. To move risk assessment practice to a higher successful implementation (e.g., integration, adoption, level, it would require mental health services to adopt satisfaction) paves the way for (more) effective risk a systematic, evidence-based approach to its imple- assessment implementation strategies. A similar mentation (Haque, 2016). Increased understanding of 312 T. L. F. DE BEUF ET AL. Preparation of the Implementation - Implementation is more than providing training. Develop a stepwise implementation plan. Get inspired by the literature, for example, Vincent, Guy & Grisso (2012). - Be prepared to make a large investment at the beginning. Invest in planning, prioritizing, and communication. Provide time (i.e., money). - Identify all direct and indirect stakeholders and their influence on the implementation. Get them involved in the process, make them feel heard, make them responsible. - Consider actively involving the service user in the implementation. - Be aware of the organization’s historical and contemporary cultural background. - Have a strong formal mandate from management, as well as their informal support. - Appoint a formal coordinator whose duties continue after the implementation until sufficient integration and sustainability (as defined in the plan) is reached. - Connect with a risk assessment expert/academic to be involved as an advisor. - Be attentive to and eliminate overlapping tasks. During Implementation - Prioritize the implementation for a dedicated period. - Be open to adjust peripheral features of the tool. Be firm about its core characteristics. - Find the right balance between planning and execution: timing is crucial. - Assess and manage expectations of staff concerning the impact of risk assessment. - Accommodate users by investing in user-friendly integrative software. - Keep reiterating the relevance of risk assessment, for example by repeatedly mentioning its value in (political, clinical, managerial) conversations at all levels. - Communicate with all stakeholders at all levels at all times. - Identify influencers among users. Give a positive influencer informal leadership, give a negative influencer individual time and attention. - Look for ways to visualize the risk assessment findings to make it more tangible. - Use the change curve to keep track of where the service, a team and/or an individual staff member is at in the change process. Be prepared to loose staff in the process. - Monitor the implementation. Monitor fidelity to the instrument’s instructions. - Persevere. Give staff time to experience the new practice. - When evaluations clearly indicate that the instrument is a misfit with the setting, be receptive to these signals and be prepared to de-implement. Include thresholds for de- implementation in the implementation plan. Figure 2. Suggestions for Implementing Risk Assessment Instruments. the implementation process and the conditions that the first, probably crucial steps in reducing adverse create a foundation for successful implementation outcomes among adolescents. (e.g., adherence to the risk assessment guidelines) is necessary to optimize risk assessment. The present ORCID study provided insight into the impeding and promot- Tamara L. F. De Beuf http://orcid.org/0000-0001- ing factors of an implementation as perceived by the 5273-8523 users of a risk assessment instrument, in this case the Corine de Ruiter http://orcid.org/0000-0002-0135-9790 START:AV. The CFIR was useful in organizing and Vivienne de Vogel http://orcid.org/0000-0001-7671-1675 synthesizing complex and multi-leveled information gathered via interviews with professional users of the References instrument. The more knowledge about risk assess- ment implementation accumulates, the better the field Bhanwer, A., Shaffer, C., & Viljoen, J. L. (2016). Short-term will understand which approach works for which Assessment of Risk and Treatability: Annotated instruments in which settings. High-quality imple- Bibliography. British Columbia University, https://doi. mentation of risk assessment instruments is one of org/10.13140/RG.2.2.36232.67845 INTERNATIONAL JOURNAL OF FORENSIC MENTAL HEALTH 313 Boeije, H. (2010). Analysis in qualitative research. Sage Research. Implementation Science, 11(1), 72–84. https:// Publications. doi.org/10.1186/s13012-016-0437-z Bonta, J., & Andrews, D. A. (2017). The psychology of crim- Koehler, J. A., L€osel, F., Akoensi, T. D., & Humphreys, D. K. (2013). A systematic review and meta-analysis on inal conduct (6th ed.). Routledge. the effects of young offender treatment programs in Borum, R., Bartel, P., & Forth, A. (2006). Structured Europe. Journal of Experimental Criminology, 9(1), 19–43. Assessment of Violence Risk in Youth. (Lodewijks, H., https://doi.org/10.1007/s11292-012-9159-7 Doreleijers, T., de Ruiter, C., & de Wit-Grouls, H., Krueger, R. A., & Casey, M. A. (2014). Focus Groups. A Trans.). Psychological Assessment Resources. (Original practical guide for applied research (5th ed.). Sage work published 2006) Publications. Braun, V., & Clarke, V. (2006). Using thematic analysis in Krueger, R. A. (2002). Designing and Conducting focus group psychology. Qualitative Research in Psychology, 3(2), interviews. Eastern Illinois University. http://www.eiu. 77–101. https://doi.org/10.1191/1478088706qp063oa edu/ihec/Krueger-FocusGroupInterviews.pdf. Chenail, R. J. (2011). Interviewing the Investigator: Langan, J. (2010). Challenging assumptions about risk fac- Strategies for addressing instrumentation and researcher tors and the role of screening for violence risk in the bias concerns in qualitative research. The Qualitative field of mental health. Health, Risk & Society, 12(2), Report, 16(1), 255–262. https://nsuworks.nova.edu/tqr/ 85–100. https://doi.org/10.1080/13698571003632429 vol16/iss1/16 Levin, S. K., Nilsen, P., Bendtsen, P., & Bulow, P. (2016). Consolidated Framework for Implementation Research Structured risk assessment instruments: A systematic (n.d.). CFIR codebook. https://cfirguide.org/tools/tools- review of implementation determinants. Psychiatry, and-templates/ Psychology and Law, 23(4), 602–628. https://doi.org/10. Crocker, A. G., Garcia, A., Isra€el, M., Hindle, Y., Gagnon, 1080/13218719.2015.1084661 D., Venegas, C., Braithwaite, E., Rot, M.-E. (2008). Masadeh, M. A. (2012). Focus group: reviews and practices. Implementing and using a systematic risk assessment International Journal of Applied Science and Technology, scheme to increase patient safety on a risk management 2(10), 63–69. http://ijastnet.com/journals/Vol_2_No_10_ unit for individuals with severe mental illness: A demon- December_2012/9.pdf stration project. Canadian Safety Patient Institute. http:// Muller-Isberner, € R., Born, P., Eucker, S., & Eusterschulte, B. crocker.mcgill.ca/docpdf/final_report.pdf (2017). Implementation of evidence-based practices in Damschroder, L. J., Aron, D. C., Keith, R. E., Kirsh, S. R., forensic mental health services. In R. Roesch & A. N. Alexander, J. A., & Lowery, J. C. (2009). Fostering imple- Cook (Eds.), Handbook of forensic mental health services. mentation of health services research findings into prac- (pp. 443–469). Routledge. tice: A consolidated framework for advancing Nonstad, K., & Webster, C. D. (2011). How to fail the implementation science. Implementation Science, 4(1), implementation of a risk assessment scheme or any other 50–66. https://doi.org/10.1186/1748-5908-4-50 new procedure in your organization. American Journal of De Beuf, T. L. F., de Vogel, V., & de Ruiter, C. (2019). Orthopsychiatry, 81(1), 94–99. https://doi.org/10.1111/j. Implementing the START:AV in a Dutch residential 1939-0025.2010.01076.x youth facility: Outcomes of success. Translational Issues Patton, M. Q., Cochran, M. (2002). A guide to using qualita- in Psychological Science, 5(2), 193–205. https://doi.org/ tive research methodology. MSF Intersectional Evaluation 10.1037/tps0000193 Group. http://evaluation.msf.org/sites/evaluation/files/a_ Desmarais, S. L. (2017). Commentary: Risk assessment in guide_to_using_qualitative_research_methodology.pdf the age of evidence-based practice and policy. Peterson-Badali, M., Skilling, T., & Haqanee, Z. (2015). International Journal of Forensic Mental Health, 16(1), Implementation of risk assessment in case management 18–22. https://doi.org/10.1080/14999013.2016.1266422 for youth in the justice system. Criminal Justice and Galdas, P. (2017). Revisiting bias in qualitative research: Behavior, 42(3), 304–320. https://doi.org/10.1177/ Reflections on its relationship with funding and impact [Editorial. International Journal of Qualitative Methods, Powell, B. J., Waltz, T. J., Chinman, M. J., Damschroder, 16(1), 160940691774899. ]. https://doi.org/10.1177/ L. J., Smith, J. L., Matthieu, M. M., Proctor, E. K., & Kirchner, J. E. (2015). A refined compilation of imple- Haque, Q. (2016). Implementation of violence risk assess- mentation strategies: results from the Expert ment instruments in mental healthcare settings. In J. P. Recommendations for Implementing Change (ERIC) pro- Singh, S. Bjorkly, & S. Fazel (Eds.), International perspec- ject. Implementation Science, 10(1), 21–34. https://doi. tives on violence risk assessment. (pp. 40–52). Oxford org/10.1186/s13012-015-0209-1 University Press. https://doi.org/10.1093/acprof:oso/ Schlager, M. D. (2009). The organizational politics of imple- 9780199386291.001.0001 menting risk assessment instruments in community cor- Hilverdink, P., Daamen, W., Vink, C. (2015). Children and rections. Journal of Contemporary Criminal Justice, 25(4), youth support and care in the Netherlands. Nederlands 412–423. https://doi.org/10.1177/1043986209344555 Jeugdinstituut. http://www.youthpolicy.nl/en/Download- Schutt, R. K. (2004). Investigating the social world. The pro- NJi/Publicatie-NJi/Children-and-youth-support-and-care- cess and practice of research (7th ed.). Sage Publications. in-The-Netherlands.pdf Sher, M. A., & Gralton, E. (2014). Implementation of the Kirk, M. A., Kelley, C., Yankey, N., Birken, S. A., Abadie, START:AV in a secure adolescent service. The Journal of B., & Damschroder, L. (2015). A systematic review of the Forensic Practice, 16(3), 184–193. https://doi.org/10.1108/ use of the Consolidated Framework for Implementation JFP-04-2013-0021 314 T. L. F. DE BEUF ET AL. Software, N. C. H. (2016). Express Scribe Transcription soft- Viljoen, J. L., Nicholls, T. L., Cruise, K. R., Desmarais, S. L., ware (Version 5.84) [Computer Software]. NCH & Webster, C. D. (2016). Short-Term Assessment of Risk Software. https://www.nch.com.au/scribe/versions.html and Treatability: Adolescent Version (START:AV) – User Tran, N. (2019, September 16). State of the field: Guide. (T. L. F. De Beuf, C. de Ruiter, & V. de Vogel, Implementation research panel. [Scientific Panel]. Global trans.). Mental Health, Law, and Policy Institute, Simon Implementation Conference. Fraser University. (Original work published 2014) Vaismoradi, M., Turunen, H., & Bondas, T. (2013). Content Vincent, G. M., Guy, L. S., & Grisso, T. (2012). Risk analysis and thematic analysis: Implications for conduct- Assessment in juvenile justice: A guidebook for implemen- ing a qualitative descriptive study. Nursing & Health tation. John D. & Catherine T. MacArthur Foundation. Sciences, 15(3), 398–405. https://doi.org/10.1111/nhs. http://modelsforchange.net/publications/346 Vincent, G. M., Guy, L. S., Perrault, R. T., & Gershenson, Vermaes, I., Konijn, C., Jambroes, T., & Nijhof, K. (2014). B. (2016). Risk assessment matters, but only when imple- Statische en dynamische kenmerken van jeugdigen in mented well: A multisite study in juvenile probation. Law JeugdzorgPlus: Een systematische review [Static and dynamic characteristics of youth in secured residential and Human Behavior, 40(6), 683–696. https://doi.org/10. care: A systematic review. Orthopedagogiek: Onderzoek en 1037/lhb0000214 Praktijk, 53(6), 278–292. ]. Vincent, G. A., Paiva-Salisbury, M. L., Cook, N. E., Guy, Viljoen, J. L., Beneteau, J. L., Gulbransen, E., Brodersen, E., L. S., & Perrault, R. T. (2012). Impact of risk/need assess- Desmarais, S. L., Nicholls, T. L., & Cruise, K. R. (2012). ment on juvenile probation officers’ decision-making: Assessment of multiple risk outcomes, strengths, and Importance of implementation. Psychology, Public Policy change with the START:AV: A short-term prospective & the Law, 18(4), 549–576. https://doi.org/10.1037/ study with adolescent offenders. International Journal of a0027186 Forensic Mental Health, 11(3), 165–180. https://doi.org/ Webster, C. D., Nicholls, T. L., Martin, M.-L., Desmarais, 10.1080/14999013.2012.737407 S. L., & Brink, J. H. (2006). Short-Term Assessment of Viljoen, J. L., Cochrane, D. M., & Jonnson, M. R. (2018). Risk and Treatability (START): the case of a new struc- Do risk assessment tools help manage and reduce risk of tured professional judgement scheme. Behavioral Sciences reoffending? A systematic review. Law and Human & the Law, 24(6), 747–766. https://doi.org/10.1002/bsl. Behavior, 42(3), 181–214. https://doi.org/10.1037% 2Flhb0000280 https://doi.org/10.1037/lhb0000280 737

Journal

International Journal of Forensic Mental HealthTaylor & Francis

Published: Jul 2, 2020

Keywords: START:AV; implementation determinants; risk assessment; consolidated framework for implementation research; focus groups

There are no references for this article.