Permalink: ‎

This paper explores the cross-national challenges of integrating evidence-based interventions into existing services for high-resource- using children and youth. Using several North American model programme exemplars that have demonstrated efficacy, the paper explores multiple challenges confronting policy-makers, evaluation researchers and practitioners who seek to enhance outcomes for troubled children and youth and improve overall service effective- ness. The paper concludes with practical implications for youth and family professionals, researchers, service agencies and policy–makers, with particular emphasis on possibilities for cross-national collaboration.

Correspondence: James K. Whittaker, School of Social Work, University of Washington, 4101 Fifteenth Avenue NE, Seattle, WA 98105-6299, USA E-mail: jimw@u.washington.edu

Keywords: children in need (services for), evidence-based practice, research in practice, therapeutic social work

Accepted for publication: January 2009

INTRODUCTION

Across many national boundaries and within multiple service contexts – juvenile justice, child mental and child welfare – there is a growing concern about a proportionately small number of multiply challenged children and youth who consume a disproportionate share of service resources, professional time and public attention.While accurate, empirically validated popu- lation estimates and descriptions remain elusive. The consensus of many international youth and family researchers, including those reported by McAuley and Davis (2009) (UK), Pecora et al. (2009a) (US) and Egelund and Lausten (2009) (Denmark) in this present volume seems to be that some combination of externalizing, ‘acting-out’ behaviour, problems with substance abuse, identified and often untreated mental

health problems, experience with trauma and challeng- ing familial and neighbourhood factors are often, and in various combinations, manifest in the population of children and youth most challenging to serve. Many of these find their way into intensive out-of-home care services, and Thoburn (2007) provides a useful window into the out-of-home care status of children in 14 countries and offers useful observations on improvements in collecting administrative data for child and family services to inform both policy and practice. Others call for a critical re-examination of the present status of ‘placement’ as a central fulcrum in child and family services policy and practice (Whittaker & Maluccio 2002).

A sense of urgency is conveyed by the fact that many child and youth clients of ‘deep-end’, restrictive (out-of-home) services disproportionately represent underserved and often socially excluded families and communities of colour, and pose additional challenges in service planning around the cultural compatibility of proffered interventions (Blasé & Fixsen 2003; Barbarin et al. 2004; Miranda et al. 2005). Important work in this area includes ethnic and cultural

Author note: Portions of this paper in earlier form were presented by the author at the 8th and 10th annual EUSARF International Conferences at the University of Leuven, Belgium, 9–11 April 2003 and the University of Padova, Italy, 26–29 March 2008.

doi:10.1111/j.1365-2206.2009.00621.x

166 Child and Family Social Work 2009, 14, pp 166–177 © 2009 Blackwell Publishing Ltd

variations on known effective practices. Lau (2006), for example, offers a nuanced and sensitive treatment of actual and potential adaptations in existing parent training models. A basic concern with questions of equity and social justice, coupled with a growing scep- ticism about the efficacy of traditional residential, ‘place-based’ services, has heightened the search for more preventive, family- and community-based, cul- turally congruent service alternatives. All of this is set against a backdrop of concern about the state’s ability to provide effective parenting oversight and support for children in care, as well as those who remain with their families (Bullock et al. 2006). Fortunately, this search is occurring at a time when researchers in many countries are shedding light on mechanisms of risk and resilience (Sameroff & Gutman 2004), change processes involved in effective interventions (Biehal 2008) and the challenges faced by parents in multiply stressed environments (Ghate & Hazel 2002; Ghate et al. 2008) that are rich in their potential for contri- butions to intervention design and evidence-informed practice.

The primary purpose of this paper is to examine some of the challenges and opportunities in incorpo- rating evidence-based strategies and interventions into existing service systems to better meet the needs of high-resource-using children and youth. The growing corpus of empirical research on promising treatment strategies offers, if not clear-cut prescrip- tions, then rich implications for future policy initia- tives and service experiments.

Indeed, the pursuit of evidence-based practice, in its many forms, increasingly attracts the attention of those who plan, deliver and evaluate critical treatment and rehabilitative services for vulnerable children and their families across national boundaries and regions. While definitions of ‘evidence-based practice’ empha- size different dimensions of that construct, the common themes of bringing ‘science-to-service’, and its reciprocal ‘service-to-science’, are increasingly evident in the child, youth and family services systems in many European countries and North America, as well as elsewhere. Simultaneously, reform efforts in the USA and many European countries press for community-based, family-oriented, non-residential alternatives to traditional residential care and treat- ment programmes for acting-out children and youth with identified mental health problems (Chamberlain 2003; Weisz & Gray 2008). However, the impulse for service reform and the availability of at least some empirically validated model interventions do not of themselves constitute a sufficient basis for system

reform, but instead serve to illuminate some of the many fault lines that exist in the child and family services field:

• The continuing tensions between ‘front-end’, pre- ventive services and ‘deep-end’ highly intensive treatment services and the unhelpful dichotomies these tend to create and perpetuate

• The tensions between a widely shared desire to adopt more evidence-based practices and the genu- inely felt resistances to these, particularly when they are used in a rigid fashion that requires strict adher- ence to established protocols with little opportunity for experimentation, customization or practitioner discretion. For example, as one family support researcher recently observed, we need much more fine-grained analyses of the actual lived experience of client families with the services offered to them (S. P. Kemp 2008, personal communication). Such analyses will almost certainly involved a ‘mixed- methods’ approach using qualitative measures and methods to augment quantitative studies

• The tension, as manifested in North America and elsewhere between evidence-based and culturally competent practices, reflects, among other things, antagonism towards certain practice strategies based on perceptions of the under-representation of ethnic minorities in the study samples on which certain models have been validated As model programmes proliferate and are increas-

ingly removed from the particular political and cul- tural niches within which they were developed, we would do well to heed the cautions offered by Munro et al. (2005) that researchers, planners and youth and family practitioners are at a moment in time when cross-national perspectives are critical in helping iden- tify new ways of both framing problems and shaping service solutions. Cross-national dialogue can help in identifying different formats for collecting, analysing and utilizing routinely gathered client information, analysing subtle local adaptations of internationally recognized evidence-based services and examining the effects of differing policy contexts on service outcomes.

THE QUEST FOR MORE EFFECTIVE INTERVENTIONS

For the remainder of this paper, I wish to do three things: (1) briefly identify where we are in our search for effective (evidence-based) interventions; (2) assess how we are doing in increasing their availability to high-resource-using troubled youth and their families;

Evidence-based intervention for high-risk youth J K Whittaker

167 Child and Family Social Work 2009, 14, pp 166–177 © 2009 Blackwell Publishing Ltd

and (3) identify some particular challenges faced by the individual practitioner, the social agency and the public policy context in furthering the shared goal of improving outcomes, and thus life prospects for troubled children. The author’s bias will soon be readily apparent. First, as one who has spent a lifetime trying to bring both the precision of research methods and the richness of research findings to the ‘shop floor’ of children’s agency practice, I am convinced that the evidence-based practice movement will not succeed until it is embraced by those closest to the children: the child and youth care workers, the social workers, teachers, family support workers and others who, with parents, toil on the front lines of helping. This is not in my view a one-way street – Science-to- Service – but presumes a vital feedback loop from Service-to-Science where the insights and hypotheses of those most directly involved in interventions (including parent and child consumers) inform and improve successive generations of applied research studies. Second, I readily acknowledge the North American bias apparent in many of my examples – I write of what I know best – while recognizing a deeply felt need in my country for European and other cross- national perspectives if we are ever to achieve success with our internal efforts at improving outcomes.

The search for evidence-based practices with chil- dren and families is now well underway on both sides of the Atlantic. Kazdin and Weisz (2003), Weisz (2004), Burns and Hoagwood (2002), Macdonald (2001), Pecora et al. (2009b) and McAuley et al. (2006) survey effective interventions in child welfare and child mental health services, as well as review current research on service populations that will inform the creation of novel interventions.

The simple, nominal definition of evidence-based practice offered by Professor Geraldine MacDonald of Queen’s University in Belfast provides a useful start- ing point:

Evidence-based practice indicates an approach to decision-

making which is transparent, accountable and based on careful

consideration of the most compelling evidence we have about

the effects of particular interventions on the welfare of indi-

viduals, groups and communities. (MacDonald 2001, p. xviii)

It is clear that debates about what constitutes the sufficiency and quality of evidence – where to set the bar for rigour, how to distinguish evidence-based vs. evidence-informed practice – continue apace both in academic and practitioner discourse even as the evidence-based practice movement as a whole contin- ues to raise its profile in policy and services. These

competing definitions and nuances are, in toto, a sign of health as they simply serve to underscore one or another aspect of what is emerging as a more fulsome understanding of what evidence-based practice con- sists of. These aspects include, but are not limited to:

• a dual focus on aetiology and outcomes • the incorporation of ethics and values as key com-

ponents

• the development of a collaborative process with affected client groups

• a commitment to transparency in processes and accountability Many practitioners and practice researchers have

participated in the work of international groups such as the Campbell and Cochrane Collaborations (Littell 2008) – originating in the health field – that attempt to sift, sort and categorize the state of the evidence around particular illnesses, socio-behavioural problems or social welfare concerns. Many have also experienced – closer to home – the increasing impact of national, state and regional initiatives designed to increase the content of proven, efficacious practices into child, youth and family service systems. Such initiatives typically use two strategies, often in combination:

Positive Reinforcement: e.g. ‘Laying Flowers Along Certain

Pathways’ by encouraging adoption of selected efficacious

model interventions. (One notes in passing that ‘efficacy’ of a

given intervention often increases in proportion to the dis-

tance from its country of origin!)

Coercion: e.g. Penalizing a programme, agency or practitioner

whose interventions do not reflect a sufficient quantity of

evidence-based practice according to an agreed-upon time

schedule. In the USA, this typically means that a practitioner

or service agency follows a prescribed protocol for interven-

tion or risks losing reimbursement for services rendered.

MOVING FROM ‘EFFICACY-TO-EFFECTIVENESS ’

In the USA at the moment, there is growing respect for the complexities involved in moving from pilot demonstrations of effective child, youth and family interventions to broad-scale application: i.e. moving from ‘efficacy’ to ‘effectiveness’ (Jensen et al. 2005; Weisz & Gray 2008). What these terms signify are: 1. That individual investigators can demonstrate sig- nificant results for novel treatments over standard (or traditional) services through carefully controlled, rig- orously conducted studies often including random- ized controlled trials: the ‘gold standard’ of clinical research. That is, they can demonstrate efficacy.

Evidence-based intervention for high-risk youth J K Whittaker

168 Child and Family Social Work 2009, 14, pp 166–177 © 2009 Blackwell Publishing Ltd

2. Yet, these impressive results do not, on close examination, appear to influence what might be thought of as routine, day-to-day practice as con- ducted in more familiar agency settings. Thus, the evidence-based practice movement, while demon- strating efficacy, cannot as yet demonstrate overall effectiveness.

What explains this disconnect? Lisbeth Schorr, an astute analyst of child and family services innovation, sums it up succinctly: ‘Successful programs’, she says, ‘do not contain the seeds of their own replication’ (Schorr 1993, quoted in Fixsen et al. 2005).

Thus, if we are truly interested in effectiveness – i.e. achieving wide-scale adoption of proven efficacious interventions, we need to look beyond efficacy studies: (1) to those contextual elements that influence prac- tice decisions and client outcomes (Kemp et al. 1997); and (2) to a different kind of research undertaking that focuses directly on the processes involved in suc- cessful adoption of proven efficacious interventions (Weisz & Gray 2008).

John Weisz, one of the nation’s leading research analysts in child mental health and a professor of psychology at Harvard University as well as President of the Judge Baker Children’s Center in Boston, points the way forward on what is needed to ultimately resolve the efficacy/effectiveness challenge:

A very important focus for the next stage of research on

interventions for children will be the effective implementation

of evidence-based practices by practitioners in service settings.

This will require an active collaboration between the research-

ers who develop and test interventions and the clinical, child

welfare, and education professionals who serve children and

families. (J.R. Weisz 2008, personal communication)

EXPLORING THE LANDSCAPE OF EVIDENCE-BASED SERVICES FOR HIGH-RISK YOUTH

Let us proceed, then, by exploring the context within which evidence-based services are nested. Here, we find some common and proximate elements familiar to all who labour in the child and family services field, as well as a few more distal forces that, nonetheless, have a potential for considerable impact on the identification, validation and eventual integration of evidence-based practices. I will refer, briefly, to more or less typical examples from within the US context.

Model intervention programmes

For purposes of illustration, I offer three interventions that have received considerable attention in children’s

mental health services in the USA, and which have been the objects of numerous community replications and research study both in North America and else- where (Whittaker 2005). These include:

• Multisystemic Therapy (MST), developed principally by Dr Scott Henggeler, a psychologist now at the Department of Psychiatry and Behavioral Sciences, Medical University of South Carolina (Henggeler et al. 1998; Schoenwald & Rowland 2002; Henggeler&Lee 2003).http://www.mstservices.com

• Treatment Foster Care (MTFC), developed in several clinical/research teams in the USA and represented here by the model (Multi-dimensional Treatment Foster Care) principally developed by Dr Patricia Chamberlain and colleagues at the Oregon Social Learning Center – a highly influential applied behaviour analysis developmental research centre – one of whose founding members is Dr Gerald Patterson (Chamberlain & Reid 1998; Chamberlain 2002, 2003). http://www.MTFC.com

• Wraparound Treatment, a novel, team-oriented, community-centred intervention developed by a variety of individuals including the late Dr John Burchard, formerly Professor of Clinical Psychology at the University of Vermont, John Van Den Berg, Carl Dennis and others beginning in the early 1980s (Burns & Goldman 1999; Burchard et al. 2002). http://www.rtc.pdx.edu/ PDF/PhaseActivWAProcess.pdf [While space does not permit in depth analysis here,

the interested reader is directed to the previously cited references, as well as to the web sites for each of these three models that include multiple references to com- pleted and in-progress research and demonstration efforts, as well as specifics on programme principles and components. A variation of the of the MTFC model designed for younger children in regular foster care is described in this present volume by Price et al. (2009)].

These three interventions are specifically designed to provide alternative pathways for children who otherwise would be headed into more costly and restrictive residential provision. Dr Barbara Burns, Professor of Psychology at Duke University in North Carolina and a principal author of the children’s mental health section of our latest Surgeon General’s Report on Mental Health (US Department of Health and Human Services 1999) provides a succinct ratio- nale for why this is warranted:

The most critical question for the future is, what will it take

to convince payers, public and private, to support the

Evidence-based intervention for high-risk youth J K Whittaker

169 Child and Family Social Work 2009, 14, pp 166–177 © 2009 Blackwell Publishing Ltd

interventions that are backed up by evidence about improved

outcomes? Assuming that the pool of dollars available for

mental health treatment will not increase, it will be necessary

to shift resources away from institutional care (which lacks

evidence of effectiveness) toward community alternatives.

This will require a reduction in funds allocated to institu-

tional care, where a significant portion of the child mental

health money is still being spent. (Burns & Hoagwood 2002,

p. 13)

While reviews of residential care in both the UK (Sinclair 2006) and the USA (Whittaker 2006) confirm a move away from residential services, recent comparative international contributions have urged critical re-examination of the multiple varieties of residential service (Courtney & Ivaniec 2009) to meet the needs of at least some high-resource- using youth. In part, this sentiment reflects the fact that theory and model development, particularly in the arena of intensive residential services has lan- guished as development of comparable family- centred services has flourished. Some have urged the development of a conceptual schema for intensive services – e.g. the ‘prosthetic environment’ – which transects more traditional residential, family and community boundaries is strengths-oriented and incorporates educational, socialization and family support services along with intensive treatment (Whittaker 2005).

In focusing here on a few programme models spe- cifically designed to serve as alternatives to residen- tial care and treatment, and other forms of intensive out-of-home service, one must acknowledge omis- sion of a great deal of promising, empirically based work that is presently being done with a wide range of family-, school- and community-centred interven- tions that is both more preventive in its focus and appropriate for a much wider population of children and families than space allows us to examine here. See, for example, Carolyn Webster Stratton’s Incred- ible Years Program (Beauchaine et al. 2005) and the work of many others whose contributions in such areas as family support illuminates a segment of ser- vices more preventive in focus (Kemp et al. 2005; Lightburn & Sessions 2006) and the contribution of Jackson et al. (2009).

What, then, are the similarities and differences of these three promising interventions? A recent review (Burns & Hoagwood 2002) yields the following: 1. All three interventions adhere to ‘systems of care’ values: The ‘systems of care’ framework derives from both our National Institute of Mental Health and

private foundation initiatives in the 1980s, and is defined as:

A comprehensive spectrum of mental health and other neces-

sary services which are organized into a coordinated network

to meet the multiple and changing needs of children and

adolescents with severe emotional disturbances and their

families. (Stroul & Friedman 1986, p. xx)

The system of care thus defined is based on three main ele-

ments. First, the mental health service system efforts are

driven by the needs and preferences of the child & family and

are addressed by a strengths-based approach. Second, the

locus and management of services occur within a multi-

agency collaborative environment grounded in a strong com-

munity base. Third, the services offered, the agencies

participating and programs generated are responsive to cul-

tural context and characteristics. [Though, as noted, this

remains a contested area with respect to some communities of

color.] (Burns & Hoagwood 2002, p. 19)

2. All three interventions are delivered in a commu- nity – home, school, neighbourhood – context as opposed to an office 3. All have operated in multiple service sectors: mental health, juvenile justice, child welfare 4. All were developed and evaluated in ‘real world’ community settings, thus enhancing external validity 5. All show preference for the model treatment con- dition in multiple randomized controlled trials 6. All lay claim to being less expensive to provide than institutional care (Burns & Hoagwood 2002, p. 7).

Differences of course exist. For example, both MST and MTFC possess a higher degree of specificity with respect to intervention components than does wrap- around. As of this writing, MST has perhaps the strongest evidentiary base, particularly in clinical trials showing positive effects, though some recent reviews, including one by Prof. Julia Littell of Bryn Mawr University in Pennsylvania conducted for the Camp- bell Collaboration, have raised critical questions about the evidence base offered in support of MST (Littell 2005, 2008). Finally, from a staffing perspective, MST appears to make higher use of master’s-level-trained professionals in service delivery than either MTFC or wraparound.

To these three model programmes, we must of course add numerous other evidence-based treatment techniques targeted to specific conditions and prob- lems, as reflected in recent reviews by Kazdin and Weisz (2003), Weisz (2004) and Chorpita et al. (2007).These model intervention programmes do not of course exist in a vacuum, but both influence and are influenced by a host of other elements in a typical state or regional context in the USA.

Evidence-based intervention for high-risk youth J K Whittaker

170 Child and Family Social Work 2009, 14, pp 166–177 © 2009 Blackwell Publishing Ltd

PUBLIC, VOLUNTARY AND PROPRIETARY SERVICE PROVIDERS

Model programmes such as MST, MTFC and wrap- around are typically adopted by some segment of the mixed system of service agencies (Public/Voluntary/ Proprietary) that make up the delivery system in a given state, county or municipality. Public service pro- viders are typically service funders as well, creating in the view of some voluntary agencies an unequal influ- ence in terms of what particular models are selected for adoption, as well as on the masking of true admin- istrative costs of programme implementation, given the public sector’s economies of scale and presumed ability to mask start-up costs. Given the wide varia- tions in state and county service systems within the USA, there are some anecdotal reports of the ten- dency of certain model programmes to bend and shape themselves into a widely varying array of funding arrangements (referred to as ‘pretzelling’) in order to gain a foothold and a leverage in a given public system (K. Blasé 2007, personal communica- tion) with the result that local service providers may be held to similar outcome and process standards while enjoying widely varying reimbursements to support their efforts.

NATIONAL, REGIONAL AND LOCAL RESEARCH CENTERS AND RESOURCE NETWORKS

In addition to evidence-based programme models that typically have their own internal capacity for pro- gramme development, marketing, training, evaluation and dissemination, a wide variety of university and institute-based resource networks and research centres play an increasingly important role in the promotion of evidence-based programmes and practices. For example, the National Implementation Research Network (NIRN) was begun at the University of South Florida as part of a larger effort to bring science-based information to the forefront of child mental health practice. Recently relocated to the University of North Carolina, NIRN has done significant work in documenting national, state and regional capacity to support model programme development, and has provided consultation to individual states and organi- zations on effective strategies for integrating evidence- based practices into the fabric of existing services (Fixsen et al. 2005). For more information, see: http:// www.fpg.unc.edu/~NIRN/. The California Evidence- Based Clearinghouse for Child Welfare Practice is

funded by the California Department of Social Ser- vices, Office of Child Abuse Prevention and guided by a state advisory committee and a National Scientific Panel. The Clearinghouse provides guidance on selected evidence-based practices in simple straightfor- ward formats, reducing the consumer’s need to conduct literature searches, review extensive literature or understand and critique research methodology (http://www.cachildwelfareclearinghouse.org/). The Clearinghouse has developed a six-tiered schema for sorting out promising programmes ranging from ‘Well-Supported – Effective Practice’ to ‘Concerning Practice’ (e.g. shows negative effects on clients and/or potential for harm).

A legislatively generated state institute, the Washington State Institute on Public Policy (WSIP) was created by the Washington state legislature to conduct cost/benefit and a range of other studies on a variety of classes of intervention, including child welfare and early intervention (http://www.wsipp. wa.gov/board.asp). Its generally thorough and well- executed analyses have achieved wide dissemination beyond the region and are frequently cited by model programme developers as confirmation of their effec- tiveness. Methodological concerns have recently been raised about the general quality of intervention research reviews (Littell 2005, 2008), including those generated by WSIP, and within local practice commu- nities, one hears anecdotally some concerns about the potential for overly concrete inferences by legislative bodies and funding sources whose attention may extend only to the executive summary section of detailed reviews of model programmes and not to the caveats and nuances contained in their appendices and footnotes.

Beyond these particular exemplars, there are a wide variety of government-, university- and institute- based research centres and clearinghouses devoted to the identification, review, evaluation and promotion of evidence-based practices. Such centres are not typically coordinated, resulting oftentimes in an over- load of information for busy practitioners desirous of identifying the most appropriate interventions for troubled youth and their families. The problem is intensified as estimates place the number of docu- mented treatments for children and adolescents in excess of 500 (Kazdin 2000). Here, the work of Dr Bruce Chorpita at the University of Hawaii offers at least a partial solution. For a number of years, Chor- pita’s research team has been refining a ‘common elements’ approach to identified evidence-based treatments and then matching these with identified