Directions Program Evaluation Literature Review and Methodology

You can read a downloadable version of the article: Directions Program Evaluation Literature Review and Methodology

Abstract

The Crisis Residential Center’s (CRC) Directions Program Evaluation for the 2006-2008 Financial Grant Years plans to look into the effectiveness of the aforementioned program. To do this, the existing literature regarding youth similar to those found at the CRC is examined. Previous research has studied at-risk youth and residential centers. This study seeks to add upon this data by looking specifically at a short term residential center, and it’s programming from multiple perspectives. To perform the effectiveness study of the Directions Program both qualitative and quantitative methods are used.

Introduction

According to the U.S. Conference of Mayors (2007), as cited by the National Coalition for the Homeless (2008), youth who are not living with their families make up 1% of the total urban homeless population. The U.S. Department of Justice’s Office of Juvenile Justice and Delinquency Prevention found there was about 1,682,900 youth who ran away or were forced out of their homes (Flores, 2002; Molino, 2007; Hammer, Finkelhor & Sedlak, 2002; National Coalition for the Homeless, 2008). Approximately over 1.5 million of these youth have been in danger while they were away from home due to physical or substance abuse. Unfortunately, caretakers only reported less than a quarter of these youth to the police or to an agency which helps find missing children. These youth were comprised of 50% male and 50% female. Most (57%) were White, non-Hispanic (Hammer et. al., 2002).

Reasons for Runaway and Homeless Youth

Youth runaway and become homeless for several reasons. Many come from homes where they are not feeling supported or nurtured. These homes are often single-parent households or broken homes (Kidd, 2006; Miller, Donahue, Este & Hofer, 2004). According to a study, which looked at more than 600 adolescents who were homeless or runaways, their families were often abusive or neglectful toward the youth. In addition, there could be parental substance abuse and issues with the law (Whitbeck & Hoyt, 1999). For example, many of the youth reported one or more of their parents had a substance abuse issue, 56% with alcohol and 15% with marijuana (Whitbeck & Hoyt, 1999). Youth who had a biological parent who abused alcohol or drugs were more likely to sell them (Whitbeck & Hoyt, 1999).

Also, many homeless youth have issues with school such as attendance. The study by Miller et. al. (2004), which interviewed 19 youth, found most dropped out of school before they were able to graduate. Many felt “while life on the street was not easy, it was preferable to the life they had left” (Miller et. al., 2004, p. 740).

Additionally, homeless youth face barriers which keep them homeless. The street youth typically lack in education which may limit their ability to get a job or at least earn enough money to support themselves. The youth in the Miller et. al. (2004) study stated fear hindered their ability to attempt to combat homelessness partly due to negative previous experiences they had utilizing services.

Consequences

There are many consequences for youth living on the street. According to Whitbeck and Hoyt (1999), support networks are incredibly important for adolescents and influence their behavior. By running away, these adolescents severed the ties with any positive social networks. Thus, they formed relationships with individuals who may not be positive influences. While on the street there is an increased chance adolescents will be exposed to violence including being sexually assaulted from those individuals they interact with (Whitbeck & Hoyt, 1999).

Adolescents who are homeless face developing depression, conduct disorder, and physical health issues (Robertson, 1989). In the study by Whitbeck and Hoyt (1999), approximately 18% of the youth stated they had a health concern which they wanted to see a doctor about. However, youth on the streets generally do not access health care. One reason for this is they typically do not see helping professionals as being effective at meeting their needs.

Practices for Working with Homeless

Programs which work with homeless and at-risk youth have a number of practices they can implement. These include things such as individual, group, and family therapy. They also often provide referrals to community services. The environment is a factor and often these programs employ a therapeutic milieu community.

Silvan, Matzner, and Silva (1999) consider the milieu community a major part of therapeutic programs. As a component of treatment, it can be used along with individual, group, and family therapy. This milieu community can also help solidify skills learned in treatment and provide a consistent structure. It provides support, encouragement, empathy, and nurtures those involved as well as offers a way to teach skills. This milieu community can also foster a peer identity. This affects the attitudes and values of the youth involved in the community.

Within the milieu therapeutic community, members of a therapy group are bonded together by a common interest (Toseland & Rivas, 2005). Zimet and Farley (1985) state groups provide the client with interpersonal experiences to help foster social skills (as cited in Silvan, Matzner, & Silva, 1999, p. 469). In residential treatment programs, groups are led by trained facilitators who teach youth and their families skills to foster independence, communication, and family reunification. They also provide opportunities for clients to help each other in learning the skills (Miley, O’Melia, & DuBois, 2004). Group therapy functions well in the milieu environment as these skills and the group cohesion are reinforced throughout the day.

In addition to providing a conducive environment for group therapy, the milieu environment fosters therapeutic relationships with staff. Both staff and clients feel this positive relationship is a key component of successful treatment. Therefore, the development of this relationship is important in the treatment process (Nebbitt, House, Thompson, & Pollio, 2007).

Apart from the milieu environment and the relationships and skill building it enhances, another key treatment is family therapy (Silvan, Matzner, & Silva, 1999). Family issues have been identified as being a main precipitating problem which leads to youth running away (Safyer, Thompson, Macio, Zittel-Palamara, & Forehand, 2004). Thus, it is interesting to note studies have found successful family reunification upon discharge has led to improvements on various aspects such as school, runaway behaviors, and family stability (Thompson, Pollio, & Bitner, 2000; Teare, Furst, Peterson, & Authier, 1992).

In addition to runaway behaviors, at-risk youth often face behavioral issues such as drug use. Research has found family interaction and reunification improved such behaviors (Schmidt, Liddle, and Dakof, 1996; Huey, Henggeler, Brondino, & Pickrel, 2000). Sunseri (2004) adds children with high-functioning families have greater improvements in behavior after treatment.

Family involvement in treatment also reduces the risk of recidivism. A study by Lakin, Brambila, & Sigda (2004) found family therapy attendance correlated with less recidivism and readmission at a children’s residential center. Johnson, Farquhar, and Sussman (1996) and Shane (1999) describe involvement in family therapy as a part of treatment, which leads to positive outcomes (as cited in Thompson, Kost, & Pollio, 2003, p. 302). This includes a decrease of problem behaviors, a reduction in the likelihood and need for readmission, and an increase in family functioning (Lakin, Brambila, & Sigda, 2004). Thus, it is agreed among many researchers family therapy is key to achieving positive outcomes.

Referrals to other services also lead to positive outcomes. Clients and families are referred to other agencies or services to foster continued improvements to the client’s individualized treatment plan. These referrals may be to agencies which offer services such as family therapy, rehabilitation services, or mental health treatment. Nebbitt, et al. (2007) found participation in referrals after discharge facilitated the successful reunification of the family. This highlights the need for continued intervention.

While there have been studies focusing on family relationships and the needs of at-risk and runaway youth, few recent studies have focused on residential treatment programs which serve this population. Even fewer studies researched the effect of the programs from the clients’ perspective. Thus, this study builds on previous research by adding a component, which looks into the perception of the clients.

Agency Information

Youth Family Adult (YFA) Connections is a private independent 501(c) (3) nonprofit agency who serves the greater Spokane area with counseling and substance abuse treatment programs. YFA Connections, originally established as the Youth Help Organization in 1969, focused on working with Spokane’s youth and their involvement in illegal and addictive drugs. Since then, the agency has come to offer a wide range of counseling and treatment services to a diverse group of clients. These clients need housing, family interventions, and substance abuse treatment. In 1997 the name was changed to the current YFA Connections to better describe the diverse clients served (YFA Connections, n.d., About us).

One program offered through YFA Connections is Spokane’s Regional Crisis Residential Center (CRC). The CRC assists youth 13-17 years-old who come from a variety situations (YFA Connections, n.d.a, Youth and family services; Christensen, 2008). Youth who stay at the CRC receive individual, group, and family sessions tailored to their personal needs. While these needs are different for each client, there are some commonalities. For example, many clients are runaways and/or homeless or their family is in crisis. They often lack skills such as emotional regulation, basic living, and interpersonal skills. In addition, family relationships need strengthening. Furthermore, these youth and families are often not linked to community services and are in need of help in planning for a healthy future (YFA Connections, n.d.a, Youth and family services).

To meet these needs the CRC has three different programs: Directions, State CRC, and HOPE (Homeless Youth Prevention / Protection and Education). Each of these programs has different legislation governing them, programmatic focuses, and duration of stay. While these three programs have many similarities, they each have a different focus. Furthermore, the structure of each program effects how each of the other programs are implemented. The various interventions and requirements of each of these programs are made and the experience of the youth living at the CRC is similar regardless of the specific program they are involved with.

State CRC Program

The State CRC Program is funded and regulated by Washington State through the Revised Code of Washington (RCW, Crisis residential centers [CRCs], 1995; CRCs, 1998; CRCs, 2000a; CRCs, 200b; CRCs, 1979). The RCW places specific expectations on agencies funded as Crisis Residential Centers. They are required to give documentation back to the Department of Social and Health Services (DSHS) regarding the clients. This includes the number, age, sex, referral source, precipitating event, services provided, ultimate dispositions, and length of stay (CRCs, 1979).

A number of services are required to be a part of the CRC. These include things such as multidisciplinary team meetings where professionals from various fields meet to assess the needs of the clients and to determine interventions. Furthermore, clients have the option to see mental health professionals when the need arises (CRCs, 2000a). The staff at the CRC are trained to effectively counsel the youth admitted to their program. They provide treatment, supervision, and structure (CRCs, 1998). Youth may stay for only five days (CRCs, 1995).

State HOPE Program

The HOPE Program is funded by Washington State through the HOPE act (Short title, 1999). Similar to the State CRC Program, the HOPE Program is given its rules and funding through the RCW (HOPE centers, 1999a, HOPE centers, 1999b; HOPE centers, 1999c; HOPE centers, 1999d; HOPE centers, 2008; Short title, 1999). Youth must either be a street youth or participating in increasingly risky behavior (HOPE centers, 2008). For example, risky behaviors may include drug abuse or reckless sexual activity.

Youth stay in the HOPE Program for 30 days. They often transfer to the HOPE Program from the State CRC Program when a suitable placement is not to be found in the allotted time. These youth often have failed foster care placements, have few resources given by DSHS, and often live dangerously. Clients are connected with education, counseling, and self-development programs to help them become more self-sufficient (Seattle post-intelligencer editorial board, 1999).

Federal Directions Program

The Federal Basic Center Grant funds the Directions Program. The Basic Center Grants’ stated purpose is strengthening community-based programs addressing the immediate needs of runaway and homeless youth and their families. These centers provide youth with emergency shelter, food, clothing, counseling, and referrals for health care. The grant funds three-year periods and allows the organization to reapply (U.S. Department of Health and Human Services [HHS], 2008). This grant’s financial year operates from October 1 to September 30. The current grant runs from 2006 to 2009. The Directions Program has been receiving the Basic Center Grant since 1986 with only a few exceptions (T. Wright, personal communication, November 3, 2008).

The Directions program has specific service requirements to be able to receive the Basic Center Grant. The HHS described these requirements and the U.S. Code puts them into law. To serve their clients they are required to; A) provide temporary shelter, B) have an individualized intake process with the proper paperwork, C) engage in case disposition, D) employ individual, group and family counseling, E) offer recreational activities, F) link clients to services in the community, and G) make aftercare services available (HSS, 2007; Basic center grant program 2006).

Agency Mission Related to the Research Project

YFA’s stated mission is to “provide prevention, intervention, education and treatment services to youth, families, and adults experiencing conflict or crisis” (YFA Connections, n.d., About us). Their vision statement is “to create an open and safe environment, devoted to enhancing community wellness through active partnerships with families, communities and agencies” (YFA Connections, n.d., About us). Although these statements are broader than just the Directions Program, there is a definite connection between the agency mission and the research project.

The research project explores how effective the Directions Program is at facilitating lasting change and meeting the needs of its clients during the current grant years. It also examines the clients’ perspectives about the impact of the CRC. Finally, part of the effectiveness study looks at what outside resources clients have used after discharging. This helps the CRC know what can be improved to be better able to reach the diverse needs of their clients in the upcoming 2009 year.

This research seeks to promote change for the Directions Program and add to the knowledge of other similar programs in Washington State and around the country. The research and recommendations will help the CRC in their mission of providing services to their clients. Because the Regional CRCs and HOPE Centers run their programs based on the same legislation and expected practices, the research findings will be applicable overall. This is also true with Basic Centers around the country.

The Directions Program, State CRC, and HOPE programs all require accountability to the various government organizations and the YFA Connections board of directors. To keep up with these accountability requirements, the CRC does programmatic evaluations, however not to the scale of this research project. Reports regarding the various programs are sent to their oversight committees (i.e. federal or state). These reports give minimal data regarding some of the activities of the program, such as how many youth are served and the various referral sources are given. However, they do not compile in-depth information that could be insightful to a program evaluation. They also do not include any reports on former clients’ and their legal guardians’ perspective regarding the program. This research project attempts to add the missing pieces of the previous effectiveness studies.

Methodology

Objectives of Research

This study (see appendix A for attached timeline) seeks to determine the effectiveness of the CRC’s Directions Program in facilitating lasting change and meeting the needs of its clients during the current financial grant years. The objective of this research project is to determine the impact the Directions Program has on its clients and their families. This will help to identify specific procedures and interventions needed to improve at mezzo and macro level. Furthermore, it is to give a voice to former clients and their families about their experience at the CRC.

An understanding of the client’s history will be obtained by examining their files. Data regarding all former clients for the 2006-2008 financial grant years will be collected in regards to demographics, programmatic statistics, client history, and the initial Global Appraisal of Individual Needs – Short Screener (GAIN-SS) score. The legal guardians’ perspective will be extrapolated through mail-in surveys. An in-depth interview and a mail-in survey will give an understanding of the effectiveness of the program through the client’s perspective. Finally, a pretest-posttest method of the GAIN-SS shows change over time in the participant’s lives. The study will use both qualitative and quantitative approaches to reach this means.

Hypotheses and Operational Definitions

The research question guiding this project seeks to determine the effectiveness of the CRC’s Directions Program in facilitating lasting change and meeting the needs of its clients during the financial grant years for 2006-2008. Four hypotheses are used to explore this research question.

Hypothesis 1: Positive Impact in Former Clients Life

The Directions Program had a positive impact on the clients who completed and participated in the program. Completion and participation in the Directions Program will be defined as the client being discharged with their advocate’s approval. A lower GAIN-SS score and the clients’ self-report in the surveys and in-depth interviews demonstrate the positive impact.

Hypothesis II: Positive Impact of Family Relationships

Completing the Directions Program has a positive impact on the client’s relationship with their families. Completing the Directions Program is defined the same as in hypothesis I. The improved relationships with the family will be defined by the self-report of the legal guardians and clients in the surveys and in-depth interviews.

Hypothesis III: Referrals and Family Relationships

The use of one or more referrals is positively correlated to lasting improvement in the client’s family relationships. The participants’ self-report on the surveys will define the use of referrals. The participants report on their lasting improvement via the surveys and in-depth interviews.

Hypothesis IV: Referrals and Client’s Lives

The use of one or more referrals is positively correlated to lasting improvement in the client’s lives. The participants self-report on the surveys will define the use of referrals. The participants report on their lasting improvement via the surveys, in-depth interviews, and a lower GAIN-SS score.

Research Participants

The participants include former clients of the CRC’s Directions Program as well as their legal guardians. There will be approximately 260 participants, 127 of which are former clients’ with ages ranging from 13-19 and 127 adults, which are their primary caregivers. The participants will be all the former clients who went through the CRC’s Directions Program between October 1, 2006, and September 30, 2008.

Data Collection Methods and Procedures

There will be five methods of data collection utilized in this research project. First, client files will be studied and pertinent information extrapolated so quantitative background data can be gathered on the former clients. This background data will include demographics, programmatic statistics, client histories, and pretest GAIN-SS scores. Second, a survey will be sent (see Appendix B) to the legal guardians asking both closed-ended Likert scale questions as well as some open-ended questions. Questions cover topics such as satisfaction with the Directions Program, referrals used and their impressions, and affect on family relationships. Third, a mail-in survey with former Directions Program clients (see Appendix C) asks both closed-ended Likert scale questions as well as some open-ended questions. The questions cover topics such as perceptions of the programs interventions, family relationships, and relationships with staff. Furthermore, the GAIN-SS questions have been adapted from the form distributed by Chestnut Health Systems to be given in the mail-in survey. In addition, the researchers will conduct a more in-depth, qualitative, informal interview (see Appendix D).

Measurement

One of the assessment tools the CRC utilizes is the GAIN-SS. The GAIN-SS is a shortened version of the Global Appraisal of Individual Needs (GAIN). There are three main uses for the GAIN-SS; 1) to quickly identify clients who would benefit from further work with a Licensed Mental Health Practitioner, 2) to be usable by staff with minimal training, and 3) as a periodic measure of change over time (Chestnut Health Systems, n.d.; Dennis, Chan, & Funk, 2006).

The GAIN-SS identifies issues regarding four areas: internalizing disorders, externalizing disorders, substance disorders, and crime and violence. Dennis, Chan, and Funk (2006) found the GAINS-SS appeared to be capable of quickly identifying people who might have a problem in the four areas of interest.

The GAIN-SS will be used to show change over time regarding former clients. All clients in the Directions Program are given the GAIN-SS upon entrance into the CRC. The questions from the GAIN-SS have been adapted and added to the former client mail-in survey. This will allow the researchers to compare the levels of change regarding the four areas the GAIN-SS looks at.

Reference

Basic center grant program, 42 USC, § 5712 (2006, January 2)

Chestnut Health Systems (n.d.). Global appraisal of individual needs - short screener (gain-ss). Retrieved October 28, 2008, from http://www.chestnut.org/li/gain/GAIN_SS/index.html

Christensen, H. P. (2008, April 28) Building a life: Nonprofit helped transform runaway. Spokesman Review. From http://www.spokesmanreview.com/ourkids/stories/?ID=242429

Crisis residential centers, 74 RCW § 13.032 (1998)

Crisis residential centers, 74 RCW § 13.0321 (1995)

Crisis residential centers, 74 RCW § 13.033 (2000a)

Crisis residential centers, 74 RCW § 13.034 (2000b)

Crisis residential centers, 74 RCW § 13.035 (1979)

Dennis, M., Chan, Y., & Funk, R. (2006, November). Development and validation of the GAIN Short Screener (GSS) for internalizing, externalizing and substance use disorders and crime/violence problems among adolescents and adults [Electronic version]. The American Journal on Addictions, 15, 80-91. Retrieved October 25, 2008, doi:10.1080/10550490601006055

Hammer, H., Finkelhor, D., & Sedlak, A. J. (2002). Runaway/thrownaway children: National estimates and characteristics. Retrieved from : http://missingkids.com/en_US/documents/nismart2_runaway.pdf

HOPE centers, 74 RCW § 15.220 (1999a)

HOPE centers, 74 RCW § 15.225 (2008)

HOPE centers, 74 RCW § 15.250 (1999b)

HOPE centers, 74 RCW § 15.260 (1999c)

HOPE centers, 74 RCW § 15.270 (1999d)

Huey, S. J., Jr., Henggeler, S. W., Brondino, M. J., & Pickrel, S. G. (2000). Mechanisms of change in multisystemic therapy: Reducing delinquent behavior through therapist adherence and improved family and peer functioning [Electronic Version]. Journal of Consulting and Clinical Psychology, 68(3), 451-467.

Kidd SA (2006) Factors precipitating suicidality among homeless youth: A quantitative follow-up. Youth and Society 37:393–422

Lakin, B. L., Brambila, A. D., & Sigda, K. B. (2004). Parental involvement as a factor in the readmission to a residential treatment center [Electronic version]. Residential Treatment for Children & Youth, 22(2), 37-52.

Miley, K. K., O'Melia, M., & DuBois, B. (2004). Generalist Social Work Practice: An empowering approach (4th ed.). Boston: Pearson Education, Inc.

Miller, P., Donahue, P., Este, D., & Hofer, M. (2004). Experiences of being homeless or at risk of being homeless among canadian youths [Electronic version]. Adolescence, 39(156), pp. 735-755.

National Coalition for the Homeless, (2008). Homeless youth [Fact sheet]. Retrieved from http://www.nationalhomeless.org/publications/facts/youth.pdf

Nebbitt, V. E., House, L. E., Thompson, S. J., & Pollio, D. E. (2007) Successful transitions of runaway/homeless youth from shelter care [Electronic version]. Journal of Child & Family Studies, 16, 545-555

Responsible living skills program, 74 RCW § 15.240 (2008)

Safyer, A. W., Thompson, S. J., Maccio, E. M., Zittel-Palamara, M., & Forehand, G. (2004). Adolescents' and parents' perceptions of runaway behavior: Problems and solutions [Electronic Version]. Child and Adolescent social Work Journal, 21(5), 495-512.

Schmidt, S. E., Liddle, H. A., & Dakof, G. A. (1996). Changes in parenting practices and adolescent drug abuse during multidimensional family therapy [Electronic Version]. Journal of Family Psychology 10(1), 12-27

Seattle post-intelligencer editorial board (1999, April 4) Act would be a big help to area's homeless youths. Seattle Post Intelligencer. From http://seattlepi.nwsource.com/opinion/hacted.shtml

Short title, 74 RCW § 15.220 (1999)

Silvan, M., Matzner, F, & Silva, R. (1999). A model for adolescent day treatment [Electronic version]. Bulletin of the Menninger Clinic, 63(4), 459-480

Sunseri, P. A. (2004). Family functioning and residential treatment outcomes [Electronic version]. Residential Treatment for Children & Youth, 22(1), 33-53.

Teare, J. F., Furst, D. W., Peterson, R. W., & Authier, K. (1992). Family reunification following shelter placement: Child, family, and program correlates [Electronic Version]. American Journal of Orthopsychiatry, 62(I), 142-146.

Thompson, S. J., Kost, K. A., & Pollio, D. E. (2003). Examining risk factors associated with family reunification for runaway youth: Does ethnicity matter? [Electronic version], Family Relations, 52, 296-304

Thompson, S. J., Pollio, D. E., & Bitner, L. (2000). Outcomes for adolescents using runaway and homeless youth services [Electronic Version]. Journal of Human Behavior in the Social Environment, 3(1), 79-97.

Toseland, R. W. & Rivas, R. F. (2005). An introduction to group work practice (5th ed.). Boston: Pearson Education, Inc.

U.S. Department of Health and Human Services. (2007, September 14) Basic center program performance standards. Retrieved October 28, 2008 from http://www.acf.hhs.gov/programs/fysb/content/youthdivision/programs/performancestandards.htm

U.S. Department of Health and Human Services. (2008, January 30) Fact sheet: Basic center program. Retrieved October 28, 2008 from http://www.acf.hhs.gov/programs/fysb/content/youthdivision/programs/bcpfactsheet.htm

Whitbeck, L. B., & Hoyt, D. R. (1999). Nowhere to grow: Homeless and runaway adolescents and their families. New York: Aldine De Gruyter.

Youth Family Adult Connections (n.d.) About us. Retrieved October 28, 2008, from http://yfaconnections.org/about.htm

Youth Family Adult Connections (n.d.a ) Youth and family services. Retrieved October 28, 2008 from http://yfaconnections.org/youthfam.htm