National Center of Juvenile Justice Study: Consistency, Fidelity, Oversight
In October 2012, the National Center of Juvenile Justice (NCJJ), completed a evaluability study on George Junior Republic. The purpose of the assessment was to determine the degree to which George Junior Republic is prepared to be evaluated. Researchers from the NCJJ reviewed program documentation, conducted site visits, interviewed key staff, and convened focus groups to assess:
The information gained through these activities allowed NCJJ researchers to develop a more responsive evaluation plan by acquiring firsthand knowledge of the organizations program model, specific programs and activities, operations, and data processing capacities.
- The strength of the George Junior Republic program model
- Fidelity of implementation of juvenile treatment
- The strength of the connections between the needs of residents and George Junior Republic programming
- The availability and accessibility of program outcome data
- Adequacy of the sample size
- Administrative support for evaluating the juvenile treatment program
A Model of Consistency
The following is a summary of the NCJJ Evaluability Assessment (October, 2012):
“George Junior Republic’s program model, as designed, is implemented with a great deal of fidelity to the model. The George Junior Republic mission is clearly articulated and broadly disseminated. The policies, procedures, and roles and responsibilities of all George Junior Republic employees are clearly described in a carefully constructed operations manual that is updated regularly. All new employees receive an extensive orientation that addresses the George Junior Republic philosophy, goals, activities and programs, and expected outcomes.
“On site observations, interviews, and focus groups indicate that the George Junior Republic program is implemented as designed with a high degree of uniformity and consistency and with a great deal of active oversight up and down throughout the organization. It is very clear that George Junior Republic is operating as it is designed to operate and with a great deal of consistency.”
The assessment indicated that George Junior Republic is a strong candidate for an outcomes evaluation. A second phase was recommended which would include an evaluation of intermediate outcomes to demonstrate accomplishment of program goals and objectives.
Second Phase: Program Outcomes Evaluation
This outcome evaluation phase of the project began in spring 2014 and will be conducted over a two year period. The purpose of the research is to conduct a comprehensive evaluation of George Junior Republic’s processes, outcomes, and impacts to document and demonstrate the strength of the program design, the integrity of program operations, the efficacy of programming for residents, and ultimately the impact of George Junior Republic on program participants.
NCJJ will conduct an outcome evaluation of youth assigned to George Junior Republic. The study will measure individual level outcomes representing the accomplishment of program and juvenile treatment objectives: average length of stay, successful completion of competency-based or treatment interventions, participation in recreational activities, completion of community service, payment of restitution, behavior and progress in school, progress in motivation system, changes in assessed risk status, and status at release from George Junior Republic. The study will also track results of behavioral/attitudinal assessments completed at intake, periodically while in the program, and at discharge.