View tables in pdf format: Program Overview Table and Effectiveness Summary Table
PROGRAM OVERVIEW TABLE
To assist in the reader's search, the Program Overview Table summarizes general information about each reviewed program. A brief description of each section appears below.
|
• |
Grade level indicates the appropriate grades in which the program is intended. For non-school-based interventions, age-grade equivalencies may be estimated as follows: PreK, three to four years of age; Kindergarten, five to six; 1st grade, six to seven; 2nd grade, seven to eight; 3rd grade, eight to nine; and so on. |
|
• |
Prevention level reflects the Institute of Medicine's classification system. Universal programs are intended for the general population within a specified age range to prevent the onset of one or more risk behaviors; Selective programs are intended for individuals with characteristics or experiences that put them at risk for engaging in one or more risk behaviors; Indicated programs are intended for individuals who have already engaged in one or more risk behaviors. |
|
• |
Target behaviors indicates which behaviors the program developers intended to affect: tobacco use, alcohol or other drug use, life skills, and/or violence/aggression. |
|
• |
Program Setting provides information about both the location and the intervention techniques: classroom programs are generally curricular prevention approaches; school-wide programs are intended to affect school climate, school policy, or both; home/family programs either occur in the home or require the involvement of one or more adult guardians; community programs take place in community-based organizations or other agencies and may include the formation of a local policy making or advisory group; and "other" settings indicates that the program has been implemented in such locations as emergency rooms and detention or mental health facilities. |
EFFECTIVENESS SUMMARY TABLE
The Effectiveness Summary Table provides a snapshot of the research and findings. Full evaluation reviews should be consulted before a school district or community seriously considers adopting or adapting a program. The columns to the right of each program name fall into three categories: research design, effects, and generalizability. The entries in these three sections are briefly explained below.
Research Design: Certain features are critical to ensure what researchers call the internal validity of a research design. A design with internal validity gives us confidence that whatever effects have been found can in fact be attributed to the intervention and not to other events or factors. In general, the interventions that have been tested with the best evaluation designs will have all of the following indicators checked:
|
• |
Evaluation design. There has been at least one evaluation with random assignment of subjects (or schools) to treatment and control conditions, or a matched comparison group. Pre-post quasi-experimental designs with matched comparison groups were considered adequate. Adequate or better research design is critical to determine whether any effects are, in fact, attributable to the intervention. |
|
• |
Sample size. There is an adequate sample size in treatment conditions given the selected outcome analyses. In general, the Violence Institute examined whether sample selection methods and sample sizes enabled generalization beyond the particular evaluation sample. In addition, Violence Institute staff examined whether the sample size was adequate to the impact analyses conducted by the evaluators. |
|
• |
Measurement. Outcome measures appear to be reliable and valid. This box is checked when previously published measures of youth behaviors, attitudes and/or norms were used or when the evaluators explicitly cite positive results of psychometric analyses (e.g., Cronbach's alpha = 0.70) |
|
• |
Sample retention. Sample attrition was minimal (= 20% at one year, = 30% at 18-month follow-ups or later); not differential between treatment conditions; or attrition-related biases were detected and corrected statistically during impact analyses. |
Effects: These indicators show whether the intervention had a positive, statistically significant impact on young people. We indicate whether an impact was found on drug use or violence/aggression, or on other factors associated with drug use or violence (e.g., norms, attitudes, cognitions). We also indicate whether effects were sustained one year or more beyond the end of the intervention. (An n/a entry in the table denotes that these effects were not tested.) For the sake of uniformity, we applied the p = 0.05 standard to assess statistical significance in all cases. In some cases, failure to achieve statistical significance may occur because the evaluation sample size was too small. In other cases, findings may be statistically significant but not educationally or clinically meaningful. We recommend that readers exercise caution in interpreting results and weigh their decision to adopt a program on more than statistical significance.
Generalizability: Certain features are critical to ensure what researchers call the external validity of a research design. A design with good external validity gives us confidence that the results of the research can be generalized beyond the study sample.
|
• |
Evaluation settings. We checked all of the settings in which evaluation was conducted. |
|
• |
Sample diversity. We checked all of the racial/ethnic groups that comprised at least 20 percent of the evaluation sample. |
|
• |
Replication. The program was replicated (and evaluated) in at least one additional site: full or modified replication (i.e., intervention was revised based on earlier evaluation findings). |