Effect Size and Clinical/Practical Significance
Statistical significance only tells the researcher how likely it is that an observed finding could have occurred by chance. It does not say anything about magnitude of the effect observed. Effect size is a name given to a group of statistics that measure the magnitude of a treatment effect. In many cases, effect size is a better measure of research outcomes than the significance level. This is because with large samples, one can observe statistically significant group differences even when only a tiny effect is present. Unlike significance tests, effect size indices are independent of sample size.
Effect Size Statistics. The most commonly used effect size estimate is Cohen's d. Cohen's d is computed by dividing the mean difference between groups by the pooled standard deviation. Cohen's d can also be computed from the value of the t-test of the differences between groups means. An alternative measure of effect size is the point-biserial correlation between the dichotomous independent variable and the continuous dependent variable. This coefficient is called the effect size correlation or effect size r.
Both Cohen's d and the effect size r can be easily computed if one has the needed information (either the group means and standard deviations or the t-test values for the difference between groups. See the University of Colorado - Colorado Springs Effect Size Calculators at: http://web.uccs.edu/lbecker/Psy590/escalc3.htm
In applying his concept (and while noting the risk of providing broadly applicable guidelines), Cohen originally proposed the following interpretation:
|
Cohen's d |
Interpretation |
|
0.2 |
Small |
|
0.5 |
meduim |
|
0.8 |
Large |
Cohen later developed more precise guidelines for interpreting effect size. As indicated in the 3rd column of the following table, an effect size can be thought of as the average percentile standing of the 'treated' subjects compared to the controls. Accordingly, an effect size of 0.0 indicates that the treatment mean is located at the 50th percentile of the control group distribution; an effect size of 0.8 would place the treatment group mean at the 79th percentile of the control group; and an effect size of 1.7 means that the mean of the treated group is positioned at about the 95th percentile of the control group.
Effect sizes can also be interpreted as the percent of nonoverlap of the treated group's distribution with that of the control group. As see in column four of the table, an effect size of 0.0 indicates that the distribution for the treated group overlaps completely with that of the control group, i.e., there is 0% of nonoverlap. An effect size of 0.8 indicates a nonoverlap of 47.4% in the distributions, whereas an effect size of 1.7 indicates a nonoverlap of 75.4% in the distributions.
|
Original Standard |
Effect Size |
Percentile Standing |
Percent of Nonoverlap |
|
|
2.0 |
97.7 |
81.1% |
|
|
1.9 |
97.1 |
79.4% |
|
|
1.8 |
96.4 |
77.4% |
|
|
1.7 |
95.5 |
75.4% |
|
|
1.6 |
94.5 |
73.1% |
|
|
1.5 |
93.3 |
70.7% |
|
|
1.4 |
91.9 |
68.1% |
|
|
1.3 |
90 |
65.3% |
|
|
1.2 |
88 |
62.2% |
|
|
1.1 |
86 |
58.9% |
|
|
1.0 |
84 |
55.4% |
|
|
0.9 |
82 |
51.6% |
|
Large |
0.8 |
79 |
47.4% |
|
|
0.7 |
76 |
43.0% |
|
|
0.6 |
73 |
38.2% |
|
Medium |
0.5 |
69 |
33.0% |
|
|
0.4 |
66 |
27.4% |
|
|
0.3 |
62 |
21.3% |
|
Small |
0.2 |
58 |
14.7% |
|
|
0.1 |
54 |
7.7% |
|
|
0.0 |
50 |
0% |
Table adapted from Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Hillsdale, NJ: Lawrence Earlbaum Associates.