ED 602

Statistical Research for Behavioral Sciences

Brian G. Smith, Ph.D.
Lesson - 13

You may also check your understanding of the material on the Wadsworth web site. Click on the Publisher Help Site button.

 

Homework - Lesson 13

Any student may may do the assignments from any area. You may run through this work an unlimited number of times. If you make errors, you will be referred to the appropriate area of the book for re-study.

  

.

 

Assessment - Lesson 13

You will have two options to take the quiz. If you fail to achieve 100% on the quiz, you will not able to advance to the next lesson. After failing on the second take, please email the instructor at ed602@mnstate.edu so remedial action can be taken.
Homework and Quizzes are on Desire 2 Learn. Click on the Desire 2 Learn link, log in, select the Homework/Quizzes icon and choose the appropriate homework or quiz.

Assignments and Information
 
Reading: Chapter 13
   Definition Page: Contains definitions arranged alphabetically.

 

In our last three lessons we discussed setting up statistical tests in a variety of situations:

  • If we are making inferences about a single score we can use the z-score test.
  • If we are making inferences about a single sample we can use
    • the z-test (if we know the population variance)
    • The single sample t-test (if we do not know the population variance)
  • If we are making inferences about two samples we can use
    • The independent t-test (if the two samples are independent of one another)
    • The dependent t-test (if the two samples are related to one another as in matched samples or a pre-test post-test situation).

When we wish to look at differences among three or more sample means, we use a statistical test called analysis of variance or ANOVA. Analysis of variance yields a statistic, f, which indicates if there is a significant difference among three or more sample means. When conducting an analysis of variance, we divide the variance (or spread-outness) of the scores into two components.

  • The variance between groups, that is the variability among the three or more group means.
  • The variance within the groups, or how the individual scores within each group vary around the mean of the group.

We measure these variances by calculating SSA, the sum of squares between groups, and SSError, the sum of squares within groups. (As a way of checking your math, you can calculate SStotal which should be the same as the sum of SSA + SSError.)

Each of these sums of squares is divided by its degrees of freedom, (dftotal or the degrees of freedom for the grand mean, dfA, degrees of freedom between, and dferror, degrees of freedom within) to calculate the mean square between groups, MSA, and the mean square within groups, MSerror.

Finally we calculate f, the f-ratio, which is the ratio of the mean square between groups to the mean square within groups. We then test the significance of f using the tables on pages 492 and 493 to complete our analysis of variance.

ANOVA

  • A test of significance
  • Uses a series of f distributions
  • The f distributions are close to normal, but have a slight positive skew
  • f is always positive
  • If the null hypothesis is true, f will be close to 1.00
  • Most often uses an alpha of at least 0.05
  • Tells you if there is a difference between all groups, multiple comparison tests are needed to find significant differences between any two specific means.

Assumptions

  • Random sampling
  • Normal distribution of scores
  • The variances of scores in the population are equal

Advantages of ANOVA

  • More powerful than a t test if there are more than two levels of the independent variable
  • Reduces work for numerous levels (5 levels = 1 test rather than 10 tests)
  • Tests the null hypothesis with minimal error if there are more than two levels of the independent variable

Multiple Comparison Tests

  • Tests of significance
  • Makes pairwise comparisons
  • Finds which pairs of samples differ significantly from each other in one-factor multilevel designs

The Tukey HSD Test

  • Used for Post Hoc comparisons after a significant difference has been found with an ANOVA test on more than two groups.
  • Used to make pairwise comparisons
  • Provides a critical difference (CD) that is used as the minimum difference between two treatment means for them to be significantly different
  • Uses the formula

Now we can apply this information to our BASC data. We will start with the two sets of scores from lesson 12, the BASC scores for the Hispanic youth and the BASC scores for the non-Hispanic youth. I have put the scores into a table that simplifies our math for us.

Hisp.RawScores Dev. Within Sqd.Dev.Within Betw.Grp. Dev. Betw.Grp.Sq.Dev. Grand Dev. Sqrd.GrandDev.
25
 -26.4
696.96
 0.42
0.1764
 -25.98
674.9604
32
 -19.4
376.36
 0.42
0.1764
 -18.98
360.2404
37
 -14.4
207.36
 0.42
0.1764
 -13.98
195.4404
40
 -11.4
129.96
 0.42
0.1764
 -10.98
120.5604
41
 -10.4
108.16
 0.42
0.1764
 -9.98
99.6004
44
 -7.4
54.76
 0.42
0.1764
 -6.98
48.7204
45
 -6.4
40.96
 0.42
0.1764
 -5.98
35.7604
48
 -3.4
11.56
 0.42
0.1764
 -2.98
8.8804
49
 -2.4
5.76
 0.42
0.1764
 -1.98
3.9204
50
 -1.4
1.96
 0.42
0.1764
 -0.98
0.9604
51
 -0.4
0.16
 0.42
0.1764
 0.02
0.0004
51
 -0.4
0.16
 0.42
0.1764
 0.02
0.0004
52
 0.6
0.36
 0.42
0.1764
 1.02
1.0404
52
 0.6
0.36
 0.42
0.1764
 1.02
1.0404
52
 0.6
0.36
 0.42
0.1764
 1.02
1.0404
55
 3.6
12.96
 0.42
0.1764
 4.02
16.1604
55
 3.6
12.96
 0.42
0.1764
 4.02
16.1604
57
 5.6
31.36
 0.42
0.1764
 6.02
36.2404
59
 7.6
57.76
 0.42
0.1764
 8.02
64.3204
60
 8.6
73.96
 0.42
0.1764
 9.02
81.3604
62
 10.6
112.36
 0.42
0.1764
 11.02
121.4404
63
 11.6
134.56
 0.42
0.1764
 12.02
144.4804
65
 13.6
184.96
 0.42
0.1764
 14.02
196.5604
67
 15.6
243.36
 0.42
0.1764
 16.02
256.6404
73
 21.6
466.56
 0.42
0.1764
 22.02
484.8804
Grp. Mean 51.4
           
             
Non-Hisp. Scores Dev. Within Sqd. Dev. Within Between Grp. Dev. Betw. Grp. Sq. Dev. Grand Dev. Sqrd. Grand Dev.
29
 -21.56
464.8336
 -0.42
0.1764
 -21.98
483.1204
34
 -16.56
274.2336
 -0.42
0.1764
 -16.98
288.3204
35
 -15.56
242.1136
 -0.42
0.1764
 -15.98
255.3604
40
 -10.56
111.5136
 -0.42
0.1764
 -10.98
120.5604
42
 -8.56
73.2736
 -0.42
0.1764
 -8.98
80.6404
43
 -7.56
57.1536
 -0.42
0.1764
 -7.98
63.6804
44
 -6.56
43.0336
 -0.42
0.1764
 -6.98
48.7204
45
 -5.56
30.9136
 -0.42
0.1764
 -5.98
35.7604
48
 -2.56
6.5536
 -0.42
0.1764
 -2.98
8.8804
49
 -1.56
2.4336
 -0.42
0.1764
 -1.98
3.9204
49
 -1.56
2.4336
 -0.42
0.1764
 -1.98
3.9204
50
 -0.56
0.3136
 -0.42
0.1764
 -0.98
0.9604
50
 -0.56
0.3136
 -0.42
0.1764
 -0.98
0.9604
51
 0.44
0.1936
 -0.42
0.1764
 0.02
0.0004
51
 0.44
0.1936
 -0.42
0.1764
 0.02
0.0004
52
 1.44
2.0736
 -0.42
0.1764
 1.02
1.0404
53
 2.44
5.9536
 -0.42
0.1764
 2.02
4.0804
57
 6.44
41.4736
 -0.42
0.1764
 6.02
36.2404
58
 7.44
55.3536
 -0.42
0.1764
 7.02
49.2804
59
 8.44
71.2336
 -0.42
0.1764
 8.02
64.3204
61
 10.44
108.9936
 -0.42
0.1764
 10.02
100.4004
62
 11.44
130.8736
 -0.42
0.1764
 11.02
121.4404
62
 11.44
130.8736
 -0.42
0.1764
 11.02
121.4404
69
 18.44
340.0336
 -0.42
0.1764
 18.02
324.7204
71
 20.44
417.7936
 -0.42
0.1764
 20.02
400.8004
Grp. Mean 50.56  
5580.16
  
8.82
  
5588.98
Grand M 50.98            

Our mean square between the groups would be

MSA = SSA/dfA = 8.82/1 = 8.82

And our mean square within the groups would be

MSError = SSError/dfError = 5580.16/48 = 116.2533

So our f would be

F = MSA/MSError = 8.82/116.2533 = 0.07586881

which we would round to 0.076. Looking at the table on page 492, we find the column for 1 degree of freedom in the numerator, and then we go down the side to 48 degrees of freedom in the denominator. We would need an f of at least 4. 04 to be significant at an alpha of 0.05, so we retain the null hypothesis, there is no significant difference between the scores on the BASC for these two groups.

Our study has only two groups, and we didn’t find a significant difference between those groups with the ANOVA test, so we don’t need to run a multiple comparison test. However, I will show you the math for the Tukey HSD test anyway, just so you can see how the formula works. On page 494 we find the q for comparing 2 means with 48 degrees of freedom. It is about 2.85. We get the MSError from earlier in the notes, 116.2533. and nA is the number of subjects in each group, 25.

 

This tells us that we would need a difference of at least 6.15 between the means of our two groups for the difference to be significant at an alpha of 0.05. Our difference is only 0.84, so we retain the null hypothesis.

SPSS Tips:
 

 

 

 

Vocabulary

One-factor multilevel design – An experiment with one independent variable and three or more levels of that independent variable.

Analysis of variance (ANOVA) – A statistical test used to analyze multilevel designs.

Grand mean – The mean of all scores in an experiment.

Mean square - The name used for a variance in the analysis of variance.

Within-groups error variance – The variance of the scores in a group calculated about that group mean.

Between groups variance – The variance calculated using the variation of the group means about the grand mean.

a – The number of levels of a factor A.

nA – Number of scores in a level of a one-factor design

Multiple comparison test – Statistical tests used to make pairwise comparisons to find which means differ significantly from one another in a one factor multi-level design.

Pairwise comparisons – Statistical comparisons involving two means.

Error rate in an experiment – The probability of making at least one Type I error in the statistical comparisons conducted in an experiment.

Post hoc comparisons – Statistical tests that make all possible pairwise comparisons after a statistically significant Fobs has occurred for the overall analysis of variance.

Critical difference – The minimum numerical difference between two treatment means that is statistically significant.