The General Linear Models Approach To MANOVA
*-----------------------------------------------------*
| glm.sas -- General linear models approach to MANOVA |
| Data from A. Anderson, Oregon State University. |
| Four different response variables are measured. |
| The hypothesis that we want to test is that sex |
| does not affect any of the four responses. |
*-----------------------------------------------------*;
Options PS=55 LS=90 PageNo=1 NoDate
FORMCHAR='|----|+|---+=|-/\<>*';
title1 'Skull Measurements by Sex -- A. Anderson';
data skull;
input sex $ length basilar zygomat postorb @@;
sex=upcase(sex);
Datalines;
m 6460 4962 3286 1100 m 6252 4773 3239 1061 m 5772 4480 3200 1097
m 6264 4806 3179 1054 m 6622 5113 3365 1071 m 6656 5100 3326 1012
m 6441 4918 3153 1061 m 6281 4821 3133 1071 m 6606 5060 3227 1064
m 6573 4977 3392 1110 m 6563 5025 3234 1090 m 6552 5086 3292 1010
m 6535 4939 3261 1065 m 6573 4962 3320 1091 m 6537 4990 3309 1059
m 6302 4761 3204 1135 m 6449 4921 3256 1068 m 6481 4887 3233 1124
m 6368 4824 3258 1130 m 6372 4844 3306 1137 m 6592 5007 3284 1148
m 6229 4746 3257 1153 m 6391 4834 3244 1169 m 6560 4981 3341 1038
m 6787 5181 3334 1104 m 6384 4834 3195 1064 m 6282 4757 3180 1179
m 6340 4791 3300 1110 m 6394 4879 3272 1241 m 6153 4557 3214 1039
m 6348 4886 3160 991 m 6534 4990 3310 1028 m 6509 4951 3282 1104
f 6287 4845 3218 996 f 6583 4992 3300 1107 f 6518 5023 3246 1035
f 6432 4790 3249 1117 f 6450 4888 3259 1060 f 6379 4844 3266 1115
f 6424 4855 3322 1065 f 6615 5088 3280 1179 f 6760 5206 3337 1219
f 6521 5011 3208 989 f 6416 4889 3200 1001 f 6511 4910 3230 1100
f 6540 4997 3320 1078 f 6780 5259 3358 1174 f 6336 4781 3165 1126
f 6472 4954 3125 1178 f 6476 4896 3148 1066 f 6276 4709 3150 1134
f 6693 5177 3236 1131 f 6328 4792 3214 1018 f 6661 5104 3395 1141
f 6266 4721 3257 1031 f 6660 5146 3374 1069 f 6624 5032 3384 1154
f 6331 4819 3278 1008 f 6298 4683 3270 1150
;
title2 'MANOVA';
proc iml;
reset nolog;
*-----------------------------------------------------------*
| Compute MANOVA using Sums of Squares Methods |
*-----------------------------------------------------------*;
use skull;
read all var{length basilar zygomat postorb}
where(sex='F') into Y1;
read all var{length basilar zygomat postorb}
where(sex='M') into Y2;
close skull;
n1=nrow(Y1); n2=nrow(Y2);
p=ncol(Y1); g=2; /* g=number of groups */
one1=J(n1,1,1); one2=J(n2,1,1);
/* compute means and variances for each group */
ybar1=Y1`*one1/n1;
ybar2=Y2`*one2/n2;
ystar=Y1-one1*ybar1`;
s1=ystar`*ystar/(n1-1);
ystar=Y2-one2*ybar2`;
s2=ystar`*ystar/(n2-1);
/* compute overall mean */
ybar=(n1*ybar1+n2*ybar2)/(n1+n2);
free ystar one1 one2;
print 'Manova using Sums of Squares Approach: ',;
print n1 n2 p,ybar1 ' ' ybar2 ' ' ybar,s1 s2,;
/* compute between sums of squares and cross products */
B=n1*(ybar1-ybar)*(ybar1-ybar)`+
n2*(ybar2-ybar)*(ybar2-ybar)`;
/* compute within sums of squares and cross products */
W=(n1-1)*s1+(n2-1)*s2;
WilkLamb=det(W)/det(B+W);
/* since g=2, lambda can be converted to an F */
num_df=p;
den_df=n1+n2-p-1;
F=den_df/num_df*((1-WilkLamb)/WilkLamb);
pvalue=1-probF(F,num_df,den_df);
print B, W, WilkLamb f num_df den_df pvalue,,;
free /; /* clear and release all matrices */
*-----------------------------------------------------------*
| Compute MANOVA using General Linear Models Approach |
*-----------------------------------------------------------*;
use skull;
/* Read in the Dependent Variables */
read all var{length basilar zygomat postorb} into Y;
/* Read in the Classes or Grouping Variable(s) */
read all var{sex} into class;
n=nrow(Y); p=ncol(Y);
/* construct design matrix (Independent Vars) */
/* consists of intercept and class dummy variables */
X=J(n,1,1)||design(Class);
/* print out the data and corresponding design matrix */
print 'General Linear Models Procedure Approach: ',;
print 'Data (Y) and Design Matrix (X): ';
print Y Class[format=3.0] X[format=3.0],;
/* number of treatment levels, the first is the intercept */
g=ncol(X)-1;
XpX=X`*X;
XpY=X`*Y;
YpY=Y`*Y;
B=Ginv(XpX)*(XpY); /* Find least-squares solution */
/* B[1,] corresponds to overall mean (intercept) */
/* B[2,] corresponds to treatment effect 1 */
/* B[3,] corresponds to treatment effect 2 */
/* Compute l, the contrast of the Beta's of interest */
l={0,1,-1}; /* T1 vs T2 */
/* Test if Treatment effect is zero */
/* First compute Hypothesis (H) and Error (E) SS */
H=(l`*B)`*inv(l`*Ginv(XpX)*l)*(l`*B);
E=YpY-B`*(XpX)*B;
/* One Test Statistic is Wilk's Lambda */
WilkLamb=det(E)/det(E+H);
Print n ' ' p ' ' g, xpx, xpy, ypy, B, H, E, WilkLamb,;
/* Now Compute Partial Correlation Matrix */
/* Construct Partial Var-Cov Matrix from E */
Cov=E/(n-g);
/* Construct diagonal matrix of Std. Dev. */
Dv=diag(sqrt(vecdiag(Cov)));
Corr=inv(Dv)*Cov*inv(Dv);
print 'Partial Correlation Matrix: ',Corr,;
/* Compute other MANOVA test statistics as well */
EinvH=inv(E)*H;
/* Pillai's trace */
Pillai=trace(H*inv(H+E));
/* Hotelling-Lawley trace */
Hotellin=trace(EinvH);
/* Roy's maximum root */
/* Note: since eigen() requires symmetric matrix */
/* we must program around it as follows: */
F=root(inv(E));
A=F*H*F`;
call eigen(chRoot,eigenvec,A);
RoyRoot=chRoot[1]; /* get the largest one */
eigenvec=F`*eigenvec; /* recover correct eigenvectors */
/* Compute Transpose so that the E'Vectors become the rows */
chvecT=eigenvec`;
print 'Other Multivariate Test Statistics for Ho: ',;
print EinvH,chRoot chvecT, Pillai ' ' Hotellin ' ' RoyRoot,;
quit;
*-----------------------------------------------------------*
| Compute MANOVA using Pre-programmed Linear Models Proc. |
*-----------------------------------------------------------*;
proc glm data=skull;
class sex;
model length basilar zygomat postorb = sex / xpx inverse solution;
manova h=sex / printe printh;
run; quit;
Skull Measurements by Sex -- A. Anderson 1
MANOVA
Manova using Sums of Squares Approach:
N1 N2 P
26 33 4
YBAR1 YBAR2 YBAR
6486.0385 6429.1515 6454.2203
4938.8846 4898.2727 4916.1695
3261.1154 3258.9697 3259.9153
1093.8846 1090.2424 1091.8475
S1 S2
23191.238 23352.125 6580.6354 4946.0846 34476.82 26469.895 6924.7235 -1539.663
23352.125 25269.466 6272.0938 3895.3862 26469.895 22244.33 5232.7273 -1852.443
6580.6354 6272.0938 5435.3062 1189.5738 6924.7235 5232.7273 3961.4053 104.10133
4946.0846 3895.3862 1189.5738 4159.2262 -1539.663 -1852.443 104.10133 2849.4394
B
47060.932 33597.045 1775.0656 3013.0798
33597.045 23985.106 1267.2286 2151.0534
1775.0656 1267.2286 66.952728 113.64871
3013.0798 2151.0534 113.64871 192.91267
W
1683039.2 1430839.8 386107.04 74382.903
1430839.8 1343555.2 324249.62 38106.472
386107.04 324249.62 262647.62 33070.589
74382.903 38106.472 33070.589 195162.71
WILKLAMB F NUM_DF DEN_DF PVALUE
0.9567081 0.6108864 4 54 0.6565704
General Linear Models Procedure Approach:
Data (Y) and Design Matrix (X):
Y CLASS X
6460 4962 3286 1100 M 1 0 1
6252 4773 3239 1061 M 1 0 1
5772 4480 3200 1097 M 1 0 1
Skull Measurements by Sex -- A. Anderson 2
MANOVA
6264 4806 3179 1054 M 1 0 1
6622 5113 3365 1071 M 1 0 1
6656 5100 3326 1012 M 1 0 1
6441 4918 3153 1061 M 1 0 1
6281 4821 3133 1071 M 1 0 1
6606 5060 3227 1064 M 1 0 1
6573 4977 3392 1110 M 1 0 1
6563 5025 3234 1090 M 1 0 1
6552 5086 3292 1010 M 1 0 1
6535 4939 3261 1065 M 1 0 1
6573 4962 3320 1091 M 1 0 1
6537 4990 3309 1059 M 1 0 1
6302 4761 3204 1135 M 1 0 1
6449 4921 3256 1068 M 1 0 1
6481 4887 3233 1124 M 1 0 1
6368 4824 3258 1130 M 1 0 1
6372 4844 3306 1137 M 1 0 1
6592 5007 3284 1148 M 1 0 1
6229 4746 3257 1153 M 1 0 1
6391 4834 3244 1169 M 1 0 1
6560 4981 3341 1038 M 1 0 1
6787 5181 3334 1104 M 1 0 1
6384 4834 3195 1064 M 1 0 1
6282 4757 3180 1179 M 1 0 1
6340 4791 3300 1110 M 1 0 1
6394 4879 3272 1241 M 1 0 1
6153 4557 3214 1039 M 1 0 1
6348 4886 3160 991 M 1 0 1
6534 4990 3310 1028 M 1 0 1
6509 4951 3282 1104 M 1 0 1
6287 4845 3218 996 F 1 1 0
6583 4992 3300 1107 F 1 1 0
6518 5023 3246 1035 F 1 1 0
6432 4790 3249 1117 F 1 1 0
6450 4888 3259 1060 F 1 1 0
6379 4844 3266 1115 F 1 1 0
6424 4855 3322 1065 F 1 1 0
6615 5088 3280 1179 F 1 1 0
6760 5206 3337 1219 F 1 1 0
6521 5011 3208 989 F 1 1 0
6416 4889 3200 1001 F 1 1 0
6511 4910 3230 1100 F 1 1 0
6540 4997 3320 1078 F 1 1 0
6780 5259 3358 1174 F 1 1 0
6336 4781 3165 1126 F 1 1 0
6472 4954 3125 1178 F 1 1 0
6476 4896 3148 1066 F 1 1 0
6276 4709 3150 1134 F 1 1 0
6693 5177 3236 1131 F 1 1 0
6328 4792 3214 1018 F 1 1 0
6661 5104 3395 1141 F 1 1 0
6266 4721 3257 1031 F 1 1 0
Skull Measurements by Sex -- A. Anderson 3
MANOVA
6660 5146 3374 1069 F 1 1 0
6624 5032 3384 1154 F 1 1 0
6331 4819 3278 1008 F 1 1 0
6298 4683 3270 1150 F 1 1 0
N P G
59 4 2
XPX
59 26 33
26 26 0
33 0 33
XPY
380799 290054 192335 64419
168637 128411 84789 28441
212162 161643 107546 35978
YPY
2.45949E9 1.87354E9 1.24176E9 415851816
1.87354E9 1.42732E9 945876976 316734980
1.24176E9 945876976 627258515 210033665
415851816 316734980 210033665 70531077
B
4305.0633 3279.0524 2173.3617 728.04235
2180.9751 1659.8322 1087.7537 365.84227
2124.0882 1619.2203 1085.608 362.20008
H
47060.932 33597.045 1775.0656 3013.0798
33597.045 23985.106 1267.2286 2151.0534
1775.0656 1267.2286 66.952728 113.64871
3013.0798 2151.0534 113.64871 192.91267
E
1683039.2 1430839.8 386107.04 74382.903
1430839.8 1343555.2 324249.62 38106.472
386107.04 324249.62 262647.62 33070.589
74382.903 38106.472 33070.589 195162.71
WILKLAMB
0.9567081
Skull Measurements by Sex -- A. Anderson 4
MANOVA
Partial Correlation Matrix:
CORR
1 0.9515161 0.5807295 0.1297859
0.9515161 1 0.5458395 0.0744171
0.5807295 0.5458395 1 0.1460686
0.1297859 0.0744171 0.1460686 1
Other Multivariate Test Statistics for Ho:
EINVH
0.0839414 0.0599262 0.0031661 0.0053744
-0.051611 -0.036845 -0.001947 -0.003304
-0.053245 -0.038012 -0.002008 -0.003409
0.0025457 0.0018174 0.000096 0.000163
CHROOT CHVECT
0.0452508 0.001819 -0.001118 -0.001154 0.0000552
6.089E-18 -0.001669 0.0023262 0.0004308 -0.000125
1.377E-18 0.000658 -0.001029 0.0020627 -0.000019
-6.67E-19 -0.000569 0.0005968 -0.000136 0.0023172
PILLAI HOTELLIN ROYROOT
0.0432919 0.0452508 0.0452508
Skull Measurements by Sex -- A. Anderson 5
MANOVA
General Linear Models Procedure
Class Level Information
Class Levels Values
SEX 2 F M
Number of observations in data set = 59
Skull Measurements by Sex -- A. Anderson 6
MANOVA
General Linear Models Procedure
The X'X Matrix
INTERCEPT SEX F SEX M LENGTH BASILAR
INTERCEPT 59 26 33 380799 290054
SEX F 26 26 0 168637 128411
SEX M 33 0 33 212162 161643
LENGTH 380799 168637 212162 2459490751 1873536863
BASILAR 290054 128411 161643 1873536863 1427322166
ZYGOMAT 192335 84789 107546 1241760351 945876976
POSTORB 64419 28441 35978 415851816 316734980
ZYGOMAT POSTORB
INTERCEPT 192335 64419
SEX F 84789 28441
SEX M 107546 35978
LENGTH 1241760351 415851816
BASILAR 945876976 316734980
ZYGOMAT 627258515 210033665
POSTORB 210033665 70531077
Skull Measurements by Sex -- A. Anderson 7
MANOVA
General Linear Models Procedure
X'X Generalized Inverse (g2)
INTERCEPT SEX F SEX M LENGTH BASILAR
INTERCEPT 0.0303030303 -0.03030303 0 6429.1515152 4898.2727273
SEX F -0.03030303 0.0687645688 0 56.886946387 40.611888112
SEX M 0 0 0 0 0
LENGTH 6429.1515152 56.886946387 0 1683039.204 1430839.7517
BASILAR 4898.2727273 40.611888112 0 1430839.7517 1343555.1993
ZYGOMAT 3258.969697 2.1456876457 0 386107.03613 324249.61888
POSTORB 1090.2424242 3.6421911422 0 74382.903263 38106.472028
ZYGOMAT POSTORB
INTERCEPT 3258.969697 1090.2424242
SEX F 2.1456876457 3.6421911422
SEX M 0 0
LENGTH 386107.03613 74382.903263
BASILAR 324249.61888 38106.472028
ZYGOMAT 262647.62354 33070.588578
POSTORB 33070.588578 195162.71445
Skull Measurements by Sex -- A. Anderson 8
MANOVA
General Linear Models Procedure
Dependent Variable: LENGTH
Source DF Sum of Squares Mean Square F Value Pr > F
Model 1 47060.93163060 47060.93163060 1.59 0.2119
Error 57 1683039.20396262 29527.00357829
Corrected Total 58 1730100.13559322
R-Square C.V. Root MSE LENGTH Mean
0.027201 2.662355 171.83423285 6454.22033898
Source DF Type I SS Mean Square F Value Pr > F
SEX 1 47060.93163052 47060.93163052 1.59 0.2119
Source DF Type III SS Mean Square F Value Pr > F
SEX 1 47060.93163052 47060.93163052 1.59 0.2119
T for H0: Pr > |T| Std Error of
Parameter Estimate Parameter=0 Estimate
INTERCEPT 6429.151515 B 214.93 0.0001 29.91250047
SEX F 56.886946 B 1.26 0.2119 45.06008952
M 0.000000 B . . .
NOTE: The X'X matrix has been found to be singular and a generalized inverse was used to
solve the normal equations. Estimates followed by the letter 'B' are biased, and
are not unique estimators of the parameters.
Skull Measurements by Sex -- A. Anderson 9
MANOVA
General Linear Models Procedure
Dependent Variable: BASILAR
Source DF Sum of Squares Mean Square F Value Pr > F
Model 1 23985.10578408 23985.10578408 1.02 0.3174
Error 57 1343555.19930067 23571.14384738
Corrected Total 58 1367540.30508475
R-Square C.V. Root MSE BASILAR Mean
0.017539 3.122939 153.52896745 4916.16949153
Source DF Type I SS Mean Square F Value Pr > F
SEX 1 23985.10578405 23985.10578405 1.02 0.3174
Source DF Type III SS Mean Square F Value Pr > F
SEX 1 23985.10578405 23985.10578405 1.02 0.3174
T for H0: Pr > |T| Std Error of
Parameter Estimate Parameter=0 Estimate
INTERCEPT 4898.272727 B 183.28 0.0001 26.72596278
SEX F 40.611888 B 1.01 0.3174 40.25989992
M 0.000000 B . . .
NOTE: The X'X matrix has been found to be singular and a generalized inverse was used to
solve the normal equations. Estimates followed by the letter 'B' are biased, and
are not unique estimators of the parameters.
Skull Measurements by Sex -- A. Anderson 10
MANOVA
General Linear Models Procedure
Dependent Variable: ZYGOMAT
Source DF Sum of Squares Mean Square F Value Pr > F
Model 1 66.95272805 66.95272805 0.01 0.9045
Error 57 262647.62354313 4607.85304462
Corrected Total 58 262714.57627119
R-Square C.V. Root MSE ZYGOMAT Mean
0.000255 2.082299 67.88116856 3259.91525424
Source DF Type I SS Mean Square F Value Pr > F
SEX 1 66.95272806 66.95272806 0.01 0.9045
Source DF Type III SS Mean Square F Value Pr > F
SEX 1 66.95272806 66.95272806 0.01 0.9045
T for H0: Pr > |T| Std Error of
Parameter Estimate Parameter=0 Estimate
INTERCEPT 3258.969697 B 275.80 0.0001 11.81659471
SEX F 2.145688 B 0.12 0.9045 17.80047830
M 0.000000 B . . .
NOTE: The X'X matrix has been found to be singular and a generalized inverse was used to
solve the normal equations. Estimates followed by the letter 'B' are biased, and
are not unique estimators of the parameters.
Skull Measurements by Sex -- A. Anderson 11
MANOVA
General Linear Models Procedure
Dependent Variable: POSTORB
Source DF Sum of Squares Mean Square F Value Pr > F
Model 1 192.91266644 192.91266644 0.06 0.8132
Error 57 195162.71445221 3423.90727109
Corrected Total 58 195355.62711864
R-Square C.V. Root MSE POSTORB Mean
0.000987 5.359188 58.51416300 1091.84745763
Source DF Type I SS Mean Square F Value Pr > F
SEX 1 192.91266643 192.91266643 0.06 0.8132
Source DF Type III SS Mean Square F Value Pr > F
SEX 1 192.91266643 192.91266643 0.06 0.8132
T for H0: Pr > |T| Std Error of
Parameter Estimate Parameter=0 Estimate
INTERCEPT 1090.242424 B 107.03 0.0001 10.18600833
SEX F 3.642191 B 0.24 0.8132 15.34416850
M 0.000000 B . . .
NOTE: The X'X matrix has been found to be singular and a generalized inverse was used to
solve the normal equations. Estimates followed by the letter 'B' are biased, and
are not unique estimators of the parameters.
E = Error SS&CP Matrix
LENGTH BASILAR ZYGOMAT POSTORB
LENGTH 1683039.204 1430839.7517 386107.03613 74382.903263
BASILAR 1430839.7517 1343555.1993 324249.61888 38106.472028
ZYGOMAT 386107.03613 324249.61888 262647.62354 33070.588578
POSTORB 74382.903263 38106.472028 33070.588578 195162.71445
Skull Measurements by Sex -- A. Anderson 12
MANOVA
General Linear Models Procedure
Multivariate Analysis of Variance
Partial Correlation Coefficients from the Error SS&CP Matrix / Prob > |r|
DF = 57 LENGTH BASILAR ZYGOMAT POSTORB
LENGTH 1.000000 0.951516 0.580729 0.129786
0.0001 0.0001 0.0001 0.3315
BASILAR 0.951516 1.000000 0.545840 0.074417
0.0001 0.0001 0.0001 0.5788
ZYGOMAT 0.580729 0.545840 1.000000 0.146069
0.0001 0.0001 0.0001 0.2739
POSTORB 0.129786 0.074417 0.146069 1.000000
0.3315 0.5788 0.2739 0.0001
Skull Measurements by Sex -- A. Anderson 13
MANOVA
General Linear Models Procedure
Multivariate Analysis of Variance
H = Type III SS&CP Matrix for SEX
LENGTH BASILAR ZYGOMAT POSTORB
LENGTH 47060.931631 33597.044862 1775.0655644 3013.0797874
BASILAR 33597.044862 23985.105784 1267.2285765 2151.0533958
ZYGOMAT 1775.0655644 1267.2285765 66.952728063 113.64871005
POSTORB 3013.0797874 2151.0533958 113.64871005 192.91266643
Characteristic Roots and Vectors of: E Inverse * H, where
H = Type III SS&CP Matrix for SEX E = Error SS&CP Matrix
Characteristic Percent Characteristic Vector V'EV=1
Root
LENGTH BASILAR ZYGOMAT
POSTORB
0.04525085 100.00 -0.00181900 0.00111840 0.00115382
-0.00005517
0.00000000 0.00 -0.00048407 0.00047922 -0.00017569
0.00232068
0.00000000 0.00 -0.00181551 0.00256794 -0.00047096
0.00000000
0.00000000 0.00 -0.00010944 0.00004495 0.00205091
0.00000000
Manova Test Criteria and Exact F Statistics for the Hypothesis of no Overall SEX Effect
H = Type III SS&CP Matrix for SEX E = Error SS&CP Matrix
S=1 M=1 N=26
Statistic Value F Num DF Den DF Pr > F
Wilks' Lambda 0.95670815 0.6109 4 54 0.6566
Pillai's Trace 0.04329185 0.6109 4 54 0.6566
Hotelling-Lawley Trace 0.04525085 0.6109 4 54 0.6566
Roy's Greatest Root 0.04525085 0.6109 4 54 0.6566