SticiGui Regression Diagnostics

This chapter studies whether regression is an appropriate summary of a given set bivariate data, and whether the regression line was computed correctly. Problems with regression are generally easier to see by plotting the residuals rather than the original data. The vertical amount by which the line misses a datum is called a residual—it is the error in estimating the value of Y for that datum from its value of X using the regression line. Plotting the residuals against X is called a residual plot.

If the data are heteroscedastic, nonlinearly associated, or have outliers, the regression line is not a good summary of the data, and it is not appropriate to use regression to summarize the data. Heteroscedasticity, nonlinearity and outliers are easier to see in a residual plot than in a scatterplot of the raw data. There can be errors of arithmetic in calculating the regression line, so that the slope or intercept is wrong. It is easy to catch such errors by looking at residual plots, where they show up as a nonzero mean or a trend.

Residual plots

A residual is the vertical difference between the Y value of an individual and the regression line at the value of X corresponding to that individual, for regressing Y on X. That is, suppose there are n pairs of measurements of X and Y:

and so on. The i^th vertical residual is the amount by which the regression line at the i^th value of X misses the i^th value of Y—the error in using the regression line to estimate the i^th datum. (For regressing X on Y, a residual is the horizontal difference between the X value of the individual and the regression line at the value of Y corresponding to that individual.)

Plotting the residuals as a function of the "independent" variable (the one that is being regressed upon) can tell a great deal about the regression, including whether it was appropriate to use linear regression in the first place, and whether the regression was computed correctly. A plot of the residuals against the corresponding values of the independent variable is called a residual plot. It is a scatterplot of the n points

A residual plot is like a scatterplot of the original data, but with (ax_i + b) subtracted from the value of y_i for each point (x_i, y_i), i = 1, 2, … , n. Subtracting the regression line from the data removes from Y any overall average and any trend with X.

Figure 10-1 shows the GMAT data again, this time with the residual plots displayed instead of the original data. (If the figure does not show Quantitative GMAT versus Verbal GMAT, select those variables from the drop-down menus.)

Figure 10-1: Residual Plot of the GMAT Data.

Data: gmat.json vs

r: 0.35

School	1st year MBA GPA	Verbal GMAT	Quant. GMAT	Undergrad. GPA

1	3.155	31	37	3.53
1	3.665	38	40	3.38
1	3.685	35	36	3.71
1	3.305	41	25	3.15
1	3.065	36	42	3.54
1	3.1	43	43	3.59
1	4.13	46	48	4
1	3.165	42	37	2.63
1	2.605	33	28	3.09
1	3.64	28	40	3.39
1	2.24	35	28	3.14
1	2.92	36	38	3.25
1	3.07	37	32	2.09
1	2.805	41	38	2.86
1	3.4	40	31	3.78
1	3.63	35	32	4
1	2.5	38	40	3.27
1	3.645	44	46	2.87
1	3.085	44	36	3.01
1	4.13	47	32	2.84
1	3.36	37	43	2.41
1	2.805	38	33	3.41
1	3.53	37	41	3.57
1	3.6	40	34	3.16
1	3.66	28	44	3.25
1	2.4	37	32	3
1	2.775	40	37	3.28
1	3.075	38	46	3.09
1	3.645	39	50	3.63
1	3.94	39	41	3.87
1	2.55	30	26	3.02
1	2.95	39	32	3.52
1	3.95	40	34	3.63
1	3.565	44	42	3.89
1	3.135	25	38	3.57
1	3.715	35	32	3.65
1	3.635	40	37	3.27
1	3.03	36	33	3.05
1	3.58	27	37	3.22
1	3.125	33	40	2.67
1	3	34	38	2.65
1	3.885	33	40	3.73
1	2.61	44	42	3.5
1	2.9	32	30	3.66
1	3.105	44	40	3.09
1	3.205	34	33	3.29
1	3.115	39	42	2.47
1	3.135	30	35	3.43
1	2.75	30	38	2.49
1	3.715	32	38	3.94
1	3.08	35	33	3.38
1	2.87	38	31	2.78
1	2.97	29	31	3.7
1	2.935	39	36	3.47
1	2.24	35	44	2.22
1	3.935	42	42	2.89
1	2.695	37	31	3.04
1	3.52	37	42	2.42
1	2.7	36	40	3.06
1	2.595	44	35	2.15
1	3.205	38	46	2.68
1	3.565	33	36	3.53
1	3.385	44	38	3.4
1	3.275	36	35	3.09
1	2.66	32	36	2.83
1	3.335	38	37	3.09
1	2.675	37	32	3.12
1	3.325	38	33	3.24
1	2.57	33	37	3.15
1	2.895	32	30	2.99
1	3.985	33	40	3.76
1	3.235	37	31	3.47
1	3.81	38	47	2.88
1	3.915	35	39	3.97
1	2.615	34	39	3.12
1	3.115	36	35	3.39
1	2.985	45	43	3.04
1	2.835	34	36	2.93
1	3.365	37	33	2.97
1	2.85	39	32	3.46
1	3.805	35	39	3.6
1	3	32	34	3.51
1	2.675	33	38	3
1	2.565	30	33	2.4
1	3.75	39	52	2.83
1	2.515	33	33	2.82
1	2.955	36	43	3.17
1	3.095	32	40	3.24
1	2.72	42	30	3.55
1	3.03	33	39	3.23
1	3.05	30	33	3.03
1	3.63	31	46	3.25
1	3.885	29	37	3.63
1	3.6	37	47	3.33
1	3.195	38	33	3.59
1	2.65	34	32	3.19
1	3.81	45	47	3.73
1	3.64	41	40	3.68
1	3.535	49	41	3.45
1	3.685	44	43	3.67
1	2.69	38	38	3.28
1	3.275	39	34	3.88
1	2.585	24	38	3.67
1	2.985	42	39	3.5
1	2.95	42	26	2.67
1	3.89	44	46	2.91
1	3.43	37	37	3.32
1	2.735	33	35	3.51
1	3.44	43	47	3.19
1	2.765	38	31	3.03
1	3.345	32	31	3.57
1	3.155	37	33	3.21
1	3	34	40	3.68
1	3.05	30	43	3.13
1	3	32	34	3.5
1	3.145	30	40	3.86
1	2.985	34	41	2.9
1	2.515	38	30	3.66
1	2.5	42	33	3.73
1	2.415	35	34	2.97
1	2.66	40	41	2.89
1	2.7	34	34	3.08
1	3	38	34	3.1
1	2.775	35	37	2.53
1	3.415	26	43	3.56
1	3.315	38	36	3.3
1	3.935	37	41	3.52
1	2.955	38	37	3.37
1	2.515	31	39	3.37
1	3.16	38	43	3.04
1	3.815	49	38	2.53
1	3.48	27	36	3.67
1	3.385	36	43	2.78
1	2.78	34	36	2.85
1	3.635	36	40	3.32
1	3.26	33	33	3.16
1	3.13	41	28	3.88
1	3.695	39	50	3.26
1	3.345	33	36	3.8
1	3.115	28	32	3.44
1	2.92	37	40	2.64
1	4.5	34	33	3.31
1	3.2	27	39	2.87
1	3.115	35	27	3.74
1	2.775	39	36	2.8
1	2.765	38	34	3.2
1	2.695	30	32	2.9
1	3.315	24	38	3.44
1	3.36	34	35	3.45
1	3.225	33	27	3.44
1	3	36	31	3.11
1	3.065	44	28	2.65
1	2.935	35	36	3.4
1	2.63	36	43	3.33
1	2.68	42	43	3.68
1	3.29	28	34	3.46
1	2.67	30	36	3.21
1	3.545	40	26	4
1	2.72	28	33	3.42
1	2.925	39	45	2.65
1	3.36	44	37	3.82
1	2.5	34	39	2.71
1	2.69	33	38	3.76
1	3.335	40	45	3.71
1	2.63	31	34	2.9
1	2.715	46	28	3.93
1	3.88	34	33	2.91
1	3.285	36	45	3.2
1	3.04	35	48	2.75
1	3.065	41	29	3.4
1	2.5	34	38	3.38
1	2.675	31	33	3.5
1	2.71	34	35	3.3
1	2.92	36	36	2.94
1	2.815	35	42	3.1
1	3	32	33	3.43
1	3.165	32	37	3.3
1	2.93	33	37	2.4
1	2.84	39	33	2.94
1	2.67	41	36	2.39
1	2.69	29	39	3.34
1	2.4	36	34	2.86
1	3.485	29	41	3.7
1	3.265	35	41	3.17
1	3.13	34	41	3.04
1	2.44	39	41	2.79
1	3.915	33	41	2.84
1	3.085	39	53	3.77
1	3	34	34	2.81
1	3.29	39	43	3.12
1	2.4	35	35	3.02
1	3.065	34	44	2.9
1	3.705	36	38	3.61
1	0.775	19	29	3.28
1	3.24	39	33	3.32
1	2.87	38	35	3.04
1	3.225	35	41	3.14
1	3.26	41	40	3.72
1	4.065	39	43	3.84
1	2.9	39	47	3.19
1	1.8	30	32	3.62
1	2.365	33	31	3.18
1	3.08	37	27	3.35
1	3.145	38	32	3.36
1	3.58	33	43	3.42
1	2.935	26	36	2.98
1	2.845	45	45	3.44
1	2.485	28	38	2.96
1	2.75	34	32	3.33
1	4.065	38	51	3.89
1	3.21	32	31	3.43
1	2.67	44	40	2.77
1	3.065	37	41	3.26
1	2.9	35	33	3.58
1	3.385	32	41	3.28
1	3.5	34	41	3.63
1	3.455	35	39	3.29
1	3.095	44	44	2.85
1	3.485	30	32	3.66
1	2.515	34	32	2.64
1	2.835	32	32	3.59
1	2.85	31	41	2.91
1	2.405	43	40	3.41
1	3.615	40	46	3.38
1	3	39	37	3.42
1	2.985	43	32	3.7
1	3.515	44	36	3.04
1	3.14	32	31	3.39
1	2.815	38	34	3.16
1	3.43	33	37	3.58
1	3.305	30	34	3.67
1	3.035	30	36	3.59
1	2.795	43	24	3.31
1	3.655	29	37	3.77
1	3	37	30	3.48
1	2.55	39	39	3.14
1	2.765	34	36	3.58
1	2.435	36	39	3.4
1	3.15	32	41	2.88
1	3.285	34	38	3.61
1	2.935	34	38	2.77
1	3.565	33	32	3.74
1	3.16	34	35	3.15
1	2.8	35	35	3.43
1	3.315	36	38	3.08
1	3.485	40	45	3.39
1	3.305	35	37	2.83
1	3.465	39	38	3.44
1	3.985	43	46	3.38
1	3.45	33	36	3.5
1	3	35	33	3.09
1	2.855	35	44	2.99
1	2.945	39	36	3.32
1	3.195	29	35	3.79
1	3	35	43	3.87
1	3.47	28	42	3.34
1	3.76	32	34	3.05
1	2.745	35	34	3.63
1	2.985	30	38	3.14
1	3.635	38	42	3.1
1	3.015	33	36	3.71
1	3.065	39	46	2.97
1	2.485	37	27	3.21
1	2.555	25	38	3.4
1	3.515	37	38	3.79
1	3.165	36	30	3.12
1	2.64	32	30	3.14
1	2.6	27	34	3.15
1	2.735	35	32	3.11
1	3.195	43	42	3.24
1	3.765	40	34	3.53
1	3.295	32	41	3.05
1	2.92	35	37	3.18
1	3.5	38	43	3.41
1	3.64	31	44	3.54
1	2.95	37	41	3.08
1	2.8	30	39	2.88
1	3.42	37	46	3.42
1	3.015	37	31	3.28
1	3.125	32	37	3.55
1	3.5	37	27	3.67
1	2.63	46	37	2.89
1	3.26	48	33	3.42
1	2.41	38	28	3.27
1	3.485	40	47	3.54
1	3.265	41	37	2.91
1	2.565	38	40	2.72
1	2.985	33	36	3.33
1	2.645	38	46	3.28
1	2.515	37	46	3
1	2.835	28	32	3.23
1	2.895	38	33	3.75
1	3.465	34	34	3.55
1	3.45	27	46	3.16
1	2.9	34	38	2.52
1	3.305	38	35	3.29
1	2.855	37	32	3.1
1	2.735	39	38	2.48
1	3.26	45	45	2.87
1	2.585	38	40	2.62
1	3.545	35	41	3.44
1	2.78	41	25	3.56
1	2.97	36	36	3.12
1	3.405	34	40	2.95
1	3.415	25	35	3.14
1	2.55	32	27	3.57
1	3.24	37	30	3.52
1	3.185	43	32	2.97
1	2.775	32	42	2.83
1	3	32	26	3.58
1	2.835	32	31	2.78
1	2.81	31	35	2.9
1	2.765	32	35	2.57
1	3.785	33	40	3.5
1	2.97	37	34	2.37
1	3.15	37	39	2.99
1	2.88	36	38	3.66
1	3.805	32	38	3.59
1	2.965	29	35	3.26
1	2.435	31	22	3.19
1	2.965	39	31	3.18
1	3.375	41	31	3.73
1	2.95	29	32	3.63
1	2.55	38	27	3.22
1	3.015	24	41	3.44
1	3.355	33	42	3.3
1	2.635	36	35	3.18
1	3.165	38	46	2.75
1	2.9	33	29	2.49
1	3.535	33	29	3.6
1	3.45	34	36	3.81
1	2.915	36	30	3.22
1	2.92	39	42	3.37
1	2.925	39	42	3.51
1	3.535	36	32	3.94
1	3.365	30	37	3.47
1	4.275	39	40	4
1	2.865	34	41	3.31
1	3.8	41	39	3.97
1	2.435	32	37	3.21
1	3.09	35	32	2.9
1	3.565	35	37	3
1	3.25	34	40	3.03
1	3.57	32	45	3.03
1	3.485	39	38	3.92
1	3.155	29	34	3.33
1	2.885	46	30	3.34
1	2.615	36	37	2.71
1	3	33	33	2.89
2	2	32	35	3.2
2	3.63	43	42	3.8
2	3.63	36	32	3.5
2	2.66	34	22	3.6
2	2.2	33	27	2.9
2	2	39	38	2.8
2	3.9	43	46	3.2
2	1.86	38	30	3.2
2	2.63	18	45	2.8
2	3	39	24	2.2
2	2.9	38	28	3.5
2	2.5	35	28	3
2	2.7	19	25	3
2	1.4	34	27	3.5
2	3.2	31	32	3.2
2	3.4	43	25	3.4
2	3.33	37	39	3.2
2	3.7	28	24	3.4
2	3.6	41	46	3.9
2	2.6	27	25	3
2	3	47	49	3.3
2	1.75	20	21	3.5
2	3.25	30	22	2.2
2	3.3	34	42	3.1
2	3.4	41	36	3.1
2	2.4	37	22	2.2
2	2.3	39	45	3.3
2	3.3	36	47	2.5
2	2.3	35	24	3
2	0.8	38	27	2.8
2	1.25	36	12	3
2	2	30	14	3
2	2.1	28	35	3.1
2	2.9	36	35	3.3
2	2.5	34	26	3.2
2	2.1	33	36	3.6
2	1.66	30	19	3.1
2	3	43	38	3.2
2	3.78	44	45	2.8
2	3	29	39	2.8
2	2.9	33	25	3.3
2	2.4	40	32	2.3
2	2.75	31	22	3.6
2	2.4	22	28	3
2	3	38	28	3.1
2	3.9	42	31	3.3
2	1.88	37	32	3.2
2	3.2	39	38	3.5
2	3	45	42	3.3
2	2.63	43	32	3.4
2	3.3	33	43	3.2
2	2.1	40	31	3.2
2	2.5	34	30	2.9
2	2	38	36	2.6
2	3.1	26	32	3.5
2	3	40	30	3.4
2	2.2	37	27	3.1
2	3	39	38	3
2	3.3	41	35	3.2
2	2.33	44	32	3.1
2	2.7	26	23	3.1
2	2.75	29	39	3.7
2	1.89	17	25	2.8
2	3.1	43	30	3.2
2	3.5	31	31	3.4
2	3.3	34	40	2.2
2	3	44	32	2
2	3.3	37	37	3.2
2	3.4	36	34	3.8
2	3.2	37	32	3.5
2	3.3	43	32	2.8
2	3	39	41	2.8
2	3.37	40	33	3.5
2	2.6	37	27	3.6
2	3	28	45	3.6
2	1	14	15	3.3
2	3.2	36	31	3.4
2	3.77	47	43	3.3
2	2.9	39	41	3
2	2.4	32	22	2.9
2	3.4	44	27	3.2
2	3.6	33	26	3.2
2	2.8	36	31	2.5
2	3.7	28	33	3.8
2	2.8	39	39	2.9
2	3.1	41	36	2.6
2	3.2	31	40	2.9
2	3.5	44	34	3
2	2.2	26	28	3.3
2	3.1	38	45	2.8
2	2.33	17	15	3.3
2	2.4	25	19	2.8
2	2.4	38	36	3.2
2	3	33	28	3.4
2	1.8	36	37	3.6
2	1.83	39	35	2.4
2	2.8	26	39	2.9
2	1.9	17	19	3.5
2	2.8	32	34	3.4
2	2.6	35	31	3.2
2	2.8	43	31	3.4
2	2.5	32	30	3.1
2	2.8	35	24	2.3
2	2.8	40	37	2.9
2	1.87	14	30	3.4
2	2.5	44	29	3.6
2	2.9	43	41	3
2	3.1	40	32	3.1
2	3.18	43	45	2.1
2	3.3	38	37	3.5
2	3.2	32	33	3.5
2	3.1	36	34	3.3
2	1.9	46	28	2.7
2	3	37	34	3
2	3.5	38	39	3.7
2	3.7	32	34	3.7
2	2.7	32	30	2.8
2	3.7	41	39	3.3
2	3.1	43	39	3.3
2	2.5	35	27	3.2
2	3.3	36	28	3.6
2	1.8	38	29	2.8
2	2.5	31	33	3.4
2	3.5	17	40	3.3
2	3.4	35	31	2.8
2	3.2	38	32	2.9
2	2.3	33	20	3.2
2	3	28	25	2.8
2	3.2	43	33	3.6
2	3.5	47	41	3.6
2	2.9	33	34	3
2	2.6	32	31	2.9
2	3.5	20	43	3.3
2	3.4	38	33	3.6
2	3.3	35	41	3.2
2	2.75	21	15	3.3
2	2.9	24	40	2.8
2	2.67	38	34	3.2
2	2.44	33	35	3.3
2	2.9	29	34	3.4
2	2.6	36	35	3
2	2.7	27	32	3.3
2	2.3	28	29	3.3
2	2	25	28	3.2
2	2.7	37	41	2.9
2	3.1	40	27	3.3
2	2.2	34	23	3.4
2	3.2	39	35	3.2
2	3.22	17	35	3.6
2	2.62	39	32	3.1
2	3	36	32	3.2
2	3	31	23	3.1
2	3.2	45	35	3.1
2	2.9	37	37	2.7
2	2.88	39	42	3.9
2	2.4	28	27	2.7
2	3.2	28	26	3.4
2	2.55	44	38	3.7
2	2.11	37	26	3.1
2	2.4	33	34	2.8
2	2.1	35	24	3.5
2	2.77	40	30	3.5
2	2.38	38	32	2.9
2	2.75	36	40	3.3
2	3.2	38	34	3.8
2	3.44	45	43	3.8
2	3.5	40	35	3.6
2	3	31	46	3.5
2	3.1	40	31	3.4
2	3.4	42	38	3
2	2	33	30	3.7
2	2	33	31	2.9
2	2.3	30	28	2.6
2	2.2	39	40	3.3
2	2.55	38	39	3.4
2	3.1	46	23	3.5
2	3	32	33	3.1
2	3.1	21	30	3.8
2	2.5	18	21	3.2
2	2.4	38	31	2.9
2	3.2	34	40	2.1
2	2.8	43	36	3.4
2	3.7	43	36	4
2	2.3	37	30	2.8
2	2.4	35	35	3
2	2.2	27	36	2.9
2	2.4	34	31	2.7
2	3	37	33	3.4
2	3.5	26	49	3.2
2	2.7	37	18	3.2
2	3	28	39	3.5
2	3.3	40	38	3.4
2	1.9	34	31	3.3
2	3.1	40	29	2.7
2	2.4	33	27	3.4
2	1.9	34	27	3.4
2	3.1	36	28	3.4
2	2.77	29	32	3.4
2	2.8	36	35	3.4
2	2.56	31	31	2.7
2	2.8	27	33	3.7
3	3.9	49	51	3.18
3	3.66	39	43	3.75
3	3.41	31	39	3.55
3	2.98	43	41	3.08
3	3.64	33	38	3.26
3	3.25	24	39	3.47
3	2.94	36	38	3.05
3	3.56	27	40	3.49
3	3.45	43	41	3.53
3	3.64	42	44	3.48
3	3.39	39	30	3.49
3	3.1	38	33	3.3
3	3.25	40	60	2.82
3	3.49	38	45	3.75
3	3.8	41	47	4
3	2.94	32	38	3.65
3	2.94	37	37	3.15
3	3.06	36	34	3.45
3	3.16	36	37	3.81
3	3.72	40	39	3.38
3	3.46	36	35	3.81
3	3.05	40	34	3.39
3	3.54	40	31	3.26
3	3.81	41	38	3.61
3	3.32	35	29	3.57
3	3.83	40	45	3.64
3	3.31	47	30	3.53
3	3.11	41	37	3.22
3	3.3	42	31	2.98
3	3.42	38	28	3.25
3	3.49	39	39	3.73
3	3.45	36	32	3.44
3	3.56	34	37	3.74
3	3.45	42	32	3.39
3	3.67	42	45	2.69
3	3.66	36	36	3.57
3	3.7	31	40	3.42
3	3.3	39	28	3.62
3	3.67	42	25	3.56
3	3.58	45	35	3.6
3	3.38	40	45	2.93
3	3.53	38	33	3.49
3	3.04	35	40	3.7
3	3.92	47	35	3.11
3	3.41	24	36	3.33
3	3.26	39	31	3.51
3	3.6	39	34	3.36
3	3.08	36	47	3.1
3	3.3	43	47	3.55
3	3.29	32	40	3.91
3	3.51	44	38	2.76
3	3.03	33	39	3.42
3	3.42	44	33	3.57
3	3.24	25	30	3.31
3	2.97	40	34	3.6
3	3.54	47	29	3.72
3	2.94	28	40	3.23
3	3.16	36	43	2.6
3	3.81	49	42	3.8
3	3.71	28	34	3.37
3	3.59	33	45	3.06
3	3.33	39	31	3.36
3	2.98	34	38	2.58
3	3.86	45	45	3.96
3	3.53	42	38	3.42
3	3.39	40	34	3.37
3	3.29	38	34	3.45
3	3.49	42	49	3.57
3	3.11	36	36	3.16
3	3.76	38	36	3.53
3	3.82	18	39	3.94
3	3.54	37	41	4
3	3.94	47	39	3.83
3	2.88	32	34	3.7
3	3.24	37	37	3.41
3	3.38	37	31	3.85
3	3.52	44	33	3.5
3	3.28	38	39	3.04
3	3.06	30	41	3.4
3	3.38	42	33	3.34
3	3.64	40	33	2.01
3	3.42	36	46	3.23
3	2.83	44	43	3.86
3	3.2	32	26	3.76
3	3.51	41	39	3.5
3	2.67	39	46	2.86
3	3.2	31	35	3.4
3	3.7	41	44	3.91
3	3.93	38	43	2.84
3	3.58	41	38	3.24
3	3.7	35	42	3.13
3	3.66	41	41	2.62
3	2.99	39	43	3.72
3	3.33	32	48	3.15
3	3.85	45	52	3.96
3	3.4	30	38	3.45
3	3	36	45	3.13
3	3.48	38	34	3.95
3	3.58	44	43	3.47
3	3	35	35	3.43
3	3.29	28	34	3.5
3	3.3	43	45	3.27
3	3.29	33	45	3.24
3	3.02	40	39	3.25
3	3.5	33	44	3.95
3	3.96	44	37	3.92
3	3.44	25	44	3.58
3	3.49	41	41	3.67
3	3.49	39	44	2.74
3	3.04	37	41	3.4
3	3.66	36	37	3.35
3	3.54	39	40	3.75
3	3.11	34	40	3.84
3	3.55	42	33	3.02
3	3.31	41	34	3.84
3	3.17	43	40	2.99
3	3.41	40	48	3.45
3	3.2	40	37	3.58
3	3.33	31	39	3.49
3	3.51	41	45	3.64
3	3.01	35	38	3.52
3	3.32	42	40	2.9
3	3.68	48	39	3.78
3	3.64	49	46	3.72
3	3.1	42	43	3.04
3	3.55	37	33	3.55
3	3.27	39	45	3.1
3	3.16	33	40	3.33
3	3.37	36	42	3.57
3	3.97	42	49	4
3	3.75	36	35	3.15
3	3.58	31	45	3.72
3	3.53	43	38	3.6
3	3.77	46	35	3.36
3	3.91	43	39	3.58
3	3.23	49	37	3.49
3	2.88	34	39	3.22
3	3.19	36	32	3.42
3	3.59	44	31	3.59
3	3.52	33	34	3.6
3	3.39	37	36	3.59
3	2.93	43	39	3.35
3	3.36	43	35	3.43
3	3.19	33	37	3.58
3	3.56	31	37	3.85
3	3.05	25	36	3.38
3	3.11	33	41	3.37
3	3.27	48	37	3.3
3	3.92	44	41	3.96
3	3.04	43	43	3.31
3	3.82	36	36	3.45
3	3.91	46	43	3.85
3	3.24	37	39	3.55
3	3.11	21	47	3.86
3	3.12	38	37	3.16
3	3.55	40	41	3.09
3	3.24	41	42	3.28
3	3.75	41	41	3.44
3	3.83	44	43	3.8
3	2.72	35	28	3.19
4	3.53	46	29	2.3
4	4	27	37	3.3
4	3.76	32	28	3
4	3.47	34	34	2.7
4	3.29	26	28	3
4	3.73	24	28	2.9
4	3.87	35	31	2.7
4	3.58	27	28	3.4
4	3	19	25	2.3
4	2.88	25	25	3.4
4	3.14	31	43	3
4	3.15	39	28	3.1
4	3.29	29	17	3
4	3.4	28	30	3.2
4	3.64	26	23	2.6
4	3.5	29	29	3.2
4	3.11	38	32	2.7
4	3.5	29	31	3
4	3.31	36	32	2.3
4	3.79	37	42	3.3
4	2.9	15	28	3.2
4	3.67	32	22	3.1
4	3	32	24	3
4	3.84	35	40	3.4
4	3.5	18	20	3.1
4	3.11	25	24	2.9
4	3.52	26	24	3.1
4	3.2	24	31	2
4	3.88	34	27	3.5
4	3.11	29	31	3.3
4	3	39	17	3.8
4	3.14	28	23	2.9
4	3.26	38	27	2.4
4	3	19	22	2.4
4	2.88	33	35	1.1
4	3	21	37	2.1
4	2.57	20	25	2.9
4	4	44	39	3.5
4	3	21	20	3.2
4	2.74	32	33	2.1
4	3.36	32	31	3.1
4	3	30	25	2.7
4	3	32	20	3
4	3.59	40	30	3
4	2.88	33	35	2.3
4	3.42	23	24	3.3
4	3.5	39	28	3.9
4	2.89	34	32	2.8
4	4	33	35	2
4	3.5	38	30	3.2
4	4	22	17	2.8
4	3.82	18	37	3.7
4	3.16	26	21	3
4	3.37	37	36	2.8
4	3.58	37	32	2.1
4	3	30	24	2.9
4	3.33	37	33	3.4
4	3.5	25	23	2.7
4	3.19	27	21	2.8
4	3.59	31	37	3.3
4	3	33	30	2.9
4	3.29	31	27	2.9
4	3.26	31	29	3.5
4	2.83	30	15	3.1
4	3.4	33	26	2.4
4	4	43	39	2.5
4	3.55	42	43	3.1
4	3.87	33	39	3.7
4	3.56	35	37	2.8
4	2.83	21	16	2.5
4	3.83	20	15	3.4
4	3	26	25	2.7
4	2.84	33	26	2.6
4	3.17	27	24	2.7
4	3.53	28	23	2.8
4	3.55	31	30	2.3
4	4	28	30	3.2
4	3.36	26	27	2.8
4	3.55	26	32	3.3
4	3.25	24	22	3
4	3.23	16	26	2.6
4	3.53	37	32	2
4	3.33	20	29	3
4	3.36	29	28	1.9
4	2.73	7	20	2.3
4	3.5	28	24	3.2
4	3.5	34	28	3.6
4	3.5	25	25	2.3
5	3.3	24	39	3.68
5	3.25	34	33	3.16
5	2.65	37	31	3.07
5	2.2	36	34	3.82
5	3.06	25	38	2.89
5	3.41	33	30	2.79
5	2.8	36	32	3.47
5	2.69	32	28	2.83
5	3.51	41	34	3.61
5	3.1	41	49	3.31
5	2.15	39	35	2.97
5	3.3	36	35	3.83
5	3.71	27	31	3.7
5	3.55	38	30	3.42
5	3.54	34	47	3.12
5	3.33	34	36	3.23
5	3.44	38	32	3.5
5	3.11	31	47	3.36
5	3.31	33	34	3.4
5	3.2	36	32	2.86
5	3.71	40	41	3.42
5	3.75	33	37	2.94
5	2.2	15	40	2.68
5	2.6	30	26	3.33
5	3.13	45	33	3.12
5	2.89	42	28	3.14
5	2.85	33	24	3.19
5	3.9	41	35	3.07
5	2.5	36	30	3.36
5	3.09	34	30	3.75
5	3.92	34	33	3.09
5	3.65	37	31	3.88
5	3.88	31	38	2.66
5	3.26	31	29	3.38
5	3.75	41	43	3.39
5	3.1	38	32	3.39
5	3.01	28	30	2.95
5	3.24	41	40	3.15
5	3.5	33	30	3.39
5	2.75	35	32	3.3
5	3.35	33	40	3.02
5	1.9	38	26	2.9
5	3.1	25	39	3.53
5	2.75	31	31	3.34
5	2.25	23	26	2.66
5	2.25	34	33	3
5	3.6	29	43	3.73
5	3.28	24	23	3.77
5	3.41	33	36	3.47
5	3.2	41	26	2.73
5	2.95	31	29	2.88
5	3	31	33	3.53
5	2.95	33	38	3.11
5	3.4	33	35	2.78
5	2.85	34	31	3.32
5	2.65	32	35	2.76
5	2.85	30	24	3.71
5	2.6	35	32	3.21
5	3.15	30	34	2.62
5	3.51	39	29	2.83
5	3.47	32	37	3.79
5	3.35	37	40	3.55
5	3.6	45	46	3.07
5	3.85	28	31	3.26
5	2.8	34	38	2.95
5	2.5	32	26	2.72
5	3.9	42	35	3.79
5	3.11	36	37	3.1
5	2.76	36	26	3.42
5	3.68	40	45	3.38
5	3.56	37	33	3.49
5	3.3	34	40	3.12
5	2.55	33	26	3.72
5	2.95	38	27	3.2
5	3.56	28	34	3.3
5	3.17	29	33	2.41
5	3	37	29	3.46
5	2.5	18	23	3.4
5	2.5	38	34	3.41
5	2.45	33	35	3.18
5	3.14	35	28	3.51
5	3.49	34	42	3.5
5	2.2	24	28	3.01
5	3.15	35	29	3.13
5	2.45	28	23	3.05
5	2.92	33	28	3.19
5	2.35	35	35	3.39
5	3.97	44	48	3.45
5	2.81	27	41	3.47
5	3.05	36	31	3.57
5	2.65	25	41	2.92
5	2.1	30	28	2.58
5	2.45	28	28	2.59
5	3.4	40	35	3.51
5	3.52	33	31	3.37
5	3.35	38	37	3.44
5	3.35	31	26	2.77
5	2.96	39	44	2.67
5	2.6	40	30	3.41
5	3.56	46	49	3.31
5	3.1	35	36	2.95
5	2.55	37	35	2.49
5	2.6	37	25	3.15
5	3.28	25	43	2.9
5	2.7	31	32	2.93
5	2.3	26	27	3.35
5	3.47	33	32	2.57
5	3.8	38	30	3.54
5	2.7	24	38	3.6
5	2.8	30	31	3.19
5	3.73	41	38	3.38
5	3.87	28	32	2.77
5	3.25	32	40	3.4
5	3.28	34	32	2.57
5	3.32	39	34	3.73

School

Cases: 913
Mean: 2.36
SD: 1.39
Min: 1.00
LQ: 1.00
Median: 2.00
UQ: 3.00
Max: 5.00

1st year MBA GPA

Cases: 913
Mean: 3.11
SD: 0.49
Min: 0.78
LQ: 2.81
Median: 3.13
UQ: 3.48
Max: 4.50

Verbal GMAT

Cases: 913
Mean: 35.09
SD: 6.22
Min: 7.00
LQ: 32.00
Median: 36.00
UQ: 39.00
Max: 49.00

Quant. GMAT

Cases: 913
Mean: 35.00
SD: 6.77
Min: 12.00
LQ: 31.00
Median: 35.00
UQ: 40.00
Max: 60.00

Undergrad. GPA

Cases: 913
Mean: 3.23
SD: 0.40
Min: 1.10
LQ: 3.00
Median: 3.28
UQ: 3.50
Max: 4.00

Clicking Plot Residuals will toggle the display back to a scatterplot of the data. Clicking Plot Residuals again will change the display back to the residual plot. Notice that for the residual plot for quantitative GMAT versus verbal GMAT, there is (slight) heteroscedasticity: the scatter in the residuals for small values of verbal GMAT (the range 12–22) is a bit larger than the scatter of the residuals for larger values of verbal GMAT.

Reading Residual Plots

Regression is a poor summary of data that have heteroscedasticity, nonlinear association, or outliers. These are easier to see in a residual plot than in a scatterplot of the original data. Figure 10-2 is the residual plot for more severely heteroscedastic data:

Figure 10-2: Residual Plot of Heteroscedastic Data.

r: 0.98

The heteroscedasticity is clearly evident—the vertical scatter is quite different in different vertical strips, large in some slices and small in others. Click Plot Data in Figure 10-2 to display a scatterplot of the raw data. You will see that the heteroscedasticity, while still visible, is less pronounced than in the residual plot. (Click Plot Residuals to show the residual plot again.)

Note that for both Figure 10-1 and Figure 10-2 , there is no trend in the residuals: the horizontal (X) axis divides the residuals into two roughly symmetrical parts. This is because the data are linearly associated. In contrast, Figure 10-3 is a residual plot for data with nonlinear association:

Figure 10-3: Residual Plot of Data with Nonlinear Association.

r: 0.71

Recall that two variables have nonlinear association if their scatterplot shows a curved pattern. The residuals have a nonlinear association with X if and only if the original observations of Y have a nonlinear association with X, but it is easier to see the nonlinear association in the residual plot than in the scatterplot, just as it is easier to see heteroscedasticity in the residual plot. Figure 10-4 is the residual plot for data that are both nonlinearly associated and heteroscedastic:

Figure 10-4: Residual Plot for Heteroscedastic, Nonlinearly Associated Data.

r: 0.91

In Figure 10-3 and Figure 10-4 the residuals are scattered asymmetrically around the x axis: They show a systematic sinuous pattern characteristic of nonlinear association. In some ranges of X, all the residuals are below the x axis (negative), while in other ranges, all the residuals are above the x axis (positive). Nonlinear association between the variables shows up in a residual plot as a systematic pattern. Toggle back and forth between plotting the data and plotting the residuals in Figure 10-3 and Figure 10-4 . Note that the nonlinearity and heteroscedasticity are easier to see in the residual plots.

Similarly, a residual plot shows outliers if and only if the original data contain outliers, but—as with heteroscedasticity and nonlinearity—it is easier to see outliers in the residual plot than in a scatterplot of the original data. In summary:

We have seen that residual plots can help identify heteroscedasticity, nonlinearity and outliers. They can also help identify computational errors in calculating the regression line. At issue above was whether regression is a good summary of the data: whether regression is appropriate. At issue here, in contrast, is whether the slope and intercept of the regression line were computed correctly.

Figure 10-5: An Impossible Residual Plot: The Mean of the Residuals is not Zero.

r: 0.05

Figure 10-6: An Impossible Residual Plot: The Mean of the Residuals is not Zero, and the Residuals have a Trend.

r: 0.96

If the residuals have a trend, the slope of the regression line was computed incorrectly. If the residuals do not have a trend but their average is not zero, the intercept of the regression line was computed incorrectly. If the residuals have a trend and their average is not zero, then the slope of the regression line was computed incorrectly, and the intercept of the regression line might also have been computed incorrectly.

If the regression line is computed correctly, then if we treat the residuals themselves as data and compute another regression line for them—that is, if we regress the residuals against X—that regression line will coincide with the x axis (a horizontal line at height y = 0).Note 10-2 ▾

The following exercise tests your ability to read residual plots for evidence of heteroscedasticity, nonlinearity, or outliers, and to determine whether the regression line was computed correctly and appropriately.

Summary

The residuals from a regression line are the values of the dependent variable Y minus the estimates of their values using the regression line and the independent variable X. If the i^th datum is (x_i, y_i) and the equation of the regression line is y = ax+b, then the i^th residual is

A residual plot is a scatterplot of the residuals versus their corresponding values of X, that is, a plot of the n points

A residual plot shows heteroscedasticity, nonlinear association, or outliers if and only if the original scatterplot does, but it is easier to see these qualitative features of bivariate data in the residual plot than in the scatterplot of the original data.

If the regression line is computed correctly, the correlation coefficient between the residuals and the independent variable is zero—the residuals do not have a trend with X—and the average of the residuals is zero. If the residuals have a trend, the slope of the regression line was computed incorrectly. If the residuals do not have a trend but the mean of the residuals is not zero, the intercept of the regression line was computed incorrectly. If the residuals have a trend and the mean of the residuals is not zero, the slope of the regression line was computed incorrectly, and the intercept of the regression line might or might not have been computed correctly.

Chapter 10

Regression Diagnostics