Examples of using the generalized likelihood ratio test.
Using the mtcars dataset, we will compare the full model to the reduced model. The full model includes four of the variables in the dataset. The reduced model includes only the variables disp and carb. The null model includes only the intercept. We will use the lrtest() function from the lmtest package to perform the likelihood ratio test.
Likelihood Ratio Test for Linear Regression
library (lmtest)#fit full model <- lm (mpg ~ disp + carb + hp + cyl, data = mtcars)
Call:
lm(formula = mpg ~ disp + carb + hp + cyl, data = mtcars)
Coefficients:
(Intercept) disp carb hp cyl
34.021595 -0.026906 -0.926863 0.009349 -1.048523
#fit reduced model <- lm (mpg ~ disp + carb, data = mtcars)
Call:
lm(formula = mpg ~ disp + carb, data = mtcars)
Coefficients:
(Intercept) disp carb
31.1527 -0.0363 -0.9557
#fit nullmodel <- lm (mpg ~ 1 , data = mtcars)
Call:
lm(formula = mpg ~ 1, data = mtcars)
Coefficients:
(Intercept)
20.09
#perform likelihood ratio test for differences in models lrtest (model_full, model_reduced)
Likelihood ratio test
Model 1: mpg ~ disp + carb + hp + cyl
Model 2: mpg ~ disp + carb
#Df LogLik Df Chisq Pr(>Chisq)
1 6 -77.558
2 4 -78.603 -2 2.0902 0.3517
lrtest (model_full, model_null)
Likelihood ratio test
Model 1: mpg ~ disp + carb + hp + cyl
Model 2: mpg ~ 1
#Df LogLik Df Chisq Pr(>Chisq)
1 6 -77.558
2 2 -102.378 -4 49.64 4.294e-10 ***
---
Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
lrtest (model_reduced, model_null)
Likelihood ratio test
Model 1: mpg ~ disp + carb
Model 2: mpg ~ 1
#Df LogLik Df Chisq Pr(>Chisq)
1 4 -78.603
2 2 -102.378 -2 47.55 4.729e-11 ***
---
Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
Check the calculations.
- 2 * log ( exp (- 78.60301 ) / exp (- 77.55790 ) )- 2 * ( - 78.60301 - (- 77.55790 ))2 * ( 78.60301 + (- 77.55790 ))pchisq (2.04522 , df = 2 , lower.tail = FALSE )
Likelihood ratio test for logistic regression
Using the infert dataset, we will compare the full model to the reduced model. The full model includes two of the variables in the dataset. The reduced model includes only the variable induced. The null model includes only the intercept. We will use the lrtest() function from the lmtest package to perform the likelihood ratio test.
education age parity induced case spontaneous stratum pooled.stratum
1 0-5yrs 26 6 1 1 2 1 3
2 0-5yrs 42 1 1 1 0 2 1
3 0-5yrs 39 6 2 1 0 3 4
4 0-5yrs 34 4 2 1 0 4 2
5 6-11yrs 35 3 1 1 1 5 32
6 6-11yrs 36 4 2 1 1 6 36
7 6-11yrs 23 1 0 1 0 7 6
8 6-11yrs 32 2 0 1 0 8 22
9 6-11yrs 21 1 0 1 1 9 5
10 6-11yrs 28 2 0 1 0 10 19
11 6-11yrs 29 2 1 1 0 11 20
12 6-11yrs 37 4 2 1 1 12 37
13 6-11yrs 31 1 1 1 0 13 9
14 6-11yrs 29 3 2 1 0 14 29
15 6-11yrs 31 2 1 1 1 15 21
16 6-11yrs 27 2 2 1 0 16 18
17 6-11yrs 30 5 2 1 1 17 38
18 6-11yrs 26 1 0 1 1 18 7
19 6-11yrs 25 3 2 1 1 19 28
20 6-11yrs 44 1 0 1 1 20 17
21 6-11yrs 40 1 0 1 1 21 14
22 6-11yrs 35 2 2 1 0 22 24
23 6-11yrs 28 2 0 1 2 23 19
24 6-11yrs 36 1 0 1 1 24 12
25 6-11yrs 27 2 1 1 1 25 18
26 6-11yrs 40 2 0 1 2 26 27
27 6-11yrs 38 2 0 1 2 27 26
28 6-11yrs 34 3 0 1 2 28 31
29 6-11yrs 28 4 1 1 2 29 34
30 6-11yrs 30 4 2 1 0 30 35
31 6-11yrs 32 1 0 1 1 31 10
32 6-11yrs 34 2 1 1 0 32 23
33 6-11yrs 42 1 1 1 0 33 16
34 6-11yrs 32 2 0 1 2 34 22
35 6-11yrs 39 1 1 1 0 35 13
36 6-11yrs 35 2 0 1 2 36 24
37 6-11yrs 36 1 0 1 1 37 12
38 6-11yrs 34 3 1 1 2 38 31
39 6-11yrs 30 3 0 1 0 39 30
40 6-11yrs 28 1 0 1 1 40 8
41 6-11yrs 39 3 0 1 2 41 33
42 6-11yrs 35 1 0 1 0 42 11
43 6-11yrs 41 1 0 1 0 43 15
44 6-11yrs 37 2 1 1 1 44 25
45 12+ yrs 30 1 0 1 0 45 44
46 12+ yrs 37 1 1 1 0 46 48
47 12+ yrs 28 2 0 1 2 47 51
48 12+ yrs 27 4 2 1 0 48 61
49 12+ yrs 26 2 2 1 0 49 49
50 12+ yrs 38 3 0 1 2 50 60
51 12+ yrs 24 3 1 1 2 51 56
52 12+ yrs 36 5 1 1 2 52 62
53 12+ yrs 27 3 1 1 1 53 57
54 12+ yrs 28 1 0 1 1 54 42
55 12+ yrs 29 2 0 1 2 55 52
56 12+ yrs 36 2 0 1 2 56 55
57 12+ yrs 28 2 1 1 0 57 51
58 12+ yrs 28 2 0 1 2 58 51
59 12+ yrs 28 1 0 1 1 59 42
60 12+ yrs 27 2 0 1 2 60 50
61 12+ yrs 35 2 0 1 2 61 54
62 12+ yrs 25 1 0 1 1 62 41
63 12+ yrs 34 1 0 1 1 63 47
64 12+ yrs 31 2 0 1 2 64 53
65 12+ yrs 26 2 1 1 0 65 49
66 12+ yrs 32 1 0 1 1 66 46
67 12+ yrs 21 1 0 1 1 67 39
68 12+ yrs 28 3 1 1 2 68 58
69 12+ yrs 37 3 0 1 2 69 59
70 12+ yrs 25 1 1 1 0 70 41
71 12+ yrs 32 1 1 1 0 71 46
72 12+ yrs 25 1 0 1 1 72 41
73 12+ yrs 31 1 0 1 1 73 45
74 12+ yrs 38 6 0 1 2 74 63
75 12+ yrs 26 2 0 1 2 75 49
76 12+ yrs 31 1 0 1 1 76 45
77 12+ yrs 31 2 0 1 1 77 53
78 12+ yrs 25 1 1 1 0 78 41
79 12+ yrs 31 1 0 1 1 79 45
80 12+ yrs 34 1 0 1 1 80 47
81 12+ yrs 35 2 2 1 0 81 54
82 12+ yrs 29 1 0 1 1 82 43
83 12+ yrs 23 1 0 1 1 83 40
84 0-5yrs 26 6 2 0 0 1 3
85 0-5yrs 42 1 0 0 0 2 1
86 0-5yrs 39 6 2 0 0 3 4
87 0-5yrs 34 4 0 0 1 4 2
88 6-11yrs 35 3 2 0 0 5 32
89 6-11yrs 36 4 1 0 1 6 36
90 6-11yrs 23 1 0 0 0 7 6
91 6-11yrs 32 2 2 0 0 8 22
92 6-11yrs 21 1 0 0 1 9 5
93 6-11yrs 28 2 0 0 1 10 19
94 6-11yrs 29 2 0 0 0 11 20
95 6-11yrs 37 4 1 0 1 12 37
96 6-11yrs 31 1 0 0 0 13 9
97 6-11yrs 29 3 0 0 1 14 29
98 6-11yrs 31 2 1 0 0 15 21
99 6-11yrs 27 2 1 0 0 16 18
100 6-11yrs 30 5 0 0 2 17 38
101 6-11yrs 26 1 0 0 0 18 7
102 6-11yrs 25 3 0 0 1 19 28
103 6-11yrs 44 1 0 0 0 20 17
104 6-11yrs 40 1 0 0 0 21 14
105 6-11yrs 35 2 0 0 0 22 24
106 6-11yrs 28 2 0 0 0 23 19
107 6-11yrs 36 1 0 0 0 24 12
108 6-11yrs 27 2 0 0 1 25 18
109 6-11yrs 40 2 0 0 0 26 27
110 6-11yrs 38 2 0 0 0 27 26
111 6-11yrs 34 3 0 0 0 28 31
112 6-11yrs 28 4 0 0 2 29 34
113 6-11yrs 30 4 1 0 1 30 35
114 6-11yrs 32 1 0 0 0 31 10
115 6-11yrs 34 2 1 0 0 32 23
116 6-11yrs 42 1 1 0 0 33 16
117 6-11yrs 32 2 0 0 0 34 22
118 6-11yrs 39 1 0 0 0 35 13
119 6-11yrs 35 2 0 0 0 36 24
120 6-11yrs 36 1 0 0 0 37 12
121 6-11yrs 34 3 2 0 0 38 31
122 6-11yrs 30 3 0 0 2 39 30
123 6-11yrs 28 1 1 0 0 40 8
124 6-11yrs 39 3 1 0 0 41 33
125 6-11yrs 35 1 0 0 0 42 11
126 6-11yrs 41 1 0 0 0 43 15
127 6-11yrs 37 2 0 0 0 44 25
128 12+ yrs 30 1 1 0 0 45 44
129 12+ yrs 37 1 0 0 0 46 48
130 12+ yrs 28 2 1 0 0 47 51
131 12+ yrs 27 4 2 0 1 48 61
132 12+ yrs 26 2 1 0 0 49 49
133 12+ yrs 38 3 1 0 0 50 60
134 12+ yrs 24 3 2 0 1 51 56
135 12+ yrs 36 5 1 0 1 52 62
136 12+ yrs 27 3 1 0 1 53 57
137 12+ yrs 28 1 1 0 0 54 42
138 12+ yrs 29 2 1 0 0 55 52
139 12+ yrs 36 2 1 0 0 56 55
140 12+ yrs 28 2 1 0 1 57 51
141 12+ yrs 28 2 2 0 0 58 51
142 12+ yrs 28 1 1 0 0 59 42
143 12+ yrs 27 2 1 0 0 60 50
144 12+ yrs 35 2 2 0 0 61 54
145 12+ yrs 25 1 1 0 0 62 41
146 12+ yrs 34 1 0 0 0 63 47
147 12+ yrs 31 2 0 0 0 64 53
148 12+ yrs 26 2 0 0 1 65 49
149 12+ yrs 32 1 0 0 0 66 46
150 12+ yrs 21 1 0 0 1 67 39
151 12+ yrs 28 3 2 0 0 68 58
152 12+ yrs 37 3 1 0 1 69 59
153 12+ yrs 25 1 0 0 0 70 41
154 12+ yrs 32 1 1 0 0 71 46
155 12+ yrs 25 1 0 0 0 72 41
156 12+ yrs 31 1 0 0 1 73 45
157 12+ yrs 26 2 0 0 2 75 49
158 12+ yrs 31 1 0 0 0 76 45
159 12+ yrs 31 2 2 0 0 77 53
160 12+ yrs 25 1 0 0 0 78 41
161 12+ yrs 31 1 0 0 0 79 45
162 12+ yrs 34 1 0 0 0 80 47
163 12+ yrs 35 2 0 0 0 81 54
164 12+ yrs 29 1 0 0 1 82 43
165 12+ yrs 23 1 0 0 1 83 40
166 0-5yrs 26 6 2 0 0 1 3
167 0-5yrs 42 1 0 0 0 2 1
168 0-5yrs 39 6 2 0 0 3 4
169 0-5yrs 34 4 0 0 2 4 2
170 6-11yrs 35 3 0 0 0 5 32
171 6-11yrs 36 4 0 0 2 6 36
172 6-11yrs 23 1 0 0 0 7 6
173 6-11yrs 32 2 0 0 1 8 22
174 6-11yrs 21 1 1 0 0 9 5
175 6-11yrs 28 2 0 0 1 10 19
176 6-11yrs 29 2 0 0 1 11 20
177 6-11yrs 37 4 0 0 1 12 37
178 6-11yrs 31 1 0 0 0 13 9
179 6-11yrs 29 3 0 0 2 14 29
180 6-11yrs 31 2 1 0 0 15 21
181 6-11yrs 27 2 0 0 0 16 18
182 6-11yrs 30 5 1 0 2 17 38
183 6-11yrs 26 1 1 0 0 18 7
184 6-11yrs 25 3 1 0 1 19 28
185 6-11yrs 44 1 1 0 0 20 17
186 6-11yrs 40 1 0 0 0 21 14
187 6-11yrs 35 2 0 0 0 22 24
188 6-11yrs 28 2 2 0 0 23 19
189 6-11yrs 36 1 0 0 1 24 12
190 6-11yrs 27 2 0 0 2 25 18
191 6-11yrs 40 2 0 0 0 26 27
192 6-11yrs 38 2 0 0 0 27 26
193 6-11yrs 34 3 0 0 0 28 31
194 6-11yrs 28 4 2 0 1 29 34
195 6-11yrs 30 4 1 0 1 30 35
196 6-11yrs 32 1 0 0 0 31 10
197 6-11yrs 34 2 0 0 0 32 23
198 6-11yrs 42 1 0 0 0 33 16
199 6-11yrs 32 2 2 0 0 34 22
200 6-11yrs 39 1 0 0 0 35 13
201 6-11yrs 35 2 0 0 0 36 24
202 6-11yrs 36 1 0 0 0 37 12
203 6-11yrs 34 3 2 0 0 38 31
204 6-11yrs 30 3 0 0 1 39 30
205 6-11yrs 28 1 0 0 0 40 8
206 6-11yrs 39 3 0 0 0 41 33
207 6-11yrs 35 1 0 0 0 42 11
208 6-11yrs 41 1 0 0 0 43 15
209 6-11yrs 37 2 0 0 0 44 25
210 12+ yrs 30 1 0 0 0 45 44
211 12+ yrs 37 1 0 0 1 46 48
212 12+ yrs 28 2 1 0 0 47 51
213 12+ yrs 27 4 2 0 0 48 61
214 12+ yrs 26 2 1 0 0 49 49
215 12+ yrs 38 3 1 0 0 50 60
216 12+ yrs 24 3 2 0 0 51 56
217 12+ yrs 36 5 2 0 1 52 62
218 12+ yrs 27 3 2 0 0 53 57
219 12+ yrs 28 1 0 0 1 54 42
220 12+ yrs 29 2 1 0 1 55 52
221 12+ yrs 36 2 0 0 1 56 55
222 12+ yrs 28 2 2 0 0 57 51
223 12+ yrs 28 2 1 0 0 58 51
224 12+ yrs 28 1 0 0 0 59 42
225 12+ yrs 27 2 1 0 0 60 50
226 12+ yrs 35 2 1 0 0 61 54
227 12+ yrs 25 1 1 0 0 62 41
228 12+ yrs 34 1 0 0 0 63 47
229 12+ yrs 31 2 1 0 0 64 53
230 12+ yrs 26 2 0 0 2 65 49
231 12+ yrs 32 1 1 0 0 66 46
232 12+ yrs 21 1 0 0 0 67 39
233 12+ yrs 28 3 2 0 0 68 58
234 12+ yrs 37 3 0 0 2 69 59
235 12+ yrs 25 1 1 0 0 70 41
236 12+ yrs 32 1 0 0 0 71 46
237 12+ yrs 25 1 1 0 0 72 41
238 12+ yrs 31 1 0 0 0 73 45
239 12+ yrs 38 6 0 0 2 74 63
240 12+ yrs 26 2 1 0 1 75 49
241 12+ yrs 31 1 1 0 0 76 45
242 12+ yrs 31 2 0 0 1 77 53
243 12+ yrs 25 1 0 0 1 78 41
244 12+ yrs 31 1 0 0 1 79 45
245 12+ yrs 34 1 0 0 0 80 47
246 12+ yrs 35 2 2 0 0 81 54
247 12+ yrs 29 1 0 0 1 82 43
248 12+ yrs 23 1 0 0 1 83 40
<- glm (case ~ induced + spontaneous, family= binomial, data= infert)
Call: glm(formula = case ~ induced + spontaneous, family = binomial,
data = infert)
Coefficients:
(Intercept) induced spontaneous
-1.7079 0.4181 1.1972
Degrees of Freedom: 247 Total (i.e. Null); 245 Residual
Null Deviance: 316.2
Residual Deviance: 279.6 AIC: 285.6
<- glm (case ~ induced, family= binomial, data= infert)
Call: glm(formula = case ~ induced, family = binomial, data = infert)
Coefficients:
(Intercept) induced
-0.71535 0.04897
Degrees of Freedom: 247 Total (i.e. Null); 246 Residual
Null Deviance: 316.2
Residual Deviance: 316.1 AIC: 320.1
<- glm (case ~ 1 , family= binomial, data= infert)
Call: glm(formula = case ~ 1, family = binomial, data = infert)
Coefficients:
(Intercept)
-0.6871
Degrees of Freedom: 247 Total (i.e. Null); 247 Residual
Null Deviance: 316.2
Residual Deviance: 316.2 AIC: 318.2
lrtest (model_full, model_reduced)
Likelihood ratio test
Model 1: case ~ induced + spontaneous
Model 2: case ~ induced
#Df LogLik Df Chisq Pr(>Chisq)
1 3 -139.81
2 2 -158.05 -1 36.487 1.537e-09 ***
---
Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
lrtest (model_full, model_null)
Likelihood ratio test
Model 1: case ~ induced + spontaneous
Model 2: case ~ 1
#Df LogLik Df Chisq Pr(>Chisq)
1 3 -139.81
2 1 -158.09 -2 36.559 1.152e-08 ***
---
Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
lrtest (model_reduced, model_null)
Likelihood ratio test
Model 1: case ~ induced
Model 2: case ~ 1
#Df LogLik Df Chisq Pr(>Chisq)
1 2 -158.05
2 1 -158.09 -1 0.0724 0.7879