BUILDING REGRESSION MODELS

1. AGENDA:
1. Review of previous examples
2. Model building
3. Time series and intervention analysis
4. Reading:
1. Agresti and Finlay, Statistical Methods, Chapter 10, pages 356 to 371



2. PREVIOUS EXAMPLES:
1. See the notes for Class 20.
1. Population growth.
2. Public expenditures data.
2. Multiple variable case.
1. Add independent variables to the list of factors or predictors.
3. One can and should obtain residual plots. Do so in the same way as before: use plotting options or store residuals and fitted values and then plot them.



3. MODEL BUILDING:
1. Problem: suppose we want to study the effects of pollution on health, and in particular on mortality.
1. Goal: find what effects pollution have on mortality after taking into account social-economic status and living conditions.
2. For this study we'll have to rely on "aggregate" data.
3. The units of analysis are American metropolitan areas
2. Data: the data we'll use come from McDonald, G.C. and Schwing, R.C. (1973) "Instabilities of Regression Estimates Relating Air Pollution to Mortality", Technometrics, vol.15, 463-482.
1. I obtained them from the "Statistic Library" at Carnegie Mellon University in the data section of the Journal of the American Statistical Association (JAMA). (See the course web site.)
2. Data
3. The number of cases is 60 American metropolitan areas
4. Variables:
1. Average annual precipitation in inches (c1)
2. Average January temperature in degrees F (c2)
3. Same for July (c3)
4. % of 1960 SMSA population aged 65 or older (c4)
5. Average household size (c5)
6. Median school years completed by those over 22 (c6)
7. % of housing units which are sound & with all facilities (c7)
8. Population per sq. mile in urbanized areas, 1960 (c8)
9. % non-white population in urbanized areas, 1960 (c9)
10. % employed in white collar occupations (c10)
11. % of families with income less than $3000 (c11)
12. Relative hydrocarbon pollution potential (c12)
13. Same for nitric oxides (c13)
14. Same for sulphur dioxide (c14)
15. Annual average % relative humidity at 1pm (c15)
16. Total age-adjusted mortality rate per 100,000 (c16)
5. We can't use all of these variables for this example.
3. A "modeling" strategy: since we have so much data we need a plan.
1. For this example we'll restrict the analysis to a subset of variables divided as follows:
1. Pollution indicators (hydrocarbons, nitric oxide, and sulphur)
2. Social and economic factors: percent of the population that is poor and median school years completed.
3. Discrimination (percent of population non-white).
4. We would use a "theory" to select these variables. That is, we wouldn't add them to our model simply because they came with the data set.
1. For example, we might hypothesize the non-white residents would be less healthy because of lack of access to health care, nutrition, and so forth.
2. First, we need to examine the distributions.
1. We want to "deal with" variables that are skewed or have outliers.
2. Using descriptive statistics (such as comparing means and medians) and stem-and-leaf plots we find that all of the pollution variables are skewed to the right with several outliers.
1. A quick glance at bivariate plots of the dependent variable, mortality, against hydrocarbons or sulphur makes the need for transformation apparent.
1. See the previous notes regarding outliers.
2. One way to make these variables more symmetric is to transform them with the natural logarithm (the log to the base e).
3. You've already done this for other variables.
4. Doing so does in fact make them more symmetric.
3. Let's also see if their are patterns of correlations between the variables.
1. Example: correlations of mortality with social and economic factors and discrimination:

1. We can see that mortality rate is negatively correlated with education and positively related to both poverty and discrimination.
1. What does this mean?
2. Example: the higher the median years of education, which is a measure of well being, the lower the mortality rate. Makes sense, right?
2. Here are the correlations of mortality with the pollution indexes:

1. Notice that the relationships are rather weak. Plots between Y (mortality) and the X's show why.
1. Multicolinearity:
1. Important note the correlation between hydrocarbons and nitric oxides: r = .984
2. This suggests among other things that two of the predictors are themselves highly correlated and that, in fact, they are most "synonymous" or redundant.
3. We'll have to do something. The most likely and easiest strategy is to drop one of them from the analysis.
2. When the variables are transformed by taking the logs we obtain stronger correlations:

1. Now the r's are a bit stronger. But note the multicolinearity
2. We can later verify that using transformed variables improves the analysis.
1. Regression analysis:
1. Let's first regress the dependent variable against the pollution indicators:

1. Note that these are partial regression coefficients.
1. 11.8 means, for example, that as "log sulphur" rate increases by one unit, mortality increases by 11.8 per 100,000 population, after the other pollutants have been held constant.
2. This suggests that each type of pollutant has an effect.
3. Yes the coefficient for log hydrocarbons is negative. That's probably because it's so highly related to nitric oxides: the hydrocarbon and nitric oxide emissions r is .97, an almost perfect correlation.
2. We can and thus should remove one of these indicators and will do so later. But for now let's just add other variables.
3. First note this model fits the data moderately well: R² is .275.
4. An aside: when using the untransformed variables the fit is slightly better, R² = .341. But this may be misleadingly high since there are several outliers that may be making the relationship seem stronger than it perhaps is.
1. Now lets add the social and economic measures:

1. Again we have partial regression coefficients. The partial betas pertaining to nitric oxides and hydrocarbons have changed substantially, suggesting that the multicolinearity is causing the estimates to be "unstable."

1. The addition of these variables increases R² to .483.
1. Other indicators suggest an improvement in fit as we'll discuss in class.
2. Now let's add the discrimination variable, percent non-white.

1. The fit for this model is quite good, R² = .682. Other statistics and indicators also suggest the fit is solid.
1. Eliminating redundancy between hydrocarbons and nitric oxides:
1. The last model includes 6 independent variables. Do we need them all?
2. It turns out that further analysis reveals that we can "explain" mortality with just one pollution variable along with education and percent non-white:

1. The multiple correlation (R²) is .642, only slightly lower than the previous one.
1. We can and perhaps will further improve this model. For now keep several points in mind.
1. Model building is an "interaction" between substantive theory and data analysis.
2. One has to proceed systematically and carefully.
3. At each step check the tenability of assumptions.
4. Realize that various fits are possible. Each has to be justified on its own terms.



1. INTERVENTION ANALYSIS:
1. Suppose the Department of Public Safety wants to investigate a "neighborhood" watch program's effects on crime rates.
1. It has collected information on the crime in an area of Chicago both before and after the start of the program.
1. The data are the monthly number of burglaries.
2. c1 is Burglary rate
3. c2 is a "dummy variable":
   1. X = 0 for time periods before intervention
   2. X = 1 for data points after the intervention.
2. The Department wants to know how effective the watch program has been.
2. This problem represents a common type of analysis called "intervention analysis."
1. There is a time series; that is, measurement collected on (equally spaced) time intervals such as months or years.
2. At some point an "intervention"--the adoption of a policy, program, or law for example--"interrupts" the series.
3. The diagram (Figure 1) provides an illustration.

1. Note that an intervention can any of several possible consequences.
1. It can, as in Figure 1, change the "level" and/or rate of change ("slope") of a series.
1. For example, one might hypothesize that the rate of burglaries will slow down or even decline after the institution of the watch program.
2. It's possible that just the level will change, not the rate or slope of the series. (See Figure 2 below.)

1. Or, the slope or rate of the series can change as in Figure 3.

1. Each of these possibilities has different policy implications.



1. A MODEL TO INVESTIGATE THESE POSSIBILITIES:
1. We can use regression analysis to determine which, if any of these, is appropriate for the data.
2. The dependent variable is measured at different time periods.
1. Hence time is in a sense the unit of analysis.
2. If the measurements have been made at N times, then the series consists of N time periods.
3. In the present example (the one pertaining to neighborhood watch) there are 78 data points, some of which pertain to the months before the program started, some of which pertained to the period after.
3. Time series plot:
1. Most programs such as MINITAB have a time series plot that can be used to plot the data points. MINITAB automatically creates an indicator variable.
2. See the attached plot for an example.
4. Dummy coding.
1. Now, suppose we create a variable, X, with the following numeric codes or values:
1. X = 0 for time periods before the intervention and X = 1 for periods or post intervention measures.
5. We can use this variable as the independent factor in a simple regression model.

1. Interpretation:
1. We could use the standard definition or conceptualization of the regression coefficient to make sense of the terms in this model.
1. A one unit change in X leads to a b₁ unit change in Y, with b₁ being measured in units of the dependent variable.
2. But what is a one unit change in X. Here it represents a change from "before" the intervention to "after."
3. That is, those units (i.e., time periods) that have "scores" of 0 are the ones that occur before the initiation of the neighborhood watch program, and those that are coded 1 have the after units.
4. Consequently if a "unit" "could" jump from before the intervention to after, the effect on the burglary rate would be b₁.
2. Example: the estimated equation for these data (using simple OLS with the intervention variable scored 0 and 1 being treated as the independent factor) is:

1. Interpretation:
1. A change from "before" to "after" leads to 3.78 decrease in monthly burglaries, on average.
1. Another view point:
1. Since there are only two values for X (0 and 1), why not substitute each of them in the equation to see what results?
2. First, here is the equation for the before "data points; that is the ones scored 0:

1. We see that the predicted (or expected) burglary rate before the intervention is 64.4.
2. Now what about after? Well, the units are now coded 1, so the equation becomes.

1. The predicted or expected rate after is now 60.4 or 3.78 fewer burglaries than previously.
1. The analysis suggests that the intervention had an effect--the regression coefficient was not zero--and the effect was to decrease crime--the sign of the coefficient is negative.
1. It turns out, however, that the model doesn't fit the data very well in the sense that this observed effect could just be a "chance" or random phenomenon. We would need more evidence before concluding definitively that the program works.
2. Moreover, we can add a couple of terms to model changes in level as well as rate.
3. We'll see more examples next time.



1. NEXT TIME:
1. Some more examples of intervention
2. Statistical estimation and inference.

Go to Statistics main page

Go to H. T. Reynolds page.

Copyright © 1997 H. T. Reynolds