Journal of Educational Leadership in Action
A global platform for ongoing dialogue in research based practice of educational leadership
This study utilized base-year and second follow-up data from the National Educational Longitudinal Study of 1988 to investigate the relationship between eighth-grade math achievement, mathematics course-taking in high school, and twelfth-grade math achievement. Results suggested the following: 1) Type of coursework can be quantified. 2) Type of coursework was more predictive of achievement than amount. 3) There were substantial ethnic achievement differences prior to high school. 4) Number of courses, type of courses, and prior achievement were not equally predictive of twelfth-grade mathematics achievement across ethnic groups. 5) Prior achievement did not equally predict course-taking over ethnic groups in amount or type. 6) Closing ethnic achievement gaps will be a function of efforts taken before high school as well as high school coursework.
This study explored ethnic differences in mathematics achievement, mathematics course-taking, and the relationship between course-taking and achievement. The first goal was to explore the relationship between amount and content of coursework in predicting ethnic gaps in math achievement, as there was evidence that type of course may be more predictive than amount. A second goal was to provide a methodologically sound approach for quantifying course content, as it has been operationalized in various ways in prior research. Next, the authors explored the relationship between prior achievement and course-taking, as previous research suggested that students may receive differential advice on course-taking based on ethnicity (The Education Trust, 1996). Finally, we explored differences in predicting math achievement as a function of ethnicity.
2. Course Taking and Achievement
Among its many suggestions, A Nation at Risk (National Commission on Excellence in Education, 1983) recommended that high school students take four years of English, three years of mathematics, science, and social studies, plus one-half year of computer science. According to Clune (1989), forty-one states had created or increased high school graduation requirements by 1984. Schiller and Muller (2003) discussed the impact of increased graduation requirements on mathematics course-taking. Those graduation requirements tended to specify the amount of required coursework, but did little to specify the content of those courses. Presumably these policy changes were intended to address ongoing issues of achievement, two of the most salient being to raise student achievement overall and to close achievement gaps among student groups. Subsequently, efforts have begun to concentrate more on type of coursework, allowing for the possibility that “all courses are not created equal” (ACT Inc., 2005; American Diploma Project, 2004; and National Governors Association for Best Practices, 2005). Note that the ACT report (ACT Inc., 2005) suggested at minimum one advanced course beyond the level of Algebra 2.
Teitelbaum (2003) examined the influence of higher graduation requirements on amount of coursework and on student achievement gains in math and science. He found in states where more math and science coursework was required, students took more courses. However, he did not find evidence for greater achievement gains in those states. He cited two possible reasons why greater gains were not found in states with higher requirements: high schools may not consistently hold students to the higher requirements, and students did not consistently meet the higher requirements by taking advanced courses as may have been intended. By itself, increased coursework may not increase achievement, but will it close ethnic achievement gaps? Minority and majority differences in amount of coursework are small, suggesting that gaps in amount may not account for gaps in achievement. Table 1 provides data from the 2007 Digest of Education Statistics (Snyder, Dillow & Hoffman, 2008) that revealed math course-taking by gender and ethnicity for seven years over the interval of 1982 to 2005. One can see a rise in math course-taking over the years. In fact, the correlation between units taken and year was 0.979, indicating a consistent rise in courses taken over time. The course-taking differences shown in Table 1 do not mirror the typical performance differences we see. For instance, the White/Black difference in Carnegie units were small, ranging from 0.13 favoring Whites (1994) to a 0.07 difference favoring Blacks (1990). The mean difference was less than 0.02 favoring Whites. Moreover, for three of the seven time points Blacks took more mathematics courses (1990, 1998, and 2005) (see Table 1).
Several longitudinal studies have been conducted to understand growth patterns for student math achievement for different student groups (Dalton, Ingels, Downing & Bozick, 2007; Ding & Davison, 2005; Flores, 2007; Ma & Wilkins, 2007; Ma & Ma, 2005; Manzo & Cavanagh, 2007; Shettle, Roey, Mordica, Perkins, Nord, Teodorovic, Brown, Lyons, Averette & Kastberg, 2007). These studies showed that, as for high school math courses, there were no detectable differences between the number of credits earned by most students, but there were differences between the highest level of math taken. Moreover, disadvantaged students had lower initial achievement and difficulty catching up to their advantaged counterparts. For example, 37% of White graduates in 2003 took pre-calculus and calculus as opposed to 19% of Blacks (Dalton et al., 2007). These results suggest that types of courses taken may contribute to variation in academic preparation and possibly mathematics achievement by the end of high school.
Several studies examined the importance of taking higher level math courses, such as Algebra I, in 8th grade to increase math achievement (Cavanagh, 2007;Ma & Wilkins, 2007; Spielhagen, 2006). Byrnes & Miller (2006) predicted 10th and 12th grade achievement from three sets of factors in the 8th grade: opportunity factors (e.g., courses), propensity factors (e.g., prerequisite skills), and distal factors (e.g., SES), demonstrated that 58-81% of the variance in achievement was accounted for by family variables, specific opportunity, and propensity factors. Wang and Goldschmidt (2003) used longitudinal data over three points (grades 9-11) and regressed with students’ 8th grade courses and test scores to find how students’ middle school math course-taking influenced later math achievement. The distribution of math courses among various student subgroups differed by grade 8 and became increasingly inequitable by grade 11. Consequently, high school achievement scores were higher for students who enrolled and performed well in advanced math classes in 8th grade versus those enrolled in regular and remedial classes.
Studies have also examined the relationship between particular high school courses and factors associated with advanced course-taking, prediction of math achievement, and college preparation (Ercikan, McCeith & Lapointe, 2005; Paul, 2005; Shettle, et al., 2007; Trusty & Niles, 2003; Tyson, Lee, Borman & Hanson, 2007). Leow, Marcus, Zanutto, and Boruch (2004) analyzed the effects of advanced course-taking on math and science achievement using propensity score methods and sensitivity analysis. Even after accounting for the effects of bias from unobserved background variables, they concluded that to some degree, advanced course-taking improved scores on basic achievement tests. Reigle-Crumb (2006) discovered that taking Algebra I or higher the first year of high school had a strong relationship with the level of math attained by 12th grade. Consequently, inadequate academic preparation and poor performance the first year of math for particular groups of students led to decreased opportunity to advance to higher math courses (Reigle-Crumb, 2006; Trusty & Niles, 2003; Paul, 2005; Tyson, Lee, Borman & Hanson, 2007).
Davenport, Davison, Kuang, Ding, Kim & Kwak (1998) examined the amount of coursework taken by ethnic groups in each of several math content areas as reflected in NAEP transcripts. Like the data in Table 1, their data showed that ethnic groups differed little in amount of high school coursework in math. Thus, policies that do no more than increase coursework would not necessarily be expected to close achievement gaps. Both coursework amount and content have been found to be important (Education Trust, 1996; Jones, Burton & Davenport, 1984; Jones, Davenport, Bryson, Bekhuis & Zwick, 1986; Lee, Burkham, ChowHoy, Smerdon & Gverdt, 1998; Rock & Pollack, 1995; Shakrani, 1996; ). If, as found by Teitelbaum (2003), raising the required number of Carnegie units in an area does not necessarily lead to more advanced content, then distinctly different policy initiatives would seem necessary for the improvement of coursework amount versus coursework content.
One way to index student variation in course content is to categorize students by the highest course completed (Burkam & Lee, 1997; Teitelbaum, 2003). This approach has at least three limitations. First, such an approach takes into account only the highest level course and not the full array of the student’s coursework. Second, such a system constitutes a polytomous variable, the analysis of which can pose problems, as when Teitelbaum analyzed his four-category variable in a hierarchical logit regression and found that his software would not permit the use of the NELS weights. Third, the highest level of coursework, coupled with the amount of coursework, may not fully capture the relationship between achievement and coursework information.
3. Ability and Course-taking
If type rather than number of courses is more predictive of math achievement as many of the above studies suggest and if advanced course-taking is consistently associated with higher achievement after accounting for background variables as suggested by Leow, et al. (2004), one should advise students to take the highest math course possible, given their ability. Researchers reviewing the relationship in one large California school district between performance on the Comprehensive Test of Basic Skills and placement in algebra found glaring inequities: 100% of Asians and 87.5% of Whites performing in the top quartile were enrolled in algebra, while only 51% African American and 42% Latino top-quartile students were so enrolled. Moreover, Asians performing in the third quartile were more likely to be placed in algebra than African Americans and Latinos scoring in the top quartile (The Education Trust, 1996). Differential advising may be a function of how counselors view students. Frame (1984) found counselors to attribute learning problems to external factors that could be addressed via remediation, if the student was White and/or affluent. For Black and/or poorer students, the counselors attributed poor performance to the student and thus did not recommend remediation. The longitudinal nature of the data for the current study gives a unique opportunity to address course-taking given prior achievement. The authors do not advocate giving all students the highest math course that their school has to offer. We are aware that the relationship between level of math and math achievement is correlational, not causal. In addition, a recent article by the Brown Center on Educational Policy at The Brookings Institution (Loveless, 2008) cautions against placing students indiscriminately in advanced math courses, as does a study by Allensworth and Nomi (2009). Our research addressed the degree to which a student’s ability (prior achievement) relates to later math course-taking. We wished to know whether students are appropriately placed in advanced math courses as predicted by prior math ability and whether this placement is related to ethnicity.
4. Differential Prediction
Finally, we assessed the degree to which there were differential prediction equations for math achievement given ethnicity. Differential prediction would suggest the relationship of course-taking and achievement differs by ethnicity. Such differences would suggest that much of the prior research suffers from a lack of complexity in the prediction model in that most previous research used a “one model fit all” approach across ethnicity.
This study used data from the National Educational Longitudinal Study of 1988 (National Center for Education Statistics, Undated; Curtin, Ingels, Wu & Heuer, 2002). Eighth-graders in 1988 were followed longitudinally. The study useddata from the base year (1988) when the students were at the beginning of their high school career and second follow-up (1992) when the students were high school seniors. In addition to providing a wealth of demographic information over a period of time for a base sample of 25,000 students in 1988, the NELS survey also gathered test data. Students were tested in four learning areas (reading, mathematics, science, and social studies) at three time points: 1988, 1990, and 1992. Finally, the data also consisted of high school transcript information for most of the students. The sample for the current study included students for whom there were at least one transcript entry per year for four grades (9–12) and who also had valid math achievement data for both 1988 and 1992. Using these restrictions, the resulting sample size was 10,240. There are more current national longitudinal data sets such as the Educational Longitudinal study of 2002 (Ingels, Pratt, Wilson, Burns, Currivan, Rogers & Hubbard-Bednasz, 2007). However, that data starts with 10th graders and thus the students are already in high school. NELS:88 starts with 8th graders and thus we have a measure of performance for the students that is un-confounded with high school attendance. Furthermore, using the NELS 8th grade scores as a covariate allowed the authors to statistically control for all other factors present at the student’s enrollment in high school, thus allowing us to more easily parse the relationship of high school course-taking to changes in mathematics achievement as measured by the student’s 12th grade mathematics tests.
Weighting and Standard Errors
All analyses reported below employed the transcript weight. This weight was designed to make the weighted sample of students for whom transcripts were collected representative of the national population of high school students in 1992. Given that the data were collected from students sampled within schools violating the assumption of independence, regular standard errors were inappropriate (Kish, 1965). Thus, all standard errors for statistical tests were computed assuming half as many subjects (design effect of 2), which was appropriate for an analysis including variables at only one level. Note that standard practice suggests using hierarchical linear modeling (Raudenbush & Bryk, 2002) or re-sampling techniques such as “jack-knifing” to produce appropriate standard errors for such cluster samples. A discussion of “Design Effects and Approximate Standard Errors” can be found in NCES’s user’s manual (Curtin, et al., 2002). As one samples more subjects from the same cluster, the design effect increases. The use of a design effect has precedence as seen in Hoffer (1997). His work in the area of student achievement likewise assumed a design effect of two (see notes to his Table 1).
Math Coursework Categories
The course content variable adopted here had advantages over previous approaches. First, it constituted a description of the students’ course content in terms of a continuous variable that accounted for all of the student’s coursework, not just the highest level of coursework. Second, the content variable was suitable for analysis as a continuous variable in readily available software. Third, and most importantly, our course content variable, coupled with an amount of coursework variable, fully captured the complete relationship between available coursework information and achievement and account for all variation in achievement. By using the amount and content variables described below, the researcher did not risk mis-specifying the variables in a way that leads to underestimation of the variation in achievement that can be accounted for by coursework information.
The procedure began by describing a taxonomy of math courses that fully captured the pattern of courses taken by individual students. Davenport, Davison, Bielinski, Ding, Kuang, Li & Seiden (1995) used multidimensional scaling to develop a reasonably concise, but comprehensive, taxonomy for math courses. This taxonomy consists of several prototypical course sequences. A prototypical course sequence is an empirically derived set of courses that are taken by a significant subset of students. If students who take course “A” are more likely to take courses “B” and “D” as well, then courses “A,” “B,” and “D” will define a prototypical course sequence and there will be a significant number of students who take each of these three courses. Any other sets of courses with elevated probabilities of being taken by a substantial number of students will also emerge as a prototypical course sequence. The final taxonomy places 56 math courses in the Classification for Secondary School Courses (CSSC) into seven identifiable course sequences plus an “Other” category. The CSSC course titles used by many of the national surveys and assessments are described in Legum, Caldwell, Goksel, Haynes, Hynson, Rust & Blecher (1993). Table 2 shows the eight empirically derived course sequences that are shared by enough students to emerge as a distinct course-taking pattern.
In the taxonomy of Table 2, Functional courses were at the lowest end of math literacy representing survival skills in mathematics. Basic courses were the minimal courses required for general math literacy. Preformal courses may prove to be terminal courses for some students, but they can provide background for other students who subsequently take more advanced math courses. The algebra sequence was composed of an Algebra 1 course given in two parts over two years. The Standard sequence consisted of Algebra 1, Algebra 2, and Geometry and was the minimal set of math courses for a student on an academic track. Unified courses represent a different packaging of algebra and geometry concepts whereby the topics are presented in an integrated manner. Courses in the Advanced sequence were usually taken by students preparing for college. Finally, the Other category contained courses that were not easily interpretable as part of any common high school course grouping or were special offerings (often unique to a small number of schools and not taken frequently by students).
High school transcript data were used to obtain course information for each student. First the researchers computed the number of Carnegie units earned for all math courses as well as the number of Carnegie units earned in each of the eight course categories. The dependent variable, math achievement, was measured at base-year (1988 when the students were in 8th grade) as prior achievement and at the second follow-up (1992 when the students were in 12th grade). Use of prior achievement as a covariate was intended to statistically control for all other potential prior differences in the students—whether these differences are demographic, academic, environmental, behavioral, etc.—prior to high school.
Coursework and True Patterns
Meehl (1950) stated that pattern (profile) was one of the most important words in the clinician’s vocabulary. This is true given the utility of score profiles in diagnoses (predictions). The authors argue that patterns of course-taking can also be used in predicting achievement. If true, then a methodology that allows one to actually utilize course-taking patterns would be useful. This is different than what others have done previously while claiming to address course-taking patterns using number of courses, highest course, etc. None of these approaches really gets at patterns of coursework taken.
Cronbach and Gleser (1953) stated there are three defining characteristics of a profile: elevation, scatter, and shape. They noted, “Elevation is the mean of all scores for a given person. Scatter is the square root of the sum of squares of the individual’s deviation scores about his own mean; that is, it is the standard deviation within the profile. Shape is the residual information in the score set after equating profiles for both elevation and scatter.” Davison & Davenport (2002) give a methodology to operationalize information from course-taking into the components defined by Cronbach and Gleser (1953).
The method used the Carnegie units (CUs) successfully completed for courses in each of the eight course categories defined above. All of the predictive information contained in the pattern of course-taking for the eight course categories was contained in two variables: amount of coursework and pattern of coursework. Amount of coursework referred to the number of CUs earned across all eight course categories. The distribution of coursework over categories with the effect of amount removed was called the pattern of the coursework. This followed from Davison and Davenport (2002) and corresponded to the scatter and shape components of Cronbach and Gleser (1953). Pattern is operationalized as the covariance of the CUs successfully taken in each course sequence with the regression weights from regressing the eight variables on the achievement measure (12th grade math test). This variable, computed for each student, is a criterion match statistic, representing the degree to which a student’s course-taking over the eight course sequences matches an optimal pattern. The optimal pattern differentiated course-taking over the eight course categories to distinguish high achieving students from low achieving students. Thus, this variable was named course content. While it may seem unusual to describe a pattern of scores with a set of numerical coefficients, this is similar to what was done in ANOVA when a set of a priori contrast coefficients is used to describe an hypothesized pattern of mean scores. Our coefficients constitute the “criterion pattern,” one that maximizes the variance accounted for in the criterion variable. Together, Content and Amount predict the same amount of variation in math achievement as the original eight course categories.
Table 3 revealed results of regressing senior math achievement onto the amount of courses taken in the eight course categories. The eight course categories accounted for 57.4% of the variation in senior math achievement. The raw regression weights are in column 2 followed by the modified standard errors of the regression coefficients assuming a design effect of two in column 3. The subsequent T values (based on these modified standard errors) follow in column 4. All T values were highly significant with the exception of algebra. Each of the Course Pattern coefficients is the un-standardized regression coefficient expressed as a deviation about the mean of the regression coefficients for the eight course categories. This latter index followed from the definition of pattern as what remained after elevation was accounted for. These coefficients specified the Course Pattern as a set of within person contrast coefficients, and like contrast coefficients in ANOVA, the coefficients sum to zero and therefore yielded both positive and negative values. Hereafter, they are called the Advanced Pattern Coefficients because more advanced course categories have the higher coefficients. These regression weights map well with our expectation that students taking higher level math courses are more apt to score higher on math tests and Figure 1 clearly showed this relationship. The pattern values relating the beta weights of the course categories to achievement showed higher weights for categories with more advanced mathematics content (see Table 3 and Figure 1).
Table 3 also showed variance inflation factors (VIF) for each predictor. VIF values in excess of 10 suggested multicolinearity problems (Kutner, Nachtsheim, Neter & Li, 2005, 409). None of the VIFs in Table 3 were above 2.5. Given such small values of VIF, the presence or absence of another course category in the model should have had little effect on the results of the remaining parameter estimates. Thus, the regression estimates for each of the categories should remain fairly stable with the inclusion or exclusion of the other categories as predictors. While we have not shown skewness indices for each category, some of the course category frequencies were fairly skewed, particularly for Functional, with a skewness index of 18.00. The next highest skewness index was 5.37 (Basic). However, dropping Functional from the regression (not shown) had little effect on the variance accounted for or any of the other regression coefficients (as expected due to the small VIF values). Given the acceptable VIFs and the small effect of inclusion versus exclusion of the most highly skewed predictor in our model, the researchers concluded that the statistical properties for all of the predictors were acceptable. Columns 7 and 8 of Table 3 show the mean and standard deviation of Carnegie units for each of the eight course categories. The low numbers for some of the course categories indicate little activity in those courses. Also, note that the Standard sequence is by far the most prevalent, as expected (hence its name).
For each student, Content was computed as the covariance of the number of CUs taken in each course category versus the regression weight for that category. A student would receive a high Content score if their course-taking reflected higher numbers of CUs for the more advanced math courses and lower numbers of CUs for the least advanced courses. Students with negative Content scores would have taken most of their coursework in less advanced categories, ones with negative coefficients. Table 4 illustrates Amount and Content values for four students. Subjects 1 and 2 differed on amount of coursework (3 units versus 6). While students enrolled in courses from the same categories, Standard and Advanced, Subject 2 enrolled in more of these classes. Thus, Subject 2’s Content score was higher, since (s)he had relatively higher CUs for the more advanced courses, matching the optimal course pattern better. In contrast, Subject 3 took more low level courses and had a pattern that was a mirror image of the optimal and thus a negative Content score. Also note that Subject 3 took more courses than Subject 1. Subject 4 had a flat pattern, taking one course each of the Basic, Standard, and Unified sequences (with no advanced coursework). Note, for all subjects in Table 4 the Math 12 score was in the same rank order as the Content score (see bottom of table). This was not true for Amount (see Table 4).
Senior Math Achievement
Correlations among prior math achievement (8th grade math test), senior math achievement (our primary dependent variable), total number of Carnegie units earned in math (Course Amount), and Course Content (course-taking pattern as operationalized by the criterion match statistic) are shown in Table 5. Three findings from this analysis are useful in understanding later results. First, both prior achievement and senior achievement were correlated with coursework (Amount and Content), but senior achievement was more highly correlated with coursework than was prior achievement. Second, Content and Amount were correlated (r = .56). This means that in general, students who took more courses showed a pattern of more advanced courses. Third, Content was more highly correlated with both math tests than Amount; indicating the extra utility of type of coursework in predicting mathematics achievement over amount of coursework. Finally, the single variable, Content (r = .76, r2 = 57%), accounted for as much variation in the 12th grade math test as did virtually all eight coursework variables (See the R2 at the bottom of Table 3, 57.4%).
Amount accounted for 22% of the variation in senior math achievement. Content, by itself, accounted for 57%, more than twice as much as did Amount. With respect to the unique increments in R2 for predicting senior achievement above and beyond the other variable, Amount added virtually nothing (0.4%) to the variance that can be predicted from Content alone. In contrast, Content added an additional 35% to the percent of variance that can be predicted from Amount alone. Because Amount makes almost no unique contribution, these data raised the possibility that Amount of coursework was associated with senior achievement largely because students taking more coursework often (but not always) progressed to more advanced courses. As stated above, Amount and Content together accounted for the same variation in senior math achievement as did all eight math course category variables in Table 3.
Residual Gains in Math Achievement
Residual gains were computed by taking the difference between the student’s actual and predicted 12th grade math achievement scores using 8th grade math achievement as the predictor. These residual gain scores were regressed onto Amount and Content. The results are given in Table 6. In this analysis we examined whether students starting at the same level of prior achievement but with differing amounts or Content of coursework made the same gains in high school math achievement in 12th grade. Amount alone accounted for 9.5% of the variation in residual gains. Content alone accounted for 13.7%. Amount added only 1.5% to the variation in residual gains accounted for by Content alone. Content, however, added 5.7% to the variation accounted for by Amount alone. Similar to senior achievement, the data suggested that Amount of coursework added to the prediction of residual gains in high school largely because students who took more mathematics tended to show a pattern of more advanced coursework. In all cases, content of coursework was more predictive than amount.
7. Achievement Gaps
Table 7 showed means and standard deviations for the 8th grade math test, the 12th grade math test, Amount, and Content by ethnicity. For both 8th and 12th grades, Asians had the highest mean achievement followed by Whites, Hispanics, Blacks, and American Indians. On the Amount variable, Asians had the highest mean number of Carnegie units (3.44) followed by Whites (3.31), Blacks (3.30), American Indians (3.19), and Hispanics (3.08). The mean Content statistic indicated that Asians (.80) displayed the most optimal pattern of coursework (as related to 12th grade math achievement) followed by Whites (.61), Hispanics (.27), Blacks (.22), and American Indians (-.16). The negative value for American Indian students suggested they take a preponderance of lower level math courses.
Table 8 showed the effect size of each of the variables in Table 7 using Whites as the focal group. Table 8 gives the distance in pooled standard deviation units between the mean of the group in question versus White students for the given variable. Cohen (1988) gives guidance on interpreting these effect size values. Values less than 0.2 are small, 0.5 represents a medium difference, and 0.8 a large difference. All of the Asian means exceeded those for Whites. For the other three ethnic groups the White means were larger. Note that all of the effects for Amount were small. The very small effect of 0.02 for the difference between number of courses taken for Whites and Blacks matches results from a host of studies given above as exemplified by the results shown in Table 1. With the exception of Amount, all of the other effects for American Indians were large, meaning that there is a large discrepancy between their means and that for Whites on the other three variables. The moderate to large effects for Content for American Indians, Blacks, and Hispanics better mirrors the difference represented in the 8th and 12th grade test scores. Thus, it again appeared that explaining performance differences can be more readily done with Content than Amount of courses.
Table 9 answered the question of whether students were taking the appropriate number and content of math courses given their initial ability as indexed by their 8th grade math test score. Amount and Content were regressed separately on the 8th grade math test and the resulting residuals kept. Negative residuals for Amount suggested that students in the group in question took fewer math courses than predicted based on their initial ability. Residuals for Content were similarly interpreted. The two main findings from Table 9 were that Asians were taking much more optimal Content and Blacks were taking much more coursework than expected by their initial test scores. For whatever reasons, Asians were taking the most optimal courses. Also, while Blacks were taking more courses than expected, they were not taking optimal courses at the same increased rate. The extreme significance for Amount and small difference for Content for Black students suggested a reduced correlation of Amount and Content for them, since they’re taking more courses while not taking correspondingly more optimal Content. Except for the correlation between Amount and Content for American Indian students which was surprisingly negative, the next lowest correlation for these two variables was for Blacks (see Table 9).
Differential Prediction Models
For Table 10, each column represents a different regression model, and each horizontal panel a different ethnic group. All effects were tested with the Type III sum of squares for the unique contribution of each variable over and above the contribution of all other variables in the model (Maxwell & Delaney, 1990). First, dummy variables were created for ethnicity, one each for American Indians, Asians, Blacks, and Hispanics. Thus, Whites served as the referent group. Each dummy variable equaled 1 if the student was a member of the target ethnic group, 0 if the student was White, and missing if the student was a member of another ethnic group. Thus, each regression run allowed the researchers to contrast the ethnic group in question versus Whites. All analyses in Table 10 were run on a subset of the students that included only students in the target ethnic group and those in the White group. In all cases the dependent variable was the senior math test score. Values in the table are standardized regression coefficients. Stars in the table mean that the variable was not included in the given model, and “n.s.” in the table means that the variable was included, but non-significant.
Model 1 consisted of solely the ethnicity flag. Its value was 1 for the ethnic group in question and 0 for Whites. This flag was significant in all instances, suggesting that each of the ethnic groups had a mean different than that for Whites on the senior math test. The Asian score was significantly higher than that for Whites while the means for the other ethnic groups were significantly lower than the mean for Whites. One should also note that the effect was largest for Blacks as evidenced by the R2s.
Model 2 subsumed Model 1 and added Amount and Content. The variance accounted for was remarkably consistent for each of the ethnic groups, ranging from 57.4% to 59.7%, a difference of less than 2.5% for the two most extreme groups. Here, the Asian students were different in that after accounting for differences in both Amount and Content their ethnicity flag was no longer significant. This suggested that the difference in course-taking for Asian students versus White students totally accounted for the difference in the two groups’ performance.
Model 3 subsumes Model 2 and added an interaction of Amount and Content to allow for differences in the relationship of number of courses and content of courses for White students versus the ethnic group in question. Neither of the ethnic groups needed an additional variable for Amount. Thus, the relationship of Amount in the model was similar for Whites and all of the other ethnic groups. The relationship of Content was more complex. Both Blacks and Hispanics needed an additional Content variable as the effect of Content on achievement was different for them than for Whites. While the effect of number of courses did not differ for Whites versus Blacks or Hispanics, the effect of type of courses did.
Model 4 subsumed Model 3 and added the 8th grade math test as an additional predictor. For all ethnic groups prior math score, Amount, and Content were significant. After adding prior achievement there was no longer an additional difference in the effect of Content for ethnicity for Blacks and Hispanics versus Whites. Now, the effect of Amount and Content in the models was similar for Whites and these two ethnic groups. Also, the ethnicity flag for Hispanics was no longer significant. The same was not true for American Indian and Black students. The achievement gap at 12th grade for those two groups could not be sufficiently explained by prior achievement and coursework. Socio-economic status (SES) was added to these two models and, while significant, it did not ameliorate the ethnic effect. Note that the results given in Table 10 are an example of moderated regression.
While not claiming that these are the only variables of importance, the researchers are proposing a model of 12th grade math achievement with three variables: prior achievement (preparation), amount of math coursework, and content of math coursework. These three variables accounted for 76% of the variance in 12th grade math achievement. Either prior achievement or course content by itself could account for more than half of the variation in the 12th grade math test. Our data were correlational and did not permit causal inferences. Nevertheless, Content was more highly related to senior math achievement than was Amount. After controlling for Content, Amount added virtually nothing to our prediction of senior math achievement. This same relationship held for the 12th grade residual gain conditioned on prior achievement. This latter result added more confidence to this finding, given that prior achievement was a surrogate for any pre-difference in students that was associated with performance on the 12th grade mathematics test. Using the 8th grade math test score as a covariate allowed one to make stronger statements regarding the relationship of course-taking to achievement.
Increasing coursework, by itself, does not necessarily improve math achievement. The results were more consistent with the hypothesis that increasing the amount of coursework increases achievement when students progress to more advanced coursework. If taking more courses influenced achievement largely through more advanced courses, states and districts will want to encourage more advanced coursework, not just more coursework. They can do so in several ways. For example, states and districts could specify advanced coursework in their requirements; e.g. students must take three years of high school math that included geometry and algebra. Simply specifying advanced courses, however, could lead to watered down content. Therefore, states and districts may also need to adopt high school content standards and/or assessments that include advanced content. This, however, was not a call to blindly increase requirements for all which may not prove efficacious (Allensworth & Nomi, 2009).
Black, Hispanic, and American Indian students entered high school with lower mean scores than did Whites. Given these prior gaps, any intervention that would close the gaps by the end of high school must produce greater gains among these disadvantaged minority students than among Whites. Especially for college purposes, early preparation and promotion of minority students to enroll in higher level and/or Advanced Placement math courses would help increase minority student participation and success (Klopfenstein, 2004). Indeed, type of high school math courses, performance in these courses, and the time in which they are taken present different opportunities to prepare for higher level classes, college, and/or the workforce.
Our evidence suggested that eradicating performance differences before high school as being the best strategy. Prior achievement was related to both 12th grade achievement and course-taking. Students who entered high school behind would be less able to profit from the highest level courses and thus their initial discrepancy is expected to continue, if not grow. Increasing the content of high school courses for all students may not be efficacious, especially without increasing resources to assist students in catching-up to their peers.
Moreover, after equalizing prior ability and amount and content of coursework, there was still something left for American Indians and Blacks. These results casted doubt on much of the previous literature as it used a one-size-fits-all approach to predicting achievement from course-taking. As with a recent article on college selection that showed a need for multiple prediction equations for college admission (Culpepper, Davenport &U Davison, 2005), it appeared that one may need separate prediction models for some ethnic groups to capture the complexity of the relationship between course-taking, prior achievement, and subsequent achievement.
One limitation of this study was that it did not investigate reasons that might explain why majority and minority students take differing amounts and levels of math coursework in high school with the exception of prior achievement. Using 1996 NAEP data, Riley (1997) found that minority students were less likely to enroll in algebra during middle school. As a result, minority students had lower potential eligibility of entering college prep courses and advanced math classes than White and Asian students due to late enrollment or poor grades in Algebra I (Paul, 2005; Cavanagh, 2007; Tyson, Lee, Borman & Hanson, 2007). Nevertheless, taking appropriate middle school math courses in preparation for high school was noted as an important input variable for predicting high school math achievement (Wang and Goldschmidt, 2003).
Results based on the new index of course content may not be replicable by others using a different index of content. It is believed that the new index may be more appropriate for many research purposes. If differences in coursework must either be differences in amount or content, then optimal measures of amount and content, taken together, should fully capture individual differences in course-taking and should account for as much variation in achievement as does the full set of information about course-taking available from student transcripts. Taken together, our measures of Amount and Content accounted for as much variation in math achievement as did our full array of course variables. Thus, the researchers can ensure that we have not underestimated the influence of content on achievement, achievement gains, or ethnic gaps.
In summary, we found high school math course content was more strongly associated with math achievement gains in high school and with end of high school achievement than was amount of coursework. After controlling for differences in course content, we found virtually no effect of coursework amount on either gains or end of high school achievement. The researchers found that after controlling for differences in course content and amount, disadvantaged minority students (Black, Hispanic, and American Indian) did not seem to make greater gains than Whites. Since disadvantaged minority student achievement means were lower than those of Whites at the beginning of high school, disadvantaged minority students would have to make greater gains than Whites during high school in order to catch up with Whites by the end of high school. Therefore, we have argued that equalizing coursework is unlikely to eliminate ethnic gaps in achievement without first eliminating achievement gaps at the beginning of high school.
9. Policy Summary
There was a relationship between course content and achievement after accounting for prior ability and thus one should counsel students to take the highest course content consistent with their ability. Effort should be expended to decrease performance differences between groups before students get to high school, as these discrepancies were related to the courses taken in high school as well as to subsequent achievement. Finally, our prediction models may need to differ for different students. A one-size-fits-all approach may be too simplistic to capture the necessary complexities for prediction models for achievement as it relates to different groups of students.
|© 2014 Lindenwood University
Journal of Educational Leadership in Action
209 S. Kingshighway - St. Charles, Missouri 63301