Summary Report of Student achievement Data for

Mathematics in Context:

A Connected curriculum for grades 5 - 8

 

 

 

 

 

 

 

 

 

Prepared by

 

David C. Webb and Margaret R. Meyer

University of Wisconsin - Madison

 

 

Table of Contents

 

Introduction *

Field Test Student Achievement Results *

Ames Community School District *

Jesus Sanabria Cruz School, Yabucoa, Puerto Rico *

Miami-Dade County Public Schools *

Student Achievement Data *

Ames Community School District *

The Impact of Mathematics in Context on Student Readiness for Algebra *

Ames Community School District (IA) *

Placer Hills Unified School District (CA) *

MiC Student Achievement Results on State Assessments *

Prince William County School District (VA) *

Miami-Dade Public Schools (FL) *

Mathematics in Context is Effective for All Students *

Michelangelo Middle School, Bronx (NY) *

Red Clay School District (DE) *

Summary *

References *

Introduction

This report summarizes student achievement evidence for Mathematics in Context as reported by districts that used pre-publication units during field testing of MiC and as reported by various schools and districts that implemented MiC since the publication of the first edition of MiC in spring 1998. Student achievement data reported during MiC field testing revealed students in MiC classrooms demonstrated substantial gains in student achievement on various norm-referenced standardized tests. Reports from various suburban, urban, and rural school districts currently using the MiC final publication version also showed impressive gains of student achievement over the first two to four years of implementation.

The rationale for developing Mathematics in Context was supported by educational research in The Netherlands that showed materials developed according to principles of Realistic Mathematics Education (RME) had a significant impact on student achievement in mathematics . Pilot studies to evaluate the use of RME materials in U.S. classrooms further confirmed that such results were not restricted to schools in The Netherlands . Gail Burrill, a participating teacher in the pilot study, remarked,

Consistently there was evidence of higher-order thinking and analysis in all of the classes, not just the honors class. One teacher commented that some of the student work on the final test demonstrated a level of maturity about mathematical reasoning that is rarely evident in students in first-year algebra (p. 158).

During the development of MiC, field test data were collected for the purpose of improving the student units and acquiring information from teachers that could be used to expand teacher guides. During the 1993-94 academic year, most of the grades five and seven field test units were used in a recommended sequence by a sample of teachers in Florida, Iowa, Missouri, Wisconsin, and Puerto Rico. Two additional sites, one in California and one in Tennessee, were added during 1994-1996. During the second year of the field testing, the grades six and eight units were taught in a recommended sequence at the same field-test sites. One member of the MiC staff traveled to each of the key field test sites and spent one week each month during the school year collecting survey and interview data. These data included internal measures of student achievement through the use of unit-specific classroom assessment instruments.

Standardized test data was not gathered during the pilot and field testing of the units since it was felt that information from such tests did not reflect the mathematics content of the units or the instructional goals of the project. However, several field test sites did share student performance data gathered from district administration of standardized tests.

 

 

Field Test Student Achievement Results

The results presented here were voluntarily submitted by three field-test sites. The Ames Community School District results reflect this districtís ongoing assessment of student achievement, and, as such, are more comprehensive and inclusive than data reported by the other two field test sites. However, the Ames CSD results are limited by the inability of the Iowa Test of Basic Skills (ITBS) to capture improvement in student reasoning and problem solving, which are primary instructional goals of MiC. The Puerto Rico and Miami-Dade County results are clearly limited by the sample size reported and in their use of limited measures of student achievement. However, they are suggestive of potential gains in student learning.

Ames Community School District

The Ames Community School District was a field test site located in a university community, which used MiC experimentally in a few classrooms from 1993 to 1995. In 1995, the district formally adopted MiC as its curriculum for grades 5-8 and began to use MiC district-wide during the 1995-96 school year.

The table below compares national percentile rankings for students in grades six, seven, and eight on three sub-sections of the ITBS before and since MiC was adopted. In all cases, the percentile rankings of studentsí scores improved after the full implementation of MiC. Given that these are national percentile rankings, this improvement is significant, especially on the computation sub-section test where improvement ranged from 12 to 20 percentile points.

Table 1: Student Achievement Results, Iowa Test of Basic Skills (ITBS)

 

1993

1996

 

(Before MiC)

(With MiC)

 

National Percentile

National Percentile

Computation

   

Grade 6

62

81

Grade 7

60

72

Grade 8

59

79

Concepts

   

Grade 6

79

93

Grade 7

81

90

Grade 8

84

90

Problem Solving

   

Grade 6

87

96

Grade 7

89

94

Grade 8

93

94

 

 

In a follow-up study of student achievement, the Iowa Test of Educational Development (ITED) was administered to ninth-grade students who had completed two thirds of the MiC sixth-grade units, one-half of the MiC seventh-grade units, and all of the MiC eighth-grade units. On the ninth-grade administration of the ITED for this cohort,

Jesus Sanabria Cruz School, Yabucoa, Puerto Rico

This field test site used Spanish translations of MiC units with native Spanish-speaking students. The Puerto Rico Department of Education administered standardized tests to all students in Puerto Rico. The results for students who were involved in the field test of MiC were compared with results for all students in Puerto Rico.

For regular-track students who participated in the MiC field test, 21 out of 23 students scored at or above the 90th percentile for all students in Puerto Rico. The other two students scored at the 82nd and 84th percentiles. Title I-track students were also tested at this school site. One Title I group participated in the MiC field test and the other used a conventional math textbook. All students in the MiC group scored above the benchmark score for Title I qualification and, in the following year, were integrated into non-Title I classrooms. Scores of Title I students in the non-MiC group did not improve in comparison to their scores from the previous year.

Miami-Dade County Public Schools

Three middle schools and one elementary school in Miami-Dade County participated as field test sites for Mathematics in Context. The student achievement results reported here are from one middle school that piloted MiC over a two-year period.

The Stanford Achievement Test 8 (SAT8) mathematics scores for students enrolled in MiC classes at this middle school showed an overall seven-point increase after the two-year pilot compared to a decline in SAT8 mathematics scores for students not enrolled in MiC classes.

In a separate study at the same school, a sample of nine students was selected at random from two seventh-grade classes. The teacher in Class A used MiC with students considered "at risk for failure." The teacher in Class B used a conventional math textbook with students who were not at risk. All students in both classes were of minority (racial or ethnic) background. When comparing each randomly selected sample of students, the mean SAT8 mathematics total scale score for the MiC sample of "at-risk" students was five points higher than the non-MiC sample of "not-at-risk" students. In addition, the mean SAT8 mathematics computation scale score for the MiC group was more than 10 points higher than the non-MiC group.

Student Achievement Data

The results presented here were drawn from publicly reported data sets and data submitted by various MiC implementation sites. Some of these results are limited by the inability of norm-referenced assessment instruments (e.g., Iowa Test of Basic Skills, Stanford Achievement Test, etc.) to document student reasoning and problem solving. When assessments such as the New Standards Reference Examination are used, the results are more indicative of gains in student problem solving and reasoning. Overall, the results reported in this section further suggest the positive impact of MiC on student computation, readiness for algebra, and understanding of mathematical skills, concepts, and problem solving.

Improved large-scale assessments are beginning to allow districts and states to document more desirable goals for student learning of mathematics. However, the aggregation of data at the district level precludes more detailed comparisons of classrooms and other sub-groups. Stronger evidence might be possible if one could control for quality of implementation and prior achievement.

Ames Community School District

The most extensive data gathered after the first edition of MiC was published came from Ames, Iowa, a university community committed to reforming their mathematics program in spite of a history of being a high-scoring district on standardized tests. In 1995, the Ames CSD formally adopted MiC as its curriculum for grades 5-8 and began to use MiC district-wide during the 1995-1996 school year. The curriculum was gradually implemented, and by the 1997-1998 school year, most teachers were teaching eight or more of the 10 MiC units at each grade level.

The data reported in Figure 1 on page six was offered voluntarily by the Ames CSD and was not part of a formal evaluation study of MiC. The Iowa Test of Basic Skills (ITBS) was administered to all students in the district annually and reported to the general public by the school district. From these district data, we selected the grade-six students who had used MiC in the 1995-1996 school year and who had completed standardized tests in each of the reported school years. Figure 1 shows the national percentile ranking of this set of students in grades 4-8 (N = 385).

Figure 1. ITBS National Percentile Scores, Ames (IA) Community School District

 

Until 1997, the ITBS was the only external assessment used to measure student achievement. Although the ITBS continues to be used, in 1997 the district also began to use the New Standards Reference Exam to provide a deeper look at studentsí conceptual understanding and problem-solving ability, district goals that were not being adequately assessed by the ITBS. The NSRE is designed to show student achievement in mathematics in three areas: skills, concepts, and problem solving. Unlike the ITBS, which consists of multiple-choice items, the NSR exam is made up of constructed-response items scored using rubrics. Thus, although the scores are labeled similarly to those in the ITBS, the questions are more complex and the scores reflect the kinds of strategies students used. For example, Part 2 of the NRS exam (to be completed in 45 minutes) contains three tasks, with two questions in each task. The questions expect students to not only produce an answer, but to illustrate and/or explain their work. Results of the NSRE are reported in five achievement levels: little evidence of achievement, below standard, nearly achieved standard, achieved standard, and achieved standard with honors. Figure 2 on page 7 shows the percentage of all eighth-grade students from 1997 to 2000 that achieved standard or achieved standard with honors compared to a national sample.

 

 

 

 

Figure 2. NRSE, Grade 8 Students Meeting or Exceeding Standard,

Ames (IA) Community School District.

 

The results on the NSRE demonstrate the exemplary achievement of eighth-grade Ames CSD students in their understanding of mathematical skills and concepts, and their ability to solve problems after using MiC for four years. This demonstrates that Mathematics in Context, a curriculum that emphasizes the development of mathematics concepts and provides students with extensive experiences solving non-routine problems, can facilitate achievement in these areas without compromising the development of mathematical skills.

 

The Impact of Mathematics in Context on Student Readiness for Algebra

Ames Community School District (IA)

One of the primary program goals for middle-grades mathematics has been to prepare students for success in algebra because of high school graduation and college entrance requirements and its application in the physical sciences. Two districts used instruments specifically designed to measure student readiness for algebra.

In the Ames Community School District, the Iowa Algebra Aptitude Assessment is administered to all seventh-grade students to identify those who are ready for first-year algebra. Figure 3 below shows that since the adoption of MiC, the percentage of students ready for algebra has doubled, based on scores on the IAAT administered to all seventh- grade students.

Figure 3. IAAT, Percentage of Grade 7 Students Ready for Algebra.

 

Placer Hills Unified School District (CA)

The Placer Hills Unified School District is located in a suburban area of northern California and includes two elementary schools and one middle school. The districtís student and teacher populations are predominantly Caucasian. Approximately 10-20% of the students are eligible for government-funded lunch programs and fewer than 20% of the students have learned English as a second language.

The Mathematics Diagnostic Placement Test (MDPT, developed by the California State University and the University of California) is administered each spring in the Placer Hills Unified School District to evaluate student readiness for algebra. As shown in Figure 4 below, in 1997, 15% of the seventh-grade students (N = 175) met the benchmark for algebra readiness (i.e., a score of 70 %). During the 1997-98 school year, the school district piloted MiC in all eighth- grade classes. After just one year of implementation, the percentage of students in eighth grade (N = 182) who met the readiness benchmark increased to 66%.

 

Figure 4. MDTP, Histogram of Student Scores in Grades 7 and 8.

 

MiC Student Achievement Results on State Assessments

Prince William County School District (VA)

The Prince William County School District is a large suburban district outside of Washington, DC, that serves over 50,000 students. Since 1997, MiC has been used in two of the 12 middle schools in Prince William County. Beville Middle School uses MiC at all three grade levels. The student population at Beville MS includes 48% minority students.

Eighth-grade students at Beville Middle School either take MiC for a third year or take algebra after completing two years of MiC. The percentage of students in these two groups that have passed the grade 8 Virginia Standards of Learning Assessment (SOL) has consistently risen since initiation of the SOL testing program in spring 1998.

Figure 5: Virginia SOL Assessment, Passing Rates for Grade 8 students.

 

At Marsteller Middle School, one eighth-grade and one sixth-grade mathematics team use MiC. Each grade level is composed of three teams. Marsteller MS has a 23% minority population, One of the three eighth-grade teams uses MiC. Of the eighth grade students at Marsteller MS taking the Virginia SOL in spring 2000, 60.2% of the MiC students passed while the passing rate for non-MiC students was 51.8%.

 

Miami-Dade Public Schools (FL)

The Miami-Dade County Public School District serves over 300,000 students. The district student population is made up predominantly minority students (33% African-American and 52% Hispanic). More than 50% of the students are eligible for government-funded lunch programs. Approximately 55-65% of the students learned English as a second language.

For the past three years, fifth-grade students in Cutler Ridge Elementary and Hialeah Gardens Elementary have taken the Florida Comprehensive Assessment Test (FCAT). Each of these fifth-grade classes used MiC. Both schools began using MiC as their core mathematics curriculum in the 1997-98 school year.

Figure 6: FCAT, Mean scale scores

 

Shown in Figure 6 above are the mean FCAT scale scores for both schools as well as the mean scale score for Miami-Dade County for spring 1998, 1999, and 2000. Both schools showed increasing scores over the three-year period despite large increases in student enrollment. For example, the Hialeah Gardens fifth-grade enrollment grew from 151 students in spring 1998 to 243 students in spring 2000 (a 60% increase). In 1998, Hialeah Gardensí mathematics score ranked 111th out of 204 elementary schools in Miami-Dade County. In 1999, their score rank increased to 85th and in 2000 its ranking moved up to 73rd.

Mathematics in Context is Effective for All Students

Michelangelo Middle School, Bronx (NY)

In the 1996-97 school year, teachers at Michelangelo Middle School received support through the New York Urban Systemic Initiative to pilot MiC. As a result of the pilot, the school decided to fully implement MiC during the 1997-98 school year. This report shows student achievement data for sixth- (N = 277) and seventh-grade (N = 377) cohorts from the spring 1997 and spring 1998 administration of the CAT5 Mathematics Exam. The student population at this school was made up of predominantly minority students (50% African-American and 37% Hispanic). Over 50% of the students were eligible for government-funded lunch programs.

Figure 7: CAT5, Mean scale scores

Students entering sixth grade at Michelangelo Middle School are organized into grade-level classrooms based on prior achievement in fifth grade. Students then move as classroom cohorts in subsequent middle-school years. For example, the sixth-grade class designated as highest based on fifth-grade scores is still designated as the top class at grade seven regardless of performance at the end of grade six.

The graph in Figure 7 on page 13 shows mean scale scores from spring 1997 (prior to implementation of MiC) and spring 1998 (after one year of MiC implementation). Note that the mean scale score at the end of grade six surpasses the mean scale score for the grade-seven cohort prior to use of MiC.

These data were then analyzed using the classroom as the unit of analysis. For the 11 sixth-grade classes and the 13 seventh-grade classes, all showed CAT5 mean scale score gains from pre-test to post-test. Some classrooms demonstrated significant gains, with effect sizes greater than 0.75 standard deviation units.

Table 2. Effect Sizes for Grade Six and Seven Classes at Michelangelo MS

Effect Size

Number of classes

D > 0.75

6

0.50 < D <= 0.75

6

0.25 < D <= 0.50

4

0.00 < D <= 0.25

8

 

Of the six classes with an effect size above 0.75, three were at grade six and three were at grade seven. It is significant that the classes with the highest effect size are distributed among all ability ranges, as reflected by their class ranking. Furthermore, the class with the highest effect size was one of the lowest achieving grade-five cohorts (D = 1.52 SD).

 

 

 

Red Clay School District (DE)

The Red Clay School District is located in a small suburban region of northern Delaware and serves over 15,000 students. The district had a 45% minority population made up of 30% African-American students and 12% Hispanic students. Approximately 30-40% of the students were eligible for government-funded lunch programs. Fewer than 20% of the students had learned English as a second language.

During the 1997-98 school year, the Red Clay School District implemented both MiC and the Delaware Student Testing Program (DSTP). In spring 1998, the DSTP was administered to 327 students. Seventy-six students used MiC and 251 students did not use MiC. The graph in Figure 8 below combines achievement results for students from two schools with similar demographics and compares the performance on the DSTP of fifth-grade MiC students with that of non-MiC students for all students, African-American, Caucasian, and low-income students. For all four groups, students using MiC had higher mean scores than students not using MiC.

 

 

Figure 8: DSTP, Mean scores by group

Summary

This report summarizes student achievement evidence for Mathematics in Context as reported by districts that used pre-publication units during field testing of MiC and as reported by various schools and districts that implemented MiC since the publication of the first edition of MiC in spring 1998. Data reported by districts representing schools in urban and suburban settings confirm the positive trend suggested by data collected from the three field-test sites. Two of the cases reported here suggest that MiC is effective at preparing students for algebra. With regard to state assessments, student use of MiC appears to result in substantial gains. These gains are not restricted to only high-achieving students, but are also evidenced for low achieving and historically disadvantaged groups of students. Student achievement on multiple assessment instruments demonstrates that students using MiC show gains in measures of conceptual understanding and problem solving without compromising their computation skills. In fact, the results presented here suggest that student achievement in computation increases through use of Mathematics in Context.

Currently, two large-scale studies of the impact of Mathematics in Context on student achievement are underway. The "Longitudinal/Cross-Sectional Study of the Impact of Mathematics in Context on Student Mathematical Performance" is designed to determine the mathematical knowledge and understanding, attitudes, and levels of performance of students as a consequence of studying units in each of the four MiC content strands over a three-year period. The study will also compare the performance of students using MiC with that of students using conventional mathematics programs. "A Study of School-Level Implementation of Mathematics in Context and Consequent Impact on Student Achievement" will analyze the relationship between varying levels of school implementation of MiC and student achievement in four large urban districts in New York City.

 

 

 

References

 

de Lange, J., Burrill, G., Romberg, T. A., & van Reeuwijk, M. (1993). Learning and testing mathematics in context: The Case Data Visualization. Scotts Valley, CA: Wings for Learning.

Romberg, T. A., & Pedro, J. D. (1996). Developing Mathematics in Context: A research process. Madison, WI: National Center for Research in Mathematical Sciences Education.

Schoen, H. L., Ansley, T. N., Hoover, H. D., Rich, B. S., Barron, S. I., & Bye, R. A. (1993). Iowa Algebra Aptitude Test. Itasca, IL: Riverside Publishing Company.

University of Pittsburgh, & National Center on Education and the Economy. (1997). New Standards Reference Examination. San Antonio, TX: Harcourt Brace Educational Measurement.