Monthly Archives: November 2015

A Comprehensive System for Student Assessment: Part 2

By Chris Robinson @Isomorphic2CRob & Jonathan Osters @callmejosters, The Blake School

What does a grade in your course represent? How does the way you calculate your students’ grades describe what you value?

Most of us think about two types of assessment on a daily basis. Formative Assessment should give students feedback on how close they are to meeting class expectations of knowledge and skill. Summative Assessment is a measure of what they know at the end of the learning process.

Another two pieces of the assessment picture that we often think about is the balance between a student knowing the skills of mathematics and students being able to solve novel problems. In the past, Mathematics education movements have swung far in each direction, at times over emphasizing skills and at times sacrificing skills practice.

As the Blake Mathematics Department renewed its discussion on teaching problem solving in 2009, we began thinking about how we could assess more effectively. We wanted to utilize frequent formative assessment but were unsure how to walk the line of making them worth enough that students took them seriously, but not worth an unfair amount as students are still in the learning process. We also wanted to assess authentic problem solving. The last post discussed our Skills Quiz System, while this post will discuss our Problem Solving Assessments.

 

Building a Culture of Problem Solving BEFORE Assessment

Being able to assess problem solving effectively requires that students are “practicing” solving novel problems on a daily basis. There is a great deal that can be said about buliding a culture in the classroom focused on problem solving and collaboration and every teacher’s experience will be a little different: see our blog 5 Things Teachers Can Do to Establish a Cooperative Classroom Environment and Carmel Schettino’s blog is a great resource as well. Carmel is a teacher at Deerfield and major proponent of problem based instruction. Suffice it to say that building a culture is more than selecting good problems but as we are trying to talk about assessment here, we shall move on.

My colleagues and I each take a different approach here but in all cases a majority of our lessons are centered around problems that explore new topics and are either scaffolded by printed leading questions or are scaffolded by interjected questions in small group discussions. Some problems are taken from the Phillips Exeter Academy Curriculum, some from Carmel Schettino, some from our current hodgepodge of curricular resources and many written by us. The goal in general is for students to encounter something approachable but new and for them to conjecture and test out approaches to solving a problem. Then students share their approaches in small group and then large group verbally or visually through doc cams or writing on the boards. There is a summary process of the approach and sometimes an urging of “if you want to remember the steps you just went through, you may want to take notes now.”

 

Problem Solving Under Pressure

The threat of assessment is real for all students, and although our skills quiz systems alleviates some pressure, solving novel or near-novel problems can be a hairy experience for most students and teachers. The constant practice of problem solving in the classroom should help prepare students. Our Problem Solving Assessments (PSA) come in a few varieties: Exploration Labs, Reflection Journals, In-Class PSAs and Take Home PSAs. Not all classes use all types but it is safe to say that all classes use at least two of the four.

Component 1: Exploration Labs

Exploration Labs come in many forms. A majority of them are guided questions utilizing a technology like Geometer’s Sketchpad, Fathom, or Desmos. In Geometry, students may be tasked with creating a diagram, manipulating the diagram and observing how things change. In algebra, students may use Desmos sliders to see how adjusting one part of an equation affects the graph of the equation. In statistics, we may create some statistic “from the ground up” that measures some particular facet of a distribution. Each of these activities puts students in new situations, where building something from scratch is required of them, but the stakes are low, so they can try different things until a “good,” “best,” or “most efficient” way to approach the problem comes up.

Pros: +Students have opportunities to conjecture in low risk situations (often using dynamic software).
        + Students often work together and develop math communication skills.

Con: – They can take a while to grade, depending on depth of expectation.

 

Component 2: Reflection Journals

Inspired by the work of Carmel Schettino and her frequent metacognitive journal assignments, we have experimented with a variety of writing assignments. My most recent Honors Algebra II written assignment was the following:

Which is your favorite representation for a line? (standard form, slope-intercept form, point-slope form) Be sure to compare and contrast your favorite to each other type regarding

  1. a) ease of graphing.
  2. b) ease of writing an equation given two points.
  3. c) ease in finding the intersection of two lines.

Students get a few nights to compose their first response. A grade and a written commentary is given to each student, and they can then revise and resubmit to regain half the points they lost.

 

Pro: + Reflection and metacognition are powerful learning tools.

Con:  – They can take a while to grade.

 

Component 3: In-Class Problem Solving Assessments

A traditional test has a mixture of skills and problem solving. Often coming in the form of 15 skill problems and 2-4 “word problems.” Those word problems were often the same as problems previously encountered with numbers changed; this is by necessity of students having so little time to approach them on a test filled with skills. And because there is so much to do, it was relatively common for students to skip or provide only a minimal attempt of the “word problems.”

It is important to have summative assessment on skills, but now that our skills quiz system is accomplishing both formative and summative assessment , we don’t need to do that on our tests. And thus was born the Problem Solving Assessment, or PSA. A PSA is essentially a traditional test with all the skill portions removed. It is a set of 2-6 problems that require synthesis of a number of skills.  

Our PSAs have novel problems in the sense that they may have the same theme as a previously seen problems, but there is a major twist. In an Algebra 2 class, for example, we may have a two-variable systems word problem, but instead of giving them the problem and they find the solution, we might give them a solution and a framework like a paint mixing problem, and ask them to write the question. In order to write the problem they will have to create a system of equations with the correct solution, and then create the sentences in the word problem. Another example might be that of a race between several racers, where speeds, head starts, and starting points all vary, leading to different equations of position for the different racers. We may ask them a straightforward problem like “who won the race?”, but we also could ask them more thought-provoking questions like “ which racers were in 2nd place at any point in the race? How long were they in 2nd place? Use the graph to justify.”

Like any exam, these in-class PSAs have a time element to them. Students must complete the problems during the period. At times we don’t predict properly how long students will take to “solve a problem,” and so we at times let them take them home to revise and at times allow them to revise in the class after teacher comments. This has worked well for us so far, since the novelty of the problems make them such that students can’t find a similar problem in their book or online.

Pros: + Students have the extra time to problem solve, compared to a traditional test.

          + They are faster to grade than a journal, students gain comforter in a timed situation.

Con:  – It’s a timed situation, and problem solving is tough with limited time.

 

Component 4: Take Home Problem Solving Assessments

Take-home PSAs are essentially the same as in-class PSAs, the only difference being the location. The advantage is students have more time to work through the problem if needed. The disadvantage is an increased risk of academic dishonesty. We might give students only one in depth problem rather than a couple of shorter problems as in the in-class PSAs.

Pro: + Students have more time to complete it.

Con: – Managing academic honesty becomes much trickier.

A teacher can use all or some of these components for a successful assessment of problem-solving. But what happens if a student has trouble even getting started? Or what if their work is haphazard and difficult to follow? Are these PSA’s graded differently than traditional exams? We will discuss those questions in next week’s post!

 

A Comprehensive System for Student Assessment (Part 1)

by Jonathan Osters @callmejosters & Chris Robinson @Isomorphic2CRob, The Blake School

What does a grade in your course represent? How does the way you calculate your students’ grades describe what you value?

Most of us think about two types of assessment on a daily basis. Formative Assessment should give students feedback on how close they are to meeting class expectations of knowledge and skill. Summative Assessment is a measure of what they know at the end of the learning process.

Another two pieces of the assessment picture that we often think about is the balance between a student knowing the skills of mathematics and students being able to solve novel problems. In the past, Mathematics education movements have swung far in each direction, at times over emphasizing skills and at times sacrificing skills practice.

As the Blake Mathematics Department renewed its discussion on teaching problem solving in 2009, we began thinking about how we could assess more effectively. We wanted to utilize frequent formative assessment but were unsure how to walk the line of making them worth enough that students took them seriously, but not worth an unfair amount as students are still in the learning process. We also wanted to assess authentic problem solving. The remainder of this post will discuss our Skills Quiz System, while the following post will discuss our Problem Solving Assessments.

 

Background

If you are reading this blog then it is fairly safe to assume that you have heard of Dan Meyer, and his blog dy/dan. He has made a name for himself by deconstructing problems into 3-acts – peaking students interest in solving real math problems. Dan Meyer is also a regular presenter at math teacher conferences like NCTM (as well as this fantastic TED talk) and is now the Chief Academic Officer at Desmos. In 2008, he wrote a blog about how he helps students develop and perfect their skills using a retakeable quiz system (http://blog.mrmeyer.com/2008/this-new-school-year/). Our department began discussing this idea in 2010 and invited Anna Maria Gaylord (A.G.A.P.E. High School) to a department meeting to share her skill system. Finding someone who had experimented with developing a skills quiz system was instrumental in our process of developing our system.

Both Dan’s and Anna Maria’s system began with a list of skills students need to master for success in their course. The teacher then develops a set of questions that assess that particular skill, with as little utilization of other skills to reach a solution. Students are then assessed a number of times on each skill, and allowed to retake each individually to show improvement and hopefully show mastery.  Dan and Anna Maria would then modify student grades based on retake scores. After we reviewed and discussed these two approaches a number of teams of teachers at Blake decided to implement a similar system in 2010-2011 school year.

 

Our Skills Quiz System

Our system takes and refines some of the details of Meyer’s and Gaylord’s systems. Under our system, there are several components. There is an in-class quizzing component, a remediation component, and a redemption component.

Component 1: Assessment

Students take a quiz each week. That quiz will contain some newer skills they have learned that week, but also older skills they have learned in previous weeks. That skill will also appear on the following week’s quiz, and the quiz the week after that, for a total of three times the student will have seen a question on that skill. Each time the student sees the question, it will be assessed on a scale from 0-4, inspired by the same scale used in the AP Statistics reading. And, like the AP Statistics reading, the scores themselves have descriptors that we can use to decide on a student’s score (4 = Complete, 3 = Substantial, 2 = Developing, 1 = Minimal). And rather than scoring the quiz as a whole, we follow the scores earned by the students on each skill.

Skills1

Figure 1: This is a set of quizzes a student might take. On Skill #2, this student received a 3 the first attempt, a 2 the second attempt, and a 3 the third attempt. This student would receive an 8 on Skill #2.

The students receive scores for each attempt they make at a skill, and at the end of the three times they have attempted that skill, they receive a score, which is the sum of the three attempts (for example, if a student scores a 3 the first time a skill is on a quiz, a 2 the second time, and a 3 the third time, then the student receives a score of 3 + 2 + 3 = 8 for that skill). The highest score a student can receive in this system is 12. These scores are recorded in a google sheet to which each student has viewing access to only their own scores.

Skills3

Figure 2: This student’s scores for Skill #2 have been entered into a google sheet.

 

Component 2: Remediation

If a student is not satisfied with their mastery (score) of a particular skill, they have the opportunity to take what we call “Redemption Quizzes” in order to show an increased level of mastery. But first, they need to to remediate their skill in order to show improvement on another assessment. Students are encouraged to review their past attempts on that skill with their teacher. In addition, on our course websites, we post “Redemption Assignments,” extra work that a student must complete in order to take a Redemption Quiz. They then check their answers against keys we post along with the assignment. We make it plain to them that if they don’t put in extra work to master a skill they have yet to master, then it is quite likely they will continue to earn the same scores they previously earned.

Component 3: Redemption

Once a student completes the Redemption Assignment, they can take a one-question Redemption Quiz, which covers the same skill as they have just remediated. This question is scored on the same 0-4 scale as the original quiz questions, and if it is an improvement over the previous scores, then the Redemption Quiz score replaces the lowest quiz score for that skill (for example, if the student who scored a 3 + 2 + 3 = 8 takes a redemption quiz and score a 4 on it, their score will improve to 3 + 3 + 4 = 10). These Redemption Quizzes are generally taken outside of class; our school has a few tutorial periods built into the schedule, but students can also take Redemption Quizzes during Study Hall, free periods in their schedule, or before or after school, contingent on there being someone available those times to proctor the student. We also allow for periodic “Days of Redemption” about once every six weeks, where students can use the class period to take Redemption Quizzes. We allow up to three Redemption Quizzes per skill, allowing for a student who scored poorly on all three in-class attempts to completely redeem themselves. We do have a rule that a student can take only one Redemption Quiz per skill, per day. We want to see sustained excellence rather than a one-time performance. This improvement gets recorded in the google sheet, and often we make a big fuss over entering a good score, as the student can actually see their score improving in real time. This incentive to improve and celebration of improvement is what made us want to create this system in the first place.

Skills2

Figure 3: This student has taken a redemption quiz on Skill #1. His score has improved from 9 to 12.

 

To make sure students stay current with their remediation and redemption, Redemption Quizzes for a particular skill are only available for about 6 weeks after the last in-class attempt. This is about two months after a student first learns the skill.

 

Reflection: Assessing the Assessment System

Benefits of using this system over a traditional quiz system
  • Students have a clear message about what they know and don’t know, and a clear path to improvement.
  • Students take ownership of their learning more willingly when they know what they need to study.
  • The system produces increased feedback among students, parents, and teachers (and makes for rather easy conferences, since the path to improvement is so clear!).
  • Not only can we pinpoint the skills with which a particular student struggles, but we also gain a clear understanding for which skills the class as a whole is struggling.

 

Challenges/Costs of this system and how we address them
  • Enumerating the Skills – this takes some time upfront, but once it is done, then the only remaining big job is to write the questions.
  • Writing good questions – This can be tricky, too, since we want to only test one skill, not multiple skills at once. It’s also important that each question for each skill be of similar difficulty without being nearly identical.
  • Security of the Redemption Quizzes – we have tried a few different strategies for this over the years. The current way we try to keep the Redemption Quizzes secure is to have students turn in the Redemption Quiz, then place the graded Redemption Quiz in a file folder for each student, so they can request their folder from their teacher in order review any of their previously taken redemption quizzes. This has worked better than any other system we have used in the past.
  • To be honest, it’s kind of a lot of grading and bookkeeping- but the results are impressive. Our students do significantly better on retakes, and it’s way more fun to grade if they can show mastery as a result of their hard work. One can reduce the grading by reducing the number of skills assessed or in finding another way to summarize mastery beyond the sum of the three best attempts. This is simply a system that has worked well for us.

In summary, skills quizzes are both formative and summative assessments. They provide effective feedback to the students, while encouraging remediation and redemption. A student’s skills quiz average is 40% of their grade, 5-10% is homework, while a variety of problem solving assessments complete the other half of their grade. Please read our next post to read about how we measure synthesis of the individual skills into a deeper understanding of mathematics.
Thanks for reading until the end!