by Jonathan Osters @callmejosters & Chris Robinson @Isomorphic2CRob, The Blake School
What does a grade in your course represent? How does the way you calculate your students’ grades describe what you value?
Most of us think about two types of assessment on a daily basis. Formative Assessment should give students feedback on how close they are to meeting class expectations of knowledge and skill. Summative Assessment is a measure of what they know at the end of the learning process.
Another two pieces of the assessment picture that we often think about is the balance between a student knowing the skills of mathematics and students being able to solve novel problems. In the past, Mathematics education movements have swung far in each direction, at times over emphasizing skills and at times sacrificing skills practice.
As the Blake Mathematics Department renewed its discussion on teaching problem solving in 2009, we began thinking about how we could assess more effectively. We wanted to utilize frequent formative assessment but were unsure how to walk the line of making them worth enough that students took them seriously, but not worth an unfair amount as students are still in the learning process. We also wanted to assess authentic problem solving. The remainder of this post will discuss our Skills Quiz System, while the following post will discuss our Problem Solving Assessments.
If you are reading this blog then it is fairly safe to assume that you have heard of Dan Meyer, and his blog dy/dan. He has made a name for himself by deconstructing problems into 3-acts – peaking students interest in solving real math problems. Dan Meyer is also a regular presenter at math teacher conferences like NCTM (as well as this fantastic TED talk) and is now the Chief Academic Officer at Desmos. In 2008, he wrote a blog about how he helps students develop and perfect their skills using a retakeable quiz system (http://blog.mrmeyer.com/2008/this-new-school-year/). Our department began discussing this idea in 2010 and invited Anna Maria Gaylord (A.G.A.P.E. High School) to a department meeting to share her skill system. Finding someone who had experimented with developing a skills quiz system was instrumental in our process of developing our system.
Both Dan’s and Anna Maria’s system began with a list of skills students need to master for success in their course. The teacher then develops a set of questions that assess that particular skill, with as little utilization of other skills to reach a solution. Students are then assessed a number of times on each skill, and allowed to retake each individually to show improvement and hopefully show mastery. Dan and Anna Maria would then modify student grades based on retake scores. After we reviewed and discussed these two approaches a number of teams of teachers at Blake decided to implement a similar system in 2010-2011 school year.
Our Skills Quiz System
Our system takes and refines some of the details of Meyer’s and Gaylord’s systems. Under our system, there are several components. There is an in-class quizzing component, a remediation component, and a redemption component.
Component 1: Assessment
Students take a quiz each week. That quiz will contain some newer skills they have learned that week, but also older skills they have learned in previous weeks. That skill will also appear on the following week’s quiz, and the quiz the week after that, for a total of three times the student will have seen a question on that skill. Each time the student sees the question, it will be assessed on a scale from 0-4, inspired by the same scale used in the AP Statistics reading. And, like the AP Statistics reading, the scores themselves have descriptors that we can use to decide on a student’s score (4 = Complete, 3 = Substantial, 2 = Developing, 1 = Minimal). And rather than scoring the quiz as a whole, we follow the scores earned by the students on each skill.
Figure 1: This is a set of quizzes a student might take. On Skill #2, this student received a 3 the first attempt, a 2 the second attempt, and a 3 the third attempt. This student would receive an 8 on Skill #2.
The students receive scores for each attempt they make at a skill, and at the end of the three times they have attempted that skill, they receive a score, which is the sum of the three attempts (for example, if a student scores a 3 the first time a skill is on a quiz, a 2 the second time, and a 3 the third time, then the student receives a score of 3 + 2 + 3 = 8 for that skill). The highest score a student can receive in this system is 12. These scores are recorded in a google sheet to which each student has viewing access to only their own scores.
Figure 2: This student’s scores for Skill #2 have been entered into a google sheet.
Component 2: Remediation
If a student is not satisfied with their mastery (score) of a particular skill, they have the opportunity to take what we call “Redemption Quizzes” in order to show an increased level of mastery. But first, they need to to remediate their skill in order to show improvement on another assessment. Students are encouraged to review their past attempts on that skill with their teacher. In addition, on our course websites, we post “Redemption Assignments,” extra work that a student must complete in order to take a Redemption Quiz. They then check their answers against keys we post along with the assignment. We make it plain to them that if they don’t put in extra work to master a skill they have yet to master, then it is quite likely they will continue to earn the same scores they previously earned.
Component 3: Redemption
Once a student completes the Redemption Assignment, they can take a one-question Redemption Quiz, which covers the same skill as they have just remediated. This question is scored on the same 0-4 scale as the original quiz questions, and if it is an improvement over the previous scores, then the Redemption Quiz score replaces the lowest quiz score for that skill (for example, if the student who scored a 3 + 2 + 3 = 8 takes a redemption quiz and score a 4 on it, their score will improve to 3 + 3 + 4 = 10). These Redemption Quizzes are generally taken outside of class; our school has a few tutorial periods built into the schedule, but students can also take Redemption Quizzes during Study Hall, free periods in their schedule, or before or after school, contingent on there being someone available those times to proctor the student. We also allow for periodic “Days of Redemption” about once every six weeks, where students can use the class period to take Redemption Quizzes. We allow up to three Redemption Quizzes per skill, allowing for a student who scored poorly on all three in-class attempts to completely redeem themselves. We do have a rule that a student can take only one Redemption Quiz per skill, per day. We want to see sustained excellence rather than a one-time performance. This improvement gets recorded in the google sheet, and often we make a big fuss over entering a good score, as the student can actually see their score improving in real time. This incentive to improve and celebration of improvement is what made us want to create this system in the first place.
Figure 3: This student has taken a redemption quiz on Skill #1. His score has improved from 9 to 12.
To make sure students stay current with their remediation and redemption, Redemption Quizzes for a particular skill are only available for about 6 weeks after the last in-class attempt. This is about two months after a student first learns the skill.
Reflection: Assessing the Assessment System
Benefits of using this system over a traditional quiz system
- Students have a clear message about what they know and don’t know, and a clear path to improvement.
- Students take ownership of their learning more willingly when they know what they need to study.
- The system produces increased feedback among students, parents, and teachers (and makes for rather easy conferences, since the path to improvement is so clear!).
- Not only can we pinpoint the skills with which a particular student struggles, but we also gain a clear understanding for which skills the class as a whole is struggling.
Challenges/Costs of this system and how we address them
- Enumerating the Skills – this takes some time upfront, but once it is done, then the only remaining big job is to write the questions.
- Writing good questions – This can be tricky, too, since we want to only test one skill, not multiple skills at once. It’s also important that each question for each skill be of similar difficulty without being nearly identical.
- Security of the Redemption Quizzes – we have tried a few different strategies for this over the years. The current way we try to keep the Redemption Quizzes secure is to have students turn in the Redemption Quiz, then place the graded Redemption Quiz in a file folder for each student, so they can request their folder from their teacher in order review any of their previously taken redemption quizzes. This has worked better than any other system we have used in the past.
- To be honest, it’s kind of a lot of grading and bookkeeping- but the results are impressive. Our students do significantly better on retakes, and it’s way more fun to grade if they can show mastery as a result of their hard work. One can reduce the grading by reducing the number of skills assessed or in finding another way to summarize mastery beyond the sum of the three best attempts. This is simply a system that has worked well for us.
In summary, skills quizzes are both formative and summative assessments. They provide effective feedback to the students, while encouraging remediation and redemption. A student’s skills quiz average is 40% of their grade, 5-10% is homework, while a variety of problem solving assessments complete the other half of their grade. Please read our next post to read about how we measure synthesis of the individual skills into a deeper understanding of mathematics.
Thanks for reading until the end!