Episode 64 How to determine acceptable evidence of learning

In this episode, Dan interview’s Jay McTighe, author of Understanding by Design, and education consultant. This is the second episode of a 3 part series looking at the Understanding by Design Framework. In this episode, Jay  focuses in on “Determining Evidence of Learning”.

Creative Commons License

Episode 64 How to determine acceptable evidence of learning (Part 2 of the Understanding by Design Series with Jay McTighe) is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License

Unit design can be difficult and time consuming for many teachers. In this series, Dan interview’s Jay McTighe and walks you through the 3 step framework from Understanding by Design. In this episode, you can discover how to determine acceptable evidence of learning. This includes success criteria, skilled performance and knowledge evidence.

Video show

Show Notes

Episode Highlights

Key Point #1: Can you quickly remind us of the first stage and the 4 points?

  • Goals and subgoals
  • Understandings not just knowledge
  • Essential questions 
  • Key knowledge and skills

Key Point #2: Well this week we are looking at determining acceptable evidence. Can you walk us through the 3 steps in this process?

  • Performance tasks – How do they differ from normal tasks?
  • Other Evidence
  • Self-Assessment and reflection – Why is this so important?

Key Point #3: How could one of our listeners apply this stage this week in their classroom?

  • Think about one performance task that requires application and explanation that matches your unit goals
  • Create some success criteria for a lesson or series of lessons with your students.

Powerful Quote from This Episode

‘Good assessment should be thought of, not just as a single picture, but as a photo album… When we think about assessment, think about it as plural. Think about it as a collection of evidence amassed over time that will enable teachers to make inferences about what students know, understand and can do.

Resources

Transcript

Well, hi everyone and welcome back to the Effective Teaching podcast. This is episode 64, and today we are continuing our discussion with Jay all about the Understanding by Design framework if you didn’t get a chance to listen to last week’s episode, do make sure you head back over to episode 63 and if you want the workbook for this week, for the whole series of episodes or podcasts that I’m doing with Jay make sure you go to teacherspd.net/64 and you can grab your workbook there for the series and obviously if you complete that and give it back to me I’ll give you three hours of NESA accredited professional development or if you’re not from NSW you’ll also get a certificate of completion for going through this.

So Jay, can you quickly just give us a really quick recap of last week’s episode all about stage one?

Yes, hello again Dan and listeners last week I described an overview of the Understanding by Design framework and then I talked about stage one, one of three stages in our quote backward design process in a nutshell Understanding by Design is a curriculum planning framework used to plan units an entire year or course or even programs of study that cut across the grades with a focus on developing and deepening student understanding, ultimately so students can transfer their learning and we use a three-stage design process to plan such curriculum.

The last session I talked about stage one of backward design and looked at the elements that are considered in identifying our goals or desired results. And these included:

  • transfer goals – that answer the question what do we want students to be able to do with their learning when confronted with new situations,
  • understanding goals – what will students need to understand in order to transfer their learning and
  • I talked about the idea of understandings are around bigger ideas concepts principles and processes as opposed to just facts or basic skills

I described the use of essential questions – open-ended questions meant to be considered over time – that helped develop and deepen targeted understandings, and then finally there are the general knowledge and skill objectives. The basics of what students should know and do in a unit. So, those are the elements that we consider in stage one of backward design.

All right, so that was last week we focused on stage one, this week we’re looking at stage two which is all about determining what is acceptable evidence of those goals of those learning intentions of those desired results make sure I get my phrasing right for you here. Can you walk us through the three steps in this process that we’re trying to get student teachers to do?

Yeah, so stage two of backward design is to me at the heart of this process in many ways it asks a very basic question: if these are the goals of the results that we’ve mapped out in stage one then what evidence will we need to see if kids have learned the facts that we want them to learn, are developing proficiency in the basic skills but also, have the understandings that we’ve targeted and ultimately can they transfer their learning. Can they apply their learning and so we need evidence for those things.

I’m an ammateur photographer and as such I’ve always been drawn to a photographic analogy when thinking about assessment. So you can think about any assessment, be it a test, a quiz, a skill check, an observation, a performance task or a long-term project or even an external test like NAPLAN assessment, can be like a photograph in that any single assessment gives you a picture of what students know and can do and understand at a moment in time. But good assessment should be thought of not just as a single picture but as a photo album. Just as a photo album is more complete and revealing than any picture within it, similarly with assessment.

And so when we think about assessment think about it as plural. Think about it as a collection of evidence amassed over time that will enable a teacher to make inferences about what students know, understand and can do now.

This isn’t a new idea and teachers that I know typically will will gather evidence over the course of a unit before they get to the point of making judgments or giving grades for student learning. But the other part of the analogy is also important to backward design because we have different types of goals in stage one we need different types of photos in our assessment album. So think about it, when there are things you want students to know factual information or basic concepts well how do we assess for those? Well typically objective tests and quizzes work well ask a student if they know it or they don’t, have them choose from given alternatives in a multiple choice true false kind of setting. If you think about it knowledge is binary you know you know it or you don’t, so objective test questions can determine very quickly what students know. When we have skills, the kind of assessment we need to use the photo of her… skills is a little different. The best way of assessing a skill is to watch the student do the skill as in dribbling a football or basketball or drawing or swimming or looking at the product of skill work as in writing or visual art.

And that we assess skills not as right or wrong as much as we assess them along a continuum from novice to expert in terms of proficiency.

When we have understanding and transfer goals we need different kinds of evidence than just right or wrong and I’ll come back to that in a little bit more detail in just a moment, so the basic idea in UbD is there’s no one best type of assessment. Rather the best assessments are the ones that give you the right evidence, based on your goals. And so clarity about goals in stage one influences the kinds of assessments that you would need to use in stage two. So that’s the basic kind of fundamental principle.

Now in terms of practice in Understanding by Design we have a planning template. Essentially it’s a graphic organizer for unit planning and it has the elements that I described in stage one and then in stage two there’s basically just three boxes. One box is for performance tasks which I’ll comment on in a moment. Below that there’s a box for other or supplementary evidence and then on the side we have a column if you will for criteria, and I’ll come back to that again in a moment. So we think that any of the photos in your photo album would be nested in one of those boxes.

Now because this is Understanding by Design let me now ask the essential question so what is understanding? What do we look for as evidence of it? What’s the difference between a student who knows a lot of stuff but doesn’t yet really understand the material deeply compared to a student who does?

When I run workshops in UbD this is one of the exercises we do and I’m not going to do it in a formal way but but for those of you listening do it as a little thought experiment in your head. I actually give people a T-chart and one side says someone with understanding can and then their bullet points below and the other side of the the other column in the T-chart says someone who may know things but doesn’t yet understand cannot and you list those.

The answers are hugely predictable and so I won’t go through the exercise except to describe what if you were to do this. I predict you would have things like this on the first column someone who understands can do things like this:

  • they can apply their learning effectively to something new that’s different than rote learning where you just give back what was told you
  • someone who understands can make see and make connections among things whereas someone who doesn’t understand takes in information as discreet bits but doesn’t see how things are connected
  • someone who understands can explain or teach someone else using new examples in fact we know this is teachers it’s hard to teach a topic well that we ourselves don’t fully understand and so you can turn that around and make an assessment have the student become a teacher or an explainer.
  • someone who understands can not only give an answer but they can support it and just and or justify it they can explain their reasoning they can cite evidence from the text etc
  • someone who understands can go beyond the information given to create something new
  • someone who understands can make a prediction a sound prediction because think about it making a prediction means that you’ve been able to discern a pattern and then you can extrapolate.

These are the things that someone with understanding can do whereas somebody without understanding can:

  • recall
  • recognize
  • give back a plug-in as in mathematics you know I memorized the algorithm I plug in the numbers they gave me but I don’t really understand the concept.

And so that exercise that T-chart and the verbs that typically people identify gives us all we need to know, to think about the evidence that we need to assess for understanding. In other words our assessments of understanding should involve those kinds of verbs.

Having students apply what they’ve learned to something new which is to me what a performance task calls for. Having them explain something in their own words in their own ways using new examples making and supporting a prediction creating something new as in design thinking. Another one is error analysis or troubleshooting. If you really understand something you’ll be able to identify an error or a flaw and correct it. And so these verbs become the genesis for the kinds of evidence we need for understanding.

Grant Wiggins and I, my co-author, like to use a judicial analogy and here it is we’re going to presume a student is innocent of understanding until convicted and the question becomes what evidence will convict them? It’s the ideas in that verb list in our assessments that if they do it well we’ll demonstrate their understanding.

The other evidence section of our template in stage two is the place where we put more traditional tests quizzes and skill checks to assess the knowledge and skills that we’ve also identified as important in the unit.

Thanks Jay. And then when I went through it there was another section that talked about self-assessment and reflection why is that important with what’s going on?

One of the qualities of a self-directed learner is that they become increasingly able to self-assess and self-adjust rather than waiting for the teacher to tell them how they did or what they have to do next. But this leads to the one other piece of stage two that I want to mention and that is when we are giving kids open-ended performance tasks for which there’s not a single correct answer, we need to have some basis for judging what they do and how well they do it, and that basis would be through a set of criteria often embedded into a evaluation tool such as a rubric. Well to me part of stage two is not only thinking about the evidence we need given our goals but what criteria will be used to judge student performance. So whether we have an existing rubric say a rubric for writing or a rubric for critical thinking or a rubric for scientific investigation or experimentation, we could have a well-developed rubric already or whether we work with students to help identify what some people call the success criteria so that they come to understand what success looks like. So, the evaluative criteria or the success criteria are intended in my view not just for the teacher to use for grading or evaluation but those are actually teaching and learning targets you know. So, whether we’re talking about the qualities of an effective argument or literary interpretation or artistic production using oil paints or proper technique in weightlifting for that matter, we should have a set of criteria that become our learning targets in the beginning as well as the evaluative criteria for judging how well students do. And when those criteria are made public and explained to students, if they don’t already know them, then the  students can make use of those for self-assessment and reflection.

In fact I have worked with many teachers who now have, as a standard classroom practice, to develop success criteria with students or present them to them in advance. Again I like to post those in the classroom so they’re visible all the way through the unit. Then when the students are working on a performance task before they complete it or turn it in, have them self-assess their own work against the criteria. Maybe even have a peer or a group give them feedback and give them time to make any adjustments or refinements before they complete their work.

You know it’s not unlike what we do in writing. We draft, we get feedback, we self-assess, we read it through, and we invariably make some revisions before we turn in our final draft, our final product. And so self-assessment is guided by criteria following the task when the task is completed and returned to the student. I think it’s so valuable to take just a bit of time to ask the class as a whole and students individually what did we learn from this experience? What were you proud of? What do you what did you do really well? What were any areas of weakness for you? What did you learn from them? And what do you what will you do next time to improve on those things? Here’s a simple technique that any teacher can use after we’ve gone through a performance task or project ask the kids what advice would you give to students next year who are going to do this same task or project? Write a how-to guide or a set of tips. That’s a very practical way of having kids reflect on their learning and learn from their mistakes, a hallmark of self-directed learning by the way.

Thanks Jay. And then in terms of what our listeners can actually begin to apply this week, I mean there are those who are listening and working through a workbook and going deeper with this but also I want teachers who are just listening to this in the car to be able to take something away and apply it this week. What can the teacher do relating to, you know, acceptable evidence in their classroom this week?

Great question. Practical tips. If you’re clear about the goals in stage one and you’ve identified what you want students to be able to do with their learning think about at least one performance task that involves application and explanation, craft that out even if it’s crude and present it to students early on in the unit, don’t let it be a mystery that surprises them at the end. Like the game in athletics, they ought to know what the game is. Also, work with them to think about success criteria and post those so that as you’re learning you’re actually preparing for the game. You have the end in mind and you’re planning with that. That invariably makes learning richer deeper and equips the students to do the performances that you’re after in the first place which are your transfer goals.

Thank you very much Jay. Well so far we’ve covered you know identifying design results and how to determine accessible evidence as we’ve been working through the Understanding by Design framework. For those listening if you haven’t yet head over to teacherspd.net/64. There’s a workbook there. There’s further notes, there’s Jay’s white paper on Understanding by Design framework and you can get more learning and stuff there as well.

There’ll be links to his books and his website. So make sure you head over there if you didn’t listen to last week’s episode please go back to episode 63 and listen to that episode as well. These all go together to help you to design really great, I actually call it gourmet units, when you use this process. So thank you Jay so much for your time again and I’m looking forward to our next episode where we’re going to be looking at you know wrapping the whole thing up as we look into learning experiences thank you so much.

Enjoy This Podcast?

Changing the world through teaching goes beyond the classroom. That’s why it’s a big no to settle for mediocre professional development. If you enjoyed this episode of Effective Teaching Podcast, don’t forget to hit subscribe and share it with your peers!

Subscribe on Google Podcasts, Android, Apple Podcasts or Spotify.

To fostering effective unit design for lifelong learning,

Dan

Leave a comment to let me know your thoughts

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.