Beyond the Bubble Sheet: A Look at Alternative Assessments

Amalie 0:00

Imagine reading everything there is to know about whitewater rafting, never setting foot in the river. Do you think that makes you ready to ride the Colorado? Welcome to the pedagogy Toolkit. In this episode, we discuss alternative assessments, what they are, when should you use them? And what makes them different from what we think of as traditional testing

Alex 0:27

you pronounce pedagogy pedagogy? Is that it?

Amalie 0:32

So we went through this in the, the first one that James and Kami and I went through and I'm gonna start them. Well, no, there's, there are two ways to pronounce it. One is the British way. And one is the American way. Okay. And I think I must do the British one pedagogy pedagogy. But I've never I remember the first time I heard the word when I was in grad school, and I was like this. I don't. This can't be right. Pedagogy. Doesn't that sounds like a made up word.

Alex 1:03

British pedagogy. Pedagogy. Pedagogy? No, right.

Amalie 1:11

So Alex, think of a doctor who makes straight A's on all of their tests, who completes all of their papers? Who does everything writes, you know, star of every class and has never done an internship? Are you comfortable seeing that doctor?

Alex 1:35

Probably wouldn't let them touch me. Yeah, I wouldn't, I wouldn't feel great if I was walking into their office. And that was known information, I'd say let me let me get somebody who has been around a little bit more and actually had some experience, someone who's actually done the things actually done the things, even if they're, I would take someone who's fresh out of medical school, and has actually gone through all of the residency and everything like that, and got done that versus somebody who's just done it on the theoretical side.

Amalie 2:06

So there's that idea that we need both the knowledge, because we also want the doctor to know what,

Alex 2:15

no, yeah, yeah. I mean, I could go do an internship. But I mean, I know anything I can,

Amalie 2:19

I can Google, I can ask Dr. Google, right. But I don't need I want, I want the doctor to know some things. But I also want him to know how to use those things, what to do with that knowledge. And so today, let's talk about assessments, how we actually gauge whether our students are learning what we're trying to teach them?

Alex 2:42

Yeah, because the connection would be the assessment, you know, they've aced every test, they've, in theory, you know, know everything based on that assessment. But by practical assessment, we we intuitively understand that's not there's still something missing there. When when the doctors analogy is played out.

Amalie 3:00

We've all known somebody who knows a whole lot of things and can't do the things.

Alex 3:05

Yeah, yeah, we probably play this and other expressions with, you know, booksmart versus street smarts, sure. You know, knowledge versus wisdom, the implementation of knowing a thing versus just the idea of knowing a thing. But we can test both of those kinds of categories, however you want to label them. And so that's kind of what we're diving into right now.

Amalie 3:27

So when we think of assessments, we often think of traditional assessments, so multiple choice short answer, essays. Things that have right and wrong answers, things that test knowledge and facts, right? Versus what we're going to refer to as alternative assessments. And those are the things that measure what a student can and cannot do with that knowledge.

Alex 3:58

Do you know anything, um, the history of like, why traditional assessments developed and why those are so common, versus the alternatives that we're going to discuss, like,

Amalie 4:07

I'm actually reading a book right now, where he talks about how we learn things, how the education system has or has not supported that. And so you have this this history of, of apprenticeships being the norm for how people learned a profession or how they, they learned things. They, they had somebody that they worked with, and they did hands on work. And then at a certain point, we made a shift and it seems to have an I don't remember if he talks about this in that book, or if this is from something else, but he there is a shift along with the Industrial Revolution. Yeah. of moving to compliance based education. Right and making sure that the the point is less about what the people in front of the teacher what they know And it's more about, can they do what they're told?

Alex 5:04

Yeah, because it went from individual craftsmanship, for any particular discipline to put this piece into this component, right for the majority of, of production and the majority of job opportunities, doesn't require the same level of skill sets requires compliance, stay on the line, and continue to put things together, stay on

Amalie 5:26

the line, it requires, come back when you hear the whistle. Yeah, leave when you hear the whistle. Yeah, that's where bells in traditional schooling yeah comes from was practice sitting in rows. It's all practice, for an industrial workforce.

Alex 5:43

And there's tiny nuggets within some of that setup that we don't want to lose, right? Like you're talking about earlier. And we want knowledge, we want to be able to facilitate a space where there is structure. And there's an ease of facilitating the basics knowledge. But that's going to fall short. If that's what we're focused on, which is really been kind of the norm, ever since the industrial revolution that we haven't really moved away from.

Amalie 6:08

We've done a lot of focus in education on those bottom couple of pieces, the base pieces of the Bloom's pyramid, where we're focusing on D, you know it? Can you repeat it? Do you know it? And not so what? Right, I know it so what, right,

Alex 6:29

and that having worked in, I worked before I worked here at Global Campus, I worked in admissions for four years, and dealt a lot with these traditional assessments are the standardized way that we measure student performance coming into college, right, the AC T, the sa t, and that's the standardized, this person knows the basics more than this other person, so they're more likely to get admitted, or they're less likely to get admitted based on those metrics. But really, we can't decipher anything more of what does that student actually like? And how successful are they actually going to be in their education just based on a number?

Amalie 7:09

And if I recall, there have been several longitudinal studies done following students that score very well on standardized tests, and then do not do well. In college. Yeah, because that's all they've focused on, they focused on getting those standardized tests, those multiple choice getting all the knowledge. And then when it then they're thrown into college, where they're told, okay, now you have to do some things fly the knowledge, you have to take that knowledge and show us what it's worth. And that's a lot of students really struggle under that because they haven't had that kind of practice.

Alex 7:46

Yeah. The other note on this was there's some interesting data we started to collect in enrollment, as the pandemic was happening, and the AC T MSAT, wasn't available for that first year and a half, students couldn't take it, most of them were left with that junior AC trs 80, when they first took it, and maybe their scores weren't so great, because they were just getting exposed to it. That's a high anxiety inducing environment. So we had a lot of students with relatively solid GPAs, who performed well over the course of their, you know, six to nine semesters at that point. But their AC T's were relatively lower than we would have been. We found that when those students started to come to campus, at least for the first year, when that post pandemic era really began, retention maintained. So they could have higher GPAs, which was a slightly better indication, there's still some some questions, still relying on a lot of traditional assessment, and it's still subjective, isn't relative based on institution and based on grade inflation, and there's a whole lot of factors that play in there. But by and large, if you're just looking at the meta analysis, the better indicator of performance metric was a GPA, which is a little bit more rounded of an assessment analysis tool than just a single test and a single environment at this one time at this one place.

Amalie 9:05

Well, I think that idea of it being a more rounded assessment is a big part of alternative assessments, is thinking about that as a more comprehensive whenever anyone talks to people who are hiring an industry, they almost universally say that what they are looking for from the people they're hiring is not the finite knowledge of that discipline are of that industry, because that changes and grows over time. It's can they do the specific or more broad tasks? Can they learn that stuff? Can they critically think can they problem solve? Can they do those pieces? It's the

Alex 9:48

soft skills are more and more valued, the emotional intelligence, the active learning versus the only the hard skills are that the technical skills? Absolutely so we've kind of laid the groundwork of understanding some of the traditional assessments and a little bit about that landscape. So then tell me a little bit more about alternative assessments and how you kind of define that kind of how we move into that direction.

Amalie: 10:10

Okay, so alternative assessments, we're looking at things that are going to have a product or a performance, they're going to be more they're going to synthesize the information, it's going to be pulling from multiple areas. It's not just one piece of the puzzle. It's, it's has the whole puzzle been been laid out and put together? Yeah, it often is contextualized. In a real world scenario, sometimes hypothetical sometimes in in real life, the more in real life it can be, the better the learning outcomes tend to tend to be. There may not be one right answer, there may be multiple ways of achieving the same outcomes.

Alex: 11:06

That right there even just kind of shows the difference, I think pretty clearly when you compare it to a traditional assessment, because I think of like a multiple choice quiz, or a multiple choice exam. That's kind of the stereotypical traditional assessment, there is a right answer. And that's, that's the metric of knowledge. So examples for that alternative type assessment might be what, how could a teacher implement these,

Amalie: 11:29

so they tend to fall into several categories. Like I said, there are authentic ones, there are performance based, there are portfolios, and portfolios have become much more popular. And those are kind of a nice way to bridge some of the traditional assessment with a reflective idea. Reflection has become finally become recognized as a major, major part of learning, and how we how we get that meta understanding of our own learning. Yeah. So in thinking about these different products and performances, we specifically work with distance education here. And I think a lot of professors, a lot of instructors tend to think products and performance. That's great for a face to face class. But how am I supposed to do this? By a distance? Sure. So there are a number of ways that you can do that. And you can look at things like videos of those performances, videos of the artifacts, pictures, details, tracking of the process of building that product, or that performance. Looking at virtual reality, that's really, I mean, technology can really allow us to do a lot of things.

Alex: 12:55

Yeah, I think like simulation and like role, role playing assessment type things where you're, you know, in a scenario where you're putting on a particular role, and then the students have to record themselves in those roles, acting those out or playing those out, in like a business setting or a engineering setting, problem solving.

Amalie: 13:13

Yeah. And I think about even when I was in college many, many years ago, and we had to do conversation for foreign languages, we would pair up, get our little tape recorders. And do do our, our discussion on our little tape recorders. We have zoom. Now we have teams, we have all these ways we have FaceTime, we have all these ways to do this, that are technology enabled that actually in some ways make it easier to do. Yeah,

Alex: 13:42

of course. Yeah, absolutely. I would 100% agree and thinking to that discussion of like the soft skills and the ability to problem solve actively. That's what employers are looking for nowadays. And that's how much of business occurs. I mean, we work with a lot of virtual meetings in our own office and department, I know, plenty of friends and peers who are working in other industries. And some of them are fully remote and are doing things over zoom over teams all the time. They're creating slide decks and presenting material and information, synthesizing data, communicating that to clients, communicating that to supervisors and peers. And so employing these kinds of alternative assessments, project based presentation based demonstration samples is going to be a very practical skill, they're going to have to continue to use more than just a multiple choice. Did I get this answer right or wrong? Absolutely forward.

Amalie: 14:35

Well, and that comes back to some things like business memos, and letters to the editor, and oral defenses. All of those. Those ways of assessing are are more comparable to how they would actually be assessed by their employers in real life in almost any industry. Yeah, yeah. I

Alex: 14:57

think of marketing courses. I helped with fairly recently where the instructor had students. It's a digital marketing course. And instead of just giving them a quiz on components of digital marketing, he has them write LinkedIn articles, and post these LinkedIn articles to their LinkedIn page saying, Hey, you're starting to build your, your base of how to market how to explain in material and information, how to put it out there, I want you doing that actively right now. So that's actively building that skill in a way that they're going to just organically use. Whenever they get further into their field of career after they graduate,

Amalie: 15:30

I see a lot of web based portfolios, and web resumes and things like that, where where they are taking all of these pieces of samples of things that they have done, put it all together. And then now there's something that that student is leaving your class with, yes. Where they can go and to their employer or to their next to to their graduate program, or whatever, and say, this is look at what I can do.

Alex: 15:57

Absolutely. I love portfolios, and he portfolios. That's actually one of the things that I tried to focus on a lot in my assessment kind of paradigm when I when I talked to instructors. And it's also something I used in my, my graduate program. And I still have today, like I have a website, that is my E portfolio website. And what's fun about that is it's It's a living document in the sense of living site where I've gone back and I've updated it, because you've kind of got two different categories, when it comes to portfolios, you can go, you can go a little bit more the career showcase focused, and that would be where there's going to be a lot more examples that we're talking about. So if a student does an internship, or does some kind of product development, the portfolio is great, because it's a great space to provide that summative information to show future employers a this is these are the skill sets I possess, here's how I've demonstrated them, and I'm putting them all in one space for you to evaluate, to potentially make me ready for the future, where you're kind of talking earlier, they can blend a lot more with traditional assessments is you can also look at showcase from a academic perspective where a student has written a lot of really good papers or has created multimedia during their time in the fine arts and whatever that might be. And they can post that on the portfolio as well. And it will have that dual purpose, it can be that space where the student will see their own reflection of how they've grown from their 101,000 level courses to their senior seminars and capstones, to put all that into a body of work in an E portfolio that really then allows them to reflect on their growth, then it also allows them to showcase that growth and showcase their skill sets to potential future employers. And so they've got this great dual purpose. And then ideally, if they're set up, I know on our campus, we're coming up with an initiative where we're using a external software where we will host students e portfolios, and then they can take those and have private domains once they graduate in WordPress, and they're able to just build them in their undergrads and then take them moving forward to actually use them in either of these ways as personal reflections, summation or career readiness.

Amalie: 18:11

And actually, this is something that I've really, really wanted to do for a long time. There's some great free VR stuff where you can build a gallery, like an art gallery, you design it, you build, you show where the walls are, you design the the floor, the the color of the walls, and then all the artifacts on the wall, they can be interactive, to go out to links to things, the little cards on there, just like you would in a museum, right. And I've long wanted to make that sort of a resume builder for students to have them. With each of those years, it was a room and you built and you added on rooms. Oh, cool. And then when they graduated, I had wanted them to have QR codes that they could because you can then view it on a phone on

Alex: 19:08

what you were just waiting for Apple vision Pro I

Amalie: 19:10

was waiting anymore. It was ready. I was ahead of the curve. Yeah, yeah. Sure. How impressed would I be as a as an employer if I had this student who could show me their? How much they can learn in four years? Yeah, who can do that in a way that is tech forward? That is that that shows them as a person and as as a worker, and as a student learner. And as a self reflective human? I'll actually just yeah, there's so much you can do with this stuff. Right?

Alex: 19:41

Because you're touching on there's there's so many value points in that one of them being the tech savvy, you want continually employees, employers will want employees who can adapt to the changing landscape of the materials and supplies and hardware and software they're using. But then, as you mentioned, the reflection piece I don't, I can't think of many industries where self evaluation is not going to be required. Right not whether on a quarterly basis, or biannual basis, annual basis, like, and then you take that to your superior and have discussions on future performance metrics. So you need to know how to reflect and think back on, okay, here's where I was, here are goals that I've set for myself, here's how I've achieved those goals. And here's how I'm going to move forward and grow for future goals. And so building that in, whether it's through a portfolio or some kind of showcase, virtual showroom gallery, things like that are great stepping stones when students are in school to begin that process, because we aren't, we are kind of mandating it in a sense that we're kind of creating that environment where, hey, we were kind of forcing you to reflect, we want you to reflect, but we want you to build that skill, just as much as any other pedagogical skill that you're putting in place, moving forward in your discipline, so that you can also then reflect on how you're maintaining growth in your discipline moving ahead,

Amalie: 21:05

which sort of jumping off that idea of teaching the student to be more reflective, and teaching some of those metacognitive skills. The, the one thing that I did forget to mention is, I often recommend to instructors to when they give students choice of of how they want to demonstrate the that they understood the concept, or they can manage the skill, that there be a wildcard option. Oh, that the wildcard option is if you know, I can say to you, you you have these options to present a research project to me, you can write a traditional research paper, you can do a research presentation, you can pair up with your, your internship or your your employer your job and and find something that you need to to do some some work on. Or if you have another idea. I'm open to it. Yeah. And I had a student one year that did, she was doing a research paper on sustainability in, in raising chickens. And she has we do we do? And she did she created a whole Minecraft world. Okay, awesome. That was a that encompassed all her research. Yeah, it shows that if you can give them that space, sometimes they can ship and this was a student who wanted to be, he wanted to do 3d animation. So it was within her her wheelhouse This was exactly the kind of thing that was going to give her an opportunity to practice skills that she would need in her career focus. Yeah. And those bigger skills of research of source evaluation of presentation,

Alex: 22:54

and how much more that shows her aptitude and skill sets versus if you just mandated a 10 page research paper, right, you're gonna miss so much of what she's good at, and available to, like, grow in. But by providing these alternatives and her assessment options, she was able to really come up with something creative, which is what you want. You want students to get creative to still demonstrate their their base level knowledge and competencies like we're talking about, but really grow beyond that one. One question from that experience. What was that like then as the instructor when it came to grading and evaluating a variety of assessment types, because I could see maybe some questions from instructors being like, well, I open up the floodgates, and I get five different types of projects or presentations or assessments. How do I do that in a way that is fair and equitable? for all students to receive that, well, this student got this grade and the student got this grade, but then they see how they did things differently than they take issue with grading when it's right. How do I how do I equalize the schema.

Amalie: 23:56

So when I've talked to instructors about personalizing basic, which is basically what that would be is personalizing the assessments for the students. They go, I can't do 80 different projects. I'm not writing up 80 different assignments, I'm not writing, you know, rubrics, that's where you get down to rubrics, rubrics are, are critical in grading these kinds of assessments. Yeah, when you have a right and wrong answer. It's Yes, it's no. When when you're doing multiple choice, and that's there's a right there's a wrong end of story. When you're grading something that is more fluid like this, you need to have a rubric and a rubric allows you to look at something on a continuum. So you can see if you know someone who didn't know anything about this, what would their project look like? Versus someone who's an absolute world class expert? What does their project look like and what are the stages in between? So that's, that's the first But it allows you to sort of see that on a growth continuum, it allows you to give feedback to the students through that. And the thing that I most love is it allows you to isolate the skills that you're asking for. Yeah. So if with, for example, a research project, if I'm, I might be looking at their source selection, that might be one thing that I'm looking at, I might be looking at their analysis of the source, that's a separate thing. I might be looking at their presentation of the information. And that's a whole other thing. So some students may do great at the presentation, pretty good at the analysis, not great at the selection of the sources, right. And by being able to parse those pieces out, you're giving students really specific feedback on where they need to put their efforts.

Alex: 25:50

Yeah, which tells them a lot more, and tells you a lot more than just yes or no right or wrong. And like I kind of made the joke, you don't have to make any rubrics, you can make one solid rubric, but it encapsulates enough. What you're saying, Is it a capsulate, enough of the components across the spectrum of alternative assessments, because they're all going to have the same basic flow and function. If you're looking at research, if you're looking at certain subject matter, you're going to look for certain core components,

Amalie: 26:16

right? So you need to that goes back to your planning as well. So this goes back to what do I need? What am I trying to assess? That is ultimately should be the question you always come to that is should be like the thesis of your course. What am I assessing? Yeah, what do I want the student to know? To do? And how can I? How will I know when they have learned it? And have done it? And what does what does that look like it all of its various stages? Absolutely. Because

Alex: 26:47

the hits on a lot of things. We talked about her lock backwards design, understanding your course level objectives, your unit level objectives, Lesson objectives, it all has to flow from there, you have to know your end from your beginning,

Amalie: 26:59

and thinking about that's also where you can ask yourself questions like, What do I need? So what am I assessing right now? Do I just need them to have the base knowledge at this point? Or do I need them to apply the knowledge at this point? This is where you can really combine it with those low low stakes testing that we've talked about in in another episode. Yeah, that you can use those very low stakes, things as milestones throughout the course, to make sure they're getting the knowledge they need. So that at the end, when they are doing that final alternative assessment, you know that they have gotten all the the foundational pieces. Now you can score just the those softer skills?

Alex: 27:47

Yeah. Yeah, this is where collaboration amongst departments and you know, chairs, and heads, really understanding the program goals and kind of what what has to happen, because, again, you if you're teaching an advanced level course, you have to understand like, well, if they have already gotten that those lower taxonomies of identification and definitions and describing certain aspects, maybe my role and instruction is to really get them to synthesize and analyze and create and evaluate. And, yeah, that's going to have practical implication, because I don't want whenever, you know, this, probably analogy falls apart at a certain point, but I don't want my med school resident to be going to the hospital and taking multiple choice assessments. I want them going to their residency to practically engage and then be evaluated by their supervisors on what they're practically doing in the hospital on a regular basis. So that I know when they're a board certified, licensed doctor, I'm getting the proper treatment that I'm that I need to be getting. And so that that is where that really comes in. Yes, you may still need to incorporate some of the the lower level taxonomies where a quiz makes sense. But that's I think low stakes is where that makes the most sense. Assume not assume but like, create a space where they can demonstrate that competency knowledge from the prerequisite, but then focus on right, what is the actual goal of this course or this lesson that's going to demonstrate their more advanced levels of learning? Well, and it allows

Amalie: 29:22

you if you have those, those low stakes tests, pieces, those little quizzes where you're making sure that they have that knowledge that gives you that opportunity as an instructor, when when you discover they don't, yeah. Or if you discover that half the class has it and the half that took this other professor that didn't know what they're doing, doesn't have it. So you need to make sure that that group gets that information. And there's that's the only way you can, you can do that as if you test what they know.

Alex: 29:52

Yeah, there's an inevitable amount of backtracking you're going to have to do when when you come into a new course or a new subject With with a group of people, you can't all assume that they all know the same thing going into it. But again, the best way to do that long term to get them to the end assessment is going to be personalized and individualize the the end goal assessments.

Amalie: 30:00

With with a group of people and you can't all assume that they all know the same thing going into it. But again, the best way to do that long term to get them to the end assessment is going to be personalized and individualize the end goal assessments. And that is that an A note, when we say individualize and personalize, it sounds so scary when you have, you know, 80 students that you're dealing with, or 50 students or hundreds, like whatever Sure. But if you build the frameworks of what you are asking from the students, if you build those frameworks,

30:34

with the right levels of specificity and brand breadth, you can apply those kind of across the board. I am thinking back to my my undergrad being in history education, I would much rather grade 50 unique different projects with a standard rubric that at least guides them and has a compass for them to be put in the right direction, then try to read 55 Page essays that are going to read regurgitate the same themes and the same beats. And the same kind of basics over and over again, I would rather see some of that nuance and see some of that creativity, that's going to be more enjoyable from a grading perspective, if I have to be the one to grade all of this. So there's a kind of a fringe benefit out of that, oh, yeah, the first year I taught writing, I had the entire class summarize the same article, right. And we sort of summarized it as we outlined it as a class and then they wrote it up.

31:32

But that was the most boring: 2550

31:51

And honestly, it kind of told me that they could do that, as a group, it didn't tell me much about what they could apply, right, you weren't measuring an individual learning experience that was taking place, right for each of them. So that eventually I evolved that into, we look at this text as a class, and then they can select one that they apply that to, they don't have unlimited choice, they have limited choice that I have some control over and get. So that's setting up those those frameworks, those boundaries, right, it's it's kind of understanding the idea that boundaries can lead to freedom, that by creating, you know, here's the fence line, I want you to not just stay in this one little section, that would be a traditional assessment, but I'm giving you the space to, to wander and explore the next farm, here's the boundary. Yeah, don't cross that fence. But within that, there's so much freedom in that and you're going to discover things within your own little perimeter that you didn't know was there before. That's great for the students. That's great for you. Because then as they start to expand those borders a little bit, get closer to the fringes, maybe they'll discover assessment ideas and opportunities that you hadn't thought of Exactly. Like like, again, the Minecraft Yeah, world. And that's a really cool benefit to is. And you can see application of the subject matter in ways that maybe you hadn't ideated before, just because we were thinking only in this traditional assessment. So there's nothing more fun as an instructor than when your student comes up with something you had never thought of. And it's, it's brilliant, and they're excited about

Alex: 33:30

it and everybody's excited. And you get to see where it where it can go. There still is there's a small chance that that could still end up in a gutter as a gutter ball. Yeah. And that's okay. Yeah, it's okay, if you have this range of alternative assessments, and a student misses the mark, because that helps you understand, okay, this is where they are at individually. And I can work with them in a more clear, identifiable way. Then, if I just saw them get XYZ wrong on a test. That's, that's useful, and that's helpful. But sometimes it's luck, right? Oh, it's there's 25% chance to get it right. So this allows a much more targeted, clarified understanding of student learning and allows students to then reflect more clearly on their learning and what they understand versus don't. So I think we really are going to need to do an episode at some point about grading, and different different ways to grade and get a little more into the weeds on rubrics. And yeah, Jason, I've touched on this in another episode, we're gonna have to do rubrics. I used to be total hater on rubrics, I hated them. And it was only because I had never used one that was well written. They can bloat to a way where they are more to manage then it would become more cumbersome than just a traditional assessment. But if they are done well written well, like you're saying, yes, they are a fantastic tool in the tool belt. So to wrap up, like we've

Amalie: 35:00

said alternative assessments aren't really meant to take the place of traditional assessments. It's one more tool. It gets overlooked a lot in online courses. I think especially because it's a little more inner. It requires more interactive, in more interactivity from both the student and the instructor. But it doesn't have to be difficult. And it can really increase your engagement with your students. It can increase, it can allow you to have a better understanding of what your students are actually learning. Yeah, we want the doctor to have the textbook knowledge and the practical experience before they start poking and prodding and diagnosing us, right. Exactly. Yes.

Transcribed by https://otter.ai