Social and emotional learning – Page 11

Assessment

By admin | April 3, 2011 - 12:57 am | Asset appreciation, New media literacies, Participatory culture & learning, Social and emotional learning

Leave a comment

(originally written June 24, 2010)

How do we know that we know what we know?

If you’re a Senegalese shopkeeper eager to prove the freshness of your bread, you grab a loaf in your unwashed hand and give it a squeeze. “See?” the non-crumbling, slow-rising crust proclaims. “Not stale!”

“Yep,” I nod, exchanging the coin in my hand for the bread in the shopkeeper’s. “So I’ll taste…”

It’s assessment, folks. It all boils down to assessment. In this case, the proof was in the pudding (or, more precisely, the yeast). But behavioral assessment, as we saw with the bread’s impressive acrobatics, is less commonly used than paper-and-pencil quizzes. Normally, we just ask people what they know. In fact, I had asked the shopkeeper what he knew — I inquired whether the bread was from yesterday. A simple, “No, it’s good,” would have satisfied me. I would’ve taken his word for it. Getting up close and personal with my future sandwich was a test I didn’t need the shopkeeper to take. Ah, but therein we celebrate cultural difference. Not everybody’s so squeamish, nor prays to the gods of plastic wrap. And you know what? Between us? I ate the bread anyway. Gobbled it. Tasted just fine. (Maybe better! I could find out by sampling a non-squeezed and freshly-squeezed roll in a side-by-side taste test, but let’s keep our eyes on the prize, shall we?)

So usually, when it comes to assessment, we ask people what they know. Then we label it and measure it. Ah, but how do we measure it? We need some metric, right? We could compare ourselves against others. We usually do… which isn’t necessarily healthy. Nor is it necessarily fair, because we’re all little snowflakes in very special snowglobes. Who knows if someone’s snowglobe was recently rocked, or whether someone else’s snowglobe was made out of double-insulated glass? Is it fair to compare Hawaiian snowglobes and Arctic snowglobes? Does everybody get where this belabored metaphor is going?

It’s best to compare ourselves against ourselves. We’re our real competition. We’re our best yard stick. How have we grown? What do we know now that we didn’t know before? That speaks to meaningful change and, hopefully, to cast it in terms of science, significant change — because this PhD shebang isn’t just a neato thing to do on a free afternoon or 1,825… I’m gunning for big kid, philosophical status. That’s DR. Felt to you earthlings, thank you very much. This is science. I better hope it’s science, otherwise this intervention is just an exercise in well-intentioned-kumbaya-guitar-strumming — super-sweet but ain’t got no legs. With no idea what worked, why, or how, it’s impossible to extract the essential elements and work its magic elsewhere. In which case it’s “Good luck, ‘social problems,’ someone else will have to solve you! But if you want to send your kid to a really fun 6-week communication camp, come on down!”

Unh-uh. Not on my watch.

So, assessment, mes amis. Assessment. This should occur pre- and post-intervention, right, so we can quantify how our participants have changed. Good. But changed according to what? Yes. Knowledge, attitude, and practice, I was thinking. Great. In terms of what? Mhmm. So we drew up a list of objectives — things that, by the end of our journey together, we want our students to know, believe, and do. These are the things we’ll need to measure, so we’ll be able to tell whether we’ve achieved our objectives.

Famous! Splendid! So I wrote some questions pertaining to those objectives. But that’s not the end of the story.

Why? Because it wouldn’t make a very good blog post… Because some of those things don’t belong on a pre-test. I don’t think. Why? Well, the knowledge items are lesson-oriented. For example, by the end of the message development lesson, we want them to know the elements of an effective message. Super. Should that go on the general pre-test? Well, it could, but we have 12 lessons, you know, so that’d make for a really long pre-test. Also, some items need to be on the pretest, Day One, before we’ve sunk deep our benevolent claws and changed the state of our participant pool. So unless we just test the living daylights out of the kids on Day One, we’ve gotta save those specific, lesson-oriented questions for their own day.

Terrific.

Or maybe… we avoid asking the questions entirely. Ah ha. This is what I want to do in terms of measuring practice. We observe. (Observe how? Do we just watch, do we videotape?) We judge performance, let participants show us what they know and can do. (What is performance? Classroom behavior (not that they’re in school, per se), completed activities?) Hmm. And how do we assess this? How much do we pre-determine (etic, like checking off a checklist) and how much do we allow to emerge (emic, like just taking notes and seeing what’s there)? Exactly.

And who should do the judging? Us? Surely not me, the white girl from the States who’s in and out in 8 weeks flat and won’t even be here for most (all?) of the training? My Canadian camarade de chambre who will arrive Sunday? The teachers as they’re teaching? The other teachers while they aren’t teaching? Other staff members? What about the participants themselves? This is a program that prizes interaction, participation, self-expression, emancipation, defiant possession of one’s own learning. Kindred spirit program Global Kids (might I be so audacious as to claim this association? All hail, Global Kids!) utilizes alternative assessment models to empower youth-directed learning. Awesome. Since imitation is the sincerest form of flattery, I’m hoping to crib that from ‘em. Dig the badges.

But then, what about the participants playing a role in the research process as well? Oh yeah, right… That seems conceptually harmonious and, more importantly, moral. There happens to be a rich body of literature pertaining to youth as research participants. So… guess we should do that, somehow…

Meanwhile, we have to add in some contextual stuff — self-efficacy, the origin of all things, whose scale I lifted from a previous study; demographics, e.g., age, grade, parents’ professions; communication behaviors, e.g., access to devices and ways of using them. Questions pertaining to the latter two categories I appropriated from the Kaiser Family Foundation’s M2: Media in the Lives of 8- to 18-Year-Olds Report, which recently published its third wave of data.

And then there’s the SEL stuff — where participants are at in terms of their social-emotional health, what they know about the five SEL competencies (self-awareness, self-management, social awareness, relationship skills, and responsible decision-making), their attitudes in terms of the importance of these things, their practices. Good, wrote those. Do we want them to know the definitions or be able to identify the phenomena? Right. Identify. So make those questions “find the best example.” All righty.

Ditto the NML stuff — what they know about the 12 NML skills (play, performance, appropriation, multi-tasking, distributed cognition, collective intelligence, judgment, transmedia navigation, networking, negotiation, simulation, visualization), their attitudes, and practices.

Then there’s the stuff that I think, and research supports, is important too: intrinsic motivation, which is associated with possible selves, which can link up with resilience, that has implications for asset-based community development, which dovetails with positive deviance, which seems awfully similar to appreciative inquiry. Collectively, all of this argues for the necessity of requesting:

asset inventories;
community maps;
communication networks; &
learning ecologies.

So… that’s cool… to write… in French… and give to Senegalese youths to fill out… in French… when their native language is Wolof… and they’re burned out on school (which lets out July 2)… and they just wanted to learn how to use a camera… (is that true? what do they want to get out of the program? what did they think it’d be about? good questions…)

So I wrote it. The first draft. And now the team just has to sift through the pages of Q’s, and weigh each item’s importance, revise with respect to cultural appropriateness, slash and reconstruct in light of grammatical atrocity, and come to some consensus. That’s what we’ve been doing (in between my last-minute dashes home to receive (or not) the Internet repairmen, who have finally deduced that my problem is due to my second-class, pre-paid service citizenship, and can only be fixed via upgrade (read: price-doubling), which I hope to suck up and purchase tomorrow morning, a 7h30). That’s what we’ll continue to do (quickly — but not too quickly — but quickly, because time’s a-tickin…).

But let’s step back and survey the big picture here: When all of this is said and done, will we know how the participants have changed? Yes, to that, I think, the answer is Yes. Good. But here comes the thornier question:

Will we truly know which theory, from this potpourri of Yes We Can scholarship, was the one that did the trick? How do we render this phenomenon of particularity — this summer assemblage of snowflakes from very special snowglobes — into transportable universality?

THAT’s what I really want — not for the sake of adding to theory, although that’d help a bookwormy brotha out, and I’d love to do him a solid. No. This isn’t a me-show (I proclaim, on my self-aggrrandizing blog…) It’s so we can say, “Here you go, ‘social problems,’ we’ve got a silver (or, okay, a little humility, bronze, or copper) bullet that we think’s gonna knock you out.”

I’m here to make the world a better place, people. I ain’t playin.

I just finished 20th grade. I’ve gone to school for YEARS in order to know so little. Ah, but maybe from knowing what you don’t know, you can begin to learn the all-important things you must?

As they say in Senegal, Insha’Allah.

Responding to classmates re: Jenkins et al (2006)

By admin | December 9, 2010 - 8:10 am | New media literacies, Participatory culture & learning, Social and emotional learning

2 Comments

IML 501 students explored the educational demands implicit in living and working in a participatory culture via Confronting the challenges of participatory culture: Media education for the 21^st century (Jenkins, Purushotma, Clinton, Weigel, & Robinson, 2006). As I detail in my response, I enter this debate with all sorts of baggage and affiliations. The following post represents my attempt to begin a dialogue with my classmates on this important subject.

A primary skills set: Response to Jenkins et al (1996)’s Confronting the Challenges of Participatory Culture. (originally posted to class wiki on November 3, 2010)

NOTE: This white paper is Biblic to me. I feel it’s important to acknowledge that right off so I will be correctly perceived as a very biased respondent. And for full disclosure’s sake, I am a member of the Project New Media Literacies/Participatory Culture & Learning Lab, and Henry Jenkins’s advisee. SO. All sorts of prejudice. ;-)

I can and have written at great length on this paper so, rather than rehash my old ditherings, I’d like to engage the potential of this Wiki by building off of some of the comments offered by my insightful colleagues. In no particular order:

TRISTA:
In order to obtain true media literacy, one must achieve political-economic literacy as well.

LAUREL:
This is a very provocative statement. I wonder what political-economic literacy is, though, especially considering the fact that we live in a world of diversity and change — each multi-national corporation and nation-state has its own political-economic legacy, and since we are living in volatile, technologically reactive times, we have thousands of moving targets on our hands.

Jenkins actually does speak about the importance of political discourse and civic engagement in his Convergence Culture which, incidentally, was published in the same year as the white paper. But his exploration is different from yours, Trista. Back to you…

TRISTA:
It is easy to say that we must educate youth about ethical norms within this tech-rich environment, but much more difficult to explain how to do so when each country has a different set of cultural norms and legal rules that guide ethical practice. This is the age-old ethical relativism vs. universalism debate.

LAUREL:
I wonder if we could reframe this “debate” to make it not a debate at all — no opposing sides, no zero-sum either/or. Could we instead acknowledge diversity, informing students that different cultures and countries possess different norms, encourage them to keep an open mind and eye towards those norms (which is the NML skill “negotiation” and the SEL skill “social awareness”), then begin with introducing them to ethical practice in our local context? That feels reasonable to me, I must admit — I asked a leading question. To me, it is too easy to say that everybody’s different so we cannot possibly tackle such a project. Yes, everybody’s different. Let’s build our knowledge slowly then, piece by piece.

More importantly, as the NML skills indicate, is the ability to recognize and do. So we can talk about the content of ethics, both the facts and idiosyncrasies that are culturally specific and the broader concept of ethics in general. But it’d be more in the spirit of the NMLs to cut the talk and walk the walk, let the practice communicate the content. For example, ask students to create a remix project or conduct some journalism. Then engage in critical inquiry around the finished product, inviting peers to ask questions about ownership and representation and compensation for contributions. Perhaps they can do some role-playing (NML skill “performance”) or go searching for cases in which people were or were not acknowledged for their creative labor (NML skill “distributed cognition”). There are lots of ways to bring ethics to life, to help students to raise their consciousness to ethical issues and help them to identify key questions and sites of contestation so that, going forward, they can proceed thoughtfully and know to ask questions.

TRISTA:
However, it gets even more convoluted when you add media to the equation because the annonymity and lack of consequences for your actions in the online life foster a kind of “anything” goes mentality.

LAUREL:
I’d like to push on this characterization of anonymity and lack of consequences, for we’re becoming much MORE “known” online due to voluntary self-disclosure, active membership in online communities and, unfortunately, the degradation of privacy. While some handily monikered flamers do hack and incite at will, I tend to think they are the exception, not the rule. On the TV show The Office, Dwight Shrute’s character was exactly himself in Second Life, no difference, save one: he could fly. Now that’s a sitcom and Dwight’s a nut, but I still think the writers have something there. While lots of people do enjoy genderbending and experimenting online, more often than not, I think we reproduce ourselves and, for better or for worse, our real world constraints. The internet could be anything and we turn it into exactly what we’ve already got. Everywhere we go, there we are.

I also disagree that there’s an “anything goes” mentality. I think, since we are sensitive to the norms of polite society, we recognize transgression, and steps are taken. Flamers lose their site privileges. Community members talk back. Jenkins (2006) documented a case of election corruption in Alphaville, the capital of The Sims, and the real life mayoral opponents were distraught by the machinations. The editor of the fake city’s real, online newspaper was punished by the corporation that owns The Sims for reporting on this blemish in its digital utopia. As we mentioned briefly in class, a NPR contributor was hauled into the federal criminal justice system for posting a threatening quote on his FB page. So I don’t think “anything goes,” especially since, embedded in these online contexts, are real life people who care and take action (both online and offline).

TRISTA:
To explain the competencies one must have, it is best to first explain why they must have them. In his effort to encourage media literacy and outline social practices in the participatory media world, perhaps Jenkins should first address human literacy.

LAUREL:
Interesting… and actually, that’s what I brought to the NMLs, a pairing with training in social and emotional learning (SEL). My students in Senegal appreciated both sets of skills and their intersections. In my opinion, SEL is fundamental — it is the base upon which one can build NMLs, because SEL forms the individual, NML refines the learner.

KIRSI:
Jenkins almost paints a world with no wars, political or economic crisis or loneliness. It is a world with free and easy artistic expression and civic engagement. At least the information technology would facilitate it. The rest depends on human ability and will.

LAUREL:
I think this is a bit too naive a vision to pin on Jenkins. Human nature is human nature. But we certainly can establish structures that support expression and engagement.

KIRSI:
When knowledge about new media and its safe use replaces fear and uncertainty, Jenkins’ advice will serve as a guidebook to the new era of education.

LAUREL:
My view is that this advice, this recommendation of NML training, is the means by which we facilitate knowledge about new media and its safe (or, I’d say, responsible) use. So implementing this training represents a new era in education as well as leads us to a new place.

In contexts of diversity and change, it’s impossible to know everything that is and will be relevant. One CANNOT know. We have to let go of “knowing” and seek “discovering.” This is research, and it relies on distributed cognition. What we need to know is: what questions to ask; and how to find their answers (because one answer/route is way too facile for our sophisticated selves).

The unknown has always scared us. We’d rather take the uncomfortable we know than the ambiguous unknown. But if we trust our deep ability to negotiate difficulty, to optimize novelty, then there’s very little to fear. We will keep ourselves safe, not because we know what’s out there, but because we know how to react protectively, no matter what.

ASTRID:
Interestingly, he points out that playing lowers the emotional stakes of failing.

LAUREL:
Thanks for mentioning this, Astrid. It is an important part of Jenkins’s argument and educational philosophy. As they say, you gotta risk big to win big. If students are afraid to step out of their comfort zone, how will they be able to author the innovations we need down the line, or make the discoveries and connections school requires in the present?

ASTRID:
Still we would need to draw the line between fiction and non-fiction, between creating a simulated sensory experience and objective serious reporting. Where does journalism end? How far can we expand the concept of journalism without loosing credibility? These questions need to be addressed and evaluated.

LAUREL:
I wonder if it’s necessary to draw the line… I wonder if we all know anyway, and make too much of didactically explicating the difference, give our students/readers too little credit…

What is the definition of journalism? Does its definition depend in any way on its objectives? In Convergence Culture, Jenkins encourages us to think beyond the device (or form) and focus, instead, on the media (or content). While cassettes may be obsolete, the hunger for recorded music lives on. So perhaps the traditional venues for journalism may give way (and already are, perhaps, as print news empires collapse), but the need for reliable, timely information persists…

As for credibility, well, that’s a subjective assessment and a moving target. Could a credible professor lecture in jeans 40 years ago? Maybe not. But with the shifting of cultural norms around attire, our credibility standards shifted too. My sense is that journalism is always a bit behind the curve, a bit conservative. I doubt journalism will lose credibility for embracing the newfangled; I find it more likely that it loses credibility for failing to adapt quickly enough and appearing irrelevant.