The Hypothetical: Between Classroom Lecture and Discussion

A perennial debate among teachers seasoned and new concerns how much lecture versus discussion to feature in class. The debate is especially relevant to teachers of liberal arts subjects, since the content of such courses is not always conducive to rote learning techniques. Liberal arts subjects often require the completion of previously assigned reading, and even if enough students read to engage in fruitful discussion there remains the risk of devolving into debate rather than dialogue. To complicate matters further, lectures and class discussions are frequently, and falsely, pitted against one another, viewed as binary opposites and sometimes filtered through a political lens which codes the former as conservative and the latter as progressive. Consider the image of a professor lecturing to a class of students dutifully taking notes versus a classroom of circled desks with students and teacher alike engaging in a dynamic conversation.

I believe there are more or less appropriate times and subjects for either lectures or discussions, and I don’t buy that either has a particularly salient political character; after all, even the teacher discussing texts in a circle still assigns grades at the end of the term. But there are challenges to both lectures and class discussions that I think frustrate new teachers in particular. Lectures aren’t especially engaging unless done well, which comes with time and practice. On the other hand, discussions can intimidate students to the point of non-participation, especially if the topic of discussion is particularly controversial. Also, the skill of facilitating and nourishing discussions is underrated and quite challenging, another pedagogical virtue that comes with time and practice. Additionally, I find neither mode particularly effective for beginning a class. Opening with a lecture (especially in the early morning) risks students nodding off to sleep, as their engagement with lectured material is entirely determined by their individual commitment. Conversely, jumping into a discussion right away often fails to take off, as any teacher can attest, since students can be reluctant to break the ice.

Enter the hypothetical. The hypothetical is exactly as it sounds: a contrived example or problem related to the day’s concepts is posed to students who must reason through its various frictions. Incorporating hypotheticals into liberal arts classes is particularly effective, as it occupies a middle ground between lecture and discussion, combines the best elements of both, and requires no prior reading. Like a lecture, a hypothetical allows the instructor discretion in guiding students more or less forcefully towards the concepts intended for learning. And like a discussion, it engages students’ creative and critical thinking muscles, prompting them to respond and participate. Hypotheticals are a great icebreaker to boot; completing the reading is not required to participate in the deliberation of the hypothetical, as its contrived parameters present enough content for participation. It also doesn’t risk intimidating students from offering controversial opinions since its hypothetical nature provides a comfortable space for intellectual exploration.

Here is one hypothetical I offer students in one of my first-year composition courses:

Here’s what typically happens when I pose it. The majority of students almost immediately answer “no,” which allows me to oppose the class consensus to encourage deeper thinking. For instance, I point out that only a 1% chance of a wrongful conviction is quite good, especially considering current estimates put the US wrongful conviction rate somewhere between 2-10%. I then ask which kind of evidence would persuade them to convict, to which they usually respond “eyewitness testimony,” “DNA evidence,” and “video footage.” All of which, I suggest, is likely less than 99% accurate. So what gives, I press.

I then switch to another hypothetical, this time involving blood pressure medication, to which the entire class always answers “yes.” (As a side note, a great technique is to juxtapose two structurally-similar hypotheticals that nevertheless induce students to opposite conclusions; the cognitive dissonance is generative.)

“So what’s the difference?” I ask. Students usually reason many of the differences pretty well, in my experience: The stakes are different for sending an innocent person to prison compared to leaving blood pressure untreated; the former involves deciding someone else’s fate and the latter your own; and there’s a matter of “trust” when it comes to doctors that feels distinct from the consequentialism of the judicial system.

The larger point I make is this: The American judicial system (in theory, at least) does not consider the probability of committing a crime as admissible evidence, knowledge of guilt. (Thank God Minority Report is just a movie.) Even if there’s only a 1% chance, you are presumed innocent until proven guilty. In American medical science, however, the probability of a drug’s effectiveness, inferred from carefully controlled experimental trials, is considered knowledge. In fact, inferring the effectiveness of medical treatments from samples is the only way to know anything in medicine at all; it is impossible to “witness” or testify to, at least rigorously and systematically, the evidence of medical efficacy. These hypotheticals therefore elegantly demonstrate the difference between two types of information: observational knowledge, obtained and verified through experience, and probabilistic belief, inferred through studies of manipulated samples.

In the larger context of the course, we are discussing the nature of knowledge, how it is we can say we know something. The goal of this class period is to demonstrate how knowledge is context-dependent; what counts as knowledge in a medical trial is not what counts as knowledge in a court of law. Does that mean knowledge is whatever we say it is? No, it means that certain domains (medicine, law, science) have developed their own rules that govern how knowledge in that domain is understood and counted. Awareness that there are differences in how we derive knowledge is a concept I discuss in my first-year composition courses because I believe the idea is essential to understand before reading and writing academic research at the college level (your mileage may vary). This hypothetical helps students to intuit the primary takeaway of the lesson, the constructed nature of knowledge, without me lecturing at them about it.

Dissertation excerpt: the politics of writing education

From Chapter 5: The Age of Automation in The Android English Teacher: Writing Education in the Age of Automation

The unique challenge of teaching writing serves as an instructive test case for education writ large. Writing embodies a series of paradoxes. It is both a science and an art, a technical skillset and a creative outlet. It is essential to every academic field—the medium through which scholarship transpires—yet its teaching is treated as a mere service to the university. Writing is inherently social, a communicative act between author and audience, and also a process that unfolds for long stretches in solitude. Writing is heavily mediated by technology, but also a fundamentally human endeavor. Finally, writing—and especially its teaching—is simultaneously progressive and conservative, associated with both radical expressivist disciplines and a traditional, prescriptivist and civics-oriented education. One of the main challenges for liberal arts scholars and writing educators is successfully balancing these contradictions, which become heightened in the age of automation.

An automated writing education fails to strike a balance between these divisions as human teachers do, and instead aligns with one side in each. With automated writing education, writing is only a science, a skill, and a nonsocial act; it is a lifeless interaction between a writer and a preprogrammed algorithm, a rote reproduction of conventions to be marked right or wrong. At root, writing is not concerned with being “right” or “wrong,” but with effective communication. The automation of writing education reduces the nuance of negotiating effective communication between author and audience to a formulaic transmission of agreed-upon conventions between a word manager and a machine. What portends to happen to writing education in the age of automation is not simply a change in the way writing is taught, but a redefining of what writing is.

Whether automated writing education will come to be defined as politically progressive or conservative remains to be seen. As higher education itself undergoes a redefinition amid public health pandemics and technological progress, its political valence has grown more significant. Pew Research (Parker, 2019) has shown for years a growing partisan divide in views of higher education (Figure 6). Choice of academic majors and course content, as well as the value of a degree, have become politically charged in a way they never have before. Writing occupies a peculiar dual position within this politicization: conservatives believe writing is an essential component of education and decry college students’ alleged declining writing ability, yet simultaneously view the very departments that teach writing as part of a domineering leftwing culture on campuses.

As universities and colleges grapple with remote and virtual learning configurations in the coming years, I fear these growing political fault lines will become ammunition in those debates. Educational technology companies may enlist progressive political rhetoric to push their products, and arguments about virtual learning or automated educational technology may end up being more about political allegiances than the pedagogical effectiveness of the tools. In the event of educational technology companies—sensing an opportunity to get their foot in the door on campuses—invoking progressive political rhetoric to sell their products, we should think critically about the pedagogical repercussions of employing virtual, and potentially automated, educational products and services independent of such rhetoric as best we can.

Impressive-sounding claims of academic personalization and customization, combined with a progressive framing of pandemic-related social distancing and educational “access,” will continue to escalate as education turns more and more virtual. My great fear is that out of a commanding paranoia of being perceived “conservative,” liberal-minded educators will thoughtlessly accept, even advocate for, corporate-led education reforms that are nominally and symbolically progressive but deeply and structurally reactionary. Many of the arguments that preserve our autonomy as writing educators have the potential to sound conservative, and perhaps some of them even are conservative in a definitional sense. “Conservatism” has become so radioactive that people forget there are many things worth “conserving”; I believe the in-person teaching of writing is worth conserving, for instance.

We must not fear being perceived as conservative if we push back against that rhetoric. In fact, much of our jobs and livelihoods as educators revolve around conserving certain elements of the current educational model that would be irreparably disrupted by the unilateral welcoming of endless technocratic reform. If we are so afraid of being perceived as conservative that we align with nominally progressive educational reforms that beget reactionary consequences, that could give technology companies with empty progressive branding the power to significantly redefine for us what higher education looks like.

A theory of educational relativity

The theory of classical relativity understands all motion as relative. No object moves absolutely; an object moves relative only to the motion of other objects. The same can be said about much of learning and education: our educational growth is frequently measured relative to that of others–our classmates, coworkers, friends, family, and so on.

Relative educational growth recalls a concept central to test theory I’ve discussed before: norm referenced assessment. When norm referencing academic achievement, individual students are compared relative to one another and to overall group averages, which are other objects in motion. Norm referenced assessment differs from criterion referenced assessment, which measures individual growth relative to established objective criteria, stationary objects; that is, for criterion referencing, performance relative to peers doesn’t matter. Think of a driving test: you either pass or not, but passing doesn’t depend on you scoring better than your neighbor, but rather on you meeting the state-established criteria required for licensure.

As it were, I would argue most of educational assessment consists of broad norm referencing often masquerading as criterion referencing. As far as I’m concerned this is not really good or bad. “Masquerading” has negative connotations, of course, but I believe the masquerade is less deliberate than inevitable. Any teacher will tell you it’s really really hard not to compare their students to one another, even if subconsciously. Try reading a stack of 20 papers and not keeping an unofficial mental rank of performance.

Although norm referencing students relative to their peers’ class performance is somewhat inevitable, I think with careful attention (and a little statistics) our assessments can prioritize a superior norm referenced comparison than that of students to their peers: the comparison between the student and themselves.

Comparing a student’s performance to themselves recalls the familiar growth vs. proficiency debate in education circles, which our current Secretary of Education is infamously ignorant about. Basically, the argument is that schools should assess growth and not proficiency, since not all students are afforded the same resources and there is incredible variation in individual academic ability and talent. Because not all students start at the same place they therefore cannot all be expected to meet the same proficiency criteria. I agree. (Incidentally, this is why No Child Left Behind utterly failed, since it was predicated on the concept of all children meeting uniform proficiency criteria.)

One way to prioritize the assessment of growth over proficiency in a writing class is to use z-scores (a standardized unit of measurement) to measure how many standard deviations students are growing by during each assignment. Writing classes are particularly conducive to such measures since most writing assignments are naturally administered as “pre” and “post” tests, or, more commonly, rough and final drafts. Such an assignment design allows for growth to be easily captured, since a student provides two reference points for a teacher to assess.

By calculating the whole class’s mean score difference (μ) from rough to final draft, subtracting that number from an individual student’s rough and final draft score difference (x), and dividing by the standard deviation of the class score difference (σ), you obtain an individual z-score for each student, which tells you how many standard deviations their improvement (or decline) from rough to final draft represents.

Why do all this? Why not simply look at each student’s improvement from rough to final draft? Because we should expect some nominal amount of growth given the assignment design of a rough and final draft, so not all improvement could actually be improvement in such a context. Calculating a z-score controls for overall class growth, so a nominal improvement in scores from rough to final draft can be interpreted in the context of an expected, quantified amount.

To assess for an individual student’s growth relative to themselves, then, you can calculate these individual z-scores for each assignment and compare the z-scores for a single student across all assignments, regardless of differing assignment scales or values. This provides a simple way to look at a somewhat controlled (relative to observed class growth) measure of growth for an individual student relative to themselves over the course of the semester. In this way, we are better able to see more carefully the often imperceptible educational “motion” of our students relative to themselves and to peers.

If we transfer what we learn, then we need a map

This past weekend I traveled to the College English Association 2018 conference in St. Pete, Florida, to give a talk about “learning transfer.” Learning transfer, often simply “transfer” in education literature, is the idea that when we talk about education broadly and learning something specifically, what we really mean is the ability to transfer knowledge learned in one context (the classroom, e.g.) to another (the office, e.g.). It’s the idea that we have only really learned something when we can successfully move the learned knowledge into a new context.

As far as theories of learning go, I think transfer is fairly mainstream and intuitive. Of course, the particular metaphor of “transfer” has both affordances and limitations, as all metaphors do. Some critics offer “generalization” or “abstraction” as more appropriate metaphors, and there might be a case to be made for those. But as long as the theory of transfer prevails I think we first need to get some things straight. This is what my talk was about.

If learning is concerned with transferring, and thus moving, between two places like the classroom and the office, then we must have a map to help us navigate that move. In the humanities, the literature on transfer disproportionately focuses on the vehicle of transfer, not a map to guide us through landscape the vehicle traverses. The vehicle is of course literacy–reading and writing. We painstakingly focus on honing and developing the most perfect and best kinds of writing prompts and essay projects, as well as assigning the most thought-provoking reading material. And this is good. If we try to move, or transfer, from point A to B but our vehicle is broken or old or slow or unfit for the terrain, we can’t go anywhere. We like to say that our students don’t just learn to write, but they write to learn as well. Literacy is a vehicle for all knowledge. All disciplines understand this. It’s why we write papers in all our classes, not just English.

But no matter how luxurious our transfer vehicles (our writing assignments and reading requirements) are, if we don’t know where we’re going, how to get there, or what obstacles lie in our way, then it doesn’t much matter. So how do we map the abstract terrain of education and cognition? Here are two simple statistics that can help us: R-Squared and Effect Size, or Cohen’s d.

R-Squared

1_KwdVLH5e_P9h8hEzeIPnTg.png

The R-Squared statistic is commonly reported when using regression analysis. At its simplest, the R-Squared stat is a proportion (that is, a number between 0-1) that tells you how much of the variance, or change, in one quantitative variable can be explained by another. An example: suppose you’re a teacher who assigns four major papers each semester, and you’re interested in which of the essays can best predict, or explain, student performance in the class overall. For this, you’d regress student performances on one of the assignments on their final course grades. Some percentage of the variance in your Y variable (final course grades) will be explained by performance in your X variable (one of your assignments). This can give you a (rough) idea of which of your assignments most comprehensively tests the outcomes for which your whole course is designed to measure. (R-Squared proportions are often low but still instructive.)

If we extend the transfer/map metaphor, R-Squared is like the highways on a map–it can help us find more or less efficient routes, or combinations of routes, to get where we want to go.

Effect Size, or Cohen’s d

Cohen-s-D-large.png

Effect Size is a great statistic, because its units are standard deviations. This means Effect Sizes can be reported and compared across studies. Effect Sizes are thus often used in Meta-Analyses, one of the most powerful research techniques we have. At the risk of oversimplification, Effect Size is important because it takes statistical significance one step further. Many fret over whether a result in an experiment, like the test of an educational intervention, is statistically significant. This just means: if we observe a difference between the control and experimental group, what is the likelihood that the difference is simply due to chance? If the likelihood falls below a certain threshold (which we arbitrarily set, usually 5%, but that’s a different discussion), then we say the result is statistically significant and very likely real and not due to chance. However, some difference being statistically real doesn’t tell us much else about it, like how big of a difference it is. This is where Effect Sizes come in.

An Effect Size can tell us how much of a difference our intervention makes. An example: suppose you develop a new homework reading technique for your students and you test it against a control group, which will be based on performance on a later reading comprehension test. If the group means (experimental vs. control group) differ significantly, great! But don’t stop there. You can also calculate the Effect Size to see just how much they differ–and, as mentioned, Effect Size units are standard deviations! So you are able say something like: my new homework reading technique is so effective that a student testing at the 50th percentile in a different class will test about one standard deviation higher in my course, around the 84th percentile.

If we again extend the transfer/map metaphor, Effect Size helps us find the really good places to visit, or transfer to. It’s like a map of the best vacation spots. After all, most of us would rather visit the beach than North Dakota in the winter, so it’s good to know where’s worth going (for certain purposes).

Stats can help teachers

Basically, what I tried to argue in my talk is not that quantitative methods/analysis are superior, but that they can do a lot of cool things, and that they are particularly important for building maps. Maps are, after all, inherently approximate and generalized, just like grades and student performance on tests and any kind of quantitative measure are merely approximations. Maps are indeed limited in many ways: looking at a (Google street) map of Paris, for instance, obviously pales in comparison to staring up at the Palace of Versailles in person. But looking at a map beforehand can help you get around once you do visit. It’s the same with quantitative measures of learning. They can help us get a lay of the land, which then allows us to use our pedagogical experience and expertise to attend to the particular subtleties of each class and each student. Teachers are experimenters, after all, and each class, each semester, is a sample of students whose mean knowledge we hope is statistically significantly and sizably improved by the end of the course.