American Idol’s Lessons on Giving Feedback

When I was growing up, my two sisters were amazing singers and I’ve always been a little in awe of them. They tried to teach me to sing, but then offered the savvy feedback that maybe I should just mouth the words.

When I came upon Kim Marshall‘s synopsis of David Slomp’s Kappan article, I couldn’t help but smile at the lessons drawn from the singing show, “American Idol.” While I don’t think I’d be a better singer if my sisters had applied these lessons to my vocal efforts, they surely do give insight into better ways to assess, discuss, and give feedback on subjective talents.

Slomp, from the University of Lethbridge, Canada, explains that assessments should be “contextual, designed in accordance with expert knowledge, and derived from and responsive to the contexts in which they are employed.”

Lesson #1: Multiple performances to demonstrate quality and growth – “American Idol” contestants perform many times in front of a variety of audiences across a range of styles and musical periods. “Seldom are students given multiple opportunities across a range of genres and contexts and over a sustained period of time to demonstrate growth and competence,” says Slomp.

Lesson #2: The primacy of expert knowledge – “American Idol” judges are some of the most successful artists in the business, but they are unabashedly subjective in their assessments of contestants. “They view each performance through the lens of their own experiences, values, and perspectives,” says Slomp. “They disagree with one another almost as much as they agree. They don’t defer to a producer’s rubric or some predetermined scoring criteria.”

Lesson #3: Defining what’s being assessed – “We need to be clear about what we are trying to measure,” says Slomp. “An effective e-mail is very different from a well-crafted academic paper or a beautifully shaped poem. And, within each of these genres, the markers of quality vastly differ.” It’s interesting that in “The Voice,” a program similar to “American Idol” but with more of a focus on vocal quality, the judges are initially seated with their backs to the performers and rate them only on what they hear.

Lesson #4: The role of dialogue in consistency – “Expert judgment is not always reliable,” says Slomp. Schools’ response to this problem has often been to train teachers to rubric-score objectively, striving for inter-rater reliability. The “American Idol” approach is quite different: the judges engage in passionate debates about their assessments. “The check on idiosyncratic judgments is not achieved through a process of norming but rather through a process of dialogue,” says Slomp. “Unlike the process of norming, which tends to strip away qualities that are contentious or difficult to measure, this process ensures that a more complete examination of the construct being assessed is undertaken.”

Lesson #5: The role of the audience – School assessments usually judge students’ work in a vacuum, on its own merits. “The problem with this approach,” says Slomp, “is that writing, like music, by its very nature is designed to affect an audience in some way. Part of the success of a written text or a musical performance is always related to whether the piece successfully evokes the desired response in the reader or listener… Taking that element out of the equation, as so many writing assessments do, necessarily limits the construct being measured. Most writing assignments are written for the teacher or the assessor rather than for real, authentic audiences.” Using computer scoring will make this even more problematic.

Lesson #6: Attending to consequences – There is almost no research on the downstream effects of high-stakes writing assessments, says Slomp. “This needs to change, even if that means taking our cue from reality television.”

“Writing Assessment in Six Lessons – from ‘American Idol’” by David Slomp in Phi Delta Kappan, February 2015 (Vol. 96, #5, p. 62-67), www.kappanmagazine.org; Slomp can be reached at [email protected].