584: For Your Reconsideration

Ira Glass

You may remember this story that we did about a year ago, about canvassers going door to door and talking to voters for 20 minutes. And in just 20 minutes, the canvassers got some voters to change their minds, to flip positions on big, hot button issues. Gay marriage. Abortion.

Which, of course, is amazing, right? That is not something that seems like it's even possible. There was a study in a well-known, peer reviewed journal-- Science magazine-- proving how remarkably effective these people were. They invented a whole new way to sway voter opinion. It seemed like a game changer, a whole new tool for politics, for liberals or conservatives to use.

And then a month after our radio story-- maybe you saw this in the news-- there was overwhelming evidence that one of the researchers, a grad student named Michael LaCour had fabricated all the data. Science magazine retracted the article.

But the team that invented this canvassing technique, they were not the ones that did the research study. That was a separate thing. And they were just as shocked as anybody. Maybe more shocked.

The week the news broke, one of the canvass organizers, Steve DeLine, told me they'd spent months meticulously going door to door and keeping perfect records of everything they were doing for this researcher. To find out that LaCour had just made up his part of the data--

Ira Glass

Then in August, three months after the retraction, four months after our story aired--

We got a visitor to our office from two of the guys who'd developed this canvassing technique, that guy Steve and his coworker Dave Fleischer. They happened to be in town from California. And they had news.

A brand new study with new researchers was looking into their work. Results were coming in. Here's Dave.

Ira Glass

What these researchers really wanted to do was publish their results in Science magazine, you know, the place that published the original, discredited study. In fact, one of the researchers was laying the groundwork for that.

Ira Glass

Well, apparently Science magazine liked what the new researchers did. This week they published the new study about these canvassers.

This one's not fake. We know now, for real, whether those canvassers are, in fact, convincing anyone of anything when they go door to door. And if anything, what the canvassers are achieving and the way that they're achieving it is even more interesting than the fake study said.

Ira Glass

From WBEZ Chicago and This American Life, I'm Ira Glass. Today on our program, "For Your Reconsideration." We have this story and two other stories where people go back and revisit evidence from the past. And just to start this off, in case you didn't hear our first story about those canvassers a year ago, here's what they were. If you heard that show, this will just be a quick reminder.

Ira Glass

This is one of the canvassers. She's young, with tattoos and a halter top. It's a warm Los Angeles day, and the voter's on the other side of a screen door. She's older, heavyset, wire rimmed glasses. She's a nurse.

And OK, just to be clear, what you're about to hear is a real canvasser with a real voter. This is not part of the discredited study. This is from a video of one of the regular canvasses these guys Dave and Steve organized. The canvasser begins by asking this voter what she believes about abortion on a scale from zero to 10.

Ira Glass

The voter says zero. No access. No abortions for anybody. She's Catholic, from Mexico.

And they start talking about how hard it can be to discuss abortion. Even with family.

Ira Glass

Notice that this conversation's just begun and already she's talking about her daughters and her mom. That's no accident. This whole technique is built around having a very personal conversation, a very real conversation about the voter's experiences. To make that happen, this canvasser does something that a lot of the canvassers do-- she talks about her own experiences.

Ira Glass

The voter tells her that when she was 11 and got her period, nobody explained what was happening. She thought she was dying.

Ira Glass

And when she was six, her mom miscarried a baby. But nobody explained what was going on to her. And so she's tried to be different with her daughters and talk to them. That's why she became a nurse, she says, to help women.

So these two strangers are swapping stories and really talking from the heart. And 15 minutes into this 22-minute conversation, the canvasser circles back to the subject at hand in a very pointed way.

Ira Glass

It's a moment that's simultaneously intimate and manipulative and honest all at once. And it works.

After this, the canvasser asks the voter again to rate on a scale of zero to 10 where she stands on access to abortion. Remember, she was a zero before.

Ira Glass

She's been standing at the door just 18 minutes. So again, OK, that's real. That is not part of the fraudulent study.

The team that invented this technique of going door to door-- including Dave Fleischer and Steve Deline, the two guys who visited our office at the beginning of this story-- they work at the Los Angeles LGBT Center, which is a big, multimillion-dollar nonprofit, the biggest LGBT organization in the world. They started doing these canvasses in 2009. They were hoping to change voters' minds to support gay marriage. This was in the wake of Proposition 8, which rejected gay marriage in California.

And they discovered through trial and error that the trick to this whole thing was they don't try to reason with people about rights and equality and big principles like that. Canvass organizer Steve Deline says if you're talking at a kind of rational, sort of intellectual level about things, you're gonna fail.

Ira Glass

This is what they learned-- to stop telling people things. The most important thing the canvasser should do is listen.

Ira Glass

So they're out doing this, trying their canvasses. And it took them less than a year to start to get decent results like you heard with that nurse. A small but noticeable number of people at the door told them that their minds had been changed.

But of course, the only way to tell for sure if their minds changed and stayed changed was to have outsiders-- scientists-- do a proper case control study. And that's why the canvassers invited in two political scientists to do the original study that was published in Science magazine. That grad student from UCLA-- Michael LaCour-- did the actual research. But the study design and results were overseen by a big deal political scientist from Columbia University named Donald Green, who's kind of legendary in that world for transforming how people do field experiments. And Green says that before they started, he expected the results were going to show--

Ira Glass

He thought that 'cause pretty much that's what always happens. People change their minds and then the effect wears off. Or much more common, they don't change their minds at all.

There's a hefty body of research documenting something called the backfire effect, which happens with most of us. When we're confronted with evidence disproving something that we personally believe, most of us generally just dig in, and we believe it more.

This is a phenomenon, by the way, that's familiar to anybody who's ever been to a Thanksgiving dinner during an election year and tried to argue with some favorite uncle's wrongheaded political opinions. Never works, right? Facts mean nothing. And by the way, Uncle Lenny, I am not talking about you.

But this is what was so revolutionary about the data that this grad student seemed to be gathering about these canvassers. He said he did online surveys with the voters who got visited by the canvassers and they showed that their minds changed and stayed changed for six months, nine months, a year after the canvasser knocked at the door.

Green told me that he and his colleagues had read through over 900 papers, never seen anything like this result, to change people's opinions and have it last like this. Though interestingly, when LaCour first showed his data to Professor Green, Green thought, this data is so perfect, these results might be real. But OK, best to be sure.

And he told LaCour to do the entire study-- canvasses, surveys, everything-- a second time. Which LaCour did. Same result.

Now, of course, later, Green's suspicions would be confirmed in spades. All that data would be shown to be fraudulent. But here's what LaCour said he found in the original study published in Science.

OK, so what I'm about to say is from the discredited results of a retracted study. This study was about gay marriage. It was about convincing people to support gay marriage. And the study claimed that a year after the canvassers visited voters' homes, the number of voters supporting gay marriage had jumped 15%. 15%.

But-- and this was key-- according to the study, the 15% increase only happened if the canvasser at the door had been gay and admitted being gay to the voter. People who talked to other canvassers only rose 3%, which is about as much as the nation as a whole changed on the issue of gay marriage that year.

It was a similar effect with abortion, according LaCour. He said the number of people who became pro-choice after the canvass rose 5%. And again, the effect only lasted if the canvasser at the door revealed that she had had an abortion. It was talking to somebody affected by the issue that seemed to be changing people's minds.

I actually interviewed Michael LaCour a year ago for the story that I did for our show about all this. I have to say, it's strange listening to that interview today, knowing what happened just a few weeks after we talked, that his data would be shown to be fraudulent. 'Cause when I talked to him, he really talked like somebody doing a real study with real surveys. Like for instance, I asked him how attitudes towards abortion changed over time among the voters that he was studying.

Ira Glass

He offered conclusions from his data. Like this one.

Ira Glass

Nice if that were true. If there were data supporting it.

Or there was this finding, which is so particular and specific that again, if there is no data supporting this, it's just a very idiosyncratic thing to decide to say is true. LaCour said that he graded each gay canvasser on how stereotypically gay they appeared. And--

Ira Glass

That is, they were able to convince the most voters to support gay marriage.

Ira Glass

This vision of the world is such a hopeful vision of how the world might work. The world LaCour fabricated is one where intolerant people hold inaccurate stereotypes in their heads and all it takes is an empathetic conversation, one sincere talk, 20 minutes long, with somebody from the stereotyped group and the scales fall from their eyes. They realize they were wrong all along.

I like that world. I wish I lived in that world. Maybe we do live in that world for all we know. But if we do, there's no data proving it.

[PHONE RINGING]

Ira Glass

Last May, a month after we ran our story about LaCour's study, six months after he published that study in Science magazine, I called his co-author, Columbia professor Donald Green. Green had just reconsidered what was going on in the data LaCour had collected and had asked Science to retract their article, which the magazine did. He seemed understandably sobered and sad.

Ira Glass

The way the fraud was discovered was actually a little detective story of its own. A detective story that began kind of exactly the way they tell you in school that the scientific method is supposed to work.

Two young researchers-- David Broockman and Joshua Kalla, both in their 20s-- tried to replicate what LaCour did. The team from the Los Angeles LGBT Center and a Miami LGBT group was organizing a new canvass operation in Miami. Broockman and Kalla would study that.

Broockman was already suspicious of LaCour's data. It seemed too perfect. Then they started their research and they noticed something funny.

The way this experiment works, like I said, the researchers do online surveys of the voters' opinions before and after the canvassers come to their doors and talk to them. And the researchers pay the voters to fill out those online surveys to motivate them to do the surveys. And tons of voters filled out LaCour's surveys, but not Broockman and Kalla's.

This made no sense at all. And they didn't know what to do. If people didn't fill out their surveys, they'd have no study. They started to panic.

So they reached out to the survey company that LaCour said he used to find out exactly how LaCour got so many people to fill in their surveys. The company informed them that LaCour was not one of their clients, that the employee LaCour said he worked with did not exist, and in fact, this company did not have the capability to do the things LaCour said they did for him.

The hundreds of thousands of dollars that was supposedly paid out to survey responders, which LaCour said came from funders like the Ford Foundation-- who LaCour thanked in his study-- never was actually granted to LaCour. Given all that, it was hard to see how any of the data could be real.

But LaCour held firm. Said the data was real. He even wrote a paper defending it. But he said that he could not show the data to anybody because he had destroyed all the original surveys to protect the privacy of the respondents.

I'm not sure anybody believed him. He'd been headed for a prestigious job at Princeton University after finishing his doctoral thesis. Now that job evaporated. His collaborator Don Green says the odd thing was the fakery went way beyond what seemed to be necessary.

Ira Glass

As if, in later surveys, their attitudes about homosexuality had changed enough that they were comfortable finally admitting to the surveyors that they were gay. Except of course, the surveys didn't seem to exist.

Ira Glass

The day after the truth came to light, one of the guys who created the canvasses, Dave Fleischer, told me that that's what he found himself trying to get his head around, the layers of embellishment. He told me this story about working with Michael LaCour.

Ira Glass

In other words, after real life canvassers came to their doors, Michael LaCour is supposedly having these voters fill out online surveys. And he's telling the canvassers, those surveys--

Ira Glass

Canvass organizers say staff and 73 volunteers worked for more than 3,600 hours, including knocking on doors and talking to over 400 voters for the fake study. Since all this went down last year, Michael LaCour still has not admitted to fabricating data in public or in private with the colleagues of his that I've spoken to. He declined my invitation to come on the radio this week and talk about this. He declined to say whether the data was real or fake.

The one chink in this position came last month, when out of the blue he texted Dave Fleischer. "Hi Dave. If you're willing, I'd like to meet with you face to face. To apologize." Dave told me he hasn't been able to bring himself to respond just yet. He's not ready.

Ira Glass

That's David Broockman, one of the guys who discovered the fraud in the original study, and the co-author of the new study with real surveys and real data that was published this week in Science magazine. This recording has him talking to me last August, three months after Science retracted the original study.

It was Broockman and his research partner remember, who discovered LaCour's fraud when they tried to replicate his methods in Miami. Now the Miami data was in, the first part of it anyway.

The canvassers in Miami went door to door trying to change voters' attitudes about transgendered people. Miami-Dade County passed an ordinance in 2014 banning discrimination against transgender people. And these canvassers were trying to convince voters that that was a good idea, and to support it if a ballot measure someday tried to strike it down. And they succeeded.

Ira Glass

In other words, the thing that was so exciting about the discredited study turned out to be true. These canvasses can change people's minds. For real.

Roughly one out of 10 voters changed their attitudes about transgender people. As the study notes, it took from 1998 to 2012-- 14 years-- for the country as a whole to have a comparable change in its attitudes towards gay people. It only took the canvassers 20 minutes or less to do this for trans people. Broockman said the only political tactic that has ever been shown to be more effective than this in affecting attitudes is personally meeting somebody who's running for office, which of course, is a very different kind of thing.

Ira Glass

They did. Three months later, attitudes stayed changed. Again, same as the discredited study.

But where it gets really interesting is the stuff that the new guys found that is different from the original study. First difference? Remember how the original study said that only a gay canvasser could convince a voter to support gay marriage? And only a woman who talked about her own abortion could convince a voter to become pro-choice? David Broockman says on these trans issues--

Ira Glass

Which is a profoundly different finding from the original discredited study. The original study seemed to say that it was meeting somebody, relating to a gay person or a trans person or whoever that changed people's attitudes. It was empathy that did the trick. But this new study says it's something else entirely.

Ira Glass

Broockman and his co-author have also been working on a study of the abortion canvasses organized by the Los Angeles LGBT Center. And here's another difference from the original discredited research done by Michael LaCour. Broockman has found the abortion canvasses failed. They didn't actually convince people.

Ira Glass

The other possibility, of course, is that canvass organizers just haven't figured out yet how to talk about abortion at the doorstep in a way that lastingly convinces anyone. We tried very hard, by the way. Somebody actually flew to Los Angeles to locate that nurse that you heard at the beginning of the program to find out if she went back to her old pro-life beliefs after the canvasser walked away from the door. But we couldn't find her.

I will say in videos of the Miami canvass, one thing that's really striking is how malleable people do seem to be on the transgender issues in this way that's very hard to imagine on abortion rights. Sometimes a canvasser has to explain to the voters exactly what transgender means. They have to show videos to be sure that it's understood what it is that they're even talking about. Or take this man, an older voter with a kind face who stands at his doorstep in a sleeveless undershirt telling that same canvasser that you heard at the beginning of the program--

Ira Glass

They talked for a while. He's from Ecuador. Doesn't get out much because he's at home caring for his wife, who has dementia.

Doesn't have many friends. And seems genuinely confused about the difference between trans and gay. The canvasser explains what is what about that to him. She even helpfully informs him--

Ira Glass

He apologizes. Says that spending all of his time with his wife, maybe he's getting demented too. Forgetting how to talk to people. And by the end, it is just like the researcher Broockman said. The voter is the one reflecting on his experience, drawing new conclusions.

Ira Glass

It's kind of hard to hear. He's saying, I'm glad to be talking to an intelligent person that made me think about my own background. And that it was very old.

Ira Glass

Listen, probably I was mistaken.

OK, one last thing. I guess this is the most obvious question probably that I could ask in this story. And I just want you to know I did ask it. Here's the recording.

Ira Glass

He's saying that unlike Michael LaCour, they gave their actual raw information of what all the survey respondents said to somebody else to look at, who also made sure that the numbers in their charts and graphs and analysis match what the actual surveys say. They're posting the original survey data without people's names or identifying information online for anybody to inspect. Science marches on.

Coming up, a stranger tries to make somebody rethink everything in his life over the phone in one hour in one conversation. That's in a minute from Chicago Public Radio when our program continues.

Ira Glass

It's This American Life. I'm Ira Glass.

Each week on our program of course, we choose some theme, bring you different kinds of stories on that theme. Today's program, "For your Reconsideration."

We have stories today of people going back and rethinking evidence, reconsidering where they've been and what has come before. We have arrived at Act Two of our program. Act Two, "Hotline Bleak."

So here's an example of a guy reconsidering his life in real time on the phone with a complete stranger. The recording comes from Chris Gethard, who's a comedian. He also does a cable TV show. He's been on our program now and then.

And the recording that I want to play you comes from this new podcast that Chris is doing called Beautiful Stories From Anonymous People. Our editor Joel Lovell is the one who heard it and loved it.

Joel Lovell

So the idea for the podcast is pretty simple. Chris is in a studio in New York, and he opens the phone line to one caller, an anonymous person calling in from anywhere for any reason at all.

The only rules guiding the conversation are it's 60 minutes long and Chris can't hang up. If the caller decides to get off the line before the hour is up, if they get annoyed or mad or tired of Chris, that's totally fine. But Chris, he's in it for the duration.

In this call you're about to hear, you can feel him at the beginning getting his bearings, so it's a little meandering. And then something changes.

[BEEP]

Joel Lovell

This kind of conversation with a sad person who has a vague wish to join an improv class or maybe go to an open mic night but probably won't, this is really familiar terrain for Chris. He hosts a show on cable and he often talks about his own mental health struggles. And he and his staff, they're always concocting ingenious ways to motivate and cheer up depressed people. Specifically, their own depressed fans.

There's this one episode where Chris says, "OK, we're going to generate some footage here for people to revisit if they ever need to feel better." And then Ellie Kemper, that really sunny actress from The Unbreakable Kimmy Schmidt comes on. And they just keep bringing out impossibly cute things. There's a basket of bunnies.

Joel Lovell

And then a kitten. And then a baby dressed in a tuxedo. And she keeps having these huge spasms of joy.

Joel Lovell

All of which is to say this guy on the line saying he's not even sure what his dreams are, for Chris, that's like someone calling Suze Orman to talk about their FICO score.

Joel Lovell

For the next 10 minutes or so, they talk exit strategies. Like "OK, let's start blue skying here. Do you have enough saved up to get out right now and buy yourself a little time to look for something better?" He's got $300 in the bank. So no, not even close.

"All right, well, what else do you want to do?" No clear idea. His current plan is to get free by just being terrible at his job.

Joel Lovell

I should say here that we tracked the caller down and talked to some of his relatives. And everything he says in this conversation-- being delivered by Ron Paul, his mom being in prison, his having been conceived at the fair-- it's all true.

So Chris says, what's next? They're at 33 minutes and the clock's ticking and you can start to hear a little urgency in his voice. And the guy walks him through high school and college. And then after college, he moves into a trailer park. And a neighbor there tells him about a job opening at the place where he's calling from now, a company with the almost diabolically generic name Bank Tech, where he spends big swaths of his time hiding out in the hallway or eating yogurt out in his car.

So there's that, I guess, the private yogurt time. But otherwise, it's soul death. Just slow, low wattage depression.

There's about 10 minutes of that. And then Chris starts to press him. He actually really tries to change the guy's life in the brief time they have left.

Joel Lovell

That was it. He and Chris never talked again.

So I asked the guy did he go to that open mic the next night? Nah, he didn't make it. Right after talking to Chris though, he set a date. March 21st. That's when he'd quit his job and start living.

But then as the date got close, he told me he tried to do everything he could to undermine his own plan. He wasn't sure why, he just felt like he couldn't go through with it.

And then that same week, this conversation went out on Chris's podcast. He had no idea they were going to run it. But he listened to it and he heard himself talk for an hour about all these things he wants to do but never does. And then he did it. He went in to Bank Tech and quit.

Ira Glass

Joel Lovell is the editor of our program. The podcast again is called Beautiful Stories From Anonymous People.

Ira Glass

Act Three, "Kids Look Back." So today we're hearing stories about people reconsidering and reexamining what happened to them in the past. And now with an entire book, I understand, of children doing this? Is that right, Brian?

Ira Glass

Always a fine report. Brian Reed is the senior producer of our program.

Ira Glass

Our program was produced today by Stephanie Foo, Zoe Chace, Sean Cole, Neil Drumming, Chana Joffe-Walt, Miki Meek, Jonathan Menjivar, Robyn Semien, Alissa Shipp, and Nancy Updike. Our senior producer is Brian Reed. Our editor is Joel Lovell. Julie Snyder's our editorial consultant. Our technical director is Matt Tierney.

Production help from Lyra Smith. Special thanks today to Shane Fletcher. Our website, thisamericanlife.org. This American Life is delivered to public radio stations by PRX, the Public Radio Exchange.

Thanks as always to our program's co-founder, Mr. Torey Malatia. You know, summer is coming up, and he is so excited for the Tilt-A-Whirl and fried dough and the flying swings.

Ira Glass

I'm Ira Glass. Back next week with more stories of This American Life.