Status Report: About That Poll…

A new NYT/Siena poll has people freaking out again. But they shouldn’t.

Image courtesy of the NYT

Leave it to the New York Times to stir up Democratic anxiety right before the State of the Union address, with a poll showing Trump beating Biden by five points nationally. A number of my friends sent the poll to me. Some of them are now filled with such gloom and doom that I wanted to lay out my thoughts on it plainly here for a wider audience.

It seems no matter how often I beat the drum about the polls being unreliable, premature and wrong, it doesn’t allay people’s fears enough. Talk of “ditching Biden” then ensues, which of course is not going to happen. We should just stop any discussion of it right now if we know what’s good for us.

So what about this poll? Isn’t the NYT / Siena a reliable indicator? They wouldn’t publish something that is basically false information, right?

Polls aren’t “false.” They report what people actually told the pollsters. But they can be misleading, and they are often faulty. Importantly, we shouldn’t trust any polls to be predictive of the final result when there are still more than 200 days to go till the election and a whole lot of unknowns. But let me focus in particular on this poll and what some experts on polling and methodology are saying about it. I hope you come away with the same conclusion I did: that this is some data, yes, but it’s not very reliable or useful except to make big headlines.

Women voters

One of the things I look for in a polling result is if any of the breakdowns, found in what’s called the “crosstabs,” raise red flags. One jumped out right away for me in this poll, and others saw it, too. As LSU professor and political historian Robert Mann noted,

I do not believe Biden is tied with women nationally 46-46… Biden got 57% of women in 2020. You're telling me that, post-Dobbs, his support among that demo group will drop to 46? Not credible.

I agree. If you see a poll and half the women are voting for Trump, something went wrong in the polling sample. If women voted like they did in 2020, which we should assume would at least be the case especially since Dobbs, that’s an 11 point difference from this poll. Assuming the poll is half men, half women, that would put the two candidates about even.

Democrats

Here’s a thing I’m sure the Dean Phillips campaign would love to see become reality: The NYT/Siena poll has Phillips at 12 percent support among Democrats.

Really? Because last time I checked, in the actual official contests that have been held, his actual vote haul averages 1.5 percent. As UCLA professor Matt Barreto noted,

There have been 3 DNC sanctioned primaries and Phillips vote:

South Carolina - 1.7%

Nevada - 0.0%

Michigan - 2.7%

So what people are *telling* the NYT/Sienna does not square with how they are VOTING.

It’s fair to ask which Democrats are bothering to respond to and answer these polls to completion. Perhaps it’s those who, on average, tend to be more disgruntled and want to voice their displeasure to a pollster? Are these Democrats also more likely to say they are unhappy with the current president? Just throwing it out there.

Young people

The news in this poll was partway decent for Biden when it came to young voters age 18-29. He leads Trump by 13 points among them, 54 to 41 percent—but that’s still around half the spread that other major polls have on this age group. But when it comes to messaging on the youth vote, the NYT prefers to emphasize the negatives, and its own data seems at odds with itself.

For example, as former pollster and turned sometime polling industry critic Adam Carlson notes, the NYT/Siena poll of swing states conducted back in late October showed Trump actually leading in this age group in AZ and GA, while being tied with Biden in MI. On this contrary, surprising and incorrect result, Nate Cohn of the Times did a whole serious write up about what it could mean.

But when the new poll shows Biden actually leading nationally within this group by 14 points, Carlson observes, the NYT analysis completely ignored this.

Perhaps that’s because it is hard to square that earlier result of Trump leading with this very different one of Trump trailing without calling one or both into serious question. Such huge swings in such a short amount of time don’t suggest that the electorate is moving quickly so much as that the polling might be way off.

Latino voters

When the Times interviewed Latino voters, an important part of the Democratic base especially in the battleground states of Nevada and Arizona, it focused nearly completely on English-speaking respondents. The result was a 46-40 skew by Latinos in favor of Trump, which defies everything we know about Latino voting patterns, even if there has been some erosion. Looking more closely, the NYT poll, which now has many now wondering if Latino support has collapsed for Biden, had a whopping 97 percent of Latinos interviewed in English.

NBC News’s Adrian Carrasquillo took strong issue with the NYT’s approach.

I’m a broken record on this but the problem with these numbers is the lack of interviewing Spanish-speaking Latinos. Univision had a poll of 1400 Latinos in the fall that showed Biden doubling Trump with Spanish-speakers but up only 46%-43% with English-speakers.

The 46%-40% Trump lead below makes a lot more sense if a confusing 97% of interviews are in English.

Older Latino voters, whose native language is often Spanish, are some of the most reliable Democratic voters out there. To ignore them in the poll and focus nearly only on English speakers understandably delivers wacky results. The Univision poll, by the way, had Biden over Trump 58 to 31 once you included Spanish speakers, who skewed 62-26 for Biden.

Some further thoughts

There were other problems with the poll that I won’t get into here, ranging from an oversampling of conservatives to a Black voter breakdown that flies in the face of historical numbers in every other recent election. With all of these flaws, which didn’t take long for folks to unearth, we really shouldn’t lose sleep over this poll.

I mean sure, it is certainly theoretically possible that this poll is correct, and that the polling average of all the reliable polls showing a dead-heat race is way off. But I just don’t believe this is very likely.

The most likely circumstance is that we are not far off from where we were in 2020, which means it will come down to a handful of swing states and a few hundred thousand votes across them, and that this will be close and hard fought. Trump has lost support from Republicans and Biden has lost support from Democrats, and there are eight months to go for each side to try and win them back and to turn out their respective bases.

One other thing: It’s not enough these days just to “ignore the polls” or just tell friends to do the same. Given that the polls are methodologically suspect at best and even weaponized against us at worst, we need to make data-driven cases for why they are unreliable and to keep these things squarely in mind as we get closer to Election Day.

When a poll like this comes out, ask some basic questions. Is the poll an outlier when it comes to key demographics, like women, young people and minorities? Does it use flawed methodology and exclude whole swaths of voters, like Spanish-speaking households? Does it square with actual election results where Democrats have shown overwhelming support for Joe Biden and no interest in his challengers?

When your friends and colleagues drop the poll link in a group thread, or tell you that they’re really worried, you can refer them to my newsletter. Ask them afterwards whether they still trust the poll.

One advantage that Democrats and independents have over Republicans these days is our ability to think critically and to question prevailing narratives using facts and data. Let’s be sure to do that regularly, including calling out the NYT, so that we can help ease minds and lift spirits. Then let’s get to work and bring this all to a resounding victory come November.

Have a restful Sunday!

Jay