On the Morning Consult Poll, Margins of Error, and the Undecideds in the Democratic Primary

Drawing by Nathaniel St. Clair

Update/Correction: Morning Consult has responded to the article via email, apologizing that our initial query went to spam. The article has been updated to reflect their insistence that they do not simply “flush” undecided voters as the piece originally suggested it may. Rather, Morning Consult strongly asks people to say which way they would lean if they had to choose (no option for undecided/don’t know on this follow up question).

Morning Consult regularly has Joe Biden a fair higher than every other polling firm at this point, and, as they have usually throughout the 2020 Democratic primary cycle to date, also have Bernie Sanders a bit higher than the average.

Morning Consult also has, by far and away, the largest sample size each week, with a reported margin of error (MOE) of 1%. So, is Morning Consult or its MOE wrong? Or are all the other polling firms missing something that Morning Consult is onto with its large sample size and, per my diving into their full tables as sent to me over the course of three weeks around Biden’s entry into the race, fair weighting of a broad array of demographics?

Based on data in this chart and further explanation below, I think neither is quite wrong. It’s just that Morning Consult is handling undecided voters in a unique way that understandably stretches Biden’s share and, to a lesser extent, Sanders’ share. (MC in the chart = Morning Consult; DK = Undecided or Don’t Know respondents.)

I wrote about this problem (and proved absolutely correct) ahead of the UK General Election in 2017 here. And I spent a bit of time on it for this 2020 Democratic cycle in this article, but without the specific focus on Morning Consult.

When polling firms simply flush “undecided” or “don’t know” voters from their sample and then report the findings without any other adjustments, they automatically boost the leader an extra amount, and where the lead is already big, the problem becomes even worse. I noted originally here that I was unsure whether or not Morning Consult was following this practice. Anthony Patterson, Morning Consult’s Director of Communications, has written to clarify:

[T]hose respondents who select “don’t know” or “no opinion” the first time around are then prompted with the lean question and provided candidate options along with an option for “someone else.” If someone selects “someone else” they are asked to specify and write-in a name. With that said, respondents are not permitted to select undecided for this particular question.

But what would happen if they allowed people to remain undecided or did not ask the lean question at all? Patterson noted that their lead data scientist is currently out of the office, and that I could expect further response once they return.

Supposing their initial undecided numbers are similar to other polling firms, Column B in the above chart are Morning Consult’s numbers as reported for June 10-16, while Column C is what you get if you put back in the current average of other polling firms’ undecided/don’t know respondents. Biden’s share drops 6.4%; Sanders’ drops 2.7%. And this brings them both into pretty close alignment with the strict average of all polls in the field at least one day in June (and RealClearPolitics’ overall average right now as well).

In fact, Elizabeth Warren at a 3.1% difference is the only candidate’s average that falls outside the 1% MOE as compared to the June strict average. Buttigieg is at 1.7%, but since I’ve found over several election cycles that a strict 10 day average is generally right within a 1% margin, on the gap between two leading candidates or parties on the final results, this hardly seems remarkable.

Now, my #10at10 average will keep Morning Consult’s results in them as presented by Morning Consult. I’m even more against unskewing polling for presentation in averages than FiveThirtyEight (which uses a non-transparent House Adjustment on polls). The basic point of this exercise is to show 1) once again, that how a polling firm handles undecideds matters quite a bit and 2) that there really may not be as big of difference as there would initially seem between Morning Consult and the rest of the recent results we have seen. Elizabeth Warren’s total is lower than the recent average, outside the MOE, on this account, but with results for Warren ranging from 5% to 19% nationally over the course of June so far, this one difference among twenty-one candidates as polled by Morning Consult is not really all that surprising.

This is a particular data choice by Morning Consult, certainly defensible, but also a data choice that goes a long way to explaining why their share for Sanders and especially for Biden is higher than average. As my previous two articles have noted, most Democrats are persuadable (considering more than one candidate, only having “soft support” for their first choice) and there is reason to believe that Biden’s support is exaggerated based on very particular analysis of polling data (including undecideds).

If you assume that Morning Consult is getting about an average number of undecided/don’t know respondents as other pollsters, but are pushing people hard to say which way they lean when they are genuinely undecided, their results are pretty consistent with the rest of the field right now: Biden in the low 30s, Sanders around 16 or 17%.