Search






Jeff's Amazon.com Wish List

Archive Calendar

November 2024
M T W T F S S
 123
45678910
11121314151617
18192021222324
252627282930  

Archives

Another example of why I don’t like horserace poll coverage [Karl]

The latest USA TODAY/Gallup Poll, showing Barack Obama ahead by 3% among registered voters, but John McCain ahead by 4% among likely voters is topping Memeorandum at the moment, in no small part because the numbers do not gibe exactly with the 8% Obama lead in the regular Gallup tracking poll for the same period.  It becomes the occasion for all sorts of ill-informed venting about polling generally, even though as an event, the polls are no big deal (given the margins of error, the two are not so far apart as the topline numbers suggest).

Ace, unfortunately, seems to be in the “polls don’t mean anything” camp — and I write “unfortunately” because his specific observations about the underlying problems with polling in this cycle are astute.  The so-called “Bradley effect,” the ultimate turnout of the youth vote, the ultimate role of the “enthusiasm gap,” etc. all make polling likely voters a much dodgier exercise this year.  (I should add that I do not include him in the ill-informed venting, which tends to appear in the comments of various sites I visit. But I digress.)

That being said, the USA TODAY/Gallup Poll (afaik) is using consistent methodology, so McCain’s gain from a 50%-44% deficit in the June poll is of some interest.

Allahpundit notes prior polling showing a big drop among the 18-29 year-olds saying they are certain to vote.  In a prior post noting the drop, I noted this could be a temporary lack of attention during the summer doldrums.  But it is the sort of thing that could account for the USA TODAY/Gallup Poll swing among likely voters.

That prior post on the last ABC News/WaPo poll also notes just how volatile the swing vote remains. That point is made by pollster Mark Blumenthal again with regard to the new polls, along with a discussion of various factors that may cause different results between the two Gallup polls.  Which brings me full circle — it always irks me when ill-informed people vent about polls when the Internet now allows people to read expert analysis from people like Blumenthal and get a better understanding of both the strengths and weakenesses of particlar polls and polling in general.

(h/t Memeorandum.)

33 Replies to “Another example of why I don’t like horserace poll coverage [Karl]”

  1. Darleen says:

    I have a confession to make.

    A few nights ago I received one of those Rasmussen telephone surveys.

    As far as Rasmussen knows, I’m a 20 something, black female, moderate, making $30k/yr, who values national defense as #1 issue, and who doesn’t trust Obama and will be voting McCain.

  2. Dan Collins says:

    OMG, Darleen. I didn’t know you were moderate!

  3. Karl says:

    I think the research tends to show that imposters are both a small percentage and tend to be a wash over time.

    For all we know, Lisa is pretending to be Darleen and voting Obama.

  4. Brother Ratso, felonious monk says:

    No, the reason horserace polls are bad is because horse racing is cruel.

  5. ccoffer says:

    Likely voters is the only sample that matters. How much a candidate is liked by non-voters is about as relevant as ,well, how much he is liked by a throng of screaming Germans.

  6. Alec Leamas says:

    I believe that the so-called “Bradley Effect” will account for several missing “O!” points on election day, which will give everyone a bit of what they want. (See also, the Goode/Rizzo effect) We will get a decent, non-Socialist as President, and the media will have a phenomenon with which to indict the American people as hopelessly racist. Added to this is what I perceive to be resistance against the Hobson’s choice that the media is trying to force upon the American electorate – “O” or you’re an irredeemable racist.

    And I certainly don’t trust anyone under 26 or 27 to be present and to know what the fuck he/she is doing for anything remotely important who hasn’t seen the wrong end of a Drill Instructor. They’re about as dependable and motivated as Sugar-free Jello.

  7. Darleen says:

    And I certainly don’t trust anyone under 26 or 27 to be present and to know what the fuck he/she is doing for anything remotely important who hasn’t seen the wrong end of a Drill Instructor.

    I tell you, Alec, #2 daughter, 27 and mom of twin 5 y/o’s has certainly matured, very interested in news/events/politics in a way completely opposite of her pre-motherhood years. She now has a stake in the future and those boys would drive a DI mad. heh.

  8. The Lost Dog says:

    Hi, Karl.

    as far as I am concerned, you are one of the most intelligent people here.

    But, please, stop giving polls legitimacy. They are bullshit, which anyone who has ever participated knows. They are nothing more than a tool for the media to push the general population in whatever direction they choose.

    Interesting enough for anyone who has not been brainwashed by the media and our schools, but a tool to push the politically unaware in a certain direction.

    I really mean it when I say that I respect your intellect, but polls have become a tool of the drive bys to shift opinion. If you have ever been called by “pollsters”, you know that there is no “in between”.

    I answered poll questions once, and felt like I had soiled myself after I hung up. There is no room in any poll for nuance. They are bullshit, and except for the very top pollsters, are meant to flummox any participant. I’m sorry, but “black or white” doesn’t come close to my outlook, and those are the only two options offered in any poll.

    I am seriously not trying to make you feel uncomfortable, but most polls are engineered at a slant,

    But, whatever. I am always impressed with your posts, but polls are meant to sway, as far as I am concerned, and can do it more easily than Elliot Spitzer could “do the do” with that wonderful little four thousand dollar trollop.

    Please don’t take this as personal, because I think you are awesome, and am always eager to read your posts. But I just think that polls are bullshit, and are almost always used by the MSM more to shift the electorate than to give a “snapshot”

  9. TmjUtah says:

    Karl, your poll analysis is your affair. The media environment is so pro-O! that I think they are frankly a waste of time this year. I predict the margin of victory will exceed a fifteen point miss with whatever the poll mean predicts.

    I don’t pay attention to polls until November after the candidates are identified. True, the Dems are still searching… but I’m so disinterested in either of their offerings I frankly don’t care.

    Obama would be easier to beat, I’m thinking. Why, you ask?

    Because the media hasn’t explored that possibility, especially with the Messiah splattering ten points up or down against Angry McOldguy between the big-name polls, that’s why.

    Bastards.

  10. Karl says:

    Lost Dog,

    I understand your point, which is why I think people need to be more educated about polling and how it works. You’re absolutely correct to note that the questions often leave no room for nuance — and can be ambiguous to boot, so that the results are subject to multiple interpretations. But given how poll-driven the media is, I think it is better to analyze the polls rather than dismiss them — the casual viewer tends to view them as way more scientific than they are; simply dismissing them makes the complainant seem “unscientific.”

    Similarly, some polls — mostly those commissioned by media outlets — are meant to have a particular agenda. The recent NYT poll on race relations mixed in with the horserace questions comes to mind. But imho, it is far better to identify those agendas and analyze the results in that context than to pretend the effort to shape opinion is not there — or to assume it is there without making the effort to demonstrate it to the casual media consumers who fail to consider it.

    This is one reason among many that I generally note my distaste for horserace polls. They generally do not tell anyone much at this point — but the internals can be interesting. I don’t see any particular agenda at work in the USAT/Gallup poll, but the internals tend to confirm (to me, anyway) just how mushy that mushy middle remains.

    Moreover, the toplines of current round of polling are interesting simply because it looks increasingly likely that the current Gallup bump for Obama will fade like the one at the beginning of his World Tour did — if Rasmussen and the USAT/Gallup poll are on target. The reason it is interesting is because the overt opinion-shapers in the media are always going to look at things like the World Tour as the event that could vault Obama into a big, long-lasting lead, making this race look more like 1980 than 1976. It increasingly looks like it didn’t happen this time, which is significant.

  11. Karl says:

    An addendum: One thing people have learned about polls over time — and mostly through the growth of alt-media (cable TV, the ‘net) — is the difference that sampling registered vs. likely voters can make (even if they don’t know how likely voters are screened). And if people reflect on it, you can see the explanation for what many used to think was a political bias in the polling.

    To wit — conservatives very often used to complain that polls always overstated Dem support throughout the campaign, then would tighten at the end to avoid embarrassment when the GOP did better than the polls would predict. In reality, most of the difference was the that the MSM would not switch from polling registered voters to likely voters until the home stretch, because the latter costs more. Some of the last-minute shifts are due to the warping effect of eekend polling — which is another aspect of polling that people have learned about. And polls right before Election Day may understate GOP support because they tend to be weekend polls — not out of bias, but because we hold elections on Tuesday.

    These are examples where I think learning more about polling helps demystify them. People can take those factors into account when hearing the MSM spin on these polls, which I think is healthy. the more people realize the level of subjectivity that goes into constructing polls and interpreting them, the less they will be treated as having been engraved on stone tablets.

  12. TheGeezer says:

    Karl is much moer informed and educated than me. He – and others – have helped me understand not to get upset about polls. They are interesting not so much because they reveal anything about elections, but because they reveal much about pollsters.

    Only one poll matters.

  13. PR says:

    Karl, my problem with polls is that they measure opinions of people who have nothing better to do than talk to a pollster. Do they adjust for the people that don’t want to participate or just disregard that segment of the populace? Thanks for the explanation of why they always lean Democrat until the last minute with registered vs. likely voters. It’s almost as if people of a certain political persuasion are more likely to have nothing better to do than talk to a pollster, even if they are not voting.

  14. JD says:

    PR – Exactly. Like jury duty.

  15. afall says:

    “In reality, most of the difference was the that the MSM would not switch from polling registered voters to likely voters until the home stretch, because the latter costs more”

    I’d guess that the model of who is a likely voter would be less stable before the ‘home stretch.’

  16. Great Banana says:

    These are the same claims made every year: “this year the polls aren’t accurate b/c we don’t know if the ‘youth vote’ will really turn out”; “this year we don’t know if the polls are accurate b/c we don’t know if the enthusiasm for X will mean higher turnout for X”.

    These same claims/arguments were made in 2004 and 2000 (and I’m sure in previous years). Gee, does anyone remember that Howard Dean was unbeatable and would win the nomination easily? Yet, every single time the youth vote does not turn out and there was no great “enthusiasm gap” that the polls failed to predict. I really see nothing that Obama is offering that will energize the “youth vote” more so than past candidates.

    Indeed, I see less if we are looking at the Iraq war as the catalyst for the “youth vote”. In 2004, if the “youth vote” was going to vote based on opposition to Iraq, there was a much stronger catalyst for that to happen based on the status of progress in Iraq at that point in time. Do we really believe that the fact that Obama is black is going to drive the “youth” to vote in record #s? Based on what?

    This year will be the same. the only valid claim is that there is a potential “Bradley effect” (which I thought was the “Wilder effect”?). There is actual historical polling data showing such an effect, and we can predict that it will likely be about 3% against Obama.

    So, this year’s polls are of the same value as every other year’s polls. the value of any poll is as good as its methodology and its sample size. And, it is only good as far as being a snapshot of today and to show the trends.

    With all that said, I find the polls to be extremely encouraging as far as the presidential election. McCain is doing better at this point in time than Bush was in 2000 or 2004. With the anti-republican mood in genderal and the media’s incredible pro-Obama saturation coverage, the fact that McCain is actually polling ahead of Bush in 2000 and 2004 is incredibly encouraging.

    I am hardly a McCain enthusiast – I vowed not to vote for him or to give him money based on many positions he has (immigration, taxes, campaign finance), but have recently (the Obama world tour) become so strongly anti-Obama that I will vote for McCain and have donated money to him.

    My guess is that the rest of america is slowly coming ’round to the same conclusion and we will continue to see McCain rise in the polls. Indeed, if the trajectory of the polls follow history, I expect McCain to win handsomely on election day based on his current polls.

  17. Great Banana says:

    As a follow-up to my last comment, I would guess that the only way to translate “youth” enthusiasm into actual votes is to capture that enthusiasm about a week out from the election.

    The problem with the “youth” is that they get wildly enthusiastic and passionate about something new – which gets other “youth” to join in the enthusiasm. But, that enthusiasm peters out very quickly.

    Thus, getting a bunch of 18-25 year olds to pay attention and volunteer and be enthusiastic in June will almost never translate to actual votes b/c those people’s interest and enthusiasm will disappear long before November.

    If you could somehow leave the “youth” alone until about mid-October, and then get them fired up, you would have a chance at actually capturing a significant number of real votes. As it stands, I doubt very much that Obama is going to see any “youth vote.” We are already seeing their attention waning in the polls, and it is only going to get worse for O!.

  18. Karl says:

    I really see nothing that Obama is offering that will energize the “youth vote” more so than past candidates.

    Obama has built on Dean’s Internet model. That whole MyBO effort, along with his other online projects, is geared toward doing what the GOP did in 2000 and 2004 — identifying True Believers and getting them to personally reach out to their friends in the GOTV effort.

    If BO keeps his youth based energized in November, it does create the potential for increases in youth turout (which has risen in the past couple of cycles, btw). It would not be a tsunami, but perhaps enough in a close state, e.g., Ohio.

    The last ABC/WaPo poll showed a big drop in 18-29ers saying they were certain to vote, but we don’t know if that’s young people figuring out BO is just another pol or whether 18-29ers pay less attention during this relatively slow period in the campaign. It’s also true that McGovern ended up losing the youth vote in 1972 — and that McGovern is one of several candidates with big youth appeal (from Goldwater to dean) that never won. But the youth factor is something which probably needs to be considered at least more seriously this year than in some past cycles. Thus, pollsters will struggle with their models of likely voters, which are based mostly on past voting history, which 18 year olds inherently do not have.

  19. Great Banana says:

    But the youth factor is something which probably needs to be considered at least more seriously this year than in some past cycles.

    Aside from everyone asserting this to be true, why? What data supports this that is different than past years where this EXACT SAME argument has been made?

    I see the claim, and I understand the emotional pull such a claim has based on media reporting of the “energized” youth. However, I see no actual facts to support this assertion.

  20. Great Banana says:

    Put another way, you are arguing that the polls this year are just as good as they always are, and that the historical data of polls accuracy should be trusted.

    I am pointing out that the same historical data you are relying on for that argument contradicts your “youth vote” argument. You cannot rely on the data for one argument and then poo poo the data for another argument.

  21. Sdferr says:

    Instead of calling it the “Bradley effect” or the “Wilder effect”, why can’t we just be honest and call it the “Black guy effect”?

  22. Karl says:

    GB,

    I explained why. Pls read again what I said about Obama’s social networking efforts. It’s a ramped-up version of the GOP 72-hour projects that were quite successful in ’00 and ’04. And those projects are all based on research showing that contact from a peer or friend is the most effective (and cost-effective) method of GOTV. Also the data does show increased youth turnout over the past couple of cycles — again, not always major increases, but increases.

    If you were a pollster (or a GOP campaign operative) it would be risky to ignore the possibility of a youth vote increase, even if you had doubts about it. And as this post is about polling, I am merely identifying the sorts of factors that pollsters consider when developing models of likely voters.

  23. happyfeet says:

    Cause there are black Republicans is why. They are unsanctified.

  24. Sdferr says:

    Or, if you like, the “White guilt effect”?

  25. Great Banana says:

    Karl,

    I thing your argument regarding the “youth vote” is unpersuasive. Yes, there have been slight increases overall in youth vote – attributable to things like MTV’s “vote or die” campaign, etc.

    There is no evidence, for instance, that Dean’s effort actually increased the “youth vote” or that any other specific candidate’s effort has really been able to do so. the evidence about the elusive youth vote is almost always anecdotal.

    I’m not saying to ignore the possibility that the youth will rise up – after 50 years of predictions that this year it is finally going to happen – and vote en masse for Obama – but I simply see no factual evidence to support it. [And by factual evidence, I mean evidence different from the same evidence in every presidential election at this point claiming that the youth vote was actually going to matter].

    And, pointing to some new tactics by Obama to say it will happen this year is not evidence that it will happen. Every cycle a campaign comes up with an alleged new tactic that will cause the “youth vote” to rise up and actually, you know, vote. Every cycle we get this same breathless alarm that the “youth vote” will rise up and vote en masse for the liberal. Every cycle it does not happen.

    Thus, again, the only actual evidence, as opposed to argument, is against it happening this year.

    Let me ask this in a different way, however. Let’s assume that the “youth vote” is actually going to appear this cycle and vote en masse for Obama. What, exactly, can McCain do about it?

    If the “youth vote” is truly overwhelmingly for Obama, and it is only a question of whether or not Obama can get the “youth” to actually vote, what can McCain do to ensure that it does not happen? I would hate to see McCain waste limited resources chasing the “youth vote” when in all liklihood they are simply not going to vote.

    I also do not buy the claim that the “youth vote” is overwhelmingly liberal. It is probably overwhelming liberal at most colleges, but in the real world I think it is much closer to the party breakdown among everyone else. My guess is that it is probably slighly more democrat (maybe 5%)then say the 26 – 45 cohort.

    As a final anecdotal point (and this shames me), I worked on Clinton’s 92 campaign and was in college at the time (gee, imagine that, I was a big lib when in college). I worked very hard to get the “youth vote” out. We managed to get about 300 college students out of about 7,000 to vote (we probably registered close to 1,500 to vote in the college’s town). There was tons of “youth vote” excitement, and tons of support for Clinton on campus, but – like always – it did not translate into actual votes. and, we actually got the local town to put a polling station on campus in the student union, so it could not have been made much easier to get the students to vote. So, I am coming at this from cynical experience. If you put the polling station in a bar and it was open from 10 p.m. – 2 a.m., you would see a huge increase in teh youth vote. Until then, I’ll believe it when I see it.

  26. Brother Ratso, felonious monk says:

    “I’d guess that the model of who is a likely voter would be less stable before the ‘home stretch.’”

    That may be true, but substituting “registered voters” makes for an even less useful indicator. Polling organizations have to fudge somewhat on who is and who isn’t a “likely voter” even within days of the election because they’re trying to build a prediction of outcome on a prediction of voter behavior, which may be contingent on the prediction of the outcome, which depends on the predicted behavior…

    Whereas, for the media who pay for polls, a volatile race presented in polls with lousy methodology and useless sample indentifiers, is worth more than an accurately predicted outcome. If people know months in advance who’s really going to win, who cares about campaign coverage?

  27. Karl says:

    GB,

    Let me ask this in a different way, however. Let’s assume that the “youth vote” is actually going to appear this cycle and vote en masse for Obama. What, exactly, can McCain do about it?

    This is really the crux of the misunderstanding here. First, I am not suggesting or predicting that the youth vote will suddenly turn out en masse, let alone for Obama. Indeed, I have previously written that while this is a demographic BO could exploit, “even a large percentage increase in youth turnout will not change their overall share of the vote by more than a percentage point or two.” As far as polls and campaigns go, it matters only in a very close race. However, this could be a close race — in which case, pollsters and pols would take a risk in ignoring it.

    The second part of the misunderstanding is thinking that any of these observations are directed to the question of what McCain can do about it. This post is limited to the question of issues pollsters face in trying to gauge voter opinion in this election. To all outward appearances, McCain appears to be ignoring it at the macro level, which may be a completely rational choice — campaign resources are limited, even at a projected $400 million. The calculation may be that they will assume some increase in the youth vote (mostly for BO) and focus attention getting out some offsetting voters. For my purposes in this post, it’s not my concern.

  28. Great Banana says:

    Karl,

    Then, it appears that we agree for the most part. I guess I’m still not clear what you are trying to say about the youth vote – other than it may skew the polls if the pollsters a) don’t account for it and it does happen or b) account for it too much and it doesn’t happen.

    But, isn’t this always the problem with trying to define who is a “likely voter” for purposes of coming up with an accurate poll?

    I guess my point is simply that – absent some major new compelling evidence to the contrary, which I don’t believe exists just yet – the wise thing for pollsters to do would be to expect the youth vote to be about the same as it always is and account for it accordingly.

  29. Great Banana says:

    Karl,

    To follow-up my last comment, I guess this is akin to “party identification”. Should pollsters take the current self-identified party I.D. #’s at face value and thus create their sample based on those percentages? Or, should pollsters assume that many republicans who are not self-identifying as republicans right now will come back to the fold at election time and thus the pollsters should use a more historic party identification #?

  30. Great Banana says:

    My suggestion to McCain – if he believes that there will be a large youth turnout and that it will be overwhelmingly for Obama – is for him to host free keggers across america during voting hours at places that are not convenient from polling stations. That will keep the youth vote down.

  31. JD says:

    Ever notice that the only time the Leftists talk about the mechanics of the polls is when it is not in their favor. When it comes to other polls, or Presidential approval ratings, the numbers are accepted without question and are the Gospel Truth, spoken from the Mount.

  32. Great Banana says:

    Ever notice that the only time the Leftists talk about the mechanics of the polls is when it is not in their favor.

    In all fairness, both sides do this. It’s human nature to accept as true those things that support your side and question the credibility of those things that do not.

  33. Brother Ratso, felonious monk says:

    “you can at least compare two registered voter polls”

    Which plays into the media’s brainless fixation on those horserace polls. They help generate interest in a campaign even if one of the candidates is a closet Maoist or an uncloseted lunatic. That’s their only value and I say to hell with them.

    Anybody who chooses how to vote based on polls should be deprived of the franchise until they figure out what matters.

Comments are closed.