Home

About
- Who We Are
- Community Guidelines
- Right to Respond
Advertising on BOR
- Advertise on BOR
- Buy on all Texas Blogs

Advertisements

Search




Advanced Search


Follow Burnt Orange Report on Twitter (@BOR) and Facebook.

Decision Analyst: Obama up 57-43 in Texas, but poll can't be trusted


by: Michael Hurta

Sat Feb 23, 2008 at 00:35 AM CST


I am pretty sure this is by far the largest Texas lead that I have seen so far.  I decided that I am not going to trust this poll after reading the article about it, but the major margin is worth this post.

Arlington, Texas (Feb 21, 2008)- Barack Obama has a double digit lead over Hillary Clinton in Texas, according to a statewide survey of 678 registered voters planning to vote in the March 4 Democratic Primary. The survey was conducted February 20 and 21 by Decision Analyst, a major national survey research firm. Decision Analyst projections indicate that if the election were held today Obama would win 57% to 43% over Clinton. The survey's margin of error is 3 percentage points, plus or minus, at a 90% level of confidence.

Already here I saw a problem (although it is a minor one).  Almost every [political] public opinion poll uses a 95% confidence level.  Usually the level of confidence is not even reported for this reason.  I do not know enough about public opinion research to say how much that would affect the Margin of Error, but to read this poll accurately -- the margin of error would clearly find itself above 3% normally.

This was just a minor issue for me, though.  On a normal poll, I would imagine that a 678 sized sample for the state of Texas wouldn't get any obscene margin of error.  What bothered me was the paragraph I read about the poll and the firm.

Results based on scientific sampling of registered voters from American Consumer Opinion® (www.acop.com), one of the largest online research panels in the world. The sample was carefully balanced by gender, ethnicity, age, and geography; the data were weighted as necessary to fully represent the different demographic groups.

About Decision Analyst (www.decisionanalyst.com), based in Dallas-Fort Worth, is a leading international marketing research and marketing consulting firm specializing in advertising testing, strategy research, new product development, and advanced modeling for decision optimization. In addition, Decision Analyst owns and operates American Consumer Opinion® Online, one of the largest research panels in the world with over 7 million members.

A "scientific sampling?"  I realize that American Consumer Opinion has over 7 million members worldwide, but I'd be willing to bet that there are over 7 million registered voters in the state of Texas (I could be way off, but I'm sure the number is in the millions).  I'm sure the number of American Consumer Opinion's Texas panelists is a minute one compared to the actual population of registered voters.

Scientific samples are supposed to be a random sample of the population, and this was not random -- instead, it was a sample of Texas registered voters who decided on their own accord to register with www.acop.com.  Some pollsters believe that public opinion research will eventually get to become electronic based, but we are not yet there.

--

Ok, so what does this survey tell us?  Well, it probably tells us that Barack Obama is winning the internet users of Texas Democratic voters by a wide margin.

As if we already did not know that.

(I'm only an amateur on analyzing public opinion.  I take a class on it at UT from Daron Shaw.  It's possible that he has taught me wrong thus far, but I doubt it.  I trust the man on this - if I remember correctly, he worked on polling for President Bush's presidential campaigns, and you have to be good if you can help get that man elected.)

ADVERTISEMENT
Tags: , , , , , (All Tags)
Print Friendly View Send As Email
I think you are right on, Michael (0.00 / 0)
Well done on the research.

The same company has Obama up 54-46 in Ohio which is, like its Texas poll, a HUGE shift.

I think we should probably be suspicious of ny poll that you have to sign up for to participate in.


Also... (3.00 / 1)
You have to look at the amount of people surveyed...I think these random polls could go in favor of anyone.  Does it release what demographic area was polled, gender, race, political party, because some republicans are voting to offset the democratic race, class structure (low income, middle, etc.), and so forth?  I never trust polls, I wait for the final results.

Margin of error (5.00 / 1)
678 is a good sample size and would give a 3.8 MoE at 95%. There is no statistical reason that 95% needs to be the standard. The MoE is usually reported on the complete poll, but when you start breaking it down by gender/age/region/whatever, the MoE at 95% gets large. When you start talking about +/- 12% @ 95%, most of these crosstabs don't tell you anything. But if you were to calculate the 75% confidence margin, you are back in reasonable focus. A 1/4 chance of an outlier isn't nearly as good as a 1/20 chance, but it's still accurate most of the time. As long as you have a valid sample size, the most likely result is perfect accuracy. You might have a 40% chance of being within 1%, compared to a 25% chance of being between 1% and 2% off. The question is how wide do you expand the margin - do you want to capture 75%? 90%? 99%?

Of course, that all assumes that the population you are sampling matches the population you are trying to assess. I think that's where this poll misses the mark. Online polling is a valid concept, and produces good results in some cases. It will become more accurate over time, but coverage is not wide enough for this sort of survey.

Texas Economics


Great analysis (0.00 / 0)
Thanks.  I didn't actually know if the MoE would get that much higher, but as I said -- that was my minor problem.

The internet-survey issue seemed to be greatest.

"Let us tenderly and kindly cherish therefore, the means of knowledge. Let us dare to read, think, speak, and write."  -  John Adams


[ Parent ]
Sample size is fine. Choice of sample isn't. (4.00 / 2)
I've taught statistics classes, and the difference between 90% confidence and 95% is easy to fix. If you're within 3% with 90% confidence, you're within 4.5% with 95% confidence. So if this were really a fair sample of likely primary voters, then Obama would have a very big lead in Texas.

But it's not. As you noted, people who register with acop are not representative of Texas. Decision Analyst can balance for gender, ethnicity, age and geography all they want, but what about income, education and techno-savviness, all of which correlate with support for Obama?

If they repeat the survey a week from now and there's movement, that would be meaningful. But a single data point doesn't tell us anything we don't already know.  


Slight correction (0.00 / 0)
IVR's 3.8% figure is correct. A truly random sample will be within 1.65 standard deviations of the mean 90% of the time, within 1.96 standard deviations 95% of the time, and within 2.33 standard deviations 99% of the time. To go from "90%" confidence to "95% confidence", just multiply the error by 1.96/1.65, and to go from 90% to 99%, multiply by 2.33/1.65. The 4.5% figure I gave earlier was for 99% confidence.

What's not clear to me is whether the published errors are for the support of each candidate, or for the difference between candidates. If everybody has a preference, then saying "Obama is at 53% plus or minus 3%" is the same as saying "Obama is ahead by 6%, plus or minus 6%", not "Obama is ahead by 6%, plus or minus 3%".  IVR, can you fill us in?

The real problem with polling is deciding what a truly random sample is, and that depends on your model of who will actually vote, and on who will respond to polls. Different polling firms use very different models, and they often disagree by a lot more than the statistical error. All error bars should be taken with a grain of salt.  


[ Parent ]
MoE is on the candidate number (0.00 / 0)
Obama @ 53% +/- 3%

Texas Economics

[ Parent ]
I would like to believe this but.... (0.00 / 0)

Decision Analyst is a marketing research company and doesn't seem to normally do political polls.  The people polled have to like to take polls and register online.  They can adjust for geography and education and age and income etc. but they can't adjust that their poll takers are more Internet savvy.  It also looks like they only adjusted for gender, ethnicity, age, and geography, not income or education, and you have to wonder what voting turnout model they used.  

I am sure I would not use Internet panels for political polls without a track record, even though some other polling organizations are experimenting with them.  I think it is obvious that Internet users are more politically savvy than the average voters and do not reflect the typical voter.  If only the Internet users could vote Bush would never have been elected.



In Top 5 on Google for Liberal News

Easter Lemming Liberal News


Its Also the Non-Standard Wording (0.00 / 0)
The wording of the article is not standard wording for a pollster: "Decision Analyst projections indicate that if the election were held today Obama would win 57% to 43% over Clinton." That word projections suggests an interpretation of results rather than a presentation of results.

And add on top of that the fact that I could not find a complete report by the company that at a minimum presented the questions used and tabulated results leads to further credibility issues. Can you say "transparency"?  


Burnt Orange Reader

Menu

Make a New Account

Username:

Password:



Forget your username or password?


Poll
Who would you vote for in the Democratic Primary for Ag Commission?
Kinky Friedman
Hank Gilbert

Results

Advertisement

Best of Texas Left
- (Complete Directory)
- A Capitol Blog
- As the Island Floats
- B & B
- Bay Area Houston
- Blue Bloggin
- Bluedaze
- Brains and Eggs
- Capitol Annex
- Collin County Democrats
- Collin County Observer
- Community Forum
- Dog Canyon
- Dos Centavos
- Easter Lemming Liberal
- Eye on Williamson County
- Feet to the Fire
- Greg's Opinion
- Grits for Breakfast
- Half Empty
- Houtopia
- In the Pink Texas
- Kiss My Big Blue Butt
- Letters from Texas
- McBlogger
- Mean Rachel
- Musings
- North Texas Liberal
- Off the Kuff
- Panhandle Truth Squad
- Para Justicia y Libertad!
- Pink Dome
- San Antonio Mayor
- South Texas Chisme
- StoudDemBlog
- Texas Clover Leaf
- Texas Kaos
- The Caucus Blog
- There..Already
- Three Wise Men
Best of Texas Right
- Blogs of War
- BlogHouston
- Boots and Sabers
- Lone Star Times
- Publius TX
- Rick Perry vs the World
- Safety for Dummies
- Slightly Rough
- Urban Grounds
Other Texas Reads
- Burka Blog
- D Magazine
- DOT Show
- Statesman Elections
- Strong Political Analysis
- Texas Monthly
- Texas Observer
- The Texas Blue
- Quorum Report Daily Buzz
Around Austin
- Austin Bloggers
- Austin Chronicle
- Austin Contrarian
- Austin Metblogs
- Austin on Two Wheels
- Austin Real Estate Blog
- Austin Statesman
- Austin Texas Bike Shit Stuff
- Austin Towers
- Austinist
- Capital MetroBlog
- Daily Texan
- Do512
- Downtown Austin Blog
- East Austinite
- Elise Hu
-
Flash Mob Austin
- Keep Austin Blue
- M1EK
- Travis County Democrats
- University Democrats
TX Progressive Orgs
- ACLU Legislative Blog
- Atticus Circle
- Criminal Justice Coalition
- Equality Texas
- Latinos for Texas
- NOW Texas
- PFAW Texas
- Public Citizen
- SEIU Texas
- Tejano Insider
- Texas AFT
- Texas HDCC
- Texas Watch
- TFN
- TSTA
- TSEU
- Texas Young Democrats
- United Ways of Texas
TX Elections/Returns
- TX Returns 1992-present
- TX Media/Candidate List

- Bexar County
- Collin County
- Dallas county
- Denton County
- El Paso County
- Fort Bend County
- Harris County
- Jefferson County
- Tarrant County
- Travis County

- CNN 1998 Returns
- CNN 2000 Returns
- CNN 2002 Returns
- CNN 2004 Returns
- CNN 2006 Returns
- CNN 2008 Returns
Traffic Ratings
- Alexa Rating
- Quantcast Ratings
-
Syndication

Burnt Orange Reporters
Publisher - Karl-Thomas M.
Editor-in-Chief - Matt G.
Staff Writer - David M.
Staff Writer - Katherine H.
Staff Writer - Michael H.
Staff Writer - Todd H.
Man of Mystery - Phillip M.
Founder - Byron L.

Powered by: SoapBlox