Home

About
- Who We Are
- Community Guidelines
- Right to Respond

Advertising on BOR
- Advertise on BOR
- Buy on all Texas Blogs

Advertisements

Search




Advanced Search


Texas Poll Watch Numbers Wrong


by: Matt Glazer

Thu Jun 19, 2008 at 09:36 AM CDT


Yesterday we wrote about a Texas Poll Watch poll, perhaps too soon.  After a day of looking at numbers and talking to pollsters, we should not have printed the poll because none of the numbers seem correct.

First let's go through the poll county by county.

Texas Poll Watch - Aransas
Hunter- 48%
Garcia- 40.80%

Texas Poll Watch-Calhoun
Hunter- 33.75%
Garcia - 48.75%

Texas Poll Watch-Nueces
Hunter- 52.06%
Garcia- 39.69%

Texas Poll Watch-San Patricio
Hunter- 45.20%
Garcia- 32%

Now let's look at 2006.

Aransas
Seaman- 47.55%
Garcia- 44.94%

Calhoun
Seaman- 46.13%
Garcia- 49.59%

Nueces
Seaman- 50.97%
Garcia- 42.84%

San Patricio
Seaman- 40.90%
Garcia- 54.61%

To take the poll on face would mean that Garcia is - 4.94% in Aransas, - 1.16% in Calhoun,
- 3.15% in Nueces and - 22.61% in San Patricio.

As Kuff pointed out in the comments, "Garcia won San Patricio County by a huge margin in 2006 (7142 to 5350, or 57.2% to 42.8%), so any poll that shows him trailing there by double digits is a little suspicious to begin with. San Patricio did go mostly R in 2006 overall, though Bill Moody won it with 52%, but the downballot statewide Rs didn't do so well, with Todd Staples and Elizabeth Ames Jones getting only a plurality. My guess is that this is a small subsample with a large MOE, and as such probably not very useful."

To add to this, because of the huge swings in San Patricio and Aransas, it looks like Texas Poll Watch is under sampling Hispanics and over sampling Nueces.  The combination of the two would skew any poll dramatically and is an amateur mistake when weighting and creating the methodology.

Ed Martin echoes both points in the comments.

In 2006, an excellent Democratic/Hispanic field effort at the end of the campaign helped drive those San Patricio numbers to 57% for Garcia. In this district, San Patricio's 12.5K votes in a House race in an "off" year make it a major county. Unless a pollster knows how to test an accurate and representative sample of Hispanic voters, their preference will typically be under-represented in most polls in South Texas counties like San Patricio. Bottom line: If you flip the San Patricio numbers, this is still a close race that could go down to the wire, but not one that has Hunter ahead, especially by 8 points.

The most important part is this poll both fails to take in unlikely voters that Garcia's campaign will obviously target and new voters identified by the 2008 presidential primary.

In Aransas that is 873 voters.  In Calhoun that is 2,653 voters. In Nueces that is 4,271 voters.  In San Patricio that is 6,639 voters.

In 2006, Garcia won with 17,607 votes. The Democratic primary alone has identified 14,436 votes compared to 2006 when Garcia had 5,401 2006 primary voters identified.  This poll would not touch a large segment of this 14,436 because it is an automated phone system with no live dials.  That same system would disproportionately hit men, voters over 35 years old, and excludes minorities.

Just looking at the county by county numbers this poll is strikingly off, and a correction in San Patricio would completely eliminate the 8 point lead Hunter has in this apparently faulty poll.

ADVERTISEMENT
Tags: , , , , (All Tags)
Print Friendly View Send As Email
San Patricio County (0.00 / 0)
An acquaintance close to Republican politics just happened to mention to me yesterday something about Gene Seaman pissing off the Rs on San Patricio County.  I know nothing more about this, but maybe they sat on their hands for Seaman, but are now ready for vote for Hunter.

Well (5.00 / 1)
Without getting into the specifics of this poll, I would disagree with your comments on the automated phone system. First, in my automated polling, I don't think I have ever had raw results that oversampled men. Women are more likely to answer the phone than men. That is the biggest skew in raw results. There is a smaller skew towards older and white voters as well. These factors can be overcome as long as the poll has an adequate number of raw data from the lower response groups, and demographics are correctly weighted on a multi-dimensional scale. I don't know if this was done here.

From a non-polling perspective, while the 'new primary voters' will be much easier to target, most are not 'new voters,' but have history of voting in general elections. I am not involved in this poll in any way, but I would assume that the pollsters would have started with a general election sample.

Texas Economics


Matt (0.00 / 0)
Thanks for doing a substantive review of the numbers. This is the kind of analysis BOR readers want to see, and it's good to see it on our front page.

Now, a very great man once said that some people rob you with a fountain pen.

I think we still should have printed it (0.00 / 0)
The MoE for the entire poll was 4.7%, I think.

For a smaller subsample like San Patricio County, the MoE would be much, much higher.

I am looking forward to seeing more of TPW's polls this year.


I'm not. (0.00 / 0)
Nothing against anyone personally, but I have a deep distrust of this polling methodology.  Professional pollsters I trust and work with have told me it is subject to major screwups, especially in races, like this one, in which the candidates have relatively low name ID.

The recent Rasmussen polls -- with their huge swings in support for Noriega -- used the same technique.  Survey USA poll also uses auto-calls and has suffered the same problems.

Frankly, I can't get past the idea that a respondent could be a 5-year-old playing with the phone, and the pollster would never know it.

The chief selling point seems to be low cost.  I worry that you get what you pay for.


[ Parent ]
Fair enough (0.00 / 0)
There are some legitimate concerns about this polling methodology, but IVR Polls has used it very successfully in races like the '06 CD10 race.

But your concerns are certainly in line with many people across the nation. Still, SUSA was one of the most accurate polling firms during the presidential primaries.


[ Parent ]
It MIGHT be okay for presidential races... (0.00 / 0)
In those races, the candidates have high name identification (90+%) and voters know a lot about them.  This year, many were staying up with the race moment-to-moment.

That's not true for a State Representative race.  In those races, the total number of those who can rate a candidate is usually below 60%.  Moreover, there is usually less voter interest in these races, especially five months from election day.

Keep in mind one other thought: Just because s poll lands on the right numbers does not mean that it was necessarily properly done.  It coulda been luck.

Most pollsters have advanced degrees in survey science.  The successful ones are seasoned professionals who understand well where the pitfalls are.

With auto-calls, I'm not sure anyone knows where the pitfalls are yet.  That's a big problem.  Until we understand them better, no one should view these polls as reliable.


[ Parent ]
"advanced degrees" (0.00 / 0)
The Master's program I'm in requires a full year of statistics, which is basically just a set up for "how to conduct proper surveys/research." It's set in a larger context, but it's easily transferable for a poll. And I haven't even taken the classes that are devoted specifically to polls.

I thought I knew a lot about polling before -- I didn't know crap. Now I know what I don't know, but I still don't know a lot.

(how's that for a sentence?

Now, a very great man once said that some people rob you with a fountain pen.


[ Parent ]
I remember the feeling (0.00 / 0)
After sitting through Advanced Quantitative Analysis for a year at the LBJ School, I got a glimpse into the world of polling and statistics but realized that I still didn't know shit!

Good luck, Phillip.


[ Parent ]
YUP! (0.00 / 0)
Same EXACT feeling I had after my class on quant research in grad school.  Sadly, I've found a LOT of pollsters don't even know the basics of a poll.  

Without getting into the tabs and seeing what's going on in this thing, I'd just like to offer one fact...  the margin of error means that the poll will be plus of minus whatever that number is 95% of the time.  The other 5% it will be outside of it, EVEN WITH A PERFECTLY DONE POLL (if you mess it up before it will inevitably be wrong... crap in = crap out).  That means for 20 polls we see, 1 will be TOTALLY wrong.  Usually in the 5% that are outliers, it's still relatively close.  But, on occasion, you'll see something ridiculous.  This happens on both live call AND robopolls.

For me, the jury is still out on robos and exactly what they can and cannot do in comparison to a live poll.  I tend to think, just from seeing some results, they're just as good as a live poll call for the horserace/dipstick.  We all know that's probably the LEAST useful number on any poll, but it tends to be what the media fixates on.


[ Parent ]
How Rumsfeldian of you (0.00 / 0)
Sounds like you have increased your known unknowns about polling.

http://www.youtube.com/watch?v...


[ Parent ]
David, my training is in statistics ... (0.00 / 0)
I have a master of science degree from UT and did all my work in the math department. I am mostly known as a fund-raiser, but I started my first business doing computer modeling of election-day turnout and I have considerable experience in targeting.

What you refer to as "Margin of Error" (MoE) is actually more correctly called sampling error. Sampling error is based on the size of the sample and the assumption that you are drawing a truly random sample from the population of interest. Sampling error is only part of the inaccuracy of a poll like this.

Here are other possible sources of error and bias:

Pulling a sample of "likely voters" is different than pulling a sample of the people who will actually vote.

Bias can be introduced by the wording of questions.

Subgroups may be undersampled because of their lack of cooperation or interest.

Subgroups may be undersampled because they are difficult to find; e.g., people with cellphones instead of landlines; people whose phones have been turned off; people who move from place to place a lot.

Turning out the Democratic base in San Patricio county involves "knock-and-drag" programs where people literally pull people out of their chairs and drive them to the polls. What's the equivalent of getting people to participate in a phone poll when all they want to do is sit in front of the TV?

The best polling organizations use automated polling in conjunction with live operators in order to measure and correct for such biases. Using the cheap component of the technology without some quality control, in my opinion, is asking for trouble.

There's a useful concept known as the marketing triangle. At each angle of the triangle is a word; the words are "fast," "good," and "cheap." The insight behind the marketing triangle is that you can have at most two of the qualities, but that you can never have three. Take food:

Fast & good: dine at a fine restaurant

Fast & cheap: McDonalds

Good & cheap: Learn how to cook, buy the ingredients yourself, prepare them at home.

This information that IVR is peddling is of the fast and cheap variety, and the claim that the  "margin of error is 4.7%" is just the tip of the iceberg. I don't think it's worth the paper it's printed on, and if you've noticed that it's not even printed on paper at all, you've gotten my point.

"The eyes of the people are fast opening! Fight on!"--Andrew Jackson


[ Parent ]
Sniff (0.00 / 0)
Now you've gone and hurt my feelings. This isn't my poll BTW.

Texas Economics

[ Parent ]
I apologize for confusing Texas Poll Watch with IVR (n/t) (0.00 / 0)


"The eyes of the people are fast opening! Fight on!"--Andrew Jackson

[ Parent ]
Polls... (0.00 / 0)
Polls are polls; wrong or not.  Regardless of the accuracy, they can have a powerful effect on media perception, campaign finances, and targeting.  

Thanks for doing the analysis, Matt.  The numbers were certainly disturbing, and I'm glad you cleared it up with some hard data.  Let's hope the next poll (if any) for that race shows a swing the other way.

If anything, this poll gives a reality check to the expected "mass movement" of voters.  Us Dems will have a lot of new people, but we can't take it for granted.  We have to work for the votes...harder than ever!


San Pat County (0.00 / 0)
I can tell you that any poll that has Hunter ahead by that kind of margin in San Pat is at best suspect. I am very engaged in my county's politics and question the methodology of any poll that shows these results. One post speaks to the "bias" the questions can introduce into poll results. I will tell you I was polled a few weeks ago and if these results are based on that poll I can assure you there was more than a little bias in Mr Hunters favor built into the questions. While I don't have the statistical pedigree some of these posters have, I can tell you after 17 hours of stat. for my marketing degree even I can make a poll say anyhtng I want.

Connect With BOR
Your source for Texas politics.

On Facebook: BOR
On Twitter: @BOR
On the Go: Mobile App

Upcoming BOR Events

"Do I Look Illegal?"
Arizona GOP Debate Watch

Wednesday, February 22
6:00-9:00 p.m.
Angie's Restaurant
1307 E. 7th Street
RSVP on Facebook

Save The Date:
Super Tuesday Super Watch Party!
Tuesday, March 6
6:00-10:00 p.m.
Scholz Garten
1607 San Jacinto



Menu

Make a New Account

Username:

Password:



Forget your username or password?


Shared On Facebook

Advertisement

Best of Texas Left
- (Complete Directory)
- B & B
- Bay Area Houston
- Blue Bloggin
- Bluedaze
- Brains and Eggs
- Capitol Annex
- Collin County Democrats
- Collin County Observer
- Community Forum
- Dog Canyon
- Dos Centavos
- Easter Lemming Liberal
- Eye on Williamson County
- Feet to the Fire
- Grading Texas
- Greg's Opinion
- Grits for Breakfast
- Half Empty
- Houtopia
- In the Pink Texas
- Kiss My Big Blue Butt
- Letters from Texas
- McBlogger
- Mean Rachel
- Musings
- North Texas Liberal
- Off the Kuff
- Panhandle Truth Squad
- Para Justicia y Libertad!
- Pink Dome
- San Antonio Mayor
- South Texas Chisme
- StoudDemBlog
- Texas Clover Leaf
- Texas Kaos
- The Caucus Blog
- There..Already
- Three Wise Men
Best of Texas Right
- Blogs of War
- BlogHouston
- Boots and Sabers
- Lone Star Times
- Publius TX
- Rick Perry vs the World
- Safety for Dummies
- Slightly Rough
- Urban Grounds
Other Texas Reads
- Burka Blog
- D Magazine
- DOT Show
- Statesman Elections
- Strong Political Analysis
- Texas Monthly
- Texas Observer
- The Texas Blue
- Quorum Report Daily Buzz
Around Austin
- Austin Bloggers
- Austin Chronicle
- Austin Contrarian
- Austin Metblogs
- Austin on Two Wheels
- Austin Real Estate Blog
- Austin Statesman
- Austin Texas Bike Shit Stuff
- Austin Towers
- Austinist
- Capital MetroBlog
- Daily Texan
- Do512
- Downtown Austin Blog
- East Austinite
- Elise Hu
-
Flash Mob Austin
- Keep Austin Blue
- M1EK
- Travis County Democrats
- University Democrats
TX Progressive Orgs
- ACLU Legislative Blog
- Atticus Circle
- Criminal Justice Coalition
- Equality Texas
- NOW Texas
- PFAW Texas
- Public Citizen
- SEIU Texas
- Tejano Insider
- Texas AFT
- Texas HDCC
- Texas Watch
- TFN
- TSTA
- TSEU
- Texas Young Democrats
- United Ways of Texas
TX Elections/Returns
- TX Returns 1992-present
- TX Media/Candidate List

- Bexar County
- Collin County
- Dallas county
- Denton County
- El Paso County
- Fort Bend County
- Harris County
- Jefferson County
- Tarrant County
- Travis County

- CNN 1998 Returns
- CNN 2000 Returns
- CNN 2002 Returns
- CNN 2004 Returns
- CNN 2006 Returns
- CNN 2008 Returns
Traffic Ratings
- Alexa Rating
- Quantcast Ratings
-
Syndication

Powered by: SoapBlox