Why Dial Idol Doesn’t Work For DWTS
It seems as though every season, some people either freak out or get excited over what they see on Dial Idol for DWTS – maybe their favorite is in the lead, maybe the couple they hate is in the bottom, maybe a high-scoring couple looks like they aren’t getting votes. While I agree that it would be great to “pull back the curtain” and get a peek at what the volume of votes for each couple might actually be, I can tell you this right now: it’s a carefully-guarded secret, and TPTB have ways to keep it that way – despite the efforts of supposedly “genius” software like Dial Idol. So let’s debunk this one, shall we?
First of all, a bit of background on Dial Idol. It’s a software program created in 2005 by Jim Hellriegel, a fan of American Idol, that uses your computer modem to automatically “speed-dial” in votes for your favorite contestant. It then measures how often a busy signal bounces back on a particular contestant’s phone number, and, assuming that a greater number of busy signals bouncing back means that more people are trying to vote for a particular contestant, then the more votes a certain contestant must be getting. The idea was pretty clever (at the time), and despite efforts by the FOX legal team to get it shut down due to copyright infringement, it’s perfectly legal, and has correctly predicted the winners of 5 seasons of Idol.
So, you ask, why can’t the same program work for DWTS?
REASON #1: We have a limited number of votes.
Keep in mind, this software program was created for Idol, and on that show, there is no limit on the number of times you can vote; so, in theory, you could get hundreds of votes in for your favorite contestant before the phone lines closed. Hence, the more votes you can dial in, the more often you either do or don’t get a busy signal, and the more accurate the Dial Idol results are going to be. We don’t have that luxury on DWTS – we get a small, limited amount of votes each week, meaning we have fewer opportunities to measure a potential busy signal. For those with a bit of knowledge in statistics, the smaller your sample size is = the less accurate your results are going to be. And the sample size for DWTS is going to be much, much smaller than it is for Idol, due to the voting limit. Really and truly, it almost seems like a waste of time to even use Dial Idol to get your phone votes in to begin with – you’d reach your limit pretty quickly, unless the phone lines are particularly busy (such as in the finale).
REASON #2: We have a larger “window of opportunity” to vote.
The voting lines on Idol don’t open until the end of the show and are open for 2 hours after that; the voting lines on DWTS are open from the beginning of the show until 30 minutes after the show is over. Not only do DWTS voters get an extra half-hour to vote by phone, but we’re not limited to waiting until we’ve seen all the performances before we cast our votes. So what does that mean? Some smart “power voters” (like Heidi) will start voting for their favorites as soon as the show starts, when the phone lines aren’t so busy. Some viewers (like my mom) will wait until one performance catches her eye midway through the show and will give all her votes to them. And some voters will wait until the end of the show to cast their votes. What does this mean? With a larger time window to vote, there’s also the potential for the voting to get “spread out” a bit more – meaning there’s less liklihood of getting a busy signal, even though there may be just as many votes being cast for a particular contestant. With Idol, everyone is scrambling to get their votes in at the same time – meaning busy signals galore, and a more accurate estimate of just how many people are voting for a particular contestant. With DWTS, there may be a bit of a scramble to get votes in after the show is over, but Dial Idol likely won’t pick up on votes that were made earlier in the show, when there wasn’t the same urgency to hurry up and get your votes in…and hence, fewer busy signals to measure.
REASON #3: Dial Idol doesn’t take text message or online votes into account.
Perhaps the biggest caveat of Dial Idol is that it only measures phone votes – and with more and more shows also opening up voting via text message and Internet (Idol included, as of this season), it’s going to become progressively less accurate. There is absolutely no easy way to quantitatively measure text message or online votes – no busy signal to bounce back, no publicly-available statistics, etc. If you think about it, phone voting is possibly the LEAST convenient method of voting for a reality show anymore – if there’s a busy signal, you just have to wait and keep hitting redial; even though the call is toll-free, you still use up minutes if you call from a cell phone; and fewer and fewer homes have landlines anymore – leaving fewer avenues to dial in your votes. I get the feeling that more and more viewers are resorting to voting online or via text – there’s no waiting, there’s a longer window of opportunity to vote (at least online), and you’re able “cheat” the system a bit by creating multiple accounts to vote online (I can’t vouch for Idol, but I think we’ve all seen what a boon online voting can be for some DWTS viewers!). Why even waste time and money on slow phone voting when you can get the instant gratification of voting via text or Internet? And thus our Dial Idol “sample size” gets even smaller.
REASON #4: Dial Idol gives itself a “safety net” of accuracy.
Ever notice how sometimes the results at Dial Idol are different colors? You may have written it off as the web designer just trying to make the site more “visually pleasing”, but the colors do have some significance – it’s basically a way to show how confident they are in the results for each contestant. And that’s where the sneaky yellow-colored results come in – officially, it’s a way to say that the results are “too close to call” between all of the contestants who’s results are yellow; unofficially, I think it’s a way for Dial Idol to save face in the event that they’re completely off-the-mark, and pad their accuracy percentage. 🙂 See the column that says “Dial Idol Range”? Those numbers from 1-whatever for each couple are where they’re predicted to be ranked in terms of votes. And when those numbers are yellow, it means it’s too close to call, and the contestant could fall anywhere between those 2 numbers. Let’s use this week’s DWTS results on Dial Idol as an example. Chelsea & Mark are in the lead with a Dial Idol score of 31.92. But all of the couples’ scores are yellow, meaning it’s too close to call, and all of the couples could potentially be as low as 11th place. What does that mean? That if there was an elimination this week, and Chelsea & Mark get sent home, it would still count as a “correct” prediction for Dial Idol, since they said it was too close to call and that Chelsea & Mark could be anywhere between 1-11th place. Inflated accuracy percentage much? Is your faith in Dial Idol wavering yet? 🙂 That sneaky yellow “too close to call” is what lulls a lot of people into a false sense of security – their favorite couple may be in the lead with their Dial Idol score, but it doesn’t necessarily mean they’re safe from elimination. And even the green & red colors, which signify “likely safe” and “likely in danger”, respectively, are no guarantee – Dial Idol has been wrong about those before, too. Why, you ask? Which brings me to my final point…
REASON #5: NOT EVERYONE USES DIAL IDOL TO GET THEIR VOTES IN!!!!
Remember what I said about sample size? The more people that use Dial Idol, the more accurate it becomes (in theory). The problem is – we don’t know how many people are actually using Dial Idol to vote, and in order to get a decent representation of what kind of votes each of the contestants is getting, there has to be people using Dial Idol to vote for each of the contestants. The same applies to any show that Dial Idol supports – how can it measure the busy signal for each contestant if there isn’t at least one person using it to vote for each contestant? Let’s say no one uses Dial Idol to vote for Chris & Cheryl. Then there’s going to be no busy signals bouncing back, and to those who don’t know the particulars about how Dial Idol works, it’s going to appear as though they got the low score and are predicted to head home. In reality, there simply isn’t any data to determine whether they’re going to stay, go, or how they stack up against other contestants. Or, on the other hand, let’s say most of the people using Dial Idol are voting for Hines & Kym. It’s going to result in an inflated score for them that may make it look like they’re safe, but without equally as diverse data for each of the other contestants, there’s no way to know for sure.
The bottom line? Short of forcing everyone voting on the show to use Dial Idol to phone in their votes, giving us limitless votes, and making us wait until AFTER the show is over to vote, and finding a way to measure the impact of online voting & text message voting, Dial Idol is NEVER going to have the same success on DWTS that it has had on Idol up to this point. I say “up to this point” because now Idol also has online voting, and I think that will impact the results even more. So who knows, Dial Idol may become obsolete soon…yet another reason not to put too much stock in it 😉
I have never used Dial Idol and I’m glad I never have now. Thanks C!!!!!
I don’t vote, but I was curious as to the particulars of Dial Idol and how it all worked because some people SWORE by it but it didn’t seem accurate most of the time, so thanks! I’ve never watched Idol so I didn’t know how their voting system differed – had no idea they didn’t have online voting before! How ghetto. Anyway – all the points make sense that it’s a terrible predictor for DWTS. Thanks!
Yeah, I just find it funny how some DWTS fans put so much stock in what Dial Idol says when it’s not really even tailored to fit the parameters of the DWTS voting process. Someone on Twitter the other night was freaking out, saying “OMG! Sugar Ray & Anna are in the bottom on Dial Idol! They’re going to go home!” And I had to just shake my head & laugh – if someone is in the bottom on Dial Idol, all it means is that not very many people are using Dial Idol to vote for that particular contestant. It is absolutely NOT an indicator of how the regular voting is going. And likewise, just because someone is getting a lot of votes on Dial Idol, does not mean they’re safe – it just means that people are using Dial Idol to vote for them 🙂 Glad I could clarify this one for you guys…I personally think Dial Idol is just a waste of time if you’re trying to vote for DWTS. You’re better off just dialing in the votes manually.
And Fig, you’re right – it’s wrong more than it’s right, and it’s “too close to call” the majority of the time. Now if anyone tries to argue about how the Dial Idol results are a good indicator of voting behavior, you can just direct them here to set them straight 😀
I had a youngster tell me on twitter that Jennifer was in danger of going home last season when she was ranked fairly low on Dial Idol. She exclaimed to me that “it’s always right” and it actually took a bit of convincing by me to get her to understand that it’s FAR from always right. The next night I think it was Audrina who went home and she needed no more convincing. 🙂 The thing is, Dial Idol isn’t doing anything “wrong” – they qualify the hell out of their data, they make it VERY clear that the numbers are often too close to call, they color code – but people just don’t READ. They look at who’s at the top and who’s at the bottom and since it’s on the internet it must be right. Oh heeellll no. 🙂
The other thing I am curious about, but have not really looked at to see if what I suspect is true, is whether age plays a part in the skewing of the results. An argument could be made that says “well, it’s still a representative sample of the voting population”. I would absolutely disagree with that assessment because of the HIGH tendency of DWTS’s serious voting fans to vote only on line. But also – what about demographics? The main demographic reportedly watching and voting on this show are women over the age of 45 or so. I think the median age was somewhere in the 50’s. Well, if you consider that the *likelihood* of someone in the common demographic using something like Dial Idol (or online voting for that matter) goes way down as their age goes up, then that is a pretty big skew to Dial Idol results, especially if they are the largest population watching the show. If you only have the younger viewers doing stuff like dial idol, is it also safe to assume that those folks are more inclined to vote for Romeo and Chelsea than they are, say, Kirstie and Ralph??
That’s just something that a nerd like me wonders about. 🙂
Courtney, thank you for this analysis of Dial Idol. I also laugh at people believing it is the tell-all regarding which couple will be eliminated.
I think you bring up another good point, Heidi – just WHO exactly uses Dial Idol? I’m 26, and I know enough to know it’s highly subjective and not worth my time. My mother is 52 and had no idea what it was. Maybe it’s something that people over the age of 20 can recognize as bogus 🙂 I do think it’s maybe something that is used more often by the younger set (maybe teens and those in their early 20s), which could skew the voting to younger contestants.
One thing that was always interesting me (and I do credit Dial Idol for being clever with) is the GeoPredictions – it shows who is using the program to vote for certain contestants by state. Again, not a finite method, since we don’t know exactly what the demographics are for the voters, but I find it funny that for the finale last season, the GeoPrediction had Bristol winning in California & New Jersey (both traditionally Democratic states), Jennifer & Derek winning in Texas & Indiana (both traditionally Republican states), and no data available for the state of Alaska. 🙂
Forgot to mention – if you look at the GeoPredictions, it will also tell you how many votes were dialed in from those particular states, as well as how many busy signals they got. Most states have less than 100 of each. PEANUTS compared to the millions of total votes that the show gets. 🙂
Thank you so much for this Courtney! To tell the truth, I never understood what Dial Idol was and tried “researching it” and figured it was inaccurate but you really went deep into explaining how it works and why it doesn’t work on DWTS. Thank you so much for that.
I also admit that last season, I did look at Dial Idol quite a bit and remember Heidi and Courtney telling everyone how it doesn’t work for DWTS, so I figured not to trust it 😉 . You folks here just are either statistical or math geniuses and I have learned not to ask questions and agree when it comes to math 🙂
So you’ve convinced me and thanks so much again Courtney!
While I understand that dialidol is not something to take seriously, I occasionally look at raw numbers. I think it gives me an idea of who’s getting phone – votes. I also think that it gives fans a reason to power vote online the next day. How much that impact the end results?… I don’t know.