● Forecasting Elections: Comparing Prediction Markets, Polls, and their Biases
Public Opinion Quarterly. 2009. Vol. 73, No. 5, pp 895-916.
Using the 2008 elections, I explore the accuracy and informational content of forecasts derived from two different types of data: polls and prediction markets. Both types of data suffer from inherent biases, and this is the first analysis to compare the accuracy of these forecasts adjusting for these biases. Moreover, the analysis expands on previous research by evaluating state-level forecasts in Presidential and Senatorial races, rather than just the national popular vote. Utilizing several different estimation strategies, I demonstrate that early in the cycle and in not-certain races debiased prediction market-based forecasts provide more accurate probabilities of victory and more information than debiased poll-based forecasts. These results are significant because accurately documenting the underlying probabilities, at any given day before the election, is critical for enabling academics to determine the impact of shocks to the campaign, for the public to invest wisely and for practitioners to spend efficiently.
Debiased Aggregated Polls and Prediction Market Prices
Chance. 2010. Vol. 23, No. 3, pp 6-7.
● Simplifying Market Access: a New Confidence-Based Interface with Florian Teschner
The Journal of Prediction Markets. 2012. Vol. 6, No. 3, pp 27-41.
Markets are a strong instrument for aggregating dispersed information, yet there are flaws. Markets are too complex for some users, they fail to capture massive amounts of their users’ relevant information, and they suffer from some individual-level biases. Based on recent research in polling environments, we design a new market interface that captures both a participant’s point estimate and confidence. The new interface lowers the barrier to entry, asks market’s implicit question more directly, and helps reduce known biases. We further utilize a novel market rule that supplements the interface with its simplicity. Thus, we ﬁnd that market participants using our new interface: provide meaningful information and are more likely to submit proﬁtable orders than using a standard market interface.
● A Combinatorial Prediction Market for the U.S. Elections with Miroslav Dudik, Sebastien Lahaie, and David Pennock
Forthcoming: Electronic Commerce. 2013. Email for Copy of Paper.
● Forecasting Elections: Voter Intentions versus Expectations with Justin Wolfers
This Draft: November 1, 2012.
In this paper, we explore the value of an underutilized political polling question: who do you think will win the upcoming election? We demonstrate that this expectation question points to the winning candidate more often than the standard political polling question of voter intention: if the election were held today, who would you vote for? Further, the results of the expectation question translate into more accurate forecasts of the vote share than the ubiquitous intent question. Our structural interpretation of the expectation question shows that every response is equivalent to a multi-person poll of intention; the power of the response is that it provides information about the respondent’s intent, as well as the intent of her friends and family. This paper has far reaching implications for all disciplines that use polling.
Interview with Mark Blumenthal on this paper after I presented it at the AAPOR conference: Video Link.
Pollster.com, May 16, 2010.
● Combining Forecasts: Accuracy and Timeliness
This Draft: July 21, 2012.
I improve forecasts for Electoral College and senatorial elections by generating, and then combining, forecasts based on fundamental data, voter intention polling, and prediction markets. I create the most efficient forecasts available for each of these raw data types by focusing on improvements over previous models in aggregating and then debiasing them with parameters determined over: election type, days before the election, and the certainty of the data. The model that combines them utilizes the shifting levels of information from the three data types; 130 days out, it is most efficient to average the forecasts from all three data types, but the fundamental model’s unique information decreases linearly until Election Day, when it is most efficient to average just the polling and prediction market-based forecasts. The combined forecast is the most accurate forecast continuously available from 130 days prior to Election Day for all Electoral College or senatorial elections.
● Expectations: Point-Estimates, Probability Distributions, and Forecasts
This Draft: September 20, 2012.
In this paper I test a new graphical, interactive interface that captures both “best estimate” point-estimates and probability distributions from non-experts. When supplementing an expectation, a standard data point is directly stated confidence of the respondent or a confidence range. In contrast to those data points, my method induces the respondents to reveal a level of precision, and there is a sizable and statically significant positive relationship between the respondents’ revealed precision and the accuracy of their individual-level expectations. Beyond creating a more meaningful individual-level estimates, researchers can use this positive correlation between precision and accuracy to create precision-weighted aggregated forecasts that are more accurate than the standard “consensus forecasts”. Varying financial incentives does not affect these findings.
● Fundamental Models for Forecasting Elections with Patrick Hummel
Email for Copy of Paper.
This paper develops new fundamental models for forecasting presidential, senatorial, and gubernatorial elections at the state level using fundamental data from six categories: past election results, incumbency, presidential approval ratings, economic indicators, ideological indicators, and biographical information about the candidates. Despite the fact that our models differ from other state-level forecasting models in that they can be used to make forecasts of elections earlier than existing models and they do not use data from pre-election polls on voting intentions, our models give rise to lower out-of-sample forecasting errors for both the binary outcomes of elections and the fraction of the major party vote received by each candidate. We further illustrate new ways of incorporating various economic and political indicators into forecasting models that enable us to obtain a better understanding of what types of fundamental data most meaningfully predict the outcomes of elections in each state. Among our results, we find that economic variables are most meaningful as trends rather than levels and that second quarter data is as predictive of election outcomes as third quarter data.
● Are Polls and Probabilities Self-Fulfilling Prophecies? with Neil Malhotra
Email for Copy of Paper.
Psychologists have long observed that people often conform to majority opinion. This bandwagon effect occurs in the political domain as people learn about prevailing public opinion via ubiquitous polls. A recent phenomenonpublished probabilities derived from prediction market contract prices and aggregated pollsmay play a similar role. Consequently, polls and probabilities can become self-fulfilling prophecies whereby majorities, whether in support of candidates or policies, grow in a cascading manner. Despite the increased attention to whether measurement of public opinion can itself affect public opinion, the existing empirical literature is quite limited on the bandwagon effects of polls and non-existent on the effects of probabilities. To address this gap, we conducted an experiment on a diverse national sample in which we randomly assigned people to receive information about different levels of support (or probability of passage) for three public policies. We find that public opinion as expressed through polls significantly impacts individual-level attitudes whereas probabilities exhibit no effect. We also posit a mechanism underlying the bandwagon effect for polls: low public support decreases support for policies but high public support does not increase support. In sum, our study shows that measuring public opinion has the potential to change public opinion.
● Non-experts and their Understanding of Moments with Daniel G. Goldstein
Email for Copy of Paper.
● The Extent of Price Misalignment in Prediction Markets with David Pennock
Email for Copy of Paper.
We examine misaligned prices for logically related contracts in prediction markets. First, we uncover persistent arbitrages for risk neutral institutional investors between identical contracts on different exchanges. Observing the trading of several thousand dollars of contracts in a randomized trial, we document how price support well beyond what is in the published order book multiplies the size of these arbitrages. Second, we demonstrate misalignment among identical and logically related contracts listed on the same exchange that cluster around moments of high information flow, when related contracts systemically shut down or fail to respond efficiently. Third, we document bounded rationality in prediction markets, examples include: consistent asymmetry between buying and selling, leaving the average return for selling higher than for buying, and persistent price lags between exchanges. Despite these signs of departure from theoretical optimality, the markets studied, on balance, function well considering the sometimes complex and subtle relationships among contracts. Yet, we detail how to improve prediction markets by moving the burden of finding and fixing logical contradictions into the exchange and providing flexible trading interfaces; both of which free traders to focus on providing meaningful information in the form they find most natural.
● Markets, Government, and Access to Local Goods and Service Establishments with Joel Waldfogel
Work in Progress.
● Voter Turnout: Are Your Friends and Family Voting?
Work in Progress.
There is a deep literature on the phenomena of over-reporting of voting by individuals in polls. This is true for both pre-election polls, where respondents state that they intend to vote and post-election polls, where respondents state they did vote. Thus, pollsters have created complicated and opaque methods of forecasting voter turnout, both for individual respondents and aggregated within a voting district. These methods rely on combining these individual-level poll responses about voting behavior with observable characteristics and stated levels political engagement (including past voting behavior). The opaqueness of these methods leads to mistrust from observers and a sizable percentage of poll error in polls of voter intention for upcoming elections, which is the main input in most forecasts of election outcomes. In this article I propose new questions that ask the respondents what they expect to happen to voter turnout in their community, as a compliment to whether they intend to vote. I demonstrate theoretically, and then empirically, that this can create valuable data in determining both individual-level voter turnout and overall voter turnout.