The Science and Art of Prediction Markets

What constitutes a good question for a prediction market? Obviously, for the question to be valuable the answer should provide information that was not available when the question was originally asked. Otherwise, why ask the question. Value, however, is only one aspect of a good question. For prediction markets to function in a useful manner the questions that are asked must also be constructed properly. There is both a science and an art to this process.

The Science

There are three criteria to keep in mind when constructing a question for a prediction market:

• The correct answer must be concrete
• Answers must be determined on specific dates
• Information about possible answers can be acquired before the settled date

Concreteness is important because it settles the question being asked - the result is not open to interpretation. An example of a question with a vague answer would be "What policy should the U.S. government enact to encourage economic growth? A) Subsidizing green energy, B) free trade, C) fiscal austerity, D) health care reform." One problem here is that the time frame to accurately answer this question could be extensive. Also, the complexities of economic growth make it difficult to tease out the individual variables that would be necessary to concretely answer the question. If two or more answers are correct (whatever that may mean) then the market may end up reflecting the value judgments of the participants, not objective knowledge. This type of question is more suited for a poll rather than a prediction.

Not only should answers be concrete, there should be some point in time when each answer can be determined to either have occurred or not have occurred. A question that never gets resolved can hamper the prediction process by reducing the incentive to invest in that market. (Can a non-expiring question be valuable? Could the ongoing process of information discovery be useful? Questions to ponder.)

This doesn't mean, however, that every answer must be determined on the same date. Wrong answers can be closed as the process unfolds. Once the correct answer is determined, however, the market should be closed. For example, take the question "Which candidate will win the 2012 Republican Party nomination for U.S. President?" If this question is asked in January of 2012 there could be several possible answers (one for each candidate). As the year progresses to the Republican Party convention, several candidates will drop out of the election. The prediction market would then close out those answers (candidates) but stay open for the remaining answers. Weeding out wrong answers over time is part of the discovery process.

The final criterion - the ability to acquire information before the settled date - is what separates prediction markets from strict gambling. If all participants are in the dark about a question until that question is settled, then there is little value in asking the question. Prediction markets are powerful because they allow participants to impart some knowledge into the process over a period of time. The resulting market prices can then provide information that can be acted upon throughout the process. If participants cannot acquire useful information to incorporate into the market, then market activity is nothing more than playing roulette where all answers are equally possible until the correct answer is determined.

A good example to illustrate the above criteria is a customer satisfaction survey. Railinc uses a bi-annual (twice a year) survey to gauge customer sentiment on a list of products. For each product, customers are asked a series of questions the answers to which range from 1 (disagree) to 5 (agree). The answers are then averaged with a final score for each product ranging from 1-5 (the goal is to get as close to 5 as possible).

The following market could be set up for Railinc employees:

What will the Fourth Quarter 2010 customer satisfaction score be for product X?

• Less than or equal to 4.0
• Between 4.1 and 4.4 (inclusive)
• Greater than or equal to 4.5

The value of this market is that Railinc management and product owners may get some insight into what employees are hearing from customers. Customer Service personnel could have one view based upon their interactions with customers, while developers may have a different view. Over time, management and product owners could take actions based upon market movements.

As far as concreteness is concerned, the final answer for this question will be determined when the survey is completed (e.g., January 2011), and it will be a specific number that falls into one of the ranges given by the answers.

This market also satisfies the last criteria regarding the ability to acquire information before the market is settled. This is important because this is where the value of the market is realized. As Railinc employees (i.e., market participants) gain knowledge over time they can incorporate that knowledge into the market via the buying and selling of shares in the provided answers.

The Art

In the example given above regarding the customer satisfaction survey, the answers provided were not arbitrary - they were selected to maximize the value of the market. This is where the art of prediction markets is applied.

If the possible answers for a customer survey are 1-5 why not provide five separate answers (1-1.9, 2-2.9, 3-3.9, 4-4.9, 5)? Why not have two possible answers (below 2.5 and above 2.5)? The selection of possible answers is partially determined by what is already known about the result. In the case of the survey, past results may have shown that this particular product has average a 4.1. It is highly unlikely that the survey results will drop to the 1-1.9 range. Providing such an answer would not be valuable because market participants would almost immediately short that position. This is still information, but it is information that is already known. What is desired is insight to what is not known. The answers provided in the above example will give some insight into whether the product is continuing to improve or whether it is digressing.

So, the selection of possible answers to market questions must take into account what is already known as well as what is unknown. What do you know about what you don't know?

Conclusion

Good questions make good prediction markets. Constructed properly, these questions can be a valuable tool in the decision making process of an organization.