- Down with Doom
- The 100 Greatest Movie Insults (NSFW)
- Another plot by Wal-Mart to tick off somebody
- Space porn
- Solace for guitar
Wednesday, July 07, 2010
Whatever
Tuesday, July 06, 2010
The Science and Art of Prediction Markets
What constitutes a good question for a prediction market? Obviously, for the question to be valuable the answer should provide information that was not available when the question was originally asked. Otherwise, why ask the question. Value, however, is only one aspect of a good question. For prediction markets to function in a useful manner the questions that are asked must also be constructed properly. There is both a science and an art to this process.
The Science
There are three criteria to keep in mind when constructing a question for a prediction market:
Not only should answers be concrete, there should be some point in time when each answer can be determined to either have occurred or not have occurred. A question that never gets resolved can hamper the prediction process by reducing the incentive to invest in that market. (Can a non-expiring question be valuable? Could the ongoing process of information discovery be useful? Questions to ponder.)
This doesn't mean, however, that every answer must be determined on the same date. Wrong answers can be closed as the process unfolds. Once the correct answer is determined, however, the market should be closed. For example, take the question "Which candidate will win the 2012 Republican Party nomination for U.S. President?" If this question is asked in January of 2012 there could be several possible answers (one for each candidate). As the year progresses to the Republican Party convention, several candidates will drop out of the election. The prediction market would then close out those answers (candidates) but stay open for the remaining answers. Weeding out wrong answers over time is part of the discovery process.
The final criterion - the ability to acquire information before the settled date - is what separates prediction markets from strict gambling. If all participants are in the dark about a question until that question is settled, then there is little value in asking the question. Prediction markets are powerful because they allow participants to impart some knowledge into the process over a period of time. The resulting market prices can then provide information that can be acted upon throughout the process. If participants cannot acquire useful information to incorporate into the market, then market activity is nothing more than playing roulette where all answers are equally possible until the correct answer is determined.
A good example to illustrate the above criteria is a customer satisfaction survey. Railinc uses a bi-annual (twice a year) survey to gauge customer sentiment on a list of products. For each product, customers are asked a series of questions the answers to which range from 1 (disagree) to 5 (agree). The answers are then averaged with a final score for each product ranging from 1-5 (the goal is to get as close to 5 as possible).
The following market could be set up for Railinc employees:
As far as concreteness is concerned, the final answer for this question will be determined when the survey is completed (e.g., January 2011), and it will be a specific number that falls into one of the ranges given by the answers.
This market also satisfies the last criteria regarding the ability to acquire information before the market is settled. This is important because this is where the value of the market is realized. As Railinc employees (i.e., market participants) gain knowledge over time they can incorporate that knowledge into the market via the buying and selling of shares in the provided answers.
The Art
In the example given above regarding the customer satisfaction survey, the answers provided were not arbitrary - they were selected to maximize the value of the market. This is where the art of prediction markets is applied.
If the possible answers for a customer survey are 1-5 why not provide five separate answers (1-1.9, 2-2.9, 3-3.9, 4-4.9, 5)? Why not have two possible answers (below 2.5 and above 2.5)? The selection of possible answers is partially determined by what is already known about the result. In the case of the survey, past results may have shown that this particular product has average a 4.1. It is highly unlikely that the survey results will drop to the 1-1.9 range. Providing such an answer would not be valuable because market participants would almost immediately short that position. This is still information, but it is information that is already known. What is desired is insight to what is not known. The answers provided in the above example will give some insight into whether the product is continuing to improve or whether it is digressing.
So, the selection of possible answers to market questions must take into account what is already known as well as what is unknown. What do you know about what you don't know?
Conclusion
Good questions make good prediction markets. Constructed properly, these questions can be a valuable tool in the decision making process of an organization.
The Science
There are three criteria to keep in mind when constructing a question for a prediction market:
- The correct answer must be concrete
- Answers must be determined on specific dates
- Information about possible answers can be acquired before the settled date
Not only should answers be concrete, there should be some point in time when each answer can be determined to either have occurred or not have occurred. A question that never gets resolved can hamper the prediction process by reducing the incentive to invest in that market. (Can a non-expiring question be valuable? Could the ongoing process of information discovery be useful? Questions to ponder.)
This doesn't mean, however, that every answer must be determined on the same date. Wrong answers can be closed as the process unfolds. Once the correct answer is determined, however, the market should be closed. For example, take the question "Which candidate will win the 2012 Republican Party nomination for U.S. President?" If this question is asked in January of 2012 there could be several possible answers (one for each candidate). As the year progresses to the Republican Party convention, several candidates will drop out of the election. The prediction market would then close out those answers (candidates) but stay open for the remaining answers. Weeding out wrong answers over time is part of the discovery process.
The final criterion - the ability to acquire information before the settled date - is what separates prediction markets from strict gambling. If all participants are in the dark about a question until that question is settled, then there is little value in asking the question. Prediction markets are powerful because they allow participants to impart some knowledge into the process over a period of time. The resulting market prices can then provide information that can be acted upon throughout the process. If participants cannot acquire useful information to incorporate into the market, then market activity is nothing more than playing roulette where all answers are equally possible until the correct answer is determined.
A good example to illustrate the above criteria is a customer satisfaction survey. Railinc uses a bi-annual (twice a year) survey to gauge customer sentiment on a list of products. For each product, customers are asked a series of questions the answers to which range from 1 (disagree) to 5 (agree). The answers are then averaged with a final score for each product ranging from 1-5 (the goal is to get as close to 5 as possible).
The following market could be set up for Railinc employees:
What will the Fourth Quarter 2010 customer satisfaction score be for product X?The value of this market is that Railinc management and product owners may get some insight into what employees are hearing from customers. Customer Service personnel could have one view based upon their interactions with customers, while developers may have a different view. Over time, management and product owners could take actions based upon market movements.
- Less than or equal to 4.0
- Between 4.1 and 4.4 (inclusive)
- Greater than or equal to 4.5
As far as concreteness is concerned, the final answer for this question will be determined when the survey is completed (e.g., January 2011), and it will be a specific number that falls into one of the ranges given by the answers.
This market also satisfies the last criteria regarding the ability to acquire information before the market is settled. This is important because this is where the value of the market is realized. As Railinc employees (i.e., market participants) gain knowledge over time they can incorporate that knowledge into the market via the buying and selling of shares in the provided answers.
The Art
In the example given above regarding the customer satisfaction survey, the answers provided were not arbitrary - they were selected to maximize the value of the market. This is where the art of prediction markets is applied.
If the possible answers for a customer survey are 1-5 why not provide five separate answers (1-1.9, 2-2.9, 3-3.9, 4-4.9, 5)? Why not have two possible answers (below 2.5 and above 2.5)? The selection of possible answers is partially determined by what is already known about the result. In the case of the survey, past results may have shown that this particular product has average a 4.1. It is highly unlikely that the survey results will drop to the 1-1.9 range. Providing such an answer would not be valuable because market participants would almost immediately short that position. This is still information, but it is information that is already known. What is desired is insight to what is not known. The answers provided in the above example will give some insight into whether the product is continuing to improve or whether it is digressing.
So, the selection of possible answers to market questions must take into account what is already known as well as what is unknown. What do you know about what you don't know?
Conclusion
Good questions make good prediction markets. Constructed properly, these questions can be a valuable tool in the decision making process of an organization.
Subscribe to:
Posts (Atom)