Identifying and Mitigating Confirmation Bias

How a Forecast Can Become Selective

Peter Wason’s 1960 hypothesis-testing experiment examined reasoning in a number-rule task. Its confirming-evidence problem motivates a question to ask in research: what observation would distinguish this explanation from an alternative? It was not a sports-market study and establishes no betting-return effect.

For a hypothetical fan of Team A, selective analysis might mean recording recent wins while omitting availability changes or stronger opponents. The issue is the choice of evidence, not the fact that the fan has a preference. Recent information can be relevant; it becomes problematic when its weight is chosen only because it supports the desired conclusion.

Do not infer that aggregate market prices are inefficient merely because individuals can reason selectively. That separate claim needs evidence about prices, information, participants, and outcomes in a defined market.

Build a Falsification Checklist

Before a simulation decision, record:

1. the outcome being predicted and the information cutoff;
2. the chosen sample window and why it is relevant;
3. evidence supporting and challenging the thesis;
4. what new information would change the estimate;
5. how the same rules would apply to the opponent.

Weigh evidence by reliability and relevance, not by giving every pro and con equal numerical weight. Do not assign separate probabilities to overlapping narrative arguments and add them as if they were mutually exclusive outcomes. Use a defined forecasting method and test its calibration.

A model’s repeatability is useful for review, but it does not remove bias in selected features, labels, or samples. Keep an untouched later test period to reduce the opportunity to choose rules based on favorable historical results.

Worked Research Example

Assume a hypothetical Team A selection at +150, decimal 2.50. A model estimates its win probability as 55%. These are invented inputs, not a historical game or a known probability.

The price implies break-even 100/(150+100) = 40%. The forecast exceeds that threshold by 15 percentage points. Expected net return per unit is 0.55 × 1.50 − 0.45 = 0.375, or +37.50% of stake. Comparing 55% with 40% identifies the probability gap; it is not the EV calculation itself.

That large modeled expectation is a reason to check the data cutoff, event definition, availability, executable price, and probability calibration. Under an alternative assumed probability of 38%, EV becomes 0.38 × 2.50 − 1 = −5.00%. The calculation is exact given the assumptions; the assumptions remain uncertain.

A supporting or opposing article is not automatically reliable because of its position. Record its original data and what it adds beyond information already included in the model.

Questions to Ask Before Locking a View

Pair this checklist with Psychology of the Bad Beat and the Post-Tournament Audit.

Q: What is an example of selective evidence?

A: Recording a favored team’s recent wins while omitting relevant availability or opponent information. The example illustrates a research risk; it does not measure its prevalence in betting markets.

Q: Does using a model eliminate confirmation bias?

A: No. Feature selection, sample choice, labels, and tuning can all preserve preferred assumptions. Document contrary evidence and evaluate later predictions.

Q: Does recency bias prove a market inefficiency?

A: No. Individual reasoning and aggregate price accuracy are different questions. An inefficiency claim requires a defined price and outcome sample.

Q: Is the 15 percentage-point gap the example’s EV?

A: No. At an assumed 55% win probability and +150 price, net EV is 0.55 × 1.50 − 0.45 = 0.375 units per unit staked, or +37.50%.