How Agora Forecasts: Measuring How Beliefs Form

The Agora team · November 2026

Polls measure what people think. Prediction markets measure what people will bet on. Agora measures how people reach their views, and uses that to forecast. Here's how it works, how it performed on the Israeli election, and the rules we hold it to.

Why this matters

Every forecasting method today looks at the answer and ignores the reasoning. That leaves real signal on the table: who actually knows the facts, who is calibrated, and whose politics is bending their judgment. Agora captures all three every day, from thousands of people playing through the news.

Early results

The Israeli election

Ahead of Israel's October 27 election, we asked players twice whether Benjamin Netanyahu would remain Prime Minister. Both times, Agora rated his chances higher than the prediction markets:

Netanyahu's bloc won the most seats of any bloc, and he remains Prime Minister while coalition talks continue.

Next: the US midterms

We're now running seven forecasts on the 2026 midterms, covering House and Senate control and key races in Ohio and Michigan. The Polymarket and Kalshi prices were recorded at the moment each forecast opened. We'll publish the full scorecard, every race, once results are certified.

How we collect the data

So far, about 2,250 players have made nearly 9,700 forecasts and answered almost 40,000 fact-check, timeline and estimate rounds.

Each day's story is a short game, about nine minutes on average. For real-world events, it opens with a forecast: players commit to a view before seeing anything else. Then they play through the story:

• Fact-checks, timelines and "spot the AI" rounds test what they actually know, with chip wagers that reveal how confident they are.
• Briefings from left, center and right outlets show which framings they engage with.
• Command mode puts them in a decision-maker's seat and asks them to trade off real priorities.
• A debate with an AI opponent makes them defend their view in writing.
• A final call asks the opening question again, after everything they've seen.

Over weeks of play, this builds a profile of each player: what they know, how well-calibrated their confidence is, where they sit politically (measured from behavior, not a survey), and how they update when the evidence changes.

How we forecast

A raw crowd average treats every answer equally. Because we see how each forecast was formed, we don't have to.

Finding the best reasoners. Some players are consistently well-informed and well-calibrated. They get the facts right, bet big when they should and small when they shouldn't, and update in the right direction. Their forecasts carry more weight.

Learning from superforecasters. We study how our most accurate players think: which sources they read, how far they move when new facts arrive, and whether a debate changes their mind for the better. We're using those patterns to train our models to spot strong forecasters before they've built a long track record.

Correcting for bias. People predict what they want to happen. For every player, we measure how far their politics has pulled their past forecasts toward the outcome they wanted, and discount their new forecasts on partisan questions by that amount. A reliably wishful partisan counts for less when their side is on the ballot.

We see this in our own data. Before the Israeli election, which Donald Trump had weighed in on by endorsing Netanyahu, about half of our Republican players expected Netanyahu to stay in power, compared with fewer than a third of Democrats and independents. The samples are small, but it's exactly the pattern a raw crowd average, or a market, can't separate out.

Games as calibration training. Every mini-game is a small calibration test: a player stakes chips on how sure they are, then finds out if they were right. Thousands of these tell us far more about someone's judgment than a handful of real-world forecasts can, and they feed directly into how much weight that player carries.

How we test it

A forecasting claim is only worth something if it can be checked. We hold ourselves to these rules:

1. 1Same-moment comparison. For every question, we record the Polymarket and Kalshi prices at the exact time our players forecast. No comparing against a market's worst day.
2. 2Closest to the truth wins. We win a question only if our probability ends up closer to what actually happened than the market's at that same moment.
3. 3Proper scoring. Across all questions, we report Brier scores for Agora, Polymarket, Kalshi, and a coin-flip baseline.
4. 4Frozen models. The weighting model is versioned and fixed before a question opens. We never re-weight after the result is known.
5. 5No cherry-picking. We don't go looking for the subgroup that happened to win. Any new way of slicing the crowd is announced before it's tested.
6. 6Independent resolution. Every outcome is settled by a named public source, using the same resolution rules as the matching market.
7. 7Everything is published. Every scored question goes on the scorecard, losses included, with timestamps, prices and sources.

Is it statistically significant?

Not yet, and we won't say otherwise until it is. A handful of events can't prove one forecaster is better than another; a coin flip wins a few in a row too. Our bar is at least 30 pre-registered questions, scored head-to-head against the markets, with a Brier score advantage whose 95% confidence interval excludes zero. The midterms are the first large batch. Until we clear that bar, we'll describe our results as early evidence.

What this means

If reasoning-weighted forecasts consistently beat money-weighted ones, it validates Agora's core idea: how people form beliefs is more valuable than what they believe.

The product that follows is a daily forecast feed. For every major story, it shows Agora's probability next to the market's, and why they differ: which evidence moved our best forecasters, how far partisanship is skewing each side, and how opinion shifted once people saw the facts. Every story becomes a forecast, including the many questions no market covers.

• Hedge funds and traders get an independent signal, with the reasoning attached, on questions markets cover thinly or not at all.
• AI labs get rich, structured data on how people weigh evidence and change their minds.
• Media companies get an engaged audience and a live read on how their readers are forming views.
• Campaigns see how beliefs actually form and shift, not just the topline number.

Limitations

• A young track record. We launched in August 2026, so the number of resolved forecasts is still small.
• Small crowds on some questions. Some forecasts have dozens of players, not thousands. Small crowds are noisier.
• A skewed player base. Our players are mostly American, young and politically engaged, so the raw crowd isn't representative. That's one reason weighting matters.
• Yes/no forecasts. Players currently pick a side rather than give a probability. A probability slider is on the way and will give sharper forecasts.
• Imperfect market matches. Some questions have no market, or only an approximate one. We flag every case where the wording differs.