Is your confidence actually calibrated?

Say “70% sure” often enough and you should be right about 70% of the time. Most people are not. Add your forecasts below and get a Brier score, a baseline comparison (Brier Skill Score) and a bucket-by-bucket calibration curve — all computed locally in your browser.

Your forecasts

#	Label	Your p %	Outcome (1/0)	Baseline p % (opt.)

Baseline = the probability a default guesser would have given. Fill it in and you get a Brier Skill Score: > 0 means you beat your own intuition, ≤ 0 means the forecast adds nothing.

Results

Brier score

—

lower is better

Brier skill score

—

vs your baseline

Accuracy

—

no rows

Mean forecast

—

Add a few rows to see your calibration.

Calibration curve

Each dot is a 10% bucket: x = average forecast, y = actual hit rate. Perfect calibration puts every dot on the diagonal.

How to read this

Brier score

The mean of (p − outcome)&code>². A forecast of 0.8 that was right scores 0.04; a forecast of 0.8 that was wrong scores 0.64. It punishes overconfidence far more than uncertainty, which is exactly the point.


      Brier skill score
      1 − your Brier / baseline Brier. Zero means you did no better than always saying “50%”. Positive means your probability estimates genuinely add information. This is the number that tells you whether forecasting is worth your time at all.


    
      Calibration curve
      The single most useful diagnostic. Points above the diagonal mean you are under-confident (you said 30% and it happened 45% of the time). Points below the diagonal mean you are over-confident — the classic trader’s mistake, and the one that costs real money.
      Minimum useful sample
      A bucket with 2 forecasts tells you nothing. Judge a bucket at 10+ resolved forecasts, and judge the whole set at 30+ before changing how you think.



  Questions
  
    Is my data sent anywhere?
No. Everything is computed in JavaScript in your browser. Nothing is uploaded, stored or logged. The shareable link keeps your rows in the URL fragment, which is never transmitted to a server either.
    What is a good Brier score?
It depends on the question. For genuinely uncertain events, 0.20-0.25 is a competent forecaster. Below 0.15 usually means the event was easier than it looked. Above 0.30 with a baseline of 0.25 means your probabilities are adding noise, not signal.
    Should I use a baseline of 50%?
Only if 50% is genuinely your honest default for that type of question. A better baseline is what you would have said before thinking hard — most people overestimate their own skill, so 50% is often too generous to yourself.