Forecasting Future with AI

Question / Claim

What kinds of real-world events can language models meaningfully forecast?

Key Assumptions

• Forecastability depends more on signal structure than reasoning difficulty(high confidence)
• News coverage provides sufficient leading indicators for institutional events(medium confidence)
• Calibration is more important than raw accuracy in forecasting(high confidence)

Evidence & Observations

• The OpenForesight paper shows strongest performance on institutional and process-driven events like elections and appointments(citation)
• OpenForesight evaluation shows highest accuracy and calibration on institutional political events such as elections, confirmations, and government appointments.(citation)
• Prior forecasting research shows calibrated probabilistic models outperform raw accuracy metrics in political and geopolitical forecasting.(citation)
• Prediction markets and superforecaster studies demonstrate that process-driven events with public signals are more forecastable than shock-based events.(citation)
• There is no methodological or empirical connection between LLM-based forecasting and astrological or religious prediction systems; modern forecasting relies on probabilistic calibration, falsifiability, and empirical signals, whereas astrology and religion operate as belief-based, non-calibrated systems.(citation)
• The main triggering work for this discussion is the OpenForesight paper, which empirically evaluates which classes of future events are forecastable by LLMs using calibrated, open-ended predictions.(citation)

Open Uncertainties

• How well does this generalize beyond news-driven domains?
• Can models forecast longer time horizons without severe degradation?

Current Position