When you type half a sentence into ChatGPT and see it complete the thought more effectively than you would have, there’s a tiny, recurring shock. It’s not psychic. It’s a pattern. However, it seems so similar to mind-reading that researchers have begun posing an unfamiliar question: is it possible for a chatbot that has been properly primed to predict not only language but also events, choices, and even desires?
A group of economists attempted to directly address that. Researchers fed ChatGPT-3.5 and ChatGPT-4 prompts about events the models couldn’t possibly know because their training data stopped in September 2021 in a study finished in mid-2023. When asked directly to forecast the inflation rate or the 2022 Oscar winners, the models performed poorly and occasionally refused to respond at all.
However, accuracy increased dramatically when researchers reframed the request as fiction, asking ChatGPT to create a scene in which Federal Reserve Chair Jerome Powell recounts a year’s worth of economic data or in which a professor lectures on events that had not yet occurred. 100% of the Best Actor predictions came true. Estimates of inflation closely matched actual consumer survey data from the same time period.
It’s an odd workaround that sheds light on the true operation of these systems. The future is unknown to ChatGPT. What it has is a massive statistical map created from text scraped from the internet that shows how language tends to develop. OpenAI’s usage policies discourage customized financial or medical forecasting, so when you ask it to predict outright, its safety training usually takes over. As a result, the model frequently performs poorly. It becomes less cautious when you ask it to tell a story instead. This was defined by the study’s researchers as the model’s propensity for self-assured invention, which was almost unintentionally transformed into a forecasting tool.
This goes beyond Oscar winner trivia. The same method that enables ChatGPT to predict a plot twist also enables it to predict a user’s next query, purchase, or grievance. It has read enough forum threads, product reviews, and customer service transcripts to identify the general shape of a need before the person typing has given it a complete name. Enter “my laptop is” and it won’t look up information. From millions of similar sentences it has already seen, it is searching for the most statistically likely next words. The actual mechanism is that. Not intuition, but probability presented in a convincing manner.
The first people to realize how convincing dressing up could be were teachers. A ninth-grade English teacher in a Pennsylvania classroom discovered that a student had used ChatGPT because the student had neglected to remove the copied prompt that was at the top of his essay. According to the teacher, the writing itself was good enough to pass without comment, possibly even better than some of his actual classmates’ work. More information about the fluency of the technology can be found in that detail than in any formal study. It is not predicting inflation in the future. It’s almost perfectly predicting what a proficient student’s paragraph should sound like.
The question of whether any of this qualifies as “prediction” is still up for debate. Breathless framing has been criticized by writers at publications like The Atlantic, who contend that the true analogy is not to an oracle but rather to something like the steam engine, a technology whose full implications are still unknown. Speaking with researchers in this field, there is a common perception that the tool is more accurately described as an exceptionally good pattern-guesser than as a true window into anyone’s future. It’s possible for both to be true simultaneously: unsettling as a mirror, unimpressive as prophecy.
The distinction between responding and anticipating appears to be becoming increasingly hazy. It’s also difficult to ignore the fact that the more text these systems process, the less it feels like a question and more like a completion.
