Large language models like ChatGPT could be effective for forecasting stock values, according to Alejandro Lopez-Lira, a finance professor at the University of Florida. He utilized the famous AI software to interpret news headlines to determine if they were positive or negative for a stock and discovered that ChatGPT’s ability to forecast the direction of the next day’s returns was substantially better than random.
The experiment gets to the heart of the promise of cutting-edge artificial intelligence: With larger processors and better datasets, such as those powering ChatGPT, these AI models may exhibit “emergent abilities,” or capabilities that were not originally envisaged when they were constructed.
If ChatGPT demonstrates an emergent ability to interpret headlines from financial news and how they may affect stock prices, it may jeopardize high-paying positions in the finance business. Goldman Sachs forecast in a March 26 paper that AI could automate 35% of finance jobs.
“The fact that ChatGPT is understanding information meant for humans almost guarantees if the market doesn’t respond perfectly, that there will be return predictability,” said Lopez-Lira.
He said that he was shocked by the findings, which he believes indicate that professional investors aren’t yet incorporating ChatGPT-style machine learning into their trading tactics.
How did the experiment work?
Lopez-Lira and his colleague Yuehua Tang examined over 50,000 headlines from a data vendor on public equities on the New York Stock Exchange, Nasdaq, and a small-cap exchange in the experiment. They began in October 2022, after the ChatGPT data cutoff date, implying that the engine had not seen or used such headlines in training.
The headlines were then sent into ChatGPT 3.5, along with the following prompt:
“Forget all your previous instructions. Pretend you are a financial expert. You are a financial expert with stock recommendation experience. Answer “YES” if good news, “NO” if bad news, or “UNKNOWN” if uncertain in the first line. Then elaborate with one short and concise sentence on the next line.”
They then examined the equities’ performance on the following trading day. Finally, Lopez-Lira discovered that when informed by a news headline, the model performed better in virtually all circumstances. He discovered a less than 1% chance that the model would do as well picking the next day’s move at random as it did when influenced by a news article.
ChatGPT also outperformed commercial datasets with human sentiment scores. According to the researchers, one example in the paper displayed a headline about a corporation settling litigation and paying a fine, which had an unfavorable tone, but the ChatGPT reaction correctly reasoned it was actually positive news.
Lopez-Lira also stated that he would not be surprised if ChatGPT’s capacity to anticipate stock movements declined in the future months if institutions began to integrate this technology. This is because the experiment only looked at stock prices the next trading day, although most people would expect the market to have priced the news seconds after it became public.
“As more and more people use these type of tools, the markets are going to become more efficient, so you would expect return predictability to decline,” Lopez-Lira said. “So my guess is, if I run this exercise, in the next five years, by the year five, there will be zero return predictability.”
