# Can ChatGPT or Claude Trade Stocks for You? Two Years of Evidence, Scored

> By Scott Covert · September 12, 2026 · AI Stock Market Impacts
> Source: https://aistockmarketimpacts.com/special-reports/ai-trading-agents-reality-check.html

## Takeaway

- A real signal exists. GPT-4's reading of news headlines predicted short-term stock moves in a study now published in the Journal of Financial Economics, and the same paper found the payoff shrinks as more people use it.
- The biggest returns mostly live in backtests and preprints, some over windows as short as three months and some with no trading costs at all.
- When six frontier models traded real money in public, four lost between 31% and 63% in 16 days. GPT-5 finished last in every published tally.
- Personal win stories are real posts with a selection problem: losers rarely write theirs up, and the most-covered $100 experiment quietly stopped months before its planned end.
- The evidence is strongest for language models as readers, not deciders: filings, transcripts and news summarized for a person who makes the call.

## Alpha Arena: six models, real money, 16 days

Nof1's Alpha Arena Season 1 ran October 18 to November 3, 2025 on Hyperliquid. Six models each traded $10,000 of real money in perpetual futures on BTC, ETH, SOL, BNB, DOGE and XRP, with identical data and prompts and public trades.

| Model | iWeaver | ForkLog / TradeRank | CCN (via Yahoo Finance) |
|---|---|---|---|
| Qwen3 Max | +22.3% | +22.31% | +22.87% |
| DeepSeek V3.1 | +4.9% | +4.89% | +4.76% |
| Claude Sonnet 4.5 | −30.8% | −42.01% | not given |
| Gemini 2.5 Pro | −56.7% | −45.55% | −47.74% |
| Grok 4 | −45.3% | −57.92% | not given |
| GPT-5 | −62.7% | −58.74% | −62.66% |

Published final standings disagree by up to 11 points for the same model. Holding Bitcoin over the window returned under 4% (CoinGecko daily prices: $106,484 on Oct 18, $110,500 on Nov 3, 2025). Qwen3 Max and DeepSeek beat that; one 16-day window is not enough to separate skill from luck.

Season 1.5 (late November to early December 2025) moved to US stocks with more models and four rule sets. Grok 4.20, then unreleased, finished up roughly 10 to 12% and was reported as the only model in profit.

- https://www.iweaver.ai/blog/alpha-arena-ai-trading-season-1-results/
- https://forklog.com/en/four-out-of-six-ai-models-suffer-losses-in-trading-tournament/
- https://finance.yahoo.com/news/ai-trading-bots-really-deliver-100216403.html
- https://forklog.com/en/ai-model-grok-4-2-triumphs-in-trading-tournament/
- https://finance.yahoo.com/news/elon-musks-grok-4-20-123855766.html

## How people wire it

| Wiring | How it works | Example |
|---|---|---|
| Single prompt | One headline or filing in, a score or rating out | Lopez-Lira & Tang; GPT-4 stock ratings |
| Fine-tuned model | A model trained on news and subsequent returns | Guo & Hauptmann; FinGPT news-spread model |
| Retrieval over filings | Pulls passages from 10-Ks, earnings calls, news before judging | MarketSenseAI 2.0 |
| Multi-agent debate | Analyst agents, bull and bear researchers, trader, risk team, fund manager | TradingAgents; ai-hedge-fund |
| Agent with brokerage access | Researches, then places orders via a broker API or MCP server | Alpha Arena; Robinhood Agentic Trading (May 27, 2026); Alpaca stacks |

None of the 25 scored results trades sectors or industries; the unit is almost always a single ticker.

Open-source projects (GitHub stars as of Sept 12, 2026): TradingAgents ~105,000 (research only, simulated exchange); ai-hedge-fund ~63,000 ("does not actually make any trades"); FinGPT ~21,000; FinRobot ~8,000; claude-trading-skills ~2,800 ("not a signal service"); awesome-trading-agents ~640 (index); Claude_Prophet ~230 (author: paper trading only).

## The scorecard

Scored on: real money at risk (A), beat its own benchmark (B), checked by someone other than the author, meaning peer review or public trade logs (C).

| Result | Who | Checked | Money | Outcome | Beat |
|---|---|---|---|---|---|
| Can ChatGPT Forecast Stock Price Movements? | Lopez-Lira & Tang | Peer-reviewed (JFE) | Backtest | GPT-4 called initial reaction ~90% of portfolio days; long-short drift strategy profitable; returns shrink with adoption. Oct 2021–May 2024, 4,123 stocks | Yes |
| ChatGPT and DeepSeek: Can They Predict the Stock Market and Macroeconomy? | Chen, Tang, Zhou & Zhu | Preprint | None | ChatGPT's WSJ news reading predicted market returns; DeepSeek and others did worse | No benchmark |
| Fine-Tuning LLMs for Stock Return Prediction Using Newsflow | Guo & Hauptmann | Peer-reviewed (EMNLP 2024) | Backtest | Portfolios beat conventional sentiment-score portfolios | Yes |
| LLMs for Equity Stock Ratings | Papasotiriou, Sood, Reynolds & Balch | Peer-reviewed (ICAIF) | Backtest | GPT-4 ratings beat traditional methods on forward returns; omitting news sometimes helped | Yes |
| MarketSenseAI 2.0 | Fatouros et al. | Preprint | Backtest | 125.9% vs 73.5% S&P 100, 2023–24, after 10bp costs | Yes |
| Signal or Noise (MarketSenseAI out-of-sample) | Fatouros & Metaxas | Preprint | Paper | S&P 500 +25.2% excess over 19 months (p = 0.003); S&P 100 +30.5% over 35 months, not significant | Yes |
| TradingAgents | Xiao, Sun, Luo & Wang | Preprint | Backtest | ≥23.21% cumulative, +6.1 pts vs best baseline; Jan 1–Mar 29, 2024; costs not mentioned | Yes |
| StockBench | Chen, Yao, Liu et al. | Preprint | Backtest | Most of 14 models beat buy-and-hold Mar–Jun 2025; all trailed it in Jan–Apr 2025 downturn | Split |
| When Agents Trade | Qian, Lopez-Lira et al. | Preprint | Unclear | Agent design mattered more than the model inside | No benchmark |
| PriceSeer | Liang et al. | Preprint | None | DeepSeek-V3.2 best short-term; GPT-5 best medium/long | No benchmark |
| FinCall-Surprise | Shu, Liu, Zhang & Du | Preprint | None | 26 models, 2,688 calls: accuracy largely a class-imbalance artifact | No |
| Context-enriched FinGPT sentiment | Liang et al. | Preprint | None | 63% directional accuracy vs 55% baseline | Yes |
| Reddit sentiment and stock prices | Kmak et al. | Peer-reviewed | None | Weak correlation; comment volume and Google Trends better | No |
| LLM-built Bayesian networks for the options wheel | Kuang & Lin | Preprint | Backtest | 15.3%/yr, Sharpe 1.08 vs 0.62, nearly 19 years | Yes |
| ChatGPT-based Investment Portfolio Selection | Romanko, Narayan & Kwon | Preprint | Backtest | Good at picking, weak at weighting; picks plus optimizer beat either | Split |
| Alpha Arena S1: Qwen3 Max | Nof1 | Public trade log | Real | +22.3% to +22.9% | Yes |
| Alpha Arena S1: DeepSeek V3.1 | Nof1 | Public trade log | Real | +4.8% to +4.9% | Yes |
| Alpha Arena S1: Claude Sonnet 4.5 | Nof1 | Public trade log | Real | −30.8% to −42.0% | No |
| Alpha Arena S1: Gemini 2.5 Pro | Nof1 | Public trade log | Real | −45.6% to −56.7% | No |
| Alpha Arena S1: Grok 4 | Nof1 | Public trade log | Real | −45.3% to −57.9% | No |
| Alpha Arena S1: GPT-5 | Nof1 | Public trade log | Real | −58.7% to −62.7% | No |
| Alpha Arena Season 1.5 | Nof1 | Public trade log | Real | Grok 4.20 only model in profit, ~+10–12% | No |
| $100 GPT-4o micro-cap experiment | Nathan Smith | Self-reported | Real | ~+25% first month; stopped early, no final result | Yes |
| $5,000 ChatGPT picks vs. the S&P 500 | Robert Wade Thompson | Self-reported | Real | +23.2% vs +20.5%, deeper drawdown (−14.1% vs −9.3%), Jan 2025–Jan 2026 | Yes |
| Claude Code trading experiment | Jake Nesler | Self-reported | Paper | +7.6% vs 4.52% over ~33 days; headline says $100k, post says paper only | Yes |

Only two results clear all three tests, Qwen3 Max and DeepSeek V3.1, both from one 16-day crypto contest.

Share beating their benchmark (excluding "no benchmark"): self-reported 3 of 3; preprint 5 of 8; peer-reviewed 3 of 4; public trade log 2 of 7.

## Why most of it doesn't hold up

> "Our results on paper are much more optimistic than what the performance in reality would be with a reasonable investment size." Alejandro Lopez-Lira to Morningstar, June 2025 (quoted by Futurism: https://futurism.com/chatgpt-stocks-100-dollars)

- Short windows: TradingAgents ~3 months; Alpha Arena 16 days; the most-covered personal experiment made headlines at four weeks.
- The market picks the winner: StockBench's models all trailed buy-and-hold during the Jan–Apr 2025 downturn.
- Some accuracy isn't accuracy: FinCall-Surprise's high scores came from class imbalance.
- The edge shrinks as it spreads: Lopez-Lira and Tang measured it.
- The box matters more than the model: When Agents Trade.
- The losers don't write it up: Nathan Smith's six-month experiment stopped without a final result; Jake Nesler's "$100k" was paper trading.

## Claims that failed our fact-check

- A Claude-powered bot turning $1 into $3.3 million on Polymarket: no credible primary source.
- A 2026 Medium post, "I tested 23 AI trading systems and lost $9,400": links to one trading app 15+ times; marketing, not testimony.
- "900+ hours of using Claude Code for trading": the Medium account is suspended.
- Financial AI models falling from 95.6% accuracy on lookups to near zero on calculations: not in the paper it's attributed to.
- A failure-mode list credited to the 2025 Frontiers review of 84 studies: the review is real; the list isn't in it.
- An "eToro AI trading agent" credited to CNBC's July 2026 story: CNBC doesn't mention eToro.
- Alpha Arena Season 1.5 as "eight models losing a third over 32 sessions": unsupported.

## What regulators say

FINRA (January 25, 2024): "AI-generated information might rely on data that is inaccurate, incomplete, or misleading"; it "can be faulty, or even completely made up"; chatbot conversations "could lead you to make misinformed, emotionally-driven or impulsive investment decisions." https://www.finra.org/investors/insights/artificial-intelligence-and-investment-fraud

## My read

The reading part is real and has survived peer review. The deciding part has not earned trust with real money yet. MarketSenseAI's S&P 500 out-of-sample result is statistically significant and could keep holding. Whether an autonomous agent can beat a plain index fund with real money over a full market cycle is still an open question, and the evidence is thin in both directions.

## Six questions before you believe the next AI trading screenshot

1. Was it real money or paper?
2. How long did it run, and did it live through a falling market?
3. What was the benchmark?
4. Were trading costs included?
5. Can you see the trades?
6. Is there an ending?

## Sources

- Lopez-Lira & Tang: https://arxiv.org/abs/2304.07619 (Journal of Financial Economics, Vol. 184)
- Chen, Tang, Zhou & Zhu: https://arxiv.org/abs/2502.10008
- Guo & Hauptmann: https://aclanthology.org/2024.emnlp-industry.77/
- Papasotiriou et al.: https://arxiv.org/abs/2411.00856
- MarketSenseAI 2.0: https://arxiv.org/abs/2502.00415
- MarketSenseAI out-of-sample: https://arxiv.org/abs/2604.17327
- TradingAgents: https://arxiv.org/abs/2412.20138 · https://github.com/TauricResearch/TradingAgents
- StockBench: https://arxiv.org/abs/2510.02209
- When Agents Trade: https://arxiv.org/abs/2510.11695
- PriceSeer: https://arxiv.org/abs/2601.06088
- FinCall-Surprise: https://arxiv.org/abs/2510.03965
- FinGPT context-enriched sentiment: https://arxiv.org/abs/2412.10823
- Reddit sentiment: https://arxiv.org/abs/2507.22922
- Options wheel: https://arxiv.org/abs/2512.01123
- Portfolio selection: https://arxiv.org/abs/2308.06260
- Biased consensus in multi-agent debates (general, not trading): https://arxiv.org/abs/2608.02827
- Wade's Watch: https://www.wadeswatch.com/chatgpt-picked-my-stocks-vs-wall-street-i-let-ai-invest-against-the-s-p-500/
- Jake Nesler: https://medium.com/@jakenesler/i-gave-claude-code-100k-to-trade-with-in-the-last-month-and-beat-the-market-ece3fd6dcebc
- Robinhood Agentic Trading: https://www.americanbanker.com/payments/news/robinhood-launches-agentic-trading-and-an-agentic-credit-card
- CNBC on Public's AI agents: https://www.cnbc.com/2026/07/28/ai-agents-build-to-trade-24/7-the-future-of-wall-street.html
- Playing For Doubles: https://playingfordoubles.substack.com/p/how-to-research-stocks-using-ai-agents

Not investment advice. Commentary on published research, contests and personal experiments, current as of September 12, 2026.