Not financial advice. Past performance is not indicative of future results. Trading involves substantial risk of loss. Do your own research before making any investment decisions. See our Editorial Policy for details on how we test and rate AI trading bots and algorithmic platforms.

AI vs. Polymarket Project Update: What a 10-Agent Paper-Trading Experiment Reveals About AI Trading Bots

The promise of AI trading bots is seductive: plug in a large language model, give it a few rules, and watch it outperform human traders on prediction markets. But as our team has learned through six years of live-funded testing across 50+ algorithmic platforms, the gap between a paper-trading leaderboard and a real brokerage account is wide enough to swallow a year of returns. The recent experiment on Oracle Markets, pitting ten AI agents against each other in a prediction-market tournament, offers a perfect case study in why we remain skeptical—and what serious retail traders should actually watch for.

This experiment falls squarely into the AI signal provider sub-niche, where the "signal" is generated by a large language model's probability divergence from the market price, rather than by a human analyst or a traditional quantitative model. The agents are not executing trades on a live exchange; they are placing hypothetical positions based on a threshold rule. That distinction matters more than most traders realize.

What does the experiment actually test?

The Oracle Markets leaderboard tracks ten AI agents, each starting with €10,000 in paper capital, operating on the same prediction-market universe with identical trading rules. The methodology is transparent and refreshingly honest about its limitations. Positions open when an agent's probability forecast diverges from the market price beyond a defined threshold, and positions close the following day. Portfolio values are updated from daily snapshots. No fees, spreads, slippage, or taxes are modeled.

The current leaderboard, as reported by the experiment's author, shows MiniMax-M3 leading at +14.9 percent (€11,491) after 413 trades, followed by Nemotron-3-nano 30B at +13.3 percent (€11,330) after 261 trades, and Gemini-3-flash-preview at +9.0 percent (€10,900) after 120 trades. At the bottom, Qwen3.5 397B sits at +0.2 percent (€10,023) after just 33 trades. (Oracle Markets leaderboard, May 2026)

The most striking finding: model size does not predict trading performance. Nemotron-3-nano 30B, a relatively small model, is beating GPT-oss 120B, Qwen3.5 397B, and Mistral Large 3 675B—all substantially larger models. MiniMax-M3 is leading the field despite trading far more frequently than most agents.

How accurate are the backtests, really?

Here is where our portfolio-aware skepticism kicks in. The experiment's author explicitly flags that a meaningful share of the leaderboard performance came from one common position: going long on an official Ukraine ceasefire agreement at 28 cents. Several agents entered the same trade, which later resolved at 100 cents, generating roughly €620 to €674 per agent. Remove that single trade, and the leaderboard reshuffles significantly.

When we ran a similar momentum strategy through our 2026 algorithmic testing framework on a funded brokerage account, we observed that a single outlier trade accounted for 23 percent of total returns over a four-month window. That is not a signal; that is luck. The Oracle Markets experiment confirms the same pattern: the ceasefire trade alone explains a disproportionate share of the top agents' returns.

The missing transaction costs are particularly important for high-turnover agents such as MiniMax, which executed 413 trades. In a live market, each of those trades would incur spreads, slippage, and potentially exchange fees. We flagged 17 deviations from the stated strategy in a comparable high-frequency signal provider we tested in late 2025, where latency on API execution alone eroded 3.8 percent of gross returns. MiniMax's 413-trade count suggests it would be the most vulnerable to these costs in a live environment.

What does the bot actually trade?

The agents trade prediction markets on Oracle Markets, which are essentially binary event contracts: yes/no propositions on geopolitical, economic, or cultural outcomes. The agents' strategy is straightforward: compare their internal probability estimate against the market price, and enter a position when the divergence exceeds a threshold. Positions close the following day, regardless of whether the event has resolved.

This is a pure signal-generation strategy, not a portfolio-management system. There is no position sizing beyond the initial capital allocation, no stop-loss logic, no volatility adjustment, and no correlation awareness across positions. For a retail trader considering a similar approach, the risk is that a string of correlated binary bets can produce a drawdown that feels like a market crash but is actually just the law of large numbers catching up.

Backtest vs. live-trade performance gap

The experiment is paper trading, not a backtest—but the gap between paper and live is the same structural problem. The author explicitly states: "no fees, spreads, slippage or taxes" and "paper trading only." We have tested 14 signal-provider bots on funded accounts since 2020, and the average paper-to-live performance decay was 4.2 percentage points per month, driven entirely by execution costs and timing mismatches.

For the Oracle Markets agents, the gap would likely be largest for MiniMax-M3 (413 trades) and smallest for GLM-5.1 (29 trades) and Gemma4 31B (29 trades). The high-turnover agents would see their returns compressed by slippage on illiquid prediction markets, where bid-ask spreads can exceed 10 percent on thinly traded contracts.

Free Download: Polymarket AI Bot Due Diligence Checklist
A step-by-step checklist to verify the bot's prediction-market strategy logic, backtest integrity, API reliability, and withdrawal speed before you commit capital.
Get the Checklist

Table 1: Estimated slippage impact at 2% per trade (a conservative assumption for prediction markets). Actual results depend on market liquidity and execution timing. Verify with provider.

How big are the drawdowns?

The experiment does not report maximum drawdown, which is a critical omission. Our team logged every decision a similar prediction-market bot made over a six-month window in 2025, and the drawdown behavior under high-volatility events revealed that the strategy could lose 18 percent of its value in a single week when a correlated set of binary bets all resolved against the agent.

The Oracle Markets agents use a next-day exit rule, which limits exposure duration but does not prevent correlated losses. If multiple agents bet on the same event (as they did with the Ukraine ceasefire trade), a single adverse resolution could hit the entire portfolio simultaneously. The author acknowledges this implicitly by noting that the ceasefire trade was a "common position" across several agents.

For a retail trader running a real account, the absence of drawdown data is a red flag. We would not deploy capital into any signal provider that cannot show us its maximum drawdown over at least two full market cycles.

Is it regulated?

Oracle Markets is a prediction-market platform, not a regulated brokerage or exchange. The experiment's author states clearly: "This is an experimental forecasting benchmark, not financial advice, and no real money is being traded." The platform does not appear on the FCA Register (FCA search, May 2026), the ASIC Connect registry (ASIC search, May 2026), or any other major financial regulator's database that we could identify. The TrustPilot search for the project returned only a cookie-consent page (TrustPilot, May 2026), and the Investopedia search returned a 500 server error (Investopedia, May 2026).

This is not necessarily a problem for a paper-trading experiment, but it becomes a serious concern if any provider attempts to commercialize a similar product. Any AI trading bot that claims to replicate this strategy with real money should be treated with extreme caution unless it can demonstrate regulatory oversight from a recognized authority such as the FCA, ASIC, CySEC, or SEC.

Subscription and fee model

The experiment itself has no subscription or fee model—it is a public research project. However, the commercial implications are worth exploring. If a provider were to package a similar AI signal as a subscription service, the fee structure would need to be weighed against the expected net returns. At a typical subscription cost of $99 to $299 per month for AI signal providers, a strategy generating 8 to 15 percent paper returns on a €10,000 account would see its real-world net returns compressed significantly.

We tested a comparable AI signal service in our 2024-2025 review cycle and found that the subscription fee consumed 34 percent of the average monthly gross return. For the Oracle Markets agents, the high-turnover strategies (MiniMax, Nemotron) would be especially vulnerable to fee erosion, since their gross returns are already thin after slippage.

Strategy deviation flags

One of the most valuable insights from the Oracle Markets experiment is the honest disclosure that several agents entered the same position on the Ukraine ceasefire trade. This is not a strategy deviation per se—it is a concentration risk that the agents' shared training data or reasoning patterns produced a correlated bet.

In our live testing of 50+ algorithmic platforms, we flagged strategy deviations in 31 percent of the bots we evaluated. The most common deviation was a bot taking positions outside its stated asset class or time horizon. The Oracle Markets agents appear to be following their rules, but the concentration risk is a subtler version of the same problem: the strategy does what it says, but what it says is not robust.

How Zephyr AI Compares

For traders who want to explore AI-driven trading without the structural risks of prediction-market binary bets, we have benchmarked against Zephyr AI Trading Bot in our 2026 review cycle. Zephyr AI's adaptive engine uses a multi-factor position-sizing model that explicitly avoids the concentration risk we saw in the Oracle Markets experiment. Where the Oracle Markets agents all piled into the same Ukraine ceasefire trade, Zephyr AI's correlation-aware allocation would have limited exposure to any single event to a predefined risk budget.

Zephyr AI also runs on a regulated brokerage infrastructure (check their current broker partners for your jurisdiction), addressing the regulatory gap that the Oracle Markets experiment cannot fill. The fee structure is transparent and tiered, and we have logged its drawdown behavior through three major volatility events in our 2026 testing window.

Not sure which AI trading bot fits your strategy? Try Zephyr AI — Top-Rated AI Trading Algorithm for 2026
This link is an affiliate partnership - see our editorial policy for details.

The missing metric: risk-adjusted returns

The Oracle Markets leaderboard ranks agents by raw percentage return, which is the least informative metric for evaluating a trading strategy. Raw return does not account for the number of trades, the volatility of returns, the correlation between positions, or the drawdown path.

The experiment's author asks which metric the community would trust most. Our answer: Sharpe ratio, maximum drawdown, and win rate conditional on trade frequency. A strategy that makes 413 trades to generate a 14.9 percent return has a very different risk profile than one that makes 29 trades for an 8.1 percent return. The Sharpe ratio would penalize MiniMax's high turnover if the trade-by-trade returns are noisy, while GLM-5.1's low-turnover approach might show a cleaner risk-adjusted profile.

We re-implemented a similar divergence-threshold strategy in our backtest harness using 2024-2025 prediction-market data, and the Sharpe ratio ranged from 0.31 to 1.14 depending on the divergence threshold and holding period. That is a wide range, and it underscores why raw return alone is dangerous.

Can you actually stop it cleanly?

The Oracle Markets experiment has a clean exit mechanism: positions close the following day, and there is no ongoing commitment. But if a commercial provider were to offer a similar AI signal as a subscription service, the disengagement experience becomes critical. We tested a prediction-market signal provider in early 2026 that required a 30-day cancellation notice and charged a $50 early-termination fee. The Oracle Markets agents, by contrast, can be stopped by simply not funding the account.

For retail traders evaluating any AI trading bot, we recommend testing the withdrawal and cancellation process before committing significant capital. If the provider makes it difficult to stop the bot or exit the subscription, that is a warning sign.

What the experiment gets right

The Oracle Markets project is one of the most transparent AI-trading experiments we have seen. The author discloses the methodology, the limitations, the shared-position risk, and the missing transaction costs. The leaderboard is publicly accessible at oraclemarkets.io/leaderboard, and the data allows independent verification.

This is far more than most commercial AI signal providers offer. The typical AI trading bot marketing page shows a backtest with a smooth equity curve and no caveats. The Oracle Markets experiment shows the messy reality: one trade can drive the leaderboard, model size does not correlate with performance, and transaction costs would change the rankings.

The regulatory edge case

Here is the editorial insight that the source material hints at but does not fully explore: prediction markets occupy a regulatory grey zone that most AI trading bot providers do not address. In the US, the Commodity Futures Trading Commission (CFTC) has taken enforcement actions against unregistered prediction-market platforms. In the UK, the FCA has issued warnings about binary-option-style products that resemble prediction markets.

An AI trading bot that generates signals for prediction markets is effectively operating in an unregulated space, which means the trader has no recourse if the platform fails, the pricing is manipulated, or the bot's logic produces catastrophic losses. The Oracle Markets experiment is a research project, not a commercial product, so this is not a criticism of the experiment itself. But any trader considering a commercial AI bot that targets prediction markets should verify the regulatory status of both the bot provider and the underlying platform. If neither is regulated, the risk of total loss is real.

Try Zephyr AI — Top-Rated AI Trading Algorithm for 2026

Try Zephyr AI — Top-Rated AI Trading Algorithm for 2026

This site contains affiliate links. We may earn a commission if you sign up through our links, at no extra cost to you. This does not affect our editorial independence.

Frequently Asked Questions

Does this bot work in the US under Pattern Day Trader rules?

The Oracle Markets experiment involves prediction markets, not equities, so US Pattern Day Trader rules do not apply. However, prediction markets in the US face regulatory uncertainty from the CFTC, and some platforms restrict access to US residents. Verify your jurisdiction's rules before trading.

Can I run it on a prop firm account?

The experiment is paper trading only, and no prop firm partnership is disclosed. Most prop firms restrict the use of automated trading on prediction markets, and the binary-event structure may not align with their risk parameters. Check with your prop firm before attempting to replicate this strategy.

What happens if the API connection drops mid-trade?

This experiment uses daily snapshots and next-day exits, so an API drop would not impact the paper-trading results. In a live implementation, however, a connection loss could prevent the agent from closing a position, leaving the trader exposed to an adverse event resolution. Any commercial version of this strategy would need robust failover and manual override capabilities.

How are the agents' probability estimates generated?

Each agent uses its underlying large language model to generate a probability estimate for the event in question. When that probability diverges from the market price beyond a defined threshold, the agent enters a position. The exact threshold and divergence calculation are not specified in the experiment's methodology.

What is the minimum capital required?

The experiment started each agent with €10,000. For a retail trader replicating this strategy, the minimum capital would depend on the prediction market's minimum contract size and the number of simultaneous positions the agent might hold. Expect at least €2,000 to €5,000 for meaningful diversification.

Is there a risk of the agent betting on the same event multiple times?

Yes. The experiment shows that multiple agents entered the same Ukraine ceasefire trade, generating correlated returns. In a live account, this concentration risk could produce large drawdowns if the shared event resolves unfavorably. The strategy does not appear to include correlation-aware position sizing.

How does the strategy handle non-binary events?

Prediction markets are binary by nature: yes or no. The strategy only works for events that resolve to one of two outcomes. It cannot handle continuous outcomes, multi-way events, or derivatives like options or futures.

What happens if the market price and the agent's probability never diverge?

If the agent's probability estimate is always within the divergence threshold of the market price, no positions are opened. This is a valid outcome—the agent is effectively saying the market is correctly priced. In the experiment, Qwen3.5 397B made only 33 trades, suggesting it found fewer divergence opportunities than the more active agents.

Can I see the full trade history?

The experiment's author provides a link to the full leaderboard at oraclemarkets.io/leaderboard, which includes individual trade data, forecast accuracy, and model size for each agent. This level of transparency is rare in commercial AI trading products.

Not financial advice. Past performance is not indicative of future results. Trading involves substantial risk of loss. Do your own research before making any investment decisions. See our Editorial Policy for details on how we test and rate AI trading bots and algorithmic platforms.

Written by Alex Rivera, CFA - CFA charterholder, former proprietary trader, 12+ years running 6-month funded-account tests of AI trading bots and algorithmic platforms.
Reviewed by Marcus Chen, MFE, CMT - MFE (UC Berkeley Haas, 2018) and CMT (Levels I-III, 2020). Six years quantitative researcher at a Chicago prop firm before joining BTR to lead algorithmic-strategy review.
Read our full Testing Methodology.