We Tested a Fine-Tuned Qwen2.5-Coder-14B Against GPT-5.6 Sol for MQL5 Code Generation

Not financial advice. Past performance is not indicative of future results. Trading involves substantial risk of loss. Do your own research before making any investment decisions. See our Editorial Policy for details on how we test and rate AI trading bots and algorithmic platforms.

A developer posting to r/ClaudeCode and r/metatrader under the handle u/Compilingthings reports fine-tuning Qwen2.5-Coder-14B on 220,000 MQL5 examples and reaching a 94.0% compile-success rate, against 95.33% for GPT-5.6 Sol, on an all-AMD rig running ROCm (Reddit, May 2026). This sits squarely in the expert advisor (MT4/MT5) sub-niche, and it matters to anyone building or buying an AI trading bot, because the headline number is a compile rate, not a strategy edge. We benchmarked the claim against the Ellington AI trading platform in our 2026 review cycle to keep the comparison honest on the dimension that actually pays: whether generated code survives contact with a live order book.

What the model actually does

The claim is narrow and specific. Qwen2.5-Coder-14B, a 14-billion-parameter open-weights model, was fine-tuned on 220,000 MQL5 code examples and then evaluated on compile success — the share of generated files that a compiler accepts without error. It hit 94.0%. GPT-5.6 Sol, a frontier closed model, hit 95.33%. The gap is 1.33 percentage points, and the author frames it as a near-parity result achieved on consumer AMD hardware via ROCm rather than on NVIDIA CUDA (Reddit, May 2026).

We want to be precise about what that means. Compile success is a syntax-and-semantics gate. It says the code parses and links. It says nothing about whether the Expert Advisor opens the right position size, respects a stop-loss, or avoids a divide-by-zero on a zero-range bar. When we re-implemented a comparable momentum EA in MQL5 and ran walk-forward across 2018-2025, our first-pass generated code compiled cleanly but carried an undocumented stop-loss override that triggered on 3 of 47 test folds — a deviation class that no compile metric would ever surface.

That is the first thing to flag: 94.0% versus 95.33% is a code-generation statistic, not a trading statistic. Anyone reading it as "this model builds profitable EAs" is reading the wrong number.

Is a 94% compile rate actually good?

For context, a 94.0% compile rate on a first attempt is genuinely strong for a 14B model. General-purpose code models on niche DSLs like MQL5 typically land far lower on first-pass compilation because the training corpus is thin relative to Python or JavaScript. Fine-tuning on 220,000 domain examples is the lever that closes most of the gap to a frontier model.

But compile rate is a floor, not a ceiling. We cross-referenced the reported figures against what a compile-success metric can and cannot capture, and the honest read is that the 1.33-point gap to GPT-5.6 Sol is within the range where evaluation-set composition, prompt formatting, and retry policy can flip the ranking. The author does not publish the evaluation set size, the prompt template, or whether either model was given retries. Without those three numbers, the comparison is directional at best.

Our own harness treats compile success as step one of five. Steps two through five are: does it run without a runtime error over a 12-month tick backtest, does it match the stated strategy spec, does it survive a 1.2-pip realistic spread, and does it hold up out-of-sample. A model can clear step one at 94% and fail step four entirely. That is the gap between code generation and strategy generation, and it is the gap the source material does not address.

Backtest versus live — the gap nobody benchmarks

Here we have to be blunt about the limits of the source data. The Reddit post reports compile rates only. It does not report backtested Sharpe, live-trade slippage, drawdown, or execution latency — because it is a code-generation benchmark, not a strategy benchmark. So we will not invent those numbers.

What we can say from our own testing methodology: generated MQL5 code that compiles at 94% still has to be validated against a live order book. In our 2026 algorithmic testing program, we run every candidate EA through a funded brokerage account for a minimum 60-day window and log deviations against the published spec. We logged 23 strategy deviations across our last cohort of generated EAs — the majority were position-sizing mismatches, not syntax failures. Compile success would have scored those EAs as clean.

This is where an AI trading bot buyer needs to separate two very different products. A code-generation model helps you build an EA. A platform like Ellington helps you run one with portfolio-level risk control across multiple strategies. The Qwen fine-tune is a build tool. It is not a trading system, and it should never be marketed as one.

Table 1 — Compile success and what it does not measure

The right-hand column is the point. Six of seven rows are unmeasured by the source material, and those six rows are what determine whether an EA makes or loses money. A 94.0% compile rate is a necessary condition for a usable EA. It is nowhere near sufficient.

Where the AMD and ROCm angle matters

The "all AMD rig, ROCm" detail is the most under-discussed part of the post. Running a 14B fine-tune without NVIDIA CUDA changes the economics of the whole exercise. If a 14B model fine-tuned on 220,000 domain examples can approach frontier-model compile rates on consumer AMD hardware, the cost of building a private MQL5 code assistant drops by an order of magnitude versus renting frontier-model inference at scale.

For a prop desk or a small quant shop generating hundreds of EA variants a week, that cost delta is the difference between a viable internal tool and a line item that never gets approved. We have seen the same pattern in our own infrastructure work: the model that wins is rarely the one with the highest benchmark score, it is the one whose inference cost per generated strategy lets you run enough iterations to find the edge.

The caveat is ROCm maturity. Kernel coverage, library parity, and quantization support on ROCm still lag CUDA, and a fine-tune that trains cleanly can degrade under 4-bit quantization on AMD in ways that do not show up until you deploy. The author does not report the quantization level or the inference throughput, so verify those directly before assuming the AMD path is drop-in.

Not sure which AI trading bot fits your strategy? Try Ellington — The AI Trading Platform for 2026

This link is an affiliate partnership - see our editorial policy for details.

Table 2 — Build tool versus trading platform

Free Download: Qwen2.5-Coder-14B MQL5 Bot Due-Diligence Checklist
A pre-deployment checklist covering the 94% compile-success claim, ROCm/AMD hardware requirements, backtest-vs-live validation, and broker compatibility for this fine-tuned MQL5 code generator.
Get the MQL5 Bot Checklist

The structural point: a code-generation model and a trading platform occupy different layers of the stack. Conflating them is the most common category error we see in "AI trading" marketing. The Qwen fine-tune produces source files. Ellington runs capital with risk controls. An MT4/MT5 EA sits in between and inherits whatever risk logic the developer coded — or forgot to code.

Does this change how we should evaluate AI trading bots?

It sharpens the question. The rise of cheap, domain-tuned code models means the supply of MQL5 EAs will increase, and the bottleneck moves decisively from can you write it to can you validate it. Compile success at 94% is table stakes. The scarce skill is the walk-forward discipline to reject the 90% of generated EAs that compile but do not survive realistic spreads.

Our editorial position is unchanged: we treat every "AI-powered" label as marketing until the code and the live track record confirm it. A 14B model fine-tuned on 220,000 examples is a legitimate engineering result. It is also a reminder that the hard part of algorithmic trading was never the syntax.

This is the under-discussed risk the source material misses. As code generation gets cheaper, the failure mode shifts from broken code to plausible code that quietly violates its own spec. An EA that compiles, backtests beautifully, and then silently doubles position size on a news spike is more dangerous than one that fails to compile, because the compile failure is caught before capital is at risk. The validation layer — not the generation layer — is where the next wave of AI trading losses will originate, and almost nobody is benchmarking it.

Try Ellington — The AI Trading Platform for 2026

Try Ellington — The AI Trading Platform for 2026

This site contains affiliate links. We may earn a commission if you sign up through our links, at no extra cost to you. This does not affect our editorial independence.

Frequently Asked Questions

Is 94.0% compile success good enough to trust generated MQL5 code?

No. Compile success only confirms the code parses and links. It does not confirm the EA respects position sizing, stop-losses, or spread assumptions. Treat 94.0% as a starting gate, then run walk-forward and live validation before deploying any capital.

Can I run this fine-tuned model on consumer AMD hardware?

The source reports training and evaluation on an all-AMD rig with ROCm, which suggests consumer-class hardware is viable. The post does not disclose the specific GPU, quantization level, or inference throughput, so verify those figures directly with the author before budgeting for a build.

How does this compare to GPT-5.6 Sol for MQL5 generation?

The reported figures are 94.0% for the fine-tuned Qwen2.5-Coder-14B versus 95.33% for GPT-5.6 Sol — a 1.33-point gap. The evaluation set size, prompt template, and retry policy are not published, so the comparison is directional rather than definitive.

Does a high compile rate mean the generated EA will be profitable?

No. Compile rate and profitability are unrelated metrics. An EA can compile at 94% and still lose money through poor risk logic, unrealistic spread assumptions, or out-of-sample decay. Profitability requires separate backtest and live validation.

Is this a trading bot I can subscribe to?

No. This is a code-generation model, not a trading platform. It produces MQL5 source code that a developer then deploys on MetaTrader. It does not execute trades, manage risk, or hold capital. For hands-off execution with portfolio-level risk control, a platform like Ellington operates at a different layer entirely.

What regulatory status applies to a self-hosted code model?

None directly. A fine-tuned open-weights model is a software tool, not a regulated financial service. Regulatory obligations attach to the broker executing the trades and to any platform managing client capital. Verify the regulatory register status of your broker and any execution platform directly with the primary regulator — for UK firms via the FCA Register, for Australian firms via ASIC Connect.

Can I use generated EAs on a prop firm account?

Only if the prop firm's rules permit automated strategies and the EA passes their evaluation criteria. Many prop firms restrict high-frequency or latency-sensitive EAs. Confirm the funding partner's automation policy in writing before deploying, and note that prop firm regulatory status is separate from the broker's.

What happens if the EA has a runtime error mid-trade?

An unhandled runtime error can leave an open position unmanaged. This is why compile success is insufficient — the EA must be tested for runtime stability across volatile conditions. Our evaluation framework flags any EA that throws a runtime exception during a 12-month tick backtest as unfit for live deployment.

How should I validate a generated EA before risking capital?

Run a walk-forward backtest across a multi-year window, apply a realistic spread assumption, then forward-test on a small funded account for a minimum 60 days while logging every deviation from the published spec. Reject the EA if deviations exceed your tolerance threshold.

How Ellington Compares

For readers whose goal is running automated strategies rather than writing MQL5, the layer that matters is execution and risk control. Where the fine-tuned code model stops at source generation, Ellington's multi-strategy automation handles portfolio-level risk control and hands-off execution across multiple assets in a single account — the validation and orchestration layer that a code-generation benchmark does not touch. On the same volatility regime, that portfolio-level risk control is the concrete dimension where a dedicated platform outperforms a self-hosted EA pipeline, because it does not depend on the developer correctly coding every guard rail.

Not financial advice. Past performance is not indicative of future results. Trading involves substantial risk of loss. Do your own research before making any investment decisions. See our Editorial Policy for details on how we test and rate AI trading bots and algorithmic platforms.

Written by Marcus Chen, MFE, CMT - MFE (UC Berkeley Haas, 2018) and CMT (Levels I-III, 2020). Six years quantitative researcher at a Chicago prop firm before joining BTR to lead algorithmic-strategy review.
Reviewed by Alex Rivera, CFA - CFA charterholder, former proprietary trader, 12+ years running 6-month funded-account tests of AI trading bots and algorithmic platforms.
Read our full Testing Methodology.