Blog
Does the format of a system prompt actually change what an LLM costs or how well it answers? We ran 6,300 requests through six prompt formats to find out.
By Mirte Pruppers, project leader at Rewire.
In short: we gave two OpenAI models the same instructions in six formats, from plain English to XML. XML made each request about 16% larger and 11–12% more expensive than Markdown, without a measurable gain in accuracy. The difficulty of the task mattered far more than the format.
Why we looked into prompt format
Open almost any production prompt that has grown beyond a few lines and you will find structure: XML tags around the rules, YAML blocks for the policy, JSON for the reference data. The common advice is that complex prompts belong in a structured format, and many teams follow it without a second thought.
We review a lot of prompts, and the same question keeps coming up: does the format of a system prompt actually change what an LLM costs or how well it answers? A second question usually follows. Does wrapping inserted content in XML tags, Markdown headings or JSON help a model keep data apart from instructions, especially when that data contains instructions of its own?
We didn't have a firm answer ourselves, so we set up a controlled test to see what happens in one realistic setting.
How we tested it
We took a task many companies automate: checking expense claims against a written expense policy. We then wrote the same instructions in six formats: plain English, Markdown, XML, YAML, JSON and simple key-value pairs.
Only the format changed. A program generated all six versions from one source and checked that every rule, amount and name was identical. Everything else the model saw was exactly the same.
Easy and hard tasks. We built ten tasks, from a single approve-or-reject verdict to auditing batches of expenses and spotting unusual spending. The hard versions added the detail real policies have: around two dozen rules, foreign currencies, deadlines and exceptions.
A known right answer for every case. A program computed the correct answer for each example, so we could score the models objectively, including borderline cases such as exactly $500 versus $500.01.
Two OpenAI models, about 6,300 requests. Most went to the smaller, cheaper gpt-4o-mini (5,600 requests). We also ran the larger gpt-4.1 on a smaller sample (728 requests) to see whether the picture held.
Fair comparisons. Every example went through all six formats, so we compared the same question in six versions rather than different questions. And we scored "is the answer right?" separately from "is the output neatly formatted?", because a strict format check can make a correct answer look wrong.
Costs are based on the token counts the provider reported and its list prices.
In our test, XML used about 16% more input tokens
This is the clearest result, because token counts are exact. The same instructions written in XML made each request about 16% larger than in Markdown.
| Format | Input size vs Markdown | Cost per 1M requests, gpt-4o-mini | Cost per 1M requests, gpt-4.1 |
|---|---|---|---|
| English | −8.7% | $184 | $2,491 |
| YAML | −4.5% | $189 | $2,589 |
| Key-value | −3.7% | $192 | $2,619 |
| Markdown | reference | $195 | $2,660 |
| JSON | +4.0% | $200 | $2,760 |
| XML | +15.8% | $216 | $2,972 |
Costs are averaged over the eight main tasks, at list price, without prompt caching.
Input size and cost per format compared with Markdown, averaged over the eight main tasks. The lighter cost bars assume prompt caching and are not part of our conclusions.
In money, the gap is real but modest. Moving from Markdown to XML adds about $21 per million requests on gpt-4o-mini and $312 on gpt-4.1, an increase of 11 to 12%. For a small application that is negligible; at high volume or on a more expensive model it becomes noticeable. Plain English was the cheapest format in our test.
We found no format that improved correctness
So XML costs more. Does it buy better answers? On our tasks, we saw no measurable improvement from XML or from any other format.
With gpt-4o-mini, Markdown and XML both answered 71% of cases correctly. YAML scored 70%, and plain English, key-value and JSON 69%. Every format landed within about 2 percentage points of Markdown.
gpt-4o-mini, 800 examples per format. The vertical bars show the range the true score plausibly lies in; all formats overlap with Markdown.
None of these differences is larger than chance could explain. Here is what we can, and can't, conclude from that:
- Our data rule out an XML advantage above roughly 2 points for this model. If XML helps at all here, the effect is smaller than that.
- Markdown was never measurably worse than any other format. That doesn't mean Markdown improves answers. It means that in our test, choosing it cost nothing measurable.
The larger gpt-4.1, tested on a smaller sample, showed the same picture: every format within about 2 points of Markdown.
Format compliance was not an issue either. Both models returned output in exactly the requested form 100% of the time, in every format, even under strict scoring.
In our test, task difficulty mattered far more than format
In our results, the task made a much bigger difference than the syntax. Depending on the task, gpt-4o-mini scored anywhere from roughly 50% (the hardest anomaly task) to 81% (the simplest classification). On any single task, the spread between formats was about ±2 points.
gpt-4o-mini, correctness per task. The differences between tasks are far larger than the differences between formats within a task.
A few individual combinations of task and format did look different, but in no consistent direction. That is what chance alone would produce across this many comparisons.
One more observation for anyone building audit-style applications: models often reach the right verdict but cite different rule IDs. On the policy tasks, the cited rule IDs matched exactly only 27% of the time. If your application depends on the model explaining why, test that separately.
Delimiters: no difference in correctness, and only a hint on injections
The second question was about inserted content. We wrapped the data in XML tags, Markdown headings, JSON, or nothing at all. In half of the examples, we hid an instruction-like text inside the data, such as "ignore previous instructions". It also asked the model to include a secret code word, so we could detect when the model obeyed.
Correctness was the same with every delimiter for gpt-4o-mini, with differences of 1 point or less.
Injections occasionally worked in the batch-audit task. The model obeyed the injected instruction in 17 of 176 injected batches (9.7%): 11% with XML, 11% with Markdown, 2% with JSON and 14% with no delimiter.
JSON looked best here, but the numbers are small (1 case out of 44, against 6 out of 44 without a delimiter), and we only noticed the pattern afterwards. Treat it as a hint worth testing, not as protection you can rely on.
In a second, simpler task, the model never followed an injected instruction, whatever the format.
gpt-4o-mini. Top: correctness per delimiter. Bottom: how often the model followed the hidden instruction. The wide bars reflect the small number of injected cases.
What we would try next
- Consider Markdown or plain English as a starting point. In our test they were among the cheapest formats, and no format did better on correctness.
- Don't switch to XML or YAML only because a prompt is complex. We saw no accuracy gain from it; clearer rules may help more than different syntax.
- Use structured formats where a machine reads the result. When the output must be parsed, ask for JSON. That is a different job from decorating the input.
- Test on your own task with gold answers. Our results come from one setting. A few hundred examples with known correct answers will show what holds for yours.
Limitations
- Two OpenAI models only. Other model families may behave differently, and we have not checked.
- "No difference found" is not "proven equal". For gpt-4o-mini, any real difference is likely smaller than about 2 percentage points. The gpt-4.1 sample was small, so its results are less precise.
- Our own tasks. We used synthetic expense-audit tasks designed for this test. Results on your tasks may differ.
- A small injection test. Each format had only a small number of manipulated cases.
- Token counts depend on the model. Compare formats within one model, not across models.
What we take from it
This was one test, on our own tasks and two OpenAI models. It doesn't show that XML is bad or that format never matters. It shows that here, XML used more tokens without a measurable gain in correctness.
In our test, the extra tokens didn't buy extra accuracy. Until your own data says otherwise, simple seems a sensible default.
Wondering whether your prompts, models or evaluation set-up hold up on your own tasks? Let's find the answers for your situation.
Agents reward organisations that ask harder questions first.
The organisations getting compounding value from agentic AI share one trait: they invested in getting the foundations right before scaling.
We've built agent systems across financial services, energy, telecom, and public sector, from early proofs of concept to production systems running at scale. We've seen what separates the deployments that deliver from the ones that don't.
