Does prompt format matter? We tested XML, YAML, JSON, Markdown and plain English.

By Mirte Pruppers, project leader at Rewire.

For years, prompt engineering advice has followed a familiar pattern:

Simple prompts? Use plain language.
Complex prompts? Use structured formats like XML or YAML.

The reasoning seems intuitive. As instructions become more hierarchical, nested, and rule-heavy, a format with explicit structure should help the model understand the relationships between different parts of the prompt.

XML tags look like they should make things clearer:

1
2
3
4
5
6
7

<rules>
    <policy>
        <condition>
        ...
        </condition>
    </policy>
</rules>

Compared to:

1
2
3
4
5

## Rules

### Policy

- If the condition is...

The XML version feels more precise. But there is a hidden cost: every tag is a token. And tokens are money. So we tested a simple question:

Does the additional structure provided by XML, YAML, and JSON actually justify the extra token cost?

And so we ran an experiment.

The experiment

We compared six different prompt formats:

• Plain English
• Markdown
• XML
• YAML
• JSON
• Key-value format

The goal was not to find a universal "best prompt format". Instead, we wanted to isolate one specific question:

How much does the representation format itself affect token usage and reliability?

To keep the comparison fair:

• The underlying instructions were identical.
• Only the formatting style changed.
• The same model was used throughout.
• The same tasks were evaluated.

The benchmark used:

• Model: gpt-4o-mini
• Three tasks with increasing complexity
• 10 examples per task and format

The tasks were:

1. Simple classification
   A straightforward classification task with a small number of rules.
2. Complex policy compliance
   A hierarchical policy evaluation task with multiple categories and nested decision rules.
3. Expense anomaly reasoning
   A multi-step reasoning task requiring the model to identify suspicious expense patterns.
4. Bulk expense audit handbook
   A substantially larger prompt, included in our token benchmark.

The expectation was:

• Simple tasks → plain English should perform well.
• Complex hierarchical tasks → XML/YAML might justify the overhead.

The data did not support that assumption.

The results

Finding 1: XML's structure consistently used more tokens

The strongest pattern from the benchmark was simple: XML used more tokens every time.

Across all three scenarios, XML was the most expensive option. The most interesting result was the second experiment: policy compliance complexity.

This was intentionally designed as the scenario where XML should shine. The task involved:

• nested rules
• multiple policy categories
• conditional logic
• hierarchical instructions

If there was ever a case where XML's explicit structure should have paid off, this was it. Well, it didn't. XML consumed 36% more tokens than the cheapest alternative. And it did not provide a reliability improvement.

Finding 2: Plain English and Markdown were the unexpected winners

The most consistent performers were not the heavily structured formats. They were the simplest ones:

• Plain English
• Markdown

Across all three tasks, these formats were effectively tied for lowest token usage. More importantly, they did not sacrifice reliability.

In two out of the three experiments, English and Markdown achieved the same 100% success rate as XML. This suggests something important about modern language models: the model already understands structure.

A human may look at:

1
2
3
4
5

<instruction>
    <rule>
        Reject claims above $500
    </rule>
</instruction>

and think: "That structure makes the relationship clearer."

But the model may simply see additional tokens describing a relationship it could already infer from:

1
2
3
4
5

## Instruction

### Rule

Reject claims above $500.

The explicit tags are not necessarily adding intelligence. They are adding text.

Finding 3: JSON was the most ambiguous result

JSON produced the most complicated results. It was generally middle-of-the-pack for token usage. However, its reliability numbers were less impressive in some tests, particularly the complex policy task. But this result needs caution: a benchmark is only as good as its measurement.

The two 0% success results are suspiciously clean. A true failure rate of 100% is possible, but it is also possible that the validator was too strict. For example, the model might return:

1
2
3

{
  "decision": "approve"
}

inside a Markdown code block:

1
2
3
4
5

```json
{
  "decision": "approve"
}
````

A strict parser using:

would reject this. But a human would likely consider the response correct. So the conclusion is not:

JSON is unreliable.

Instead, the more accurate conclusion is:

JSON formatting results require better validation before making strong claims.

This is actually one of the most interesting follow-up areas for the experiment.

What does this mean for prompt engineering?

The three practical takeaways are simpler than expected.

#1. Use Markdown or plain English by default.

For system prompts:

• clear headings
• bullet points
• short sections
• explicit instructions

are probably the best default. For example:

1
2
3
4
5
6
7
8
9

## Role

You are an expense review assistant.

## Rules

- Never invent missing information.
- Flag suspicious expenses.
- Explain your reasoning.

This style is:

• cheap
• readable
• easy to maintain
• understood well by modern models

#2. Do not use XML/YAML just because the prompt is complex.

Complexity alone is not a reason to add more syntax. A common assumption is that more structure helps the model reason.

Our results suggest a more nuanced view: More structure helps humans organize information, but the model may not need that structure expressed as additional tokens.

The model already learned patterns from enormous amounts of structured and unstructured text. Adding XML tags may simply duplicate information that was already obvious.

#3. Use structured formats when the output needs structure.

This does not mean XML, YAML, and JSON are useless. They are still valuable when the output needs to be consumed by another system. Examples:

• API responses
• database inserts
• workflow automation
• tool calls

If your application requires:

1
2
3
4

{
  "customer_id": 123,
  "risk_score": 0.87
}

then JSON is the right choice.

The lesson is not:

"Never use structured formats."

The lesson is:

"Do not pay the token cost of structured input unless it provides a real benefit."

Limitations

This experiment has at least three important limitations.

#1. One model

• The benchmark only used: gpt-4o-mini
• Different models may behave differently.
• A larger reasoning model may benefit more from certain structures.
• A smaller model may struggle more with ambiguous formatting.
• More models are needed before making broader claims.

#2. Small sample size

Each test used:

• 3 tasks
• 10 examples per format

The XML pattern is interesting because it appeared consistently across all three tasks. However, smaller differences between formats should not be overinterpreted.

#3. Reliability measurement

The benchmark measured whether outputs passed validation. That is useful, but incomplete. Future experiments should include:

• human evaluation
• semantic similarity scoring
• LLM-based grading
• reasoning quality assessment

A format could cost more tokens but produce better answers. Cost alone is not the whole story.

The bigger lesson

The most interesting result was not that XML was expensive. Everyone expected XML to use more tokens. The surprising part was this:

The extra tokens did not buy better results.

At least for this benchmark, the additional structure from XML did not improve reliability enough to justify its cost. A 30% token premium barely registers when testing ten requests. It looks different in an application serving thousands or millions of them. Our benchmark gives us no reason to routinely pay that premium for XML. Markdown and plain English were cheaper, easier to work with and, in these tests, just as reliable.

That is enough for us to keep things simple until the data tells us otherwise.

Agents reward organisations that ask harder questions first.

The organisations getting compounding value from agentic AI share one trait: they invested in getting the foundations right before scaling.

We've built agent systems across financial services, energy, telecom, and public sector, from early proofs of concept to production systems running at scale. We've seen what separates the deployments that deliver from the ones that don't.

Let's find the answers for your situation