The Mittelstand – the dense network of small and medium-sized enterprises widely regarded as one of the defining strengths of the German economy – is built on world-class craft and specialization. Many Mittelstand companies are experimenting with AI. The most successful ones treat it not as a project, but as a capability they build themselves. That way they win twice: operational impact in the short term, independence in the long term. Their experience holds lessons for companies of all sizes — in Germany and beyond.

In many mid‑sized companies today, a similar picture emerges: several AI initiatives run in parallel, often for years, without being successfully absorbed into daily operations. At the same time, every one of these companies has pockets where AI quietly but reliably creates value. The difference between the two rarely lies in the tool, but in the approach.

Companies that generate tangible value with AI rarely have the fastest technology or the largest budget. They have something else: a clear view of where AI truly changes something in their processes, and the discipline to start exactly there.

The decisive question is therefore less “Which AI tool should we buy?” and more: How do we build the capability to integrate AI into our work for the long term and steer it ourselves? Those who answer this question win twice: operational effects visible in the first months, and a resilient independence that pays off year after year.

When AI really delivers in the Mittelstand

Before turning to the typical pitfalls, it is worth looking at what demonstrably works. Out of hundreds of projects, three factors consistently come together in successful AI applications.

First: a clear goal inside the process. Which concrete decision or action should AI make better, faster, or more reliable? “Cut the time from customer enquiry to finished quote in half so that sales can respond faster and with greater accuracy” is a goal teams can tackle immediately. And one against which success can be measured.

Second: the right data, understandable and reliable. The Mittelstand typically has rich operational data from ERP, MES, CRM, lab, and field systems. The leverage appears when the data points relevant to the specific use case are cleanly defined and stably available. That is usually far more focused and reachable than the much‑invoked “data cleanup across the whole company.”

Third: results where the decision is made. For classic operations use cases such as maintenance, quality, or planning, this usually means: directly in the ERP, MES, or CRM, i.e. where people are already working. For agentic applications in sales, procurement, technical support, or medical affairs, by contrast, the point of decision often sits outside the core systems: in the email inbox, the ticketing system, the document repository. Good agentic AI orchestrates those sources, prepares the decision, and flows the result back into the ERP or CRM. The rule is therefore not “always into the core system,” but: wherever the human decides.

Where these three elements come together, impact emerges that is measurable and scalable. That is not theory; it is the pattern behind the most successful AI initiatives we accompany in the Mittelstand.

Beyond the Mittelstand: four patterns that mark successful AI delivery

However different industries and maturity levels may be, companies that are making visible progress with AI attend to the same four themes. Each one is a step closer to the double win of operational impact and independence.

1. Enable the right people, not only the technologists

A common assumption is that AI competence is primarily a matter of additional data scientists. In practice, the larger leverage emerges elsewhere: with the people who truly know processes, customers, and equipment. Process owners, key users, team leads, shop‑floor foremen.

The greatest leverage does not sit in IT, but at the process.

This group brings exactly what makes AI projects take off: context. They know how a manufacturing order moves through the ERP, what sales decides first thing in the morning, and which supplier data can be trusted. When these people are deliberately enabled, in short, practice‑oriented sessions and on real use cases, a technical initiative becomes a working tool.

A typical pattern in practice: Data scientists and the business form joint tandems. Domain knowledge flows into requirements, AI results flow back into the process. The time from idea to first productive version typically halves, without additional tools or additional headcount.

2. Focus on a few, genuinely viable use cases

“We have twenty proofs of concept.” That sounds like momentum, and in the early phase often is. The next step, however, decides whether experiments turn into impact: prioritize clearly, hand over ownership clearly, scale clearly.

Two running use cases beat twenty planned ones.

Successful portfolios in the Mittelstand are deliberately lean: three to five use cases with visible leverage on core processes, each with a named owner and clear success criteria. Equally important is the culture of finishing things. When a pilot has exhausted its potential, or impact fails to materialize, it is deliberately closed and makes room for the next.

An illustrative picture from sales: Mid‑sized industrial equipment suppliers often have an inside sales team that simultaneously handles quote requests, spare‑parts enquiries, and service coordination. Instead of working on many ideas in parallel, successful approaches concentrate on one clearly defined step, for example pre‑qualifying incoming enquiries through an agentic assistant that pulls customer data from the CRM and proposes a first draft offer. The benefit shows up less in a single KPI than in the inside sales team being noticeably relieved and enquiries being answered more consistently.

3. Anchor AI directly where decisions are made

Perhaps the most important lever sits in the “last mile”: how does the AI result reach the person who has to work with it? The companies that are consistent here do not build AI as a separate dashboard, but as a natural part of the workflow.

The “last mile” decides between “AI introduced” and “AI in use.”

In practice: a notification in the existing system rather than an additional interface. Clear accountability for who decides on the basis of the forecast. A defined escalation when the model is uncertain. A “human‑in‑the‑loop” where experience makes the call. AI then becomes not an add‑on, but an extra pair of eyes and hands in daily operations.

An illustrative picture from procurement & supply chain: In technical wholesale, supplier risk is increasingly monitored agentically. An agent scans news, commercial register updates, and logistics data, evaluates changes, and on critical signals creates a task directly in the procurement system, including a recommended action and point of contact. The buyer still decides, as always. But knows sooner where to look.

4. Treat AI as a product, not as a project

This is a shift at first, but not a major hurdle when planned for from the start. Successful companies give their AI systems after launch the same attention they give their products: a responsible owner, an operating concept, regular quality checks, versioning, and cost control.

Software projects have an end. AI systems do not.

The effect is remarkable: the model stays reliable even as inputs or processes change. Trust in the systems grows steadily, and with it the willingness to extend AI into further areas.

A typical pattern in practice: Operating concept and ownership are built into development from the start, not bolted on after going live. Those who do this incur little extra effort. Those who have to add it later pay noticeably more. Experience points to two to three times the original development cost.

Doing AI, not just having AI

The four patterns share one thing: they shift the emphasis from “buying AI” to “doing AI.” And this is exactly where the Mittelstand plays to its strengths: closeness to the process, pragmatism, clear accountability, fast decisions.

Companies that stay ahead with AI build capability: teams that understand and steer AI; data products that are maintained and reliable; systems that run in everyday use, not in the test environment; and an operating logic that secures the value tomorrow as well.

That does not require a corporate apparatus. It requires focus: three to five use cases with real leverage on core processes, the right people behind them, and the decision to treat AI as a capability you develop, rather than a project you finish.

So the arc closes: those who build AI as a core capability win twice. In the short term through operational effects that become measurable in weeks and months: less downtime, faster quotes, sharper decisions. In the long term through an independence that makes them resilient to vendor fashions, market cycles, and the next technology wave. The Mittelstand brings everything needed for this path: depth of craft, closeness to execution, and a culture in which things count as finished only when they are running in operations.

Which of these four themes is currently moving you most in your AI work? We would be happy to exchange views with you, as peers and from the perspective of delivery.

---

This article was written by Philipp Diesinger, Partner at Rewire. His focus is on AI transformations that make the leap from pilot to productive operations: in life sciences, industry, energy, and the German Mittelstand.

Building your AI capabilities, end-to-end

Our strategies elevate the impact of AI from marginal or tactical to transformational successes.

Explore our AI strategy services

Last week, Rewire hosted the third edition of the Data Leadership Roundtables. We gathered a select group of industry pioneers – from retail giants like Albert Heijn and financial leaders like Nexent Bank to biochemical innovators like Corbion – to tackle a pressing question: How do we bridge the gap between fast-moving agentic AI and the scalable rigor of Data Management?

Ontologies talk: coffee is a great way to build relations.

Bridging the gap: context as the universal language

The session opened with a thesis: agentic AI moves fast, but data management scales. The true value lies in the intersection. A shared approach to context and metadata can unify these two worlds. Bridging this gap requires three strategic shifts:

• Investing in new capabilities: Prioritizing Knowledge Graphs and Model Context Protocol (MCP) integration for Data Products.
• Powering AI with trust: Leveraging Data Products to provide AI agents with context built on verified, trusted metadata.
• Democratizing context: Distributing semantics across use cases and domains to ensure AI agents speak the same "business language."

The CDO perspective: data ownership at the source

A highlight of the event was a fireside Q&A with Gabriela Filip (ex-CDO at Knab). Gabriela offered a masterclass in balancing speed with integrity. Her advice: Data ownership must live where the decisions happen.

Key takeaways from her perspective included:

• Local flexibility, global alignment: Keep operational definitions flexible at the local level. Mandate "strong alignment" only for critical, cross-domain business decisions.
• The "Knowledge-First" approach: Combine business and data expertise to effectively capture knowledge, supported by roles like Information Architects and coordinated by leadership (e.g., CDO).

Gabriela Filip and Freek van Gulden sparring over data strategies

Insights from data practitioners: the three recipes for success

Following the plenary, participants broke into deep-dive roundtables covering data governance, operational ownership, and organizational change. Here are some of the "recipes" that emerged:

1. Start small, deliver value first: The path to structured governance and knowledge starts by linking data governance to business value. Start with a specific use case, let governance evolve around it, and use visible wins to reduce resistance, drive adoption, and scale.
2. Flexibility over rigid architecture: With AI evolving rapidly, the only certainty is that rigid "end-state" architectures will become obsolete before they are even finished. Focus instead on decoupled, modular setups. (For deeper insights, check out this blog post.)
3. Connecting defensive with offensive strategies: exploit defensive data management initiatives – like fixing metadata, glossaries – to comply with regulatory compliance. Then use this as an opportunity to create a scalable knowledge foundation that provides a fly-wheel for offensive value creation.

Participants sharing the challenges and lessons learned.

The dialogue doesn't end here

The day’s discussions reaffirmed that navigating today’s Data & AI landscape isn’t just about tools - it’s also a bit about community: Together, we’re better equipped to tackle the complexities and turn challenges into opportunities.

Until the next Roundtable!

Building your data capabilities, end-to-end

Our strategies elevate the performance of your data management from marginal or tactical results to transformational successes.

Explore our data management services

You’ve got the prerequisites in place:

• A strategic AI roadmap. Check.
• The ability to develop data products. Check.
• Leadership alignment. Check.

The real question is: How can you build a data foundation that delivers value early without creating long-term problems, such as inconsistent metrics, fragile pipelines and technical debt.

It’s tempting to design the full data foundation upfront. In practice, this often leads to over-engineering, slow delivery, and frustrated stakeholders. A more effective approach is to start from value—while making deliberate design choices that allow the foundation to scale.

Why building a data foundation is so difficult

First, let's be clear about what we mean by a data foundation: A data foundation is the strategic layer of reusable, trusted data products, governance, and architecture that breaks down silos and turns raw data into actionable information aligned with business outcomes.

This enables organizations to accelerate analytics and AI use cases by providing consistent, high-quality data that supports both current needs and future growth. (More on data management fundamentals here.)

Building a data foundation is not just a technical exercise. Yes, you need platforms, pipelines, and integrations. But the real challenge is balancing (1) business impact, (2) trust in data, and (3) long-term scalability. Most organizations struggle because they oscillate between two common approaches to data foundation strategy.

Let's review them.

Top-down vs bottom-up data foundation approaches

Typically, one can consider starting top-down (i.e. from a value pool perspective) or bottom-up (i.e. from a source system perspective).

1. The top-down approach

It starts by addressing a specific business problem or use case.

For example: developing a price optimization tool for your sales team by stitching together CRM data, transaction history, and external market data into a customized pipeline built solely to serve that one application.

• Pro: strong business alignment. Data products are directly tied to measurable outcomes.
• Con: risk of fragmentation. Solutions become use-case specific, difficult to reuse, and often turn into silos.

2. The bottom-up approach

It starts by building the technical foundation first.

For example: unlocking all data from your ERP and CRM systems by ingesting and modeling large volumes of operational data before concrete business use cases have been defined.

• Pro: strong reusability. Source-aligned data products improve consistency and quality.
• Con: slow time to value. It’s unclear which use cases will be enabled, delaying impact and eroding trust.

Figure 1. The top-down vs. bottom-up approach 

3. The problem with either approach

Simply put, neither approach works well on its own:

• Lean too far top-down, and quick wins collapse because the underlying foundation cannot scale.
• Lean too far bottom-up, and you risk building a technically sound foundation that no one actively uses.

As a result, many organizations end up switching back and forth between these extremes, never fully unlocking the value of their data.

The solution: start top-down, but design for scale

To kickstart a data foundation effectively, we recommend a top-down, value-driven start combined with bottom-up design principles.

In practice, this means:

1. Selecting priority use cases using a top-down approach, based on strategic importance and reuse potential.
2. Applying scalable design principles when building data products, so today’s solution can evolve into tomorrow’s foundation.

This approach allows you to demonstrate value early while avoiding the common pitfall of creating one-off data solutions. The remaining risk is that a top-down start can still lead to fragmentation if design choices are made solely in service of the first use case.

The remainder of this article focuses on concrete design principles that mitigate these top-down pitfalls, as illustrated on the left-hand side of Figure 2. These principles ensure that value-driven data products can grow into a cohesive, reusable data foundation rather than a collection of isolated solutions.

Three design principles for scalable data products

As discussed in Data as a Product: Why Just Having Data Is Not Enough, data products are more than just data or pipelines. They encapsulate business logic, definitions, and assumptions. The way this logic is implemented determines whether a data product can evolve beyond its initial use case. The following design principles help ensure a top-down start remains sustainable.

1. Decouple data transformations to enable growth

Structure data products using clearly separated source → processing → sink layers so that ingestion, transformation, and delivery are explicitly isolated. This modular setup ensures that as new use cases emerge, individual parts of the data product can be extended, split, or reused—without rearchitecting the entire pipeline.

In practice, this means:

• Clearly separating source and sink logic, including all storage interactions and integrations with upstream or downstream data products.
• Isolating individual data transformation steps into distinct, composable blocks, each with a clear responsibility and output.

Why this matters:

• Enables faster creation and evolution of data products as scope expands.
• Keeps transformation logic readable, testable, and trustworthy for consuming teams.

2. Clearly document transformation logic and scope

Explicitly document the inputs, business rules, filters, and assumptions embedded in each data product so that its meaning and boundaries are unambiguous. This documentation should evolve together with the data product as scope and logic change.

In practice, this means:

• Listing all input sources and upstream data products the logic depends on.
• Describing applied business rules and filters, including rationale where relevant.
• Defining the intended scope and known exclusions to avoid misinterpretation by consumers.

Why this matters:

• Creates transparency around definitions and inclusions.
• Makes data products easier to extend, validate, and adapt as requirements evolve.

3. Design outputs for reuse across domains

Instead of optimizing outputs for a single business domain, deliberately design data product outputs to support multiple consuming domains from the start. This requires making consumption explicit and turning it into concrete design input.

In practice, this means:

• Identifying all potential consuming domains that are likely to use the data product now or in the near future.
• Capturing archetype consumption patterns and translating these patterns into detailed output requirements.
• Implementing these requirements in a consumer-aligned data model, defined through entities, attributes, and relationships that reflect shared business concepts rather than domain-specific needs.

Why this matters:

• Encourages reuse across the organization, accelerating new use case development.
• Reduces future rework by aligning outputs with real consumption needs upfront, making data products easier to extend and scale.

Figure 2. Before and after - the result of starting top-down while designing for scale.

Conclusion: building a data foundation that lasts

By starting with high-value use cases and applying scalable data product design principles, you can deliver immediate business impact without compromising the future. Done well, this approach results in a data foundation that grows organically from real business value rather than abstract architecture.

In short: don’t try to solve everything upfront—and don’t chase only quick wins either. The key to building a strong data foundation is delivering value today while deliberately designing for tomorrow.

---

This article was written by Bram van Koppen, Senior Data Engineer, and Phebe Langens, Data Scientist at Rewire.

Building your data capabilities, end-to-end

Our strategies elevate the performance of your data management from marginal or tactical results to transformational successes.

Explore our data management services