AI ARCHITECTURE

AI is not a feature. It is a systemic risk with value creation potential—both at the same time.

And that is precisely why so many projects fail.

Not because of the model. Not because of the API. But because of the (missing?) thinking behind it.

WHY AI PROJECTS REALLY FAIL

The figures are unpleasant:

• Up to 95% of all GenAI pilots do not reach production (MIT, 2025).
• 30% of all GenAI projects are discontinued without replacement after the PoC (Gartner, 2024/25).
• Gartner already predicts a dropout rate of around 40% for Agentic AI.

“Changing Anything Changes Everything.” — Sculley et al., Hidden Technical Debt in Machine Learning Systems, NeurIPS

This is no coincidence, but a structural problem. The industry is currently producing masses of PoC corpses because AI is misunderstood as plug-and-play. The result is not a product, but hidden technical debt.

LLMs have not solved this problem—they have accelerated it.

THE REAL PROBLEM: ARCHITECTURE

In AI systems, there is usually:

• 5% model code
• 95% glue code (data preparation, prompt logic, retrieval, evaluation, monitoring, logging)

Today, this glue code is often located directly in the business logic – in other words, prompt engineering in the middle of the business logic.

Consequences:

• No interchangeability of models
• No clean testability
• No explainable decision chain
• No compliance capability

In short: a system that implodes at the first model update.

THE NEW REALITY: AMBIGUITY TOLERANCE

There is a second, often overlooked reason for failure: the wrong mindset. GenAI systems are not deterministic. They are probabilistic. The same input might deliver a different output tomorrow.

Anyone approaching this with classic “Requirements Engineering” (rigid specification documents, binary approvals) will fail. We must learn to navigate in the fog.

Blurriness is not the problem, but the material, the blood in the veins of the models.

It’s no longer about “Finished” or “Perfect,” but about “Good Enough” and the question: Is the answer helpful in the context? It’s best to involve domain experts for this, as I am a big fan of the domain-centered approach (keyword: Domain Driven Design).

Requirements are no longer specified, but negotiated in dialogue with the system.

Product Owners and architects must become “Uncertainty Specialists.” They must decide how much freedom the model gets and how much control is necessary. Without this ambiguity tolerance—the ability to endure and productively use ambiguity—every GenAI project becomes a frustration machine.

MY APPROACH: ARCHITECTURE FIRST.

I work as an AI architect at the interface of:

• Software and cloud architecture
• AI engineering
• Data and AI governance
• Law (EU AI Act, GDPR, Data Act)

Not theoretically, but in ongoing projects – including at GÖRG in AI-supported systems in the legal environment and in other regulated contexts.

I don’t build “smart demos,” but AI-enabled architectures that:

• scale
• are measurable (keyword: Build-Measure-Learn Circle)
• remain explainable
• are legally compliant

WHAT AN AI ARCHITECT MUST BE ABLE TO DO TODAY

TECHNOLOGY: THE FACTORY, NOT THE MODEL

An AI architect is not a data scientist. They do not build the model, but the production line.

• Model strategy
  • Proprietary vs. open source (Llama, Mistral) vs. small language models
  • → Decision based on risk, cost, need for control
• RAG vs. fine-tuning
  • RAG: flexible, but expensive to operate
  • Fine-tuning: expensive at the beginning, cheaper in inference
  • → TCO calculation before project start, not after
• Evaluation & Quality
  • LLM-as-a-Judge, RAGAS, Bias Checks
  • Quality is measured – not felt

GOVERNANCE: LAW AS CODE

The EU AI Act is not just a compliance PDF, but a technical specification with legal consequences.

• Art. 12 – Logging & Traceability
  • Architecture requires immutable protocols: Inputs, outputs, model version, data source, vector references.
• Art. 10 – Data Quality
  • Representativeness, bias control, documented thresholds.
• Art. 14 – Human Oversight (The human in the loop)
  • Human oversight is not a switch that you just flip (“Human in the Loop: Check”). It is a promise to the person affected that a human decision actually takes place – and not a simulation.
  • Architecture solution: Instead of simple asynchronous events (“Fire & Forget”), I implement state machines with “Pending” status.
  • Frozen State: The human sees exactly the data snapshot that the AI evaluated. If the data changes during the review, the process is invalidated.
  • Circuit Breakers against Rubber-Stamping: If a reviewer “waves through” 500 cases per hour, the fuse blows. Because those who only stamp are not exercising oversight (prohibition of rubber-stamping).
• Risk tiering (ISO 42001)
  • High-risk modules are technically isolated, monitored differently, and documented differently than harmless assistance functions.

Compliance here is not achieved by lawyers at the end, but by architecture at the beginning—preferably in collaboration with a lawyer who is involved from the start: me.

WHAT I DO SPECIFICALLY

• Design of resilient AI system architectures
• Clean separation of model, retrieval, prompt layer, business logic
• Governance and logging concepts by design (including Art. 14 patterns like Pending States & Circuit Breakers)
• Establishing ambiguity tolerance as a process: away from binary approvals, towards impact-based metrics
• Decision-making bases for costs, risk, model selection, operability
• Integration of EU AI Act, Data Act, GDPR, and thus data governance from the start. Because believe me, you’ll have to bear the costs for it anyway—only later they will be exorbitantly higher. Like technical debt. The longer it “rots away,” the more difficult and, above all, the more expensive it becomes to clean up afterwards.

I prevent:

• PoC graveyards
• Untestable prompt monsters
• Compliance retrofitting under time pressure
• Unpleasant legal surprises (e.g., due to rubber-stamping or lack of human oversight)

IN A NUTSHELL

I create AI architecture for organizations that really want to use AI – not just talk about it. Because doing is like wanting, just more intense.

For decision-makers who understand that a system is not good because it impresses, but because it can still be explained two years down the line.

And for projects that are better built cleanly today than defended expensively tomorrow.

Are you looking for someone who reads code, understands laws, and takes architecture seriously?
Then we’re probably a good fit.