The headlines from AI software providers would have you believe that “anyone can build anything in an afternoon”, with convincing marketing such as “we built a production-ready app with no coding experience”. As an IT evangelist (and an inner nerd who likes to get hands on with new tools) this felt like a challenge worth testing! I started out with a PoC to try to build something a bit more real, and thereby understand not just what the tools produce, but also what they hide and to strike a balance between human-centric and AI-first engineering.
We all know that it’s not quite as easy as the marketing would have you believe, but here are some insights into where it aligns with the hype and where it doesn’t. I aimed to create a tool that would show Retrieval Augmented Generation could be used to answer Architecture questions, as well as automate elements of the production of a template-based Solution Design. For example:
How many applications do I have that have a status of “Invest”?
How many Risks do I have associated with a domain?
If I want to make a change to Application X, what are the impacts of doing so?
Can I build an “As is” diagram as a starting point?
The first thing I had was a vague idea that I wanted to develop and understand, so I started assembling with these building blocks:
A UI framework: Streamlit
I wanted to use APIs to make things accessible: FastAPI
I wanted to create an “Agent”: OpenAI via API (with Gemini as an alternative)
I wanted to understand MCP
I easily created code and components, which when combined formed something resembling a real architecture: a UI with interfaces to some data. It joined the dots and looked OK, though that didn’t last long. I put all my code together into a simple table, showed people you can run it using Streamlit, and we were all enthused. Then I wondered...
Where could I put my app to run it?
How do I update the data and move to a database?
How do I add more features?
Who should have access to this, and how do I set up those permissions?
In other words, could I move from simple queries to something that resembled enterprise-grade decision support?
The takeaway here is that the first 50% or so was fairly easy, but as I wanted to make it more useful, I needed to consider all the “real-world” issues around hosting, data storage and how things actually worked at the code level. Things that sit well outside of the scope of the marketing straplines.
When I wanted to add data connectivity and UI pages, suddenly every prompt needed to add more libraries, new files were created, and the LLM said it needed to refactor the code (to something more complex that I don’t understand fully). This is where AI-first started to fail: code I did understand (and could step through) became code that didn’t work. As I asked for help (which, in hindsight, made things worse) the code would get re-written, but still wouldn’t work, so I'd change the prompt, the LLM would rewrite... and the code still wouldn’t do exactly what I wanted. What we’d gained in speed we had lost in clarity: I couldn't reason through the generated code, changes got harder rather than easier, and I became unfamiliar with my own application. We'd traded control for velocity, and the trade wasn't worth it. The result was frustration, and many hours wasted.
A strong theme from recent industry conversations is that AI adoption is outpacing governance. Common concerns include:
Shadow IT emerging through individual experimentation
Hidden costs from uncontrolled usage
Loss of ownership of generated solutions
Data leakage into external models
Our experience touched on several of these.
The problem with this is that now a fancy spreadsheet can become a fully working app that connects to other components and quietly become critical to how a team operates, without anyone else knowing it exists or how it works. This is good from the innovation and education point of view, but a risk generally, and usually this will be beyond IT oversight and risks the supercharged re-emergence of uncontrolled and ungoverned IT. This experiment has reinforced the view that IT is much more accessible to those who are willing to take the time to get involved - but this has to have some level of engineering discipline.
When you start to create Agents there are more risks, depending on which LLM backend you are using. Are you sure they’re not capturing your data is the obvious one, but you can quickly make documents, API endpoints, data sources much more discoverable and public than you intended, and that can pollute the answers you generate from your data. Similarly, you can expose all kinds of Confluence resources which may be treated as authoritative even if they’re incomplete or unapproved, and this can have you scratching your head for some time on where the answer has come from... so again, if you’ve vibe-coded an app, how do you know? How do you trace it, or audit what has been accessed and what decisions are made on that information - a key design tenet of any Agentic solution?
Costs weren’t a big issue for what we set out to prove: we used simple API calls and low-cost subscriptions to LLMs, but the conversations this exercise kicked off outlined two startling things:
AI and LLMs introduce a new kind of cost model which is opaque for now, but the compute, the tokens and energy used are real for each prompt, Agent action, or background task. If you’ve not coded something, how do you know it’s doing things efficiently?
AI costs, or will cost, more than you think, as the cost is no longer just infrastructure – it’s behaviour-driven, so badly-designed prompts and Agent loops can scale cost faster than traditional systems ever did. Anecdotally, a customer chat revealed a large organisation had managed to run up a £2M monthly AI bill for replacing call centre staff with AI Agents. That’s a lot of call centre staff before it breaks even.
AI doesn’t remove the need for Architecture, it amplifies it
Data needs to be controlled for quality and completeness
The focus moves from writing code to understanding the business need and the systems - do not trust the AI without understanding these. AI is good accelerator, but not necessarily a good creator
Architects become translators between intent and implementation
Guardrails matter more than ever, not to slow things down but to prevent uncontrolled acceleration (and if written consistent in markdown can maybe be applied by Agents)
We did manage to build some working components where we took simple building blocks and LLM prompts, and created Agents using MCP. We also managed to do it with people whose primary focus wasn’t coding. The reality though is that whilst we did build some functioning components, when we turned it to a bigger set of components we found some problems, so the reality is:
It’s more complicated and takes longer than you think
You need a working understanding of code and how to run it, even if you didn’t write it
Hosting and deployment challenges don’t go away
Costs are harder to see but inflate quickly and need integration to FinOps
Data becomes more critical, not less
Governance, oversight, visibility and accountability become essential
Someone will need to own, supports and maintain these applications when they are live
In most organisations, Architecture data already lives in EA tools such as Ardoq or Orbus. These platforms act as the system of record and provide visualisation and insight. What we are exploring here is not a replacement for that ecosystem, but an extension of it. The opportunity lies in what those tools don’t yet do well: particularly in solution design and automation.
As an Architect I would say this, but when you’re using AI you need to consider the key pillars of what you’re doing and why. Is it just an educational piece of work, or will you use it in a business process? Education is one thing, but promoting a PoC to Production is another thing entirely. Considerations to make when using AI include:
Data: quality and completeness, even in a small sample set of use cases, helps you visualise what you want to create. You will require a baseline of defined and structured architecture data, whether in an EA tool or a data pipeline populating a data repository.
Design: rather than chasing every LLM suggestion, a focused approach and clear prompts kept things on track with what I was trying to achieve. You also need to understand what the code is trying to do to the extent you can step in, validate it and follow it to the data it is using.
Documentation: when you generate a lot, you quickly forget what does what (and No, I didn’t do this, but I should have done, as it would reinforce which bits relied on which relationships / interfaces!)
Alignment: does what you’re building actually fit with anything else in the organisation, or are you creating another isolated application that solves one problem but ignores (and adds to) the broader problems?
Discipline: a nod to Engineering discipline is important. Manage the code you do have and update small parts, unit test and commit to source control. That way you can more easily go back to a working base when things go wrong.
Structuring and governing architecture data so it can be used effectively
Defining safe patterns for AI adoption
Integrating with existing EA tooling and pipelines
Applying AI in the solution design space, where current tools fall short
This is less about introducing new tools, and more about unlocking the value and innovating on what you already have. If that sounds familiar or you’re starting to see similar challenges, feel free to reach out and talk to us about how we may be able to help you with those issues. And, with rudimentary coding skills, LLM help and some common sense, we did manage to build a demo tool that shows how well-documented and defined Architecture data can be used to validate architecture decisions and accelerate architecture processes.