Over the last couple of years at Icon, we’ve been assisting several of our clients with their exploration and adoption of Architecture-as-Code, and their differing approaches have been both informed by, and impelled, our own internal investigations on the subject. Real-world use cases vary, in terms of both function and relevant data, because everyone is starting from their own position, including their own unique set of motivations and problems in the Architecture domain. In developing our exemplars for these real-world use cases, to be able to demonstrably deliver value within our client organisations, we concluded that we needed to roll up our sleeves and build a reference implementation for Architecture-as-Code ourselves.
This exercise proved to be interesting, rewarding and challenging: even in our own environment, where we had full control of all solution design, development and technology decisions, we still encountered some lurking gremlins to contend with. This blog is to share some of the insights we’ve gained along the way.
Evolving data needs a flexible storage medium
A non-negotiable requirement for codification of any sort is alignment to a data model, and transparent storage in a data repository. However, this must be balanced with extensibility and flexibility: the data embodying the architecture will change and grow, both in quantity and in dimensionality, to support additional use cases. In our case, even with our aggregated expertise in Architecture and its codification, we found that the requirements mutated as we worked: without the ability to non-destructively extend the data model, we’d not have been able to include new data or augment existing data, without re-developing the data componentry.
It’s impossible to escape complexity in data, but our experience has demonstrated that it’s far more beneficial to face into it and make a concerted effort to tackle it natively. As well as the extensible model referred to above (which we easily grew to iteratively include capabilities, applications, components, interfaces, technologies, risks, etc.), data points can be combined to make best use of graph features: using fundamental graph concepts of nodes and edges accelerates, in turn, visualisations and further connections into additional architecture tools and functions. We are working with multiple connected source datasets, and this scenario made the power of a graph database very obvious.
AI technology is useful, even transformative: but it isn’t magic
Our solution implementation has obvious “left” and “right” sides, with the abovementioned data repository in between them. The left side is the data pipeline, by which all the disparate architecture data sources are standardised, cleansed and referentially linked; the right side hosts the architecture functions which are enabled by, and act on, the architecture data. We’ve used AI technology on both sides, with quite distinct results.
For the data pipeline on the left, AI wasn’t quite the silver bullet we’d hoped: to link up two datasets by finding the connections between them, AI needs to have meaningful descriptions, in natural language, on both sides: if this doesn’t exist, then AI has limited use in this scenario: you can’t escape the application of human expert knowledge. So, our data pipeline is a combination of manually coded transformers, alongside some programmatic invocations of AI, that were prototyped and refined by continual review, until their outputs were acceptable.
Conversely, for the functions “on the right” acting on the architecture data, we have used a single AI component, which is RAG-loaded with Icon collateral. The solution uses it to support different functions, and while these are variations of a common pattern, all required separate, detailed, prompt-engineering. The difference is that this AI component is acting on good data, and the benefits of AI usage within these functions are obvious, when one sees the results from the functions’ execution. That said, the development effort was not trivial, and nor are the results perfect: for all the functions, AI provides support and information for the user, which the user then needs to act on. AI can’t replace the application of real intelligence.
So to use AI successfully, you need to figure out how your data fits together beforehand, to then allow you to get benefit from applying AI tools as part of the use cases: indeed, we’ve found in earlier activities that AI can interpret well-organised data (as well as natural language) and can apply RAG and associated analysis to it.
Scope creep can be self-inflicted, even with the best of intentions
We started out with a simple high-level idea, but it grew appendages rapidly! And because we’re problem-solvers, there was always the temptation to extend functions to meet new requirements. Even though we were our own stakeholders, with full control of what we chose to build or add to the backlog, it still required considerable discipline to resist the natural urge to bolt-on additional functional augmentations. Keeping true to what I’d loosely refer to as “the UNIX principle” (i.e. each component should do one thing well and maintain a bounded context with clean interfaces) has ultimately led to a reasonably clean and well-structured implementation: for instance, keeping the diagram generation in a separate component from data querying.
It brought me up short when I realised the parallels between the above and the current architectural state for our clients: aggregated tactical changes over time, and no tracking of the resulting technical debt, is very likely why so many organisations struggle to remediate legacy complexity. The functional scope of applications was silently and opaquely expanded, blurring the boundaries between them, and entombing them in a self-perpetuating lattice of point-to-point integrations: the monstrous spaghetti of deadlock. To some extent, this brings us full circle in our journey: here is the clearest reason why Architecture is a necessity: without its governance, chaos and entropy are the only logical outcomes.
(As an aside, I believe that the above failure to maintain architectural “purity” is also one reason that Architecture-as-Code adoption initiatives conflict with established Enterprise Architecture software: those commercial products do not maintain a clear separation between different functional boundaries, as well as locking the architecture data into fixed, proprietary stores and formats.)
There’s life in the old dogs yet!
This activity has been very rewarding and has brought us some varied benefits:
-
Primarily, we have a working demonstration of the possibilities that are enabled by Architecture-as-Code: it brings it all to life and is resonating very well with those colleagues and clients we’ve shared it with.
-
We are also starting to use this tooling internally for modelling and maintaining our reference materials: by codifying our collateral, we have enabled the same use cases – such as diagram generation – for our own reference models and architectures.
-
Practical, empirical validation of our experience and opinions regarding AI: it requires understanding to use it effectively, and it needs decent data to operate on. Put another way: AI isn’t magic, and it can’t be used reliably without oversight from real intelligence.
-
A bright refresh of somewhat rusty engineering skills: new methods and tools that every architect really ought to be familiar with. We’ve been moonlighting as engineers, but this is a good thing! To really understand something, you have to use it: getting your hands dirty will mean you make exactly the tools you need, component-by-component, from the appropriate technologies – providing you maintain your understanding of the limitations and benefits of good design.
We’d be very interested to hear about other peoples’ adventures with Architecture-as-Code, so please reach out to us with opinions, suggestions, or questions. And if you’d like a longer discussion, we’d be happy to share our experiences along with a demo of the application itself, which we’ve named Kato: the embodiment of a useful side kick.
(And the story isn’t over: we’re continuing to develop and deepen Kato’s functions. One current focus is with generating diagrams involving large numbers of objects and relationships, and how to make the output coherent and consumable. This is neither a trivial nor a new problem, so I’m sure there will be lots of sage advice out there, which maybe we’ll be able to add to before too long...!)
