The path to architecture Nirvana

Architecture is a worthy activity, and we’ve all burned many hours trying to make the IT world a better place. The trouble is, it’s always been done manually, and we’re at a crunch point because the current way of working doesn’t scale. Diagrams diverge from reality almost as soon as they are drawn. Impact assessments take weeks because the knowledge is scattered throughout the organisation (and in some cases, lost). And now the pressure has increased: AI has supercharged IT delivery, so it’s suddenly starkly obvious that traditional architecture is slowing things down.

You can’t be without architecture: without it, chaos and incoherence are inevitable. But how does architecture accelerate to keep up with the new AI-augmented world, whilst maintaining quality, and not increasing complexity or deepening existing fragmentation? In other words, how does an organisation achieve its “architecture Nirvana”, where architecture is:

• Seen as not just value-adding, but essential
• Fast, flexible and outcome-oriented
• Data-driven and AI-enabled: automated where appropriate, whilst leveraging the expertise of architects to add maximal value and maintain accountability

And how do you get a large organisation, with an incumbent EA tool it has already paid for and a change portfolio that is already oversubscribed, to fund the journey?

The instinct is to fix the data first and build the useful things afterwards, because the data really is the foundation. It’s a reasonable approach, but it almost never gets off the ground, because it asks an organisation for a long stretch of invisible progress on trust. The approach that can work is to advance on two fronts at once (the data you hold, and the functions people can use). This is a roadmap discussion, not a technology problem, and roadmaps that attract budget tend to be built in the same order: “Why”, “What”, and “How”.

Why: understand the issues and quantify the benefits first

Funding follows use cases that quantify measurable value, so begin with the problem space. What do you need to improve (aligned to Architecture-as-Code and AI), specified in terms that a portfolio board recognises:

• Faster, cheaper change assessment: how long does it currently take to answer the questions like “What breaks if we retire this technology?”, and how much architect time is consumed producing the answers? If the model can be interrogated, this collapses from days of human archaeology to minutes.
• Design assurance at scale: manual design review is a bottleneck and prone to inconsistency. Automating checks with encoded architecture principles transforms governance.
• Model correctness: what percentage of your model reflects reality right now? Most organisations don’t know (which is itself the finding).
• Regulatory and audit evidence: in financial services, this is often the strongest single argument. Operational resilience and third-party risk obligations require you to demonstrate, on demand, how critical business services are delivered. A versioned, audited model is far cheaper to produce evidence from than a scattered amalgam of documents.
• Accelerating architecture delivery: reusing validated architecture patterns and populating standardised documentation from the model.

This needs to be fleshed out with deeper detail, which comes from the following two disciplines:

• Firstly, specify the processes you are automating: “automate impact assessment” is meaningless. Write down what happens today, and then refine from a modern, data-centric perspective: don’t just blindly automate what organically grew without any sort of process analysis or definition. You can’t automate a process you haven’t described, and furthermore, this detail will make it obvious which data matters, and where AI can be appropriately deployed.
• Secondly, let the use cases drive data priority: this is the single most useful piece of sequencing logic in the whole exercise. Every large organisation will have more data quality problems than it can fix at any one time, and more entity types than it can afford to model. The functional use cases will highlight which data dimensions to bring into scope first, and which problems with data quality need to be fixed urgently (and, by inference, which can be left for now).

Baseline the metrics before anything is built and be honest (especially where it’s embarrassing), because the worst pain points highlight the impacts which propel your business case. Choose metrics that can be instrumented, or you can’t be sure of success, and don’t waste time with a detailed fact-finding exercise: keep this at the minimum level of detail to show the issues and demonstrate the improvements.

One more critical thing to include is analysis of the running cost at scale. AI is not free, and trivial costs from a pilot will not remain insignificant when it’s part of the architecture process across a large estate. As well as the financial impact, compute has an environmental cost that is everyone’s problem. Designing the use of AI across processes end-to-end, aligning to points of human involvement and control, can help with this: fragmented adoption of AI can lead to one AI consuming the output from another, which is both an expensive waste of tokens, and a sure-fire way of introducing error and opacity.

Finally, be precise about what you expect AI to do: “AI-powered architecture” is not a use case.

What: architecture transformed by architecture

Treat your own architecture function as the system being transformed. You’d never propose a target state for any other business capability without documenting the current state: maintain traceability from the requirements and associated use cases through to the deliverables, and hence demonstrate the value delivered. (Especially crucial in the current world: uncontrolled use of AI worsens existing opacity.)

Describe the transformation of architecture in two orthogonal dimensions: the data modelling the architecture, and the architecture functions which act upon said data.

Data architecture

Your architecture model is a data product, so describe it as one:

• Define the target data model up front: this is the one thing that genuinely does have to come first, but it’s something that can be articulated in days or weeks rather than months. Define the target metamodel (entity types and permitted relationships) before starting any data cleansing or migration activities. This should be a short, focussed activity, which we’ve undertaken for several clients: the results vary slightly in each case, but all broadly align to the customer- and vendor-agnostic model we’ve developed. (An important point here about EA tooling: most incumbent EA tools embed an opinionated metamodel that’s likely to have been fettled locally for a decade at least. There will be parts of this that fit, but also parts that don’t, or there will be gaps: reuse only what makes sense.)
• Start with the data you currently have: every organisation starts from a different place with a varying initial set of data (so there is no magic data pipeline that works for everyone).
• Map where your data is located, how it’s maintained, and how its access is governed: some might be in the EA tool; some in the CMDB; some in the service catalogue; some in business-owned spreadsheets; some only in the heads of experts. Mapping how these fragmented, disparate datasets should be joined and connected is the genesis of an architecture data pipeline.
• Be explicit about ownership, including what you don’t own: some entity types belong to architecture, but many don’t. This needs to be deliberately managed as a master data problem, based on defined data ownership. This can evolve over time, dataset by dataset, as more data comes into scope.

Functional architecture

This perspective can be capability-centric, but it may be more helpful to base it on the functional componentry that supports the defined, prioritised use cases specified in the business case.

Either way, assess each element honestly, particularly incumbent EA tooling, which should also be analysed functionally, rather than as a single entity. Most of the vendor products out there do at least some things well, and it’ll be far easier to continue to use those bits of existing tools which are fulfilling their function well, rather than replace them with something novel. (If the EA tool has a usable suite of APIs, then it ought to be reasonably simple to maintain it whilst inverting the usage model, so that the code-based model is the authority. This is a critical dependency, so it needs to be tested and confirmed early.)

If this is managed well from the functional view, you’re also going to be in a much better position when it comes to understanding the application architecture of the architecture functions: “up” to capabilities and “down” to technologies and vendor products.

HOW: track and plan progress in both dimensions

Ideally, the architecture data will be at a good level of breadth and quality before any functions are implemented which use it. But in the real world data ownership and quality remediation are slow, and they produce no benefit in and of themselves. There is a clear need to show return on investment, by the delivery of measurable business value: this is predicated on functional improvements in architecture.

Therefore, the roadmap should be two-dimensional, and the progressions along each axis, whilst they can be tracked independently, should be viewed as a combination.

Data Axis

Once the data model is defined, the progress along the Data Axis combines the following:

• Dimensions: start narrow (typically the application catalogue) and then extend as the use cases require, with interfaces, business capabilities, technologies, etc.
• Quality, driven by ownership: completeness, accuracy, currency, etc.

Both can be improved incrementally, and neither is ever likely to be complete. But a partially populated model with known gaps is a perfectly reasonable thing to build on, provided the gaps are understood and accepted.

Functional Axis

Deliver functions quickly that use the data immediately available, and that really improve how people work day-to-day. An application catalogue with generated views is unglamorous, but if the current alternative is a spreadsheet and a set of stale Visio files, it’s a visible improvement in weeks rather than quarters.

From there the Functional Axis extends in alignment with the defined use cases: automated derivation from live sources; publication to consumers via multiple channels; governance checks in the pipeline; analytics; AI-assisted assessment; etc.

A Roadmap in 2D

Delivering a usable function, even based on imperfect data, will drive consumption. Users will find the errors, far faster and more cheaply than a data improvement initiative would. If the functions are helping them to do their jobs, they’ll be invested, and they will demand high-quality data to support the functions they have come to depend on.

(The inverse is the embodiment of the failure we all recognise: a data remediation programme with no consumers, producing quality improvements nobody can feel, defended in steering committees by increasingly abstract arguments.)

This approach makes it plain how to incorporate AI into the transformation. Capabilities that leverage AI simply track the Data Axis: AI can do genuinely useful work as soon as the data underneath it is good enough (and no amount of prompting compensates when it isn’t). Data improvements will be iterative, so use AI early, as soon as the data supports it. Indeed, the long-term journey when using the roadmap approach needs to be iterative: demonstrating value, building confidence, and thereby ensuring the investment for the next phase.

The argument above has an important converse: if AI runs far ahead of progress on the Data Axis, some short-term benefits will be derived, but they will quickly be outweighed by the loss of comprehension and control. Opacity gets baked into the artefacts produced, which compounds rapidly. A key point of this endeavour is to make the estate more understandable: be careful not to make the existing incomprehensibility of the IT landscape worse.

Everyone needs their own bespoke approach

Each organisation has followed its own journey and therefore will have a unique set of legacy architecture issues, including the level of disconnect between reality and the data describing their IT.

Icon Solutions has a long history of helping organisations improve their architecture capabilities, both in terms of architecture data and architecture functions: our reference assets and accelerators encapsulate our expertise and experience, and so whilst everyone’s roadmap (both the path and the plan) will be individual – albeit typically with some common elements – we can expedite the transformation process towards your own architecture Nirvana. Get in touch with us to discuss your specific issues to see how we can help.