Do agile projects need Solution Architects?

Increasingly software development in large organisations is moving to agile methods, primarily scrum.  Agile empowers developers to design and build the system – where does solution architecture fit in?

Agile and scrum is now almost ubiquitous as the development methodology to use, overtaking traditional waterfall methods and adopted by large organisations almost as the ‘magic stick’ to lower costs and speed up delivery.

By sub-dividing the project into smaller chunks (sprints), developers can design, build and deliver within each sprint and provide incremental business value. This enables a ‘fail-fast’ mentality which refactors and restructures code as and if required.

The impact on Solution Architects

This poses an obvious challenge to the role and relevance of a Solution Architect.

• Firstly, a delay up-front waiting for architectural design to be undertaken seems at odds with a fail-fast mentality.
• And secondly, is the cost of failing and refactoring going to be less than the cost of having a solution architect on the project?

My experience of this, working in a large organisation on large scale solutions, is that despite the delivery method changing, all the issues and tasks that architects address are still present.

Primarily, the biggest issue with agile is that it is aimed at small teams, building one piece of software.  It has very little to say about solutions spanning multiple technologies, potentially against existing organisation platforms, encompassing multiple development teams in multiple locations.  This is the reality facing most large enterprises.

Given a set of business requirements to fashion a solution from in this environment, then solution architects and enterprise architects are still required to fulfil their traditional role of identifying the platforms involved, or new platforms to be built, and how those platforms would be implemented.   There will be stakeholders to manage, presentations and architectural options to discuss and present, and other programmes to collaborate and align with.  Potentially a solution will require new software to be assessed and bought.

Setting the right foundations

Solution Architects are key to making sure that the project starts out with the right fundamentals in place to prevent mistakes being identified downstream which require considerable cost to rectify.  Architecture in this sense is comparable to drafting the blueprint for a building.  An agile approach is akin to building a floor at a time with a minimal blueprint, if at all, and refactor at an acceptable cost if a cosmetic failure occurs – e.g. the client asks for an internal wall to be moved.

However, the ‘benefits’ of the fail-fast approach become somewhat irrelevant if you are building a skyscraper this way and at floor sixteen discover that the foundations which were thought to be sufficient can’t actually support the rest of the building.  At that point your strategic options are limited, and you need to be mindful of the fact that the agile approach means that fifteen stories of the building are now occupied! On the one hand you could describe the teetering building as a ‘legacy’ and start a new building next to it, or you could evacuate the fifteen occupied storeys of the  structure and look for ways to architect around the issues, or raze the building to the ground and start again. Aside from the famous Tower of Pisa, such failures generally result in massive reputational and financial impact. Architects are key for planning the IT “foundations” such as large-scale structure and approach, resilience, availability, recovery, scalability, extensibility etc as opposed to just creating code to deliver user stories to deliver the Minimal Viable Product (MVP).

But then again, agile does not infer that the role of an SA should not be confined to this upfront investment. Once the solution is defined, then Solution Architects are likely to move into another typical role of technical oversight.  They are likely to work with individual development teams clarifying out what is required or providing guidance.   Because they understand the big picture, they are likely to be involved in high level project planning or analysing new requirements to identify how they should be implemented and which developer teams to engage.  They can also present technical issues to senior stakeholders, solve inter-platform design issues, help with difficult technical issues, and provide an intermediary between technical aspects and the high level, protecting developers from interference from stakeholders and project managers.

Note that there are agile processes for large scale projects (e.g. SAFe and LeSS) to follow.  However, these introduce quite a lot of process and in my experience is that waterfall is used to control multiple teams each delivering in an agile way.

Size does matter

In smaller, more agile companies, the situation may be different.  Developers may take on a more involved role, especially if building software products as opposed to multi-platform solutions.  There is still likely to be a role for architects, in the early stages of the project as before.  Architects may also be required if also delivering services as well as a product, e.g. helping clients to implement the product within their estate, or in designing new componentry to add to the product.  Certainly at Icon Solutions we have architects involved in this way in the production of our payments platform IPF, especially in planning out the future roadmap and developing new services for clients.

In short, architects are likely to be required no matter what delivery process is used.  The ways of working may change with delivery methods and styles, but there will always be the same problems of thinking in the bigger picture, designing solutions across multiple platforms and technologies, that sit at a level above the actual software development.  Ignoring the importance of this role could mean financial and political folly.