It's time for another tale from the coal face, and this time we're talking about cookie cutters! This is a classic software engineering tale - as old as time itself - of realising when you should bite the bullet and move stuff into a common module.
IPF is primarily a payments-oriented platform which allows you to build payment flow orchestrations. Alongside this, though, we also offer some additional optional modules (AOMs), which are ready-made services that you can easily integrate into your bespoke flows.
One type of AOM we develop and maintain is a scheme pack, a.k.a. clearing and settlement mechanism (CSM) service. This is an adapter between IPF and a payment scheme that implements aspects of a payment scheme, such as:
Transport and/or gateway requirements (message format, headers, crypto, HMAC, etc.)
Validation (schema validation, mutual exclusivity, disallowed fields, etc.)
Mapping between the scheme's message format and IPF's ISO 20022 canonical model
Timing around retries and inquiry for track-and-trace
We currently support about 10 scheme packs. You are probably familiar with the classic software engineering mantra that premature optimisation is the root of all evil. When we built one, two or three scheme packs, we were sort of trying to convince ourselves that all this copy-and-pasting and find-and-replacing that we're doing is totally fine for two reasons:
The work required to extract common functionality would take as long as, if not longer than, the time it would take to just copy it all one more time. Why? Because these payment schemes are similar, but just different enough to make it not economically viable
What's the likelihood that there are sweeping changes that require us to change all common functionality that is currently duplicated across various codebases?
I know some of our readers are visually oriented, so here's a diagram showing what the situation looked like:
Turns out that the two arguments above are okay for two or three scheme packs. But what about 50? Or 100? It became clear that if we were to accelerate development of scheme packs such that we could make a very large number of them, we'd invest some time into identifying common functionality. I believe scheme pack development follows the 80-20 rule: 80% of the functionality can realistically be delivered by 20% of the code, which, in this case, would be the basic happy and unhappy scenarios for an inbound and outbound payment. So how much of that 20% can we automate? And can we go even further than that?
We decided that while we're already identifying and extracting common functionality, we should also provide a fast and reproducible way to make new scheme packs. A scheme-pack-shaped cookie cutter, if you will. So my colleague Ben Savage and I set out to individually tackle these challenges: he would identify and extract common functionality into a new common CSM module, and I would make the cookie cutter.
Scheme packs mostly consist of sending and receiving messages to/from payment schemes. Using the various stages of ourConnector framework, Ben implemented what I would call a visitor pattern for each stage: have the common module do something opinionated and sensible at each stage (e.g. mapping from IPF canonical to the scheme format or vice versa). If you wish to do something different inside your specific scheme pack, you can override that functionality and supply your own.
He also made this capability globally overridable - is that a word? - for an entire scheme pack, or a single message that the scheme pack supports. For example, if a scheme doesn't use XML (our sensible default) as its wire format, you can override the common message conversion to JSON instead of XML, or protocol buffers, or whatever. Alternatively, if you don't like what the common module is doing with a single message (e.g. how it's mapped from canonical to the target), you can just override that one part.
Second, he created a new domain specific language (DSL) for defining IPF canonical-to-payment scheme message mappings. He did this using the JetBrains MPS language workbench which we use to develop our IPF Flow Designer.
I keep calling it a cookie cutter, but I might as well refer to it by its official title: a Maven archetype called csm-service-archetype. An archetype is a way to create a Maven project from a template1. This allowed me to define a project template with the correct structure, pre-written BDD tests, and sensible defaults already baked-in.
Something I didn't know about Maven archetypes is that, in addition to just specifying the structure and doing basic templating, you can specify a post-generation script in Groovy which has access to the initial properties and can do anything Groovy can do. This means that you can extend the archetype generation process to pretty much do anything.
One of the limitations of the archetype is that it can't be used to generate new files. You specify a structure containing a set of files (which are actually Apache Velocity templates), and those templates are processed when you run the archetype. However, since we've discovered that we can use Groovy to do anything in a later step, we can just create a bunch of new files ourselves!
This allows me to ask the user for a directory on the file system that contains a set of scheme-specific ISO 20022 XML schema definitions (XSDs), from which we can make scheme-specific code that can't be made common. So you could say that the code writes itself. See? Not a clickbait title.
Now that we have:
New common functionality with overridable sensible defaults
A new scheme pack DSL
A Maven archetype
We can combine these things together and make an initial shell of a project, based on some guiding inputs from the user, containing only the parts that are specific to this payment scheme, and optionally some overrides of the default functionality. Of course the idea is to avoid adding any overrides, since the point is to try to churn these out faster than you can say "clearing and settlement mechanism."
We ran an experiment with the FedNow scheme to compare how long it takes to bootstrap a scheme pack from nothing to MVP-level functionality using our fancy new cookie cutter approach vs. how we did things in the past. And the results are in:
| Scheme | Approach | Percentage improvement | Comments |
| RT1 | Copy-and-paste bonanza | (baseline) |
|
| TIPS | Copy-and-paste bonanza | 71% |
|
| SIC5 | Copy-and-paste bonanza | 86% |
|
| FedNow | Improved cookie cutter process | 97% |
|
I wish I could compare FedNow to RT1 so that I could say we can now do it 97% faster! But realistically RT1 is much more feature-rich than the other ones due to the fact it implements the full array of that scheme's messages, compared to SIC5 and TIPS which only implement the messages our customers are interested in. But, still, 71 to 14 is still an 80% improvement and more realistic for being able to do this at a much bigger scale.
The above results make it clear that it was worth spending the time to do this. Sometimes you have to slow down to speed up: the other scheme packs had to be delivered to meet client deadlines, and slowing down to identify commonalities between them seemed crazy at the time because, hey, deadlines!
Don't take my word for it; how about this McKinsey article I read a while ago about their experience with getting teams to slow down to speed up as well. It's not unique to IPF, or Icon, or even software engineering. Try to isolate a small subset of the team from the usual daily activities, so that they can investigate how the current process can be improved upon. In the aforementioned article, they had a few two-day sessions away from their usual daily activities In our case, Ben and I had to let go of some of our commitments for a few weeks to write all of this new stuff.
I've found that getting new joiners or junior members of the team involved in this process to be very valuable, as they are more likely to bring new ideas because they are not as ossified in their ways as the rest of us old timers who have been around the block, and are - at times - a little too cynical.
My closing advice would be to not be afraid to let the one(s) signing the cheques know that perhaps it's time to take a break and look at things from another angle, so we can see how we can continue to identify ways of becoming faster and more efficient.
1 Actually, I'm not sure that it has to specifically be a Maven project, or even a Java project. You can integration test your archetype by having it create a test project from your archetype and running goals you specify in a goals.txt file. But if you don't do that then you can theoretically make any sort of project?