Personal API Experience

Personal API Experience

Daily musings

Sharing my day to day lessons, ideas and contemplations of architecting, developing and maintaining services (API) in the global marketplace we call the internet.

The Hollywood Principle

APIPosted by Craig Hughes Thu, September 06, 2018 12:24:37

Inversion of Control: Don't call us, we'll call you.

Remember when you were a child, going on a long trip with your parents. Remember the frustration; “Are we there yet?”, “Are we there yet?” As an adult now - imagine or remember your own frustration from being nagged!

For both parties, this pattern is not ideal; children are frustrated with not knowing, while parents are frustrated with constantly being nagged. So, why do we think this pattern is OK in software architecture? Why do we ask systems to constantly check other systems for updates? Why do we allow this tight coupling?

Fortunately, computers are not human; they have no emotion nor a concept of time. They don’t mind constantly asking other systems for an update and other systems don’t mind being asked. All they do is check again and again so they can do something. This constant chatter between systems can, and does, have an impact. So, instead of asking systems to constantly "check for an update", why not configure them to react to an event – tell them; “I’ve done something you may want to know about”.

When something happens in Hollywood – the world knows about it; whether we choose to do something about it is our own decision. A reporter writing the story about an event does not know who will read the story or what they will do with the information they receive. The reporter only expects the newspaper will publish the story for all its subscribers to read.

In software architecture, we can publish business events using the publish-subscribe pattern (PubSub). Interested systems can subscribe to events based on an event topic and decide, for themselves, what to do with the data received. In essence, when the source system done it's bit, it can publish data about the event via PubSub to the subscribers. Decoupling is important; the source system does not, and should not, care about what the subscribers do with the data – they have already completed their task.

Since the subscriber consumes the event data for their own purposes, it is their responsibility to complete their task, within their bounded context. So what happens if the subscriber fails to react to the event (via an error or broken communication)? Is it viable to ask the publisher to replay the event? No! The subscriber cannot assume they are the only audience of the event. Just like the publisher is not aware of the subscribers, subscribers should also not be aware of one another. Imagine republishing an event that leads to duplicate communications to customers or duplicate actions!

We can:
1. publishing event data via persistent queues to ensure delivery, and/or
2. expose APIs for retrieving published event data for reconstitution queries in the event of an error by the subscriber.

While PubSub does seem like a simple pattern, implementation can be challenging. Publishing a single event using Change Data Capture (CDC), off multiple tables, is just one of those challenges. That's a whole new ball-game, and a subject for another day.

Photo provided by

  • Comments(0)//

The Butterfly Effect

APIPosted by Craig Hughes Wed, September 05, 2018 21:11:36
We are currently doing some domain driven design to understand our primary business domains and identify the appropriate core APIs required. This is an exciting yet daunting time - some domains look far too simple while others seem overly complex and almost impossible to model. One such domain is the payments domain.

A few weeks ago, during one of the payment domain workshops, one of my colleagues compared payments to a butterfly and challenged me with the question: "How do you model a butterfly?" My knee-jerk reaction was to rise to the challenge and prove it possible, but I soon ran into the problem. Butterflies are the end state of metamorphosis!

The butterfly has four stages to it's lifecycle; egg, caterpillar, pupa, butterfly. Each stage is different and has a different goal. The challenging part is, while the creature is the same, its life stages are completely different. The egg is simply a sphere with something growing inside. The caterpillar is a long cylindrical creature with a lot of legs. The pupa is a mass of thread woven into a cocoon shape. The butterfly has wings, and six legs. How can the same creature be modeled if it is a completely different object during the stages of it's life?

This stuck with me for a while as I consciously left it to percolate in the back of my mind. Yesterday, while on my way home on the metro, I had the "Aha" moment!

The answer is you can model a butterfly - by the individual stages of the creature at any moment. The question leads you to assume you need to model all stages of the creature as one model since it is one creature. This you cannot model. The relationship between metamorphosis and the butterfly is the same as payment to transaction. Both are a process, not an entity.

So, simply put, model the stages of the payment process as separate entities and serve these as core domain APIs. Next create process APIs that manage and orchestrate these core domain APIs to fulfil the payment process.

It's just as important to phrase the question as it is to understand it.

Photo by

  • Comments(0)//

"Making coffee" to explain APIs and Domains

APIPosted by Craig Hughes Wed, September 05, 2018 20:08:17

I was invited to a meeting recently, to explain APIs and why they are important for our digital blueprint. I had no idea who the audience was nor their level of understanding of the basics of APIs. So, with no handy deck prepared for the meeting I entered the room, armed with my laptop full of previous presentations.

I was introduced to the team and the scene was set - "we've heard about APIs and need to plan to do them next year". This prompted the obvious question: "does anyone know what an API is?". Being an honest audience, the response was "no, but we've heard they are reusable and will make our life easier when we have them".

I first explained the simple stuff; that an API is simply a resource made available to a computer in the same way a website is made available to us humans - via an address. Next came the challenging part; time for coffee.

Making coffee is second nature to us, but imagine trying to get a computer to make you some coffee? You have to tell it explicitly what you need done and the more information we give the computer, the more questions may be raised: What is coffee? How much milk? Do you want milk? What is Milk? Where does milk come from? What is a cow? Where do I put the coffee? What do you do with coffee? These are some of the notions we take for granted as humans, because we've learnt about coffee. Now, computers can also learn about coffee, but that is a completely different topic; not one we will cover here. Suffice to say, getting a computer to make coffee requires a lot of attention to detail.

So, how do we do this? Well, we do that same thing we do with most problems; we break them down into smaller pieces. Making coffee involves a number of "things"; water, cup, teaspoon, coffee grains, sugar, milk, kettle, etc. Lets call these domains; the 'coffee grain' domain, the 'cup' domain, the 'milk' domain, and so on. Now, we can take each domain and define the attributes (properties) of it. For example 'milk'; it's white in colour, is a liquid, is usually cold. The 'cup'; it's a solid, it can hold a certain volume, it can hold liquids, it has a handle. Once we have defined these domains and their properties, we can now tell the computer how to use them to make coffee in simple steps.

  1. Boil water in the kettle
  2. Put one teaspoon of coffee in a cup
  3. Put one teaspoon of sugar into the same cup
  4. When the water in the kettle has boiled, pour 200ml of the water from kettle into the same cup
  5. etc.

APIs can be created in the same way. We define the basic domains of the functionality we want to expose as services (another word for API). In our world, these domains could be customer, account, product, transaction, etc. These form the reusable core APIs. Now, we can use these core APIs to do more complex stuff, like creating a payment API (process API) which orchestrate the core APIs.

  1. Authenticate the customer
  2. Get the customers accounts
  3. Select the right account
  4. Create an instruction to pay the customers registered beneficiary from the selected account
  5. Authorise the payment
  6. View the transaction

With these basic core domain APIs we can now perform multiple other processes. Using most of the domains associated with making coffee, I can replace the 'coffee grains' domain with 'tea bag' domain and have the computer can make me some tea instead.

Photo by Jessica Lewis from Pexels

  • Comments(0)//

Why is Collaboration Across IT Teams so Difficult?

CollaborationPosted by Craig Hughes Fri, August 31, 2018 13:50:32

Different groups within an IT department will have different agendas and levels of understanding. The Architecture team is sometimes seen as being too hypothetical, while the Delivery team may be seen as being too focused on delivery. This drives a different approach to solving the same problem. The Architecture team may envisage an approach that’s too complicated, while the Delivery team may want to implement an approach that’s overly simplified, but gets the job done quicker. Somewhere in the middle, we need to find the common ground, and maintain the “workable” bigger picture.

So, how do we collaborate given these sometimes disparate ideologies? How does the Architecture team educate the members of the Delivery team on the bigger picture approach and the reasons for it. At the same time, how does the Delivery team communicate with the Architecture team members to share with them their pain points and pressures – driving the direction of the bigger picture to something that this easier to accomplish, or at least evolutionary?

The crux of the matter, I believe is grounding or frame-of-reference. We all have history, ideas, notions, experiences; and these help us interpret the message we receive. As groups, we have different responsibilities and therefore carry different messages; our way of speaking and the vocabulary used is based on those responsibilities and frames-of-reference. In other words, what we are trying to communicate across groups and disciplines may not be received in the way it was intended to be delivered because our frame-of-references are different!

One final view to consider is that of the people. In order for a message to be received and understood by the collective, it is important to ensure the individual understands. And this could be the biggest blocker in proper collaboration. Consider an individual who is the master of what they have been doing for a while; every one listens to them and asks their advice because they are the gurus. This elevates their position of seniority in the team (perhaps virtually) and the perception within the team is that if they (the guru) agrees, then it must be so. This individual could be in the Architecture or the Delivery team. Now, consider the scenario where this individual is being challenged on their specific approach and told they need to re-skill in order to allow the organisation to adopt a new methodology. It will be very hard for this individual to let go of their “guru” status and become a “learner”. These individuals will naturally challenge any change to protect their status and often discourage or confuse the other team members.

Albeit from a delivery perspective, I found this article very interesting:

Photo by from Pexels

  • Comments(0)//
« Previous