A Seven-Step Guide to API-First Integration – InfoQ.com

Live Webinar and Q&A – Kubernetes Ingress 101 (Live Webinar Sept 30th, 2021) – Save Your Seat Save Your Seat
Facilitating the spread of knowledge and innovation in professional software development

The Excel team announced LAMBDA, a new feature that lets users define and name formula functions. LAMBDA functions admit parameters, can call other LAMBDA functions and recursively call themselves. With LAMBDA, the Excel formula language is Turing-complete: user-defined functions can thus compute anything without resorting to imperative languages (e.g., VBA, JavaScript).
In this episode, Thomas Betts speaks with Tammy Bryant Butow, principal SRE at Gremlin about training new site reliability engineers. The discussion covers a formal SRE Apprenticeship program Tammy led at DropBox, and gets into ideas about the best way to teach people new technical skills.
In this second edition of the Modern Data Engineering eMag, we’ll explore the ways in which data engineering has changed in the last few years. Data engineering has now become key to the success of products and companies. And new requirements breed new solutions.  
Sprint 0 can be a great mechanism in Agile transformations to reset existing teams which are not delivering value, exhibiting a lack of accountability, or struggling with direct collaboration with customers. This article shares the experiences from doing a Sprint 0 with an existing team which was struggling to deliver, helping them to align to a new product vision and become a stronger team.
The panelists discuss monitoring and observability methods that DevOps and SRE teams can employ to balance change and uncertainty without the need to constantly reconfigure monitoring systems.
Learn how to apply containerized applications to improve application speed, reliability and deployment. Virtual Event on September 21th, 9AM EDT / 3PM CEST
Learn how to apply Microservices and DevSecOps to improve application security & deployment speed. Virtual Event on Oct 19th, 9AM EDT/ 3PM CEST
Turn advice from 64+ world-class professionals into immediate action items. Attend online on Nov 1-12.
InfoQ Homepage Articles A Seven-Step Guide to API-First Integration
Nov 10, 2020 12 min read
Asitha Nanayakkara
reviewed by
Thomas Betts
From the time you wake up to the time you go back to bed, how many digital services do you think you interact with within a given day? All digital media — e.g., mobile applications, web sites, social media applications, everyday banking applications, and ATMs — use APIs, and, directly or indirectly, we interact with APIs throughout the day.
According to Akamai, the world’s leading content delivery network (CDN) provider, 83% of its traffic came through APIs, in contrast to HTML traffic in 2019. For comparison, the company reported API traffic of around 47% in 2014.
Thus, as integration specialists, we can’t ignore the presence of APIs. We have to embrace it and develop our integration use cases with an API-first integration mindset. In this article, we discuss how we can run an integration project in an API-driven manner with a seven-step execution plan.
Choreo: Low-Code Cloud Native Engineering for Professional Developers.
Engineer in low-code and pro-code simultaneously. All in hours, not weeks. Try it now — it’s free!
Before we delve into the details of "API-first integration," let’s review the benefits of using APIs.
We know that API is an acronym for Application Programming Interface. In an API-first integration strategy, we have to think of APIs similar to a business contract. When we get into any serious business with another party, we initially make a contract or an agreement between the two parties. We don’t start doing business until we get the contracts right. Similar to that, in integration use cases, we need to get the APIs done first. With that in mind, we design the APIs to expose a predefined set of capabilities of the backend systems to the frontend applications such as websites, mobile applications, and IoT devices.

Figure 1: The role of APIs
Properly designed APIs are a key enabler for any enterprise’s digital presence. The following are a few other examples.
All the above benefits depend on how we design and implement our APIs and the rest of the integration use cases.
Let’s look at how we can achieve the above benefits in our integration projects.
Nowadays, we predominantly see two approaches taken by enterprises when developing their APIs, be it RESTful or some other. These can be described as integration-first or API-first.
An integration-first approach is a traditional bottom-up approach where we build the integration logic first on top of backend services and then expose the integrated services (also known as composite services) as managed APIs to users.
In an API-first approach, we start with the API design and then move into the implementation of APIs as well as the integration logic. Since we begin with API design, we call this the API design-first or, for brevity, API-first approach. Throughout this article, we will use these two terms interchangeably to denote the same approach.
Let’s look at those two approaches within the context of a slightly abstracted-out example of an integration project.
Project Requirement:
Provide a digital experience to customers using a mobile and web application by exposing the company’s current capabilities through an online portal. There are backend and frontend development teams involved in this project.
This approach adversely affects the project timelines and costs. There is less communication between the teams and the design flaws are identified at a much later phase of the project.

Figure 2: An integration-first approach
This approach drastically reduces project delays and cost overruns due to miscommunication between frontend and backend teams leading to changes in APIs and backend systems.
After designing the APIs, it can take some time to get the live backend systems up and running for the frontend teams to make API calls and test the system. To overcome this issue, frontend teams can set up dummy services, called mock backends, that mimic the designed APIs and return dummy data. You can read more about it in this API mocking guide.
There can be instances where the requirements are vague or the development teams aren’t sure about the right approach to design the APIs upfront. In that case, we can design the API for a reduced scope and then implement it. We can do this for several iterations, using multiple sprints until the required scope is implemented. This way, we can identify a design flaw at an earlier stage and minimize the impact on project timelines.

Figure 3: An API-first approach
In software engineering, the façade design pattern is used to provide a more user-friendly interface for its users, hiding the complexity of a system. The idea behind the API façade is also the same; it provides a simplified API of its complex backend systems to the application programmers. As mentioned earlier, what we design using the API design-first approach is the Edge APIs that fronts this API façade.
The following diagram shows how it all glues together in an enterprise system.

Figure 4: An overview of an enterprise deployment
According to the above diagram, we have different backend systems of the organization and the user base (mobile, web, and direct API consumers) on either side. And, we use a combination of Edge APIs and API façades to connect those two entities.
If we further analyze the API façade, we can see it consists of utility APIs as well as domain APIs.

Figure 5: An API façade
If your backend logic is complex or communicates in a different protocol, you can design a utility API to expose that backend to the rest of the integration logic. It isn’t a must to expose all your backend services through utility APIs. If you have a RESTful SaaS application like Salesforce, you can directly connect to it if there are no complex interactions involved. Otherwise, you might design a simplified utility API to front the backend service.
Once you have your backends exposed directly or through your utility APIs, you can use a domain API that contains the message mediation logic or the business logic. This message mediation can contain things like message transformation, routing, service chaining, etc.
Another aspect of designing an API-first integration solution is to know your existing digital assets. In fact, this is one of the initial tasks of an integration project. There are different types of digital assets, and the way we integrate with them varies for each type.
Application silos are mostly the systems designed to be used by a specific department or a unit. These were never intended to be used organization-wide or exposed to the outside world. Mostly these systems are tightly-coupled to a specific department’s operating procedures and practices. Exposing these kinds of systems through the mediation layer needs special care and attention.
These are already developed as services to be used by the organization, hence need minimal effort. These are typically exposed through RESTful APIs.
When it comes to data-at-rest, organizations use different formats to store them. We need to get an idea of the data storage mechanisms, formats, and any sensitive data that we need to transfer from one entity to another through our integration layer.
We need to identify any workflow systems in the existing system. These may have different approval processes or human interventions.
There can be existing systems that work in proprietary data formats. For instance, old database management systems, CRMs, etc.
Understanding the existing capabilities of the system as well as the new requirements is key to a successful integration project. For instance, understanding the required capabilities will be a decisive factor in either using an existing integration product or developing one in-house to cater to these capabilities.
Let’s look at some of the key integration capabilities we see in most integration projects.
If you lack the proper experience or knowledge in designing integrations with the capabilities identified, it would be best to get help from an integration vendor. These integration vendors have vast experience working in many integration projects across the globe in different business domains. They will be able to assist you throughout the integration project from API design and implementation to even maintaining the system once it is up and running.
When selecting the most suitable integration vendor for your project, you need to look at whether the capabilities you require are provided by that vendor. For instance, you might be looking at a hybrid integration solution to gradually move your existing IT infrastructure to the cloud that would lead to a reduction in IT infrastructure costs.
We discussed several aspects to be aware of when we undertake an enterprise digital transformation project. Now we’ll describe a seven-step execution plan that follows the API-first strategy, to bring order to the chaos and deliver a successful enterprise integration project.
1. Identify the project objectives
By working with stakeholders and gathering requirements, we have to formulate the project objectives. Based on that, we decide the project scope.
2. Understand the enterprise ecosystems
a.    Identify any application silos
b.    Understand the data
Once we identify the project objectives, we can go through the existing system to identify application silos, different backend systems, data handling requirements, etc., as we discussed earlier under Identifying digital assets.
3. Identify the possible integration points for each system
With the objectives of the project, we might not need to touch some of the existing systems in the enterprise, or we might need to think about future expansions and select an integration vendor based on that information as well. Therefore, it is important to identify the systems we need to interact with and the specific functionalities we need to expose from those systems.
4. Identify the integration capabilities required
With the understanding of the system’s digital assets and the overall objectives, we can decide on the integration capabilities required as discussed under Key integration capabilities.
5. Design the edge APIs
These are the APIs that frontend teams will interact with. By first designing the edge APIs, which mostly cover a particular system functionality, we avoid leaking complexities of backend systems into frontend applications. This leads to a much cleaner API design for application programmers.
6. Design utility APIs and domain APIs
Once the edge APIs are designed, the backend team can design the utility APIs and the domain APIs to suit the requirements of the edge APIs. If unforeseen issues are encountered, they can go back and update the edge APIs as required.
7. Implement the frontend and integration logic
Once the edge, domain, and utility APIs are designed, both the frontend and backend teams can start implementing the system.
These steps can be followed for several sprints by breaking down the business requirements into a set of smaller problems, rather than trying to implement the whole business requirement in one go with a set of edge APIs.
In this article, we discussed an API-first approach to integration projects that reduce the project costs and timelines, increase collaborative work between teams, and improve the overall quality as well as room for future expansions. Also, we discussed how we can formulate this approach as a strategy with a seven-step execution plan.
Gartner says, through 2020, integration work will account for 50% of the time and cost of building a digital platform. So it is of utmost importance to build your digital platform adhering to proper development best practices.
Asitha Nanayakkara is a Software Engineer with a bachelor's degree in computer science and engineering. Currently, he works as a technical lead at WSO2 providing expertise to build integration products. Asitha has experience working with international companies, including fortune 500 companies, as an integration consultant. He likes to share the knowledge he gains while working on those integration projects with others.

A round-up of last week’s content on InfoQ sent out every Tuesday. Join a community of over 250,000 senior developers. View an example

We protect your privacy.
You need to Register an InfoQ account or or login to post comments. But there’s so much more behind being registered.
Get the most out of the InfoQ experience.
Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

A round-up of last week’s content on InfoQ sent out every Tuesday. Join a community of over 250,000 senior developers. View an example

We protect your privacy.
Focus on the topics that matter in software development right now.
Deep-dive with 64+ world-class software leaders. Discover how they are applying emerging trends. Learn their use cases and best practices.
Stay ahead of the adoption curve and shape your roadmap with QCon Plus online software development conference.
InfoQ.com and all content copyright © 2006-2021 C4Media Inc. InfoQ.com hosted at Contegix, the best ISP we’ve ever worked with.
Privacy Notice, Terms And Conditions, Cookie Policy
Is your profile up-to-date? Please take a moment to review and update.
Note: If updating/changing your email, a validation request will be sent