API centric – The evolution of reuse and API Management

 

The Evolutionary Cycle

From the earliest days of the computer there has always been the idea that by reusing software we could achieve greater productivity.   We have collectively tried to do this first by packing operating systems with more and more stuff and realized at some point we really shouldn’t put so much stuff into that, what emerged eventually was linux – simplicity.   We then created more and more powerful languages that got more and more obscure until we settled on simpler languages with libraries of reusable functions – Java.  Those functions then evolved into larger and larger things that eventually became their own applications.   We connected the applications with messaging so we could talk between large applications.  (Yes, some of these movements overlap)   This evolved into smaller applications with interfaces that we called services,  and the movement to re-use services came about.

This proved not entirely satisfactory so we went to frameworks, large batches of functionality again built alongside languages that enabled more productivity.   As the internet emerged and the web, the cloud and SaaS emerged using these paradigms it became necessary to connect companies together over the web and SOAP emerged.  It proved to be heavy weight and people yearned for the simplicity of http, so RESTful APIs emerged.    As you can see this is a repeating thread of things becoming more and more complex and then simplicity again on top of a new foundation.

 

The APIs grew more and more plentiful.   This leads us to where we are today.   A web of tens of thousands becoming quickly hundreds of thousands of APIs growing by leaps and bounds.   Each of these APIs representing services that we can re-use.

Have we reached nirvana?

Do we have the ultimate development environments now?  A hundred thousand re-usable services?   I am preparing another blog to go over the diversity and range of services available today.  However, the purpose of these services so far hasn’t been to build applications utilizing dozens of reusable services, rather to enable applications to talk to each other and for mobile applications to talk to server applications in a one to many fashion.    So, the re-use is not happening the way we thought  … at least yet.

The Immediate Future

Now we get to prognostication.   This is where you can choose to agree with me or think otherwise.   This is what I believe will happen over the next 10 years.

1) APIs will continue to massively expand along with mobile and the need for automation and connectivity between enterprises grows, hence the flourishing of API and an API Manager to help people publish APIs to external consumers.

 

2) Simultaneously we will see the growth of iPaaS, the ability to take APIs from one application usually SaaS and flow information or combine information into or out of another (usually enterprise on-premise) application.   This is the equivalent to what we saw in the growth of messaging within the enterprise.

These two things are well underway.    One of the discussion points that people have about iPaaS is, should an enterprise use it’s existing Enterprise Service Bus or to use iPaaS in the cloud to do this integration.   There are good arguments to do the latter as the requirements are new and the capacity usually doesn’t exist in existing enterprise messaging infrastructure.

The Longer Term Future

However, that happens, the next obvious steps (in parallel) are:

First, massive competition among API vendors to gain mindshare and to dominate the vernacular for the API spec.  There will be standards to consolidate the number of APIs and the types of services by the APIs will consolidate.  At the same time there will be growth in the number of services and types of services.   We already see that happening with the IaaS vendors as consolidation seems to be gaining amongst the leading cloud infrastructure vendors already.   This movement has to continue.  The growth of industry consortiums in the airplane industry, car industry, entertainment industries, ad industry and others shows that in order to gain market share and to promote innovation companies are trying to consolidate rather than differentiate their APIs.   This is a “gain some and lose some” situation we always see where there is motivation by some to consolidation and motivation by others to be differentiated.  Eventually there is always a consolidation after the winners are picked.

 

Next,  Connecting two or more wholly cloud based applications together using iPaaS and then to grow and connect applications regardless of whether they are cloud based services or on-premise applications.  We have done this with enterprise messaging over the last 20 years.   We will see messaging bring the cloud and the enterprise closer.   The use cases for this are clear and the motivations.   Most companies cannot sustain an all on-premise application infrastructure.   Even the biggest companies are moving to SaaS based applications for some things.  This trend is unstoppable and inevitably more and more of enterprise applications will be SaaS (Cloud) based.  Further more and more enterprises will move their proprietary and even on-premise COTS applications into the cloud to be run.  When those applications move to the cloud the data will follow and security issues abound (but that is a different topic).

 

3) Growth of  API Manager WITHIN Enterprises to provide the same benefits of re-use, automation, publishing, SLA standards, accounting and communication between portions of the development stack inside any company.    This is API management turned inward.  If an API Manager makes sense for external use it makes sense inside a company.   The point of APIs is to make them easily re-usable by external parties.   The same need exists within the enterprise.  How we didn’t see this before and the need for this is not clear.   SOA clearly spelled out the need for service directories but it was one of the last things companies did with SOA.   The service directories were hard to use, did not make it easy to reuse services.   The evolution of the API Manager was to make it easy to reuse and this will continue to evolve through the next point of evolution.

4) Growth of api-centric programming where the apis become the main way development within the company starts to happen.   This is the ultimate dream of the service oriented architecture.  A huge enabler of this is the AppStore.   With the AppStore we have seen an easy way to manage a plethora of applications.   With the growth has come and will come a massive explosion of applications in the enterprise.    The Enterprise App Store is the only way to manage all this and the API store is a natural extenstion of the App Store.  API Centric programming means a number of things simultaneously.

a) Fracturing of Enterprise Applications into smaller and smaller “services”, the long tail of application growth we expect

b) Publishing all services in the enterprise and all applications into an enterprise store that manages the security, distribution, upgrading and SLA for all enterprise services

c) API Centric graphical programming by dragging APIs into applications and development environments and trivially combining services that makes programming into more of a graphical exercise of dragging pieces onto each other in a way that we have dreamed of for a long time.   Once we have services implemented like this we can start to fabricate more and more sophisticated ways to build software much like we use spreadsheets and graphical editors for constructing business processes.   Programmers will simply connect services to form all the functionality for any new application or service and services will become composed of other services.

d) More and more tools to help us compose services in reliable ways since any application that depends on n services has a reliability which is the multiplication of the percentage reliability of each of the individual n services.   If each of the n services is 99% reliable then the combined reliability of a service using n of these services is .99 to the nth power.  If n=10 (not unreasonable) then the reliability of the new service is only 90.4% or the service will be down 10% of the time.   Netflix has noticed this phenomenon and started to build what is called resilient services which deal with service failures.

It’s pretty obvious if you see this like I do that over the next 10-20 years this transforms the way we do application development and how we use applications.   Applications will be forced to move to the cloud because all the services the application uses will be in the cloud already and the performance in the cloud is 100 times the performance of running from the cloud into interprise (inside the 4 walls)  and enterprise applications (outside the 4 walls).   It is clear the new paradigm for APIs is more than a passing fad or about exposing external APIs to the world.   API Centric is a new way to think about how we program, what we can use to build applications and who can build applications.