Componentized Composable Platform (Enterprise Quality Lincoln Logs)

As an enterprise architect we are faced with construction solutions for our companies that must be implemented quickly but need to scale to almost arbitrary capacity quickly when demand materializes and must stand the test of time because it is almost impossible to rebuild enterprise applications once they are successful and in production.

There are many examples of the history of really successful Enterprise Applications and how they evolved.  At TIBCO we had to build several generations of our publish/subscribe technology.  Each time we had a major architectural change which reflected our need to scale and interact with more things than the previous generation could possibly handle.   Each time we did this was a company “turning point” because a failed attempt could mean the death of the company.  So, it is with great pride that during those years we made these transitions successfully. It is a very hard thing to do to build software which is designed and architected so it is flexible enough to adapt to changing technology around it.

Having built many enterprise applications we start with good ideas of how to break things into pieces but after 5 years or more it becomes readily apparent that we broke some things into pieces the wrong way.  The book “Zen and the art of Motorcycle Maintenance”  discusses how you can look at a motorcycle engine from many vantage points giving you a different way to break down the structure of the engine which gives you different understanding of its workings.  This philosophical truth applies to motorcycle engines, string theory and software development.  There are always many ways to break a problem down.  Depending on your purpose one may work better but it is hard in advance to know which way will work best for problems in the future you have not anticipated.

This article depends on a chemical to software analogy so let me clarify the terms I will use and what they mean

particles … the lowest level classes in the structure usually not dealt with except carbon engineers or contributors

elements … higher level components composed of particles that define the building blocks of carbon software platform (bundles)

molecules … visible products that are pre-built to perform well known usually commodity functions

mixtures or compounds … combinations of molecules, elements or other mixtures that form the architecture of applications

Physical Objects … mixtures or compounds formed into a shape that is useful for some purpose corresponds to applications

This is an arbitrary grouping (see Zen reference above) but for purposes of understanding how you can compose things in WSO2 world of Carbon components it is workable analogy.

WSO2 built its Enterprise Application Platform on OSGi and calls its platform “Carbon.”  The name is very apt because like carbon it can be formed and utilized in many compounds that solve real problems.  Carbon is the underlying structure of all WSO2 products.   Composed of 200+ components it is possible to compose almost anything out of the pieces.  This is a list of a fraction  of these “elements” of carbon:

• endpoint/     event/    event-processing/    eventing/    gadget-ide/
• gadgets/   gauges/   governance/   hdfs/   health-monitor/   ldap-server/
• load-balancer/   localentry/   logging/   mapred/   mashup/   mediation/
• mediation-initializer/   mediation-statistics/   mediation-tracer/
• …

These components are truly components and are written to operate as bundles in OSGi-component-speak, i.e. they can be each independently combined without worry about interference of one component with another or confusion of one component with another.  You can independently stop, start, upgrade each component and use that component anywhere you want with limited worry about what things it depends on.  This gives us a framework for building the higher level things that follow.

Since the best practices in software architecture have promoted SOA and similar ideas these Carbon pieces have been composed into molecules that look familiar to software architects in the middleware / SOA space.  Message Broker, Enterprise Service Bus, Business Process Server, Business Rules Server, Governance Registry, Identity Manager, Cloud Gateway, Elastic Load Balancer, Complex Event Processor, Data Services Server, Storage Server and others are the first order molecules that were build from the basic elements of Carbons componentized architecture.    This gives us the family of products depicted in this diagram :

These products have the great characteristic sought by almost all software architects that define them as components as well:

a) they can be instantiated independently and stand alone or interoperate with each other and in groups, clusters or scale smoothly

b) they all operate similarly using the same mechanisms for logging, administration, fault tolerance, clustering, security, configuration and many other things that make it easy to compose them into larger pieces.  Essentially they use each other to do all the same things so they all can be interchanged easily.

c) they all are written to work in OSGi containers

d) they all have APIs defined and published in a store so you can see them and interact with them easily

It would take a lot of time to go through and enumerate the great characteristics of a componentized platform written in this way.  As far as I know no other platform of this scope has been written like this before.  It’s a tremendous achievement.

The next level of composition is the one I wanted to focus this article on.  Now that we have a set of components that have the characteristics I outlined above it is easy to compose them into bigger pieces.  In the chemical analogy we are going from molecules to mixtures or compounds.  Mixtures have the characteristic that the more you mix in of one molecule in the mixture the more the mixture takes on the characteristics of that molecule.  This is unlike the lower level molecules from which the behavior of the molecule cannot be inferred from the elements that make it up.   So, if you mix in more ESB molecules you get linearly higher performance ESB from the mixture.    If you mix in some BPS into the mixture you get some BPS character.   You can scale the mixture to meet your needs in a predictable way.

The WSO2 platform contains some mixture products.   Examples of mixture products include:

WSO2 Private PaaS composed of Cloud Controller, Deployment Synchronizer, and other pieces

WSO2 API Manager composed of Enterprise Store, BAM, ESB, ELB, DSS, SS, …

WSO2 Data Platform with Bigdata comprised of BAM, DSS, SS, CEP, UES, …

WSO2 Enterprise Mobility Management comprised of MDM, MAM, Enterprise Store, …

WSO2 App Factory comprised of Jenkins, Maven, Git, WSO2 Private PaaS, BPS, ESB, ELB, Enterprise Store, …

The real power of WSO2 comes from the ability to form mixtures of our products to solve customer problems.  In this blog series I will go through and show you examples of mixtures that form use cases.   Use cases like Business Transaction Monitoring, Account Creation, Enrollment, Cloud Gateway, Order Processing, Billing, …   The ease to combine WSO2 components into solutions that solve customer problems easily and quickly is astonishing.   It is differentiated from the way we used to compose applications which is clunky, involves creating large integration projects, has substantial friction in deployment and operation, limitations on scalability or complexity in operation.  Usually combining products from one or multiple vendors that were not designed to work together, to be components requires substantial new code development just to get the pieces to interoperate, usually requires different tools to manage and they will not scale or know how to scale independently or automatically.   There may be issues in security, role definitions and authorization that complicate the combination.

With WSO2 you can form your own mixtures of components easily and combining these mixtures with other things from the menu or just combine basic molecules to form your solution.  Let’s say you want to form a medicare enrollment application.  You will need a combination of some ESB’s for integration with data sources, you need some BPS for some business process definitions of the process of data collection and enrollment.  You need a bigdata infrastructure to keep tract of all the information that may affect elligibility and claims processing.   You need a message broker to process all the applications.  You need some user interface components for building the screens.  You need identity management component to authenticate and process authorizations for all the operations.  We need a governance registry to keep all the policies, processes, configuration information and services we will have.   I will walk you through this in following blog articles to show how componentization facilitates rapid application development, deployment and operation.

All you need to do to finish this application is to write the business processes, connect the data sources to the ESBs and the data translations to the master data model, establish the policies and configure the roles.  Hardly any code would have to be written to implement the application itself.

After we do that, we would like the system to be able to deploy itself onto hardware automatically and to scale the number of instances of each component needed as the demand on the system grows.  That is what WSO2 Private PaaS does.

What we have now with this WSO2 platform is a composable set of true components that are easily combined into any configuration you want to solve any application you want.  You have a mechanism to grow the solution automatically as the demand on it grows.

I think this represents a leap in the way people think about building applications.

The componentization of WSO2 Carbon means that WSO2 can more easily compose atoms of its 100+ components faster to build new components and products than anyone else. This is the only way in my mind to explain how a small company was able to build the entire suite of technology WSO2 has with such speed and quality.  It’s the only explanation how a small company was able to build an API management solution that leads the field in months,  followed by building a private PaaS DevOps platform, an Ecosystem PaaS that manages lifecycle of application development and Mobile Device Management, Mobile Application Management, Complex Event Processing, Enterprise Store and more in just 12 to 18 months and have these products be industry leading enterprise grade products almost out of the starting gate.

So, WSO2 has eaten it’s own dogfood.  We use the composability to produce these higher level products with agility and quality not seen in the industry before.  We help our customers build solutions in faster time than they could have imagined doing before.

The next blog on this topic will describe the way to think about how different components fit along different axis of complexity in the problem space of enterprise applications.  For instance, how does duration of transaction affect which component is appropriate to solve the problem, how does frequency of changing rules affect which components to use, how does scale or need for speed (real-time-ness) affect which component is appropriate to use when building your solution.  Using this dimensional analysis I plan to show how the WSO2 Carbon toolkit for the enterprise is “complete” and comprehensive.

Following blogs will discuss different standard mixings of components to solve enterprise problems.  I hope you will find this series of blogs interesting, thought provoking and you will pass this on to others.   The next Blog on this topic:  component-completeness-what-is-a-complete-platform

Here are some interesting links related to this topic:

Composable Software:   http://www.forbes.com/sites/reuvencohen/2013/11/25/the-state-of-enterprise-software-in-2013/

More on composable platform:  http://wso2.com/whitepapers/wso2-carbon-the-composable-platform-advantage/

The next Blog on this topic:  component-completeness-what-is-a-complete-platform