The technology “disruption” occurring in today’s business world is driven by open source and APIs and a new paradigm of enterprise collaboration

Disruption and Reuse

It is my contention that 90% of costs are being eliminated from the traditional software development process and the time to market reduced dramatically by leveraging the reuse capable today via open source, APIs, fast deployment and resource sharing with PaaS. The cost of Enterprise Software was magnified by an order of magnitude by the lack of reuse prevalent in the old paradigm of software development. This is apparent as we see how fast we are able to build technology today. This is a major reason for the massive adoption of disruptive technologies of open source and APIs we see today.

The closed source world of yesteryear

Almost every enterprise has over the years been rebuilding the same IT technology over and over that other enterprises built. Within the same enterprise it is not uncommon to find that they have many applications which have lots of similar functionality which were built almost from the scratch up each time. This happens for lots of reasons I talk about in another blog about “inner source.” Inner source is a way larger enterprises concerned with IP or secrecy of their code can try to gain the benefits of collaborative open source development. I highly recommend everyone understand this model. Please check out that article.

Even if 90% of the cost of software development can be reduced by leveraging reuse this is not the important benefit of reuse! The most important benefit of reuse is the increased innovation we are seeing and speedier time to market. Open source, inner source and building reusable public or private APIs, services and components enables an organization to leverage all the talent in the organization and creative people outside the company to create disruptive value and then to distribute that value more rapidly to the enterprise and to the market faster than ever before.

Each new technology, service, open source project provides a way for you to piggy-back on all the invention and creativity of everyone else who is moving that open source or service forward. This is not just motivational speaker gobbledy-gook or marketing speak. This is happening and creating an unmistakable tsunami of change.

Tsunami of technological change unleashed by key companies leveraging open source to create a new paradigm of compeitition

By any measure of technological change we have been and continue to be in a tsunami of technology innovation that dwarfs previous times. This cannot be denied. I have statistics and examples later in this blog. It’s hard to imagine that it was literally a handful of years ago that Yahoo and Google, Facebook, Twitter and others started down a path to bigdata with HBase, Hadoop and other bigdata technologies. The story is worth a book (which to my chagrin hasn’t been written yet.) These companies reused each others technologies and learned from each other quickly. Constantly improving the underlying technology so that they could provide greater and greater value, grow faster, improve their services by orders of magnitude while increasing their customers by many orders of magnitude in a matter of a few years. We have never seen companies in other industries do this with such openness. There was always a “stealing” of innovations or talent that occurred in corporations when some disruptive innovation came into the market. Some copied others business models, some hired talent from the innovating organization to replicate the new innovation inside their company. Some companies did it more nefariously undoubtedly. The only thing that differs with the open source model was that the companies in the Yahoo, Google, Twitter, Netflix, Facebook world did was to do so openly with full support of their organizations encouraging sharing with competitors. They allowed their engineers to pretty freely share the underlying technologies. The result has been a more rapid technological pace of change that has left everyone else in the dust. This change was needed so that these companies could grow to the scale they have and to support billions of users, to provide the kinds of services their CTO’s demanded, to adapt to the mobile revolution and the social revolution. Each of these technology advancements simply sparked more innovation in the other areas creating a virtuous circle where they fed each other:

The same open source contribution model repeated for mobile apps, back end as a service, mobile application development, cloud technology (IaaS and PaaS) and other areas of technology. It is true for social technology like Twitter, Facebook and similar companies. A storm of open source projects (one named storm 🙂 ) in all these spaces and more has created massive disruption. Cloud computing platforms such as OpenStack have enlisted broad industry participation and created massive value and a 100 billion dollar market for the cloud in a few short years.

Culture is important ( Surprising finding: More people than you think are honest )

Culture is a critical component of any successful disruption. I believe, for instance, that the basically honest hard working technological worker culture of Silicon Valley was responsible for the success of the VC industry here and the valley in general. You could invest in a company in silicon valley and with almost no exceptions the entrepreneurs and people practically worked themselves to the bone doing everything they could to succeed. This is not the story you may hear of the profligate profits of the ultra successful companies. What is not mentioned is the thousands and thousands of companies that sold themselves for break-even or ended up closing shop. Those companies generally speaking gave it their best shot. If this didn’t happen many investors would never have funded the thousands of companies needed to create the multi-billion dollar successes that we all know about and the miracle of silicon valley would never have happened. The transparency and honesty of the underlying engineers was a critical factor in my opinion in making this model work.

An Example

At one point during TIBCOs financial focused years we were building stock exchanges and the thought occurred to me before the creation of Ebay that we could take our stock exchange technology and put it on the internet to allow people to exchange anything. We were thinking of this before Ebay. However, I could never imagine how you could get a person to part with their cash not knowing if the product would be shipped to them. Vice versa, why would anybody send a product to someone if they didn’t know if the check was really going to be coming. In my opinion the brilliance of Pierre Omidyar (founder of Ebay) was encapsulated in this one word: Transparency. Who would guess that getting a good feedback from a buyer or seller would be so important to people? I have done hundreds of Ebay transactions over the years and I have not had a single case of fraud. I have a 100% positive feedback score and I’m proud of that and guard it religiously. So do the vast majority of Ebay’ers. I never guessed that people would be so trustworthy. 🙂 There are bad apples everywhere but they are fewer than many of us think surprisingly. Unfortunately, it doesn’t take many bad apples to get the whole bushel discarded. Open Source has a culture of contribution and giving back, honesty and help. Why? Why do this? Why do something for nothing? A lot has been written on this topic so I won’t belabor that. I will simply say that the culture of open source has contributed tremendously to the success of the movement and to open source. The companies involved in many of the successful projects did so for selfish reasons as well no doubt but the overall benefit on everyone from opening up the source code of everything has been surprising. It unleashed another massive wave of technological innovation greater than any before.

Some interesting statistics and thoughts on the pace of change

Open source software lines of code has been multiplying by a factor of 2 every 12 to 15 months according to a comprehensive survey in *1. While this survey doesn’t measure up to today it seems highly unlikely given the number of projects and companies I know about that this growth rate has changed. The number of open source projects is doubling every thirteen months according to the same survey. In *2 Coverity found that open source software quality exceeds proprietary software quality. Black Duck in a survey in 2014 found that respondents increased by 50% in the latest survey to their open source survey, a measure itself of the growing interest. Results have moved remarkably from thinking open source software is the cheap alternative to becoming the best quality alternative. *3 In *4 Survey found that 1/2 of all software acquired and used over the next several years will be open source origin.

I don’t need surveys to see what I hear from everyone I talk to and the stories in this industry. It is clear there is a massive increase in the pace of change. Just keeping tract of new interesting projects and companies, technologies is a challenge these days. How do you keep up?

Some people have commented that they believe the open source model is dead. They point out that only Redhat and a few other companies went public and the value of open source companies is far lower than non-open source companies. This is fallacious for many reasons. Open source movement started with things like linux and XML and a few other marginal technologies. Today openstack, cassandra and numerous other open source projects which were created as part of the cloud and latest innovation spiral have just started to see wider adoption that I believe presages the next phase of open source company success. We are seeing accpetance in enterprises of open source technology just really getting going. I think we are really only at the beginning of this movement not the end.

Is Open Source always better?

We have always assumed that the intellectual property of the source code was so important that to give it away you were killing yourself and your company. Were people wrong about this? Is there any merit to guarding IP and secrets? There are places where IP protection makes sense. I don’t have the general rule for it. I think an economist must have written a paper that could elucidate the societies cost / benefit tradeoff from guarding IP or not guarding it. It’s pretty clear that if someone shares something voluntarily then they are helping society however, they may be hurting themselves in the process if nobody else reciprocates. There are clearly cases as seen in the open source movement where giving the code away did not harm yourself. A good part of that must be if others also share their improvements to your openness. If you are the only one being open then it’s certainly possible you will lose out. If some reciprocate then the net benefit of collaboration may be greater than the value of holding proprietary IP. There are many other aspects of this that I could delve into but I will keep this post short.

Where is this all going?

The next phase of change will come from APIs in the cloud and the growth of what some are calling IoT and the next phase of what I call the network effect in the cloud. Whatever you want to call it, the connection of thousands of new services in the cloud will spur a technological and disruptive value in the effect of combining and using combinations of these services never imagined or possible before. In the same was millions of devices in the real world will at first work independently but eventually the greatest value will come from the ability to leverage multiple devices to create disruptive value. I call this the network effect. It will take 10 years for this movement to become very powerful force however I am certain that the value of individual services and individual devices will be dwarfed by the value we can create eventually from the combination of all the services and devices in ways we have not imagined yet.

Uber is a good example of how connecting services in the cloud, devices in the real world (cell phones and cars) has created disruptive value. Uber is worth $17 billion and all they do is have an app. No physical hardware themselves. Yet they provide massive value to people able to earn a living like never before and people able to have convenience that is marked improvement over existing approaches. Why should taxis be roaming the streets wasting gas and time when consumers of the taxi service can so easily today coordinate their location and desired service? The obvious value to both the driver and the consumer is so real it is causing massive disruption in many places. I can’t even imagine how all the information and devices, services eventually available via the cloud and with IoT will change our world but I am certain it will. We are just at the beginning of all this change. If you are scared of change or not prepared I am sorry. Nothing will stop this.

Hope you appreciate the ideas I have brought up.