In this series of blogs I will talk about the changes in software development over the last 10 years that have revolutionized our industry. I will also talk about what I see as the important trends and technologies. You can find at the end of this blog a list of the articles in this series and pointers to each.
This series of blogs is pretty comprehensive and will be delivered roughly daily for the next few weeks. In it I will refer to specific projects, technologies that are key.
Introduction
When I started programming all you needed was a basic understanding of the C language and possibly assembler languages. Your skill was determined by your ability to generate clever algorithms to perform tasks efficiently. The tools at your disposal were essentially stone knives. There were basic tools that made programming easier like the C runtime library so you didn’t have to rewrite the most rudimentary algorithms but basically it was a mathematical-like talent to construct algorithms and write software.
Today software development is about leveraging technology that has been developed already. Productivity is a matter of the systems you employ to make development faster and more efficient not as much about clever algorithms. It is about clever use of other people’s code.
Sure, there are people writing the basic tools, the Node.js’s, the Kafka’s, the platform pieces. However, for most enterprise software development you will not be productive rewriting Cassandra yourself or Docker. You will depend on these tools and many others to create massive efficiency and scalability.
This series of blogs introduces and elaborates the tools an enterprise today needs to become productive, fast, competitive and innovative to deal with the amazing possibilities of the technology at our disposal. Essentially to be able to operate in the software world we have today.
It is my contention if you don’t adopt what I am saying in these blogs you will find yourself more and more isolated and divorced from the customers of today and are essentially legacy. It may be fine to be legacy. There are insurance firms who operate as they did 30 years ago. If you have a massive established base of older customers who don’t care about or can’t deal with change you may be able to exist and milk that customer base for many years to come and find a business that you can survive but for most of the world and businesses we need to be able to deal with the new world.
A set of platforms (stacks) of new technologies has evolved that provides a revolutionary productive development environment. It is typical that this new collection of technologies achieve 10-100x the productivity from just a few years ago.
Since 2004 a virtual revolution in software development technology has taken place. Every aspect of software development has been affected from the tools, the languages. The way we compose functionality from components and services to build applications and the way we deliver the applications has all changed.
This technology change is so important and differentiated that it deserves a label, some have used the term Platform 3. I posit that this is actually a set of platforms designed around use cases. I will talk about 4 use cases of special merit in this blog series. Here is a description of those use cases. The use cases are not necessarily orthogonal. In other words you could need all 4. There are more use cases and more combinations of technologies that are useful that I may elaborate. For instance, there is a use case of companies interested in providing high speed information to consumers.
Each of the following use cases is composed of some of the same components and some different components. I will elaborate in following blogs.
1.1 10-100x previous software development productivity
Every aspect of the development and delivery process has undergone massive automation or improvements that have all made the process of developing software more continuous.
More productive languages with many built in features, libraries of open source components to leverage, fast iteration and testing helped by high degrees of automation and the use of APIs as a way to leverage other functionality has made the front end process of software development much more productive.
The ALM (lifecycle management) has been helped tremendously by the standardization of tools.
The deployment process has typically been ½ of the entire process of software development. With new tools that integrate the cycle between development, test and production it is possible to reduce the time to deployment to less than a day and in many cases minutes.
The combination of these improvements has literally improved productivity and the ability to generate functionality by at least a factor of 10 and sometimes 100 to 1 from older technologies.
Another aspect of this improved productivity is that most of these tools are available in open source which means that large numbers of people are able to become familiar with these tools and use them meaning the problems that the initial tool orientation training time for employees can be less because many people are familiar with these tools.
1.2 Intimate connection with customers -> Direct interaction with customers
A key aspect of this new software is that it is entirely oriented to web / mobile / IoT delivery so that it assumes an intimate always available connection to the customer wherever they are.
The social applications like Facebook and Twitter epitomize the social connected aspect of the new application environment but the App Stores, Ebay and Amazon have revolutionized how people expect to interact with companies and their products. They expect to see other people’s comments and reviews for everything, they expect to be able to comment and feedback to the people and services they interact with or they won’t use them.
There is a much higher level of integration with other services and devices expected in this new environment. Consumers of services whether on the development side or consumer side expect a social environment and a higher level of intelligence and smartness that comes from knowing about the environment and the customer they are dealing with and customizing to the customers expectations.
The Enterprise is also expecting that applications today have the ability to collect usage information from end users to enable a rapid improvement of the service, to do data mining to discover new opportunities to improve the experience or open new channels to customers or users.
This Platform or Stack includes components to make building web/mobile/IoT applications easily that are social and to gather information on all the usage and activity to provide the data to drive business digitization. This is a critical aspect of this platform.
A key part of this more intimate and social aspect is preserving the customers security and privacy. OAUTH2, OPEN_ID and other protocols have been developed to support high levels of security with users having control over their information and simplifying integration across multiple tools.
1.3 Much higher rates of innovation with daily, weekly or monthly improvements possible
The highly social world of today leads to what is called the network effect which is that combinations of people, services and things with greater connectivity through networks and devices enables exponentially increasing value as people are able to leverage those things to possibilities that didn’t exist before. High levels of innovation and change are expected as consumers of your services expect rapid innovation and change or they will migrate to other applications and services.
This Platform supports increasing innovation in all phases of the creative process. Innovation can be segmented into idea creation, idea instantiation, idea promotion, idea delivery and scaling. This Platform has aspects that facilitate rapid agile development and delivery of new functionality.
1.4 The ability to scale massively to billions of users and billions of transactions may be needed
The key companies that have developed many of the Platform key components have a scale never seen before by government or industry. As a result new tools which enable rapid Cloud scaling have evolved to enable incremental growth without changing architecture or making a commitment to high fixed costs in advance.
Cloud Scale means being able to fix problems fast, introduce new features fast and to handle failures transparently at scale to billions of users.
In order to achieve this new automation was needed at being able to deploy things into production quickly but also to manage hundreds even thousands of instances of services and to load balance between these.
The New Platforms Series
- Software Development in Disruption – Introduction and motivation
- Decomposition of the new Platforms or Stacks
- The Role of Open Source in the New Software World
- Revolutionary Productivity
- Intimate Connection with Customers
- Innovation Platform
- Scaling
- Problems in this new world of Software
- The Future of Software in the next 3 years
CloudRamblings 