BigData All the time NSA for you

We are at O’Reilly’s BigData Conference Strata in Santa Clara.  The halls are bustling.  Bigdata is doing well.  The WSO2 theme for the show is “NSA for you.”  This has attracted a lot of attention.  Most people seem to like the reference to the NSA’s recent notoriety.     One person came by the booth who said he was with (past or present wasn’t clear) the NSA and said: “It’s a bit offensive”  People came over from startups that were formerly from the NSA leveraging Accumulo that were jealous they hadn’t thought of it.   It seems to be a big hit and generally piques peoples curiosity.

 NSA for you

NSA for you does not mean that you can become the NSA and spy on other countries or people willy-nilly.   Sorry, if that is your interest we don’t really provide that capability.  🙂

What “NSA for you” means is that you can collect and analyze the data in your organization, through your APIs to internal applications, external partners and customers or , through social networking and SaaS applications, from hardware infrastructure, JAVA virtual machines, logs from any application, mobile device activity and mobile application activity in short virtually anything that your organization is doing you can collect and put in bigdata.   WSO2 ESB, WSO2 BAM and related adapters and connectors provide a connect and stream anywhere to anything capability for bigdata collection.   Recently the addition of WSO2 Enterprise Mobility Management includes the capability to collect streams of data on mobile applications and mobile devices easily as well.   In short we enable you to “spy” or collect information on your organization and related organizations.   If that scares you or offends you then you really need to get the paper bag out.  It’s happening. We just make it easier for you to do it too.

BIGDATA Middleware

Twitter, Ebay, Facebook, Google all use multiple bigdata infrastructures and conventional sql databases because some applications are more suited to different data storage mechanisms.  You need the flexibility to be able to pick the data storage mechanism that is right and to eventually change your mind.  WSO2 makes that easier to use multiple storage vehicles through our WSO2 Storage Server and WSO2 Data Services Server you can use different nosql stores like MongoDB,  Cassandra and HBase as well as any conventional sql database like the big guys.

Once you do that you can perform all kinds of analytics using WSO2 BAM and WSO2 CEP .  We support hundreds of adapters and connections to virtually anything in or outside your organization that you would want to collect data on and then we provide the pieces so that you can do things in real time easily.  For instance collect statistics, counts of activity over time periods, averages, other statistical measures of anything in the data.   It may be as simple as counting transactions by region or looking at sales for products or inventory levels, machine performance or amount of time spent doing one activity or another in a period of time and how that compares with other periods.

You can also look for complex patterns of data.   For instance you may see that when a person does X, then does Y that means you should offer them Z immediately.  Our WSO2 CEP engine can look through million even billions of events and find patterns that can be instantly responded to.   You can use this capability to generate offers, detect intrusions, automate updates to registries almost anything using for instance WSO2 Business Process automation.   You can do this incrementally later after you built your system.  That’s powerful and useful.

BigData makes you look smarter

What I’ve realized is that using big data is useful for any project because it provides the ability to detect, automate and visualize things that help you finish a project well.   You may not have thought of everything you wanted to automate when you initially designed your system.  Using bigdata and WSO2 you can easily count, detect, trigger business processes or automation that makes your project look finished, look smart, look intelligent.     Do you wonder how google keeps improving their search capabilities and brilliant applications?  A good portion must be related to the utility of big data to help them understand and then automate what looks like intelligence to us.  Pattern recognition, which is what the brain does.

We use bigdata in almost every project we do now.  Not because of necessity but because it just is easier than other ways of doing things and it makes you look smarter.   The WSO2 composable application platform makes using bigdata easy.