Business Intelligence Reimagined in the Age of Artificial Intelligence

Business Intelligence (BI) tools are hot again. Just take a look at the past weeks, with Salesforce buying Tableau, Google buying Looker, and Logi Analytics buying Zoomdata. This joins other M&A deals in the past months such as Logi Analytics’ additional acquisition (yes, two in a row!) of Jinfonet in February, and Sisense acquiring Periscope Data last month.

Traditional BI is about sharp analysts who master the art and science of digging into data, intelligently querying and analyzing it to find signals amid the noise and digging up insights.

But the world has changed.
We live in the world of Big Data, where the sheer amounts of the data, the diversity of the data sources and the real-time response to the data render traditional human analysis largely impractical.

It is the age of Big Data Analytics, utilizing clever Artificial Intelligence and Machine Learning (AI/ML) algorithms to find the needle in the data haystack and surface insights. AI’s superiority was symbolically shown when Google’s AlphaGo AI defeated the world’s best Go player Sedol in 2017, some 20 years after IBM’s Deep Blue defeated Chess grandmaster Kasparov.

Big players such as Google, Amazon and Facebook have been mastering their markets for many years by using AI/ML internally. In the past few years cloud vendors also started exposing those capabilities as cloud services for everyone to use, thereby evening the (data) plane field and boosting AI/ML adoption.

This change renders older technologies irrelevant and forces consolidation.
I gave before the example of the Hadoop big data technology, where last year leading vendors Cloudera and Hortonworks had to merge to survive, and even that didn’t help them, judging by the CEO departure this month after poor reports. And their arch-competitor MapR is doing even worse now desperately looking for a buyer to avoid shutting down.
The same consolidation is facing BI tools. They need to find their place among the new AI/ML data analytics to remain relevant.


Do traditional BI tools have room in the new world?

While machines can do a fine job crunching massive amounts of data and surfacing insights, there’s still need for humans in the process to go through those insights, rank, filter, prioritize, or even simply monitor and take action items accordingly. It requires good data visualization. And that’s where BI tools can fit in. Taking complex data and insights and presenting them in a simple, intuitive and self-service user experience is the skill that BI tools have been developing for many years.

What should we expect next?

With so many BI tools out there, this consolidation will continue, where the leading tools will merge with leading analytics platforms to give an end-to-end data management experience, and smaller players will just disappear or grow into niche areas. The cloud vendors will keep leading this battle: Google will bild Looker into its AI suite, Microsoft will push its established Power BI and tighten the integration with its other Azure data services, and Amazon – the leading cloud vendor though not necessarily on this front – will probably step up its data visualization layer.

1311765722_picons03 Follow Horovits on Twitter!


Leave a comment

Filed under Big Data

The Biggest Announcement Out Of Amazon’s re:Invent

Amazon’s re:Invent yearly conference last week was packed as ever, with people and with announcements. But one announcement caught my attention, and you wouldn’t like to miss it either:

Amazon is officially stepping into hybrid cloud.

AWS Outposts is the name of Amazon’s hybrid cloud service, offering customers the ability to run compute and storage on-premises using the same native AWS APIs they know and love from the cloud. In Amazon’s own words:

AWS Outposts bring native AWS services, infrastructure, and operating models to virtually any data center, co-location space, or on-premises facility for a truly consistent and seamless hybrid cloud.

This is a groundbreaking change for AWS, the dogmatic crusader of public cloud. Back in 2016 I pointed out that hybrid cloud strategies gain traction with enterprises, one of the most lucrative sectors, and their needs and workloads cannot be met by a public cloud pure-play. Similar need for hybrid cloud arises with industrial automation, telecom network virtualization and other sectors. Now it seems Amazon finally embraced it.

The first crack in Amazon’s public cloud dogma was made in late 2016 when Amazon made a strategic partnership with VMware, which enabled Amazon to step into hybrid cloud without “getting its hands dirty”. This alliance is receiving a boost now with AWS Outposts: alongside the native AWS APIs on-prem offering, Outposts will come in another variant running VMware Cloud on AWS service.


The second crack in Amazon’s public cloud dogma was made in 2017 when the Internet-of-Things drove Amazon to offer an Edge Computing service named Greengrass, which enabled customers for the first time to run their favorite AWS services locally.

AWS is well behind in realizing the hybrid cloud: Amazon’s primary cloud rival Microsoft announced its Azure Stack in 2015. Microsoft has been pushing the adoption of Azure Stack by enterprises also through partnerships with established manufacturers and enterprise incumbents such as Dell-EMC, HPE, Huawei and Lenovo, which provide the hyperconverged hardware units and therefore have the business incentive to promote the joint solution.

Though the details are scarce, and Outposts isn’t due in general availability until second half of 2019, it seems from the announcement that Amazon is taking a different strategy, offering both the software, hardware and support by AWS in an attempt to keep all ends of the business in-house and in tight control:

Outposts infrastructure is fully managed, maintained, and supported by AWS, and its hardware and software components can be updated to deliver access to the latest AWS services.

Many other major players which had tried launching public cloud offering in the past few years have discovered Amazon’s domination in this arena and ultimately changed strategy, shut down their offering and turned to hybrid cloud, with emphasis on serving enterprise workloads. Just read the stories of  major companies such as HP, Cisco, Verizon and Rackspace.

The popular containers movement also accelerates the hybrid cloud approach, by enabling same deployment constructs through on-prem and cloud and easy mobility between them.

Hybrid cloud model is significant to many industries and workloads, taking into account various constraints such as data privacy, geo-location, and latency. This need requires more mature solutions, and Amazon weighing in will surely push the industry forward, as it has done for public cloud, serverless computing and other domains.

For more on AWS Outposts see the announcement and the product page.

1311765722_picons03 Follow Horovits on Twitter!


Leave a comment

Filed under Cloud

Why Open Source Is Eating The World

In 2011 Marc Andreessen explained in his monumental essay “why software is eating the world“. This originally controversial statement is by now a well-established fact of life. So what the next step in the evolution? I argue it’s open source.

Open Source Is Eating The World

Yes, you’ve heard it right. Open source is no longer just a topic for computer geeks running techie experiments and for bootstrapped startups looking to save some bucks. Open source has become mainstream. You don’t have to take my word for it, just take a look at this year’s multi-billion dollar mergers and acquisitions (M&A):

  • IBM is acquiring Red Hat, the first open source company to cross $1B revenues, for $34 billion.
  • Cloudera and Hortonworks, the vendors behind open source Hadoop that started the Big Data Analytics wave, are merging in a $5.2 billion deal.
  • Microsoft acquired GitHub, the popular version control system, for $7.5 billion.
  • Salesforce acquired MuleSoft, a popular open source messaging and integration middleware platform, for $6.5 billion in May. Interestingly enough, last week Salesforce made a strategic investment in Docker, another popular open source vendor for containers, alongside MuleSoft’s announced partnership with Docker. Could that mark Salesforce’s next major open source move?

ibm-red-hat-cloudy-380-1 (1)

Mergers and acquisitions, however, are not enough to make such a paradigm shift. As a young engineer I grew up programming in Java, using Solaris Unix and Open Office, all of which are open source software by Sun Microsystems. Then Oracle acquired Sun in 2009. Has Oracle changed its core DNA following that acquisition? Not exactly.

It takes more than just commercial M&A.
The big players need to embrace and commit to open source culture.
And indeed I see that happening in an increasing pace:

Microsoft is a great example: As I pointed out already back in 2015, Microsoft has made a strategic choice to embrace open source. Just last month Corporate Vice President Nat Friedman, who was recently appointed as GitHub CEO, made waves when he tweeted the open sourcing of Microsoft’s patent portfolio:

Microsoft is pledging our massive patent portfolio – over 60,000 patents – to Linux and open source by joining OIN this morning. If you’re looking for signs that we are serious about being the world’s largest open source company, look no further.

Another example from last month is IBM, which pledged its commitment to open source as an integral part of Red Hat’s acquisition:

With this acquisition, IBM will remain committed to Red Hat’s open governance, open source contributions, participation in the open source community and development model, and fostering its widespread developer ecosystem. In addition, IBM and Red Hat will remain committed to the continued freedom of open source, via such efforts as Patent Promise, GPL Cooperation Commitment, the Open Invention Network and the LOT Network.

The modern giants, who were born into the open source era, started off with open source as an integral part of their development culture. For example Kubernetes, the leading containers orchestration platform and the 2nd most active project on GitHub, originated in a project which Google open sourced to the Linux Foundation. Now you’d find even Google’s bitter rivals collaborating there. That’s the beauty of open source movement.

Even the traditional Telecommunications industry, which tends to lag behind IT and trust heavily on standardization bodies, have come to embrace open source as their new and agile way forward, reaching dozens of open source projects under the Linux Foundation alone. A prime example is Open Network Automation Platform (ONAP), which itself is a unification of open sources from east and west.

Open source movement is not just limited to software. I’m not going to delve into it in this post, but there are great examples of hardware open source, such as Open Compute Project backed by Facebook (which initiated it), Google and others.

Open source is here. Leading vendors embrace it; Enterprises and governments use it; Cloud providers and system integrators offer services for it; Communities innovate on it.

Open Source Is Eating The World.

1311765722_picons03 Follow Horovits on Twitter!


Leave a comment

Filed under Open Source

IBM Acquires Red Hat For $34B: Can It Really Make IBM A Cloud Leader?

Yesterday IBM announced its intent to acquire Red Hat in a deal worth approximately $34 billion. This is IBM’s biggest so far, and third-biggest in the history of US technology (the biggest was the Dell-EMC merger). In this acquisition IBM is trying to position itself in the cloud market, with special focus on its enterprise base (can anyone remember the days back in 2010 when IBM’s CEO Palmisano statedYou can’t do what we’re doing in a cloud“?).

Today’s public cloud market is largely dominated by Amazon, which according to Gartner’s 2017 IaaS survey holds over half of the market share (51.8%). Another quarter of the market is shared by Microsoft (13.3%), Alibaba (4.6%), Google (3.3%) and IBM (1.9%).

As I pointed out before, in principle there are 3 strategies that vendors can seek:

  1. Multi-cloud model: If you can’t beat them, join them. Support Amazon, Microsoft, Google public clouds. If done via a good generic platform, it can help avoid vendor lock-in.
  2. Hybrid model: mix the public cloud support with support for private cloud and bare-metal to offer public-private-hosted hybrid approach.
  3. Private model: concentrate on strictly private cloud. The popular open-source project OpenStack is a leading candidate for this strategy. This approach is useful for the customers insisting to run things on their own premises.

IBM chose a hybrid- and multi-cloud strategy. As Ginni Rometty, IBM Chairman, President and Chief Executive Officer, wrote:

IBM will become the world’s #1 hybrid cloud provider, offering companies the only open cloud solution that will unlock the full value of the cloud for their businesses.

IBM’s hybrid cloud business is already worth $19 billion, according to the announcement. Red Hat will join IBM’s Hybrid Cloud team, but will operate as a distinct unit, preserving its independence and neutrality, as well as Red Hat’s open source development heritage and commitment. Following the multi-cloud model, IBM stated it will continue to build and enhance Red Hat partnerships, including those with major cloud providers, such as Amazon Web Services, Microsoft Azure, Google Cloud, Alibaba and more, in addition to the IBM Cloud.

ibm-red-hat-cloudy-380-1 (1)

Just last month Red Hat, IBM and Hortonworks announced a joint project aimed to “enable big data workloads to run in a hybrid manner across on-premises, multi-cloud and edge architectures”. In an interesting timing, this month Hortonworks announced merger with Cloudera, both well-established open source companies around Hadoop and other big data projects.

Others are also targeting hybrid cloud strategy: just last month Google and Cisco rolled out a new hybrid cloud offering, Amazon teamed up with VMware, Microsoft offers Azure Stack, to name just a few. IBM should be lean and move fast to gain market share. The current acquisition and open source play should give it a serious boost, but would it be enough?

1311765722_picons03 Follow Horovits on Twitter!


Leave a comment

Filed under Cloud

The Next Chapter of Big Data Analytics: What’s Behind The Cloudera-Hortonworks Merger?

Over a decade ago a new open source project called Hadoop came to life and started the era of Big Data Analytics. It was a novel project with a scale-out, shared-nothing distributed architecture, which promised to handle the big data challenges not manageable by the standard relational databases of the time. However, it was notoriously hard to install and operate in production.

Two brands became synonymous with commercial Hadoop – Cloudera and Hortonworks – with the promise of helping enterprises leverage Hadoop. These companies, built by talents from the likes of Google, Facebook and Yahoo, grew into impressive public companies: their combined equity value after the merger is estimated at $5.2 billion (give or take the stock price changes).

But while they were bashing around (together with MapR and some others), the Big Data market has drastically changed. These emerging challenges have driven these arch-rivals to join forces. Cloudera and Hortonworks announced a “merger of equals”, though at a closer look it seems to be more of a “first among equals” arrangement, with Cloudera’s stockholders to own a 60% stake, and the new company to be called “Cloudera Inc.“.

From on-prem to the cloud

In the age of Cloud Computing, enterprises are no longer inclined to build enormous Hadoop clusters in their data centers to crunch the data (especially with Hadoop’s high upfront storage costs and painful upgrades). And so the traditional players found themselves increasingly competing against public cloud giants Amazon, Google, Microsoft and their big data services. On Gartner’s 2018 Magic Quadrant for Data Management Solutions for Analytics, the main cloud vendors shine in the Leaders quadrant (with Google a hairline away), while Hortonworks, Cloudera and MapR lag behind in the Niche Players quadrant.

From data to insights

Since the emergence of Hadoop, there’s been a big (data) bang which spawned a range of tools, platforms and services aimed at different aspects of analytics, from data lakes to data warehouses, from batch to stream, from time series to graphs and from structured to unstructured. But most disruptive are the data science driven methods of artificial intelligence and machine learning. This shuffled the competitive landscape introducing many specialized vendors such as KNIME and (both open source), which lead Gartner’s 2018 Magic Quadrant for Data Science and Machine Learning Platforms and keep pushing back even established players such as IBM, Microsoft, SAS and Teradata. Hortonworks and Cloudera are not even on that chart.

From the data center to the edge with IoT

The Internet of Things (IoT) brings a massive surge of data streaming from disperse locations, ranging from connected cars to industrial automation. This calls for different big data analytics solutions to meet regulatory, security, performance, and geo-location need, combining both cloud and edge computing. You can read more on that in this blog post.

Joining forces towards the next chapter of big data analytics

Hortonworks and Cloudera have made some moves in the right direction even before the merger to adapt their strategies to these market changes: They formed partnerships with Amazon, Google, IBM, Microsoft and started operating on their clouds, while maintaining a hybrid cloud differentiation; They launched advanced analytics and IoT offering such as Hortonworks DataFlow for streaming and IoT workloads, and Cloudera Data Science Workbench.

But it will require more than these individual efforts to keep them competitive. Therefore, joining forces is an important strategic move. Combined with the strong open source DNA shared by both companies, they will be well positioned to take a leadership role in the next chapter of big data analytics.

1311765722_picons03 Follow Horovits on Twitter!

Leave a comment

Filed under Big Data, Uncategorized

Is Dell Going to Sell To VMware In A Reverse Merger?

In 2015 Dell made the biggest take over of all times when it acquired EMC for $67 Billion. One of the main assets of that acquisition was VMware, which EMC had acquired in 2004. With the acquisition, Dell currently owns around 80% of VMware.

But now the plot thickens, as Dell is considering selling itself to the smaller VMware in a massive reverse merger that may very well be the biggest deal ever. The reason behind this strategy is Dell’s interest to get back to the public markets, after Michael Dell took the company private in 2013. This move will enable Dell to be traded publicly without going through a formal listing of a traditional IPO.

But the reason may also be found in the disruptive technology of cloud computing. Dell traditionally had a strong grasp of the enterprise IT market on the hardware side, and VMware had a similar grasp in enterprise IT virtualization solutions. But the market has been disrupted with the entrance of the public cloud vendors such as Amazon AWS and Microsoft Azure, enabling enterprises to avoid owning and managing their own data centers and instead run their workloads in the cloud. Traditional incumbents such as Verizon, HP and Dell tried fighting off by launching their own public cloud offering, but had to pull back after failing to compete. VMware also had to team up with Amazon to maintain its cloud strategy. Shifting to a more software driven approach targeted at the cloud native solutions may be part of Dell’s strategy to regain its position with the enterprises.

1311765722_picons03 Follow Horovits on Twitter!

Leave a comment

Filed under Cloud, technology

Artificial Intelligence and Bots Can Drive Self Driving Networks

Self-driving cars are set to disrupt transportation as we know it, and the basis for that is Artificial Intelligence (AI). In the age of autonomous vehicles we will tell our car something like “let’s go pick up Jane” and the car will use AI to figure out that Jane is the young daughter who needs to be picked up from kindergarten which ends in 15 minutes time, and derive the destination and best route to get there safely and on time, including all real-time driving decisions to follow law and safety of passengers and surrounding (to get a taste of just how complex that is, check out Tesla’s autonomous driving experiments).

The same concept is now applied in networking to achieve self-driving networks. While traditionally configuring networks has been very complex, laborsome and detail-heavy, new concepts such as Software Defined Networking and Intent-Based Networking arise to enable self-driving networks. In such networks, the operator will only have to provide the “how” (the intent) and let AI figure out the “how”.

An example of this new trend is Juniper’s recent announcement of AI-backed Bot applications to automate network management tasks:

  • Contrail PeerBot automates the traditionally cumbersome process of network peering to simplify policy enforcement and on-demand scaling.
  • Contrail TestBot automates the continuous auditing of design, provisioning, and deployment changes in the network.
  • AppFormix HealthBot translates troubleshooting, maintenance, and real-time analytics into an intuitive user experience to give network operators actionable insights into the health of the overall network. This one is the result of Juniper’s 2016 acquisition of AppFormix.

Juniper is but the latest announcement on the quest to fulfill the Self Driving Network vision. June this year Cisco also announced intent-based networking solutions, which Cisco says is

… the culmination of Cisco’s vision to create an intuitive system that anticipates actions, stops security threats in their tracks, and continues to evolve and learn.

The quest for self driving networks will only accelerate, driven not only by the networking giants but also by startups such as Apstra, Forward Networks, Waltz and Veriflow. So get ready to take your hands off the wheel and let AI Bots drive for your network.

1311765722_picons03 Follow Horovits on Twitter!

Leave a comment

Filed under Big Data, Real Time Analytics, SDN, Telecommunications