Category Archives: Uncategorized

The Internet of Things Drives Amazon To Edge Computing

Amazon.com is the bearer of the of e-commerce vision: buy anything online. Or as Amazon’s mission statement goes: a place where people can come to find and discover anything they might want to buy online.

But then the earth shook: a few days ago Amazon acquired organic-food chain Whole Foods for $13.7 billion, Amazon’s largest deal ever. Against its very vision and DNA, the e-commerce giant put a sound foot in brick-and-mortar, with hundreds of physical stores.

WHY? Simply put, Amazon realized that we don’t want to shop for ANYTHING online. Some products, such as groceries, people still like to smell, hand-pick, try out and buy at the store near by. That’s where it loses ground to Walmart et al. So Amazon adapted its vision, and made a serious investment to get in the game to augment its leading e-commerce play (going for M&A after some failed home-grown trials such as AmazonFresh).

Coincidentally(?), a similar earthquake happened at Amazon’s cloud around the same time: Amazon Web Services (AWS) has been the pioneer of public cloud and a strong advocate of the vision that all shall run over the web in the public cloud (hence named “web services”). Even hybrid cloud (private+public), which Microsoft, IBM and other public cloud vendors adopted well, Amazon had hard time accepting, to the point that Amazon partnered with its rival VMware to complement that piece externally.

But then a couple of weeks ago Amazon released Greengrass. Don’t let the innocent-sounding name mislead you – it is nothing short of a revolution for Amazon. Greengrass enables users for the first time to run their favorite AWS services LOCALLY, executing serverless logic and inter-device communication without necessarily connecting to AWS cloud.

WHY? Simply put, the Internet of Things (IoT). At the recent AWS Summit I heard Amazonians for the first time admitting out loud that some use cases, especially those derived from IoT, disallow connecting to a central remote cloud data center. On his blog post, AWS CTO Werner Vogels himself outlines the categories (he calls them “laws”) of these use cases:

  1. Law of Physics. Customers want to build applications that make the most interactive and critical decisions locally, such as safety-critical control. This is determined by basic laws of physics: it takes time to send data to the cloud, and networks don’t have 100% availability. Customers in physically remote environments, such as mining and agriculture, are more affected by these issues.
  2. Law of Economics. In many industries, data production has grown more quickly than bandwidth, and much of this data is low value. Local aggregation and filtering of data allows customers to send only high-value data to the cloud for storage and analysis.
  3. Law of the Land. In some industries, customers have regulatory or compliance requirements to isolate or duplicate data in particular locations. Some governments impose data sovereignty restrictions on where data may be stored and processed.

In fact it’s bigger than merely IoT. Amazon tried launching its IoT API service with direct connectivity to the cloud and it didn’t catch for many types of IoT use cases. The missing ingredient was Edge Computing. As I wrote before, IoT, Big Data and Machine Learning Push Cloud Computing To The Edge, and that’s what Amazon realized. On AWS Summit I saw this learning simply put: AWS IoT Going to the edge.

IMG_20170621_151354

The two Amazon groundbreaking stories this month come down to the same essential truth: Amazon started out from the digital internet-driven services, which matched many common use cases. But now it realizes the power of the edge, whether a physical store available down the street from my home or edge computing executing at my smart home or connected factory (or perhaps at the Whole Foods store down the street?). That’s me – living on the edge – and apparently I’m not alone.

1311765722_picons03 Follow Horovits on Twitter!

Leave a comment

Filed under Uncategorized

Oracle Launches Bare Metal Cloud Services, Challenges Amazon AWS

Oracle threw some big announcement back in September at Oracle OpenWorld conference about its plan to add Infrastructure as a Service (IaaS) to its Oracle Cloud Solutions. And now the first piece of that IaaS is announced: Oracle Bare Metal Cloud Services. Oracle’s service offers integrated network block storage, object storage, identity and access management, VPN connectivity, and a software-defined Virtual Cloud Network (VCN) – their implementation of Software Defined Networking (SDN). The new service is launched in Oracle’s new Phoenix Region (Arizona, USA), with the promise of growing to additional regions. The Phoenix region has 3 Availability Domains (similar to Availability Zones in Amazon Web Services).

oracle-iaas-logoOracle has been exploring cloud for a while and has made several startup acquisitions in that direction. With this move Oracle is going jumping heads-on to the ruthless cloud IaaS wars. In fact, it seems Oracle lured in some cloud experts from Amazon, Microsoft and Google to build its new IaaS.

One amazing thing that Oracle did with its IaaS, is that it designed its entire data center, up to the hardware stack, on its own! Oracle learned well the lesson from Amazon, Google et al. Thank to that design it claims to provide competitive pricing that will challenge the legendary AWS pricing.

For more information on Oracle Bare Metal Cloud Services see here.

1311765722_picons03 Follow Horovits on Twitter!

1 Comment

Filed under Cloud, Uncategorized

Google’s Self-Driving Cars To Become A Stand-Alone Business

Google is shifting gear with its self-driving car project. The project, which has been run as part of Google X research lab, has spun out into its own business unit under Google’s parent company Alphabet. According to Wall Street Journal, the car group would likely be expected to soon begin generating revenue, though not necessarily a profit at first. X chief Astro Teller told WSJ that Alphabet will likely roll out its self-driving cars incrementally over the next several years as they improve with more time on the road.

This is another move in the race to the holy grail of a fully operational autonomous car. Google keeps a special watch over Uber’s aggressive self-driving car plans, with its plans to disrupt transportation. While Google’s cars have been driving around Google’s home town in Mountain View CA and other US cities, Uber has been swinging through Pittsburgh. And they’re not alone in this race. In fact, you find surprising newcomers each day: Just today Samsung acquired U.S. auto-parts supplier Harman for $8 Billion to get onboard with connected cars, probably as part of Samsung’s Internet of Things (IoT) strategy. When the lines blur between cars, mobile and online services, when transportation is up for disruption, everyone is out for the take.

1311765722_picons03 Follow Horovits on Twitter!

2 Comments

Filed under Autonomous Car, IoT, Uncategorized

The Mysterious Creator Of Bitcoin and Blockchain Comes Out Of The Shadows

The birth of the virtual currency Bitcoin was accompanied with great mystery, when its creator chose to remain in the shadows, known only by his (or her?) pseudonym Satoshi Nakamoto. Rumors over the years brought up various potential “suspects”, but nothing was proven and no one confessed. Until now.

Today Dr. Craig Wright, a 45-year-old computer scientist from Australia, announced that he is Satoshi Nakamoto. In his post Wright says

If I sign Craig Wright, it is not the same as if I sign Craig Wright, Satoshi.

Wright then goes on to thank those who supported the project. Something that has started from a monumental paper by the mysterious Satoshi Nakamoto, followed by a release of an even more monumental implementation of Blockchain, the revolutionary technology for open distributed ledger underlying Bitcoin. In fact, the impact of Blockchain currently seems to surmount that of bitcoin, with blockchain-based innovation boiling up in both startups and financial institutions. The interest is so great (and skill set is so rare) that IBM and Microsoft launched Blockchain-as-a-Service offerings on their clouds to help companies innovate with Blockchain.

SatoshiNakamoto-p2pfoundation

Wright dedicated most of his post to convincing the community of the authenticity of his claim. He provides a signed evidence, supposedly signed with a private key associated with Satoshi Nakamoto (the key for block 9), and elaborates on the process of verifying cryptographic keys, basically implying on the verification of his own evidence.

Wright knew such announcement would not go about without a storm, so he summoned in advance three high-profile magazines – The Economist, The BBC and GQ Magazine – to exclusively present his claim and evidence so they can accompany his post with their own coverage. The magazines jumped on the scoop and dag deep into his claims. You can read their full review here:

The Economist: Craig Steven Wright claims to be Satoshi Nakamoto. Is he?

The BBC: Australian Craig Wright claims to be Bitcoin creator

GQ Magazine: Dr Craig Wright Outs Himself As Bitcoin Creator Satoshi Nakamoto

Is Dr. Wright the real Satoshi Nakamoto? The community will be debating that in the weeks to come, together with further evidence released by Wright. More importantly, Wright hints of new work he’s done in this field with “an exceptional group”. It may very well be that his current announcement just sets the stage for bigger announcements or releases yet to come.

1311765722_picons03 Follow Horovits on Twitter!

Leave a comment

Filed under Blockchain, Uncategorized

IBM, Microsoft Offer Blockchain In Their Cloud Services

Recently blockchain fans got major news, with two giants IBM and Microsoft announcing their support for Blockchain-as-a-Service (BaaS) in their cloud services. Are we going to see some cloud-based blockchain developments soon? sounds like it.

Blockchain emerged from Bitcoin cryptocurrency hype as the innovative distributed ledger technology behind Bitcoin. But while cryptocurrencies are well past Gartner’s peak of inflated expectations, blockchain is gaining growing interest from startups and enterprises alike. The interest in blockchain isn’t limited to just cryptocurrencies but also extends into other financial use cases, and even transcends FinTech realm into non-financial use cases such as electronic voting, smart contracts and ownership verification for art and diamonds.

Hyperledger-1024x239

The interest that blockchain drove the creation of different “flavors” of the distributed ledger notion, beyond the initial one used for Bitcoin. One interesting initiative recently launched is the hyperledger project, a community-backed open-source standard for distributed ledger. It was launched December 2015 under the Linux Foundation by big financial services names such as J.P. Morgan, Wells Fargo, London Stock Exchange Group and Deutsche Börse, as well as equally big IT players such as IBM, Intel, Cisco and VMware. As part of joining Hyperledger, IBM has open sourced a significant chunk of the blockchain code it has been working on.

blockchain

IBM launched its blockchain-as-a-service in production February. In order to encourage adoption of its new cloud service, IBM also opens garages for blockchain app design and implementation in London, New York, Singapore and Tokyo.

Microsoft was first to move in on blockchain. Last November ETH-BaaSMicrosoft launched a Blockchain-as-a-Service on its Azure cloud based on Ethereum in partnership with ConsenSys. But while IBM bet on hyperledger project, Microsoft took a different approach and spread its bet across multiple projects and partnerships. During last month Microsoft added to its blockchain partnerships Augur, Lisk, BitShares, Syscoin and Slock.it, and this month also added Storj.

I estimate IBM and Microsoft would not remain alone in this game. Other vendors will join in to offer platforms and cloud services to accelerate the development of blockchain-based applications. This will be a serious enabler for innovation around this fascinating technology, whether for young innovative startups bootstrapping on low budget, or for financial institutions (and other enterprises) lacking in-house skills in this cutting-edge technology.

1311765722_picons03 Follow Horovits on Twitter!

 

1 Comment

Filed under Blockchain, Cloud, Uncategorized

Live Video Streaming At Facebook Scale

Operating at Facebook scale is far from trivial. With 1.49 billion monthly active users (and growing 13 percent yearly), every 60 seconds on Facebook 510 comments are posted, 293,000 statuses are updated, and 136,000 photos are uploaded. And there lies the challenge of serving the masses efficiently and reliably without any outages.

For serving the offline content, whether text (updates, comments, etc.), photos or videos, Facebook developed a sophisticated architecture that includes state-of-the-art data center technology and search engine to traverse and fetch content quickly and efficiently.

But now comes a new type of challenge: A few months ago Facebook rolled out a new service for live streaming called Live for Facebook Mentions, which allows celebs to broadcast live video to their followers. This service is quite similar to Twitter’s Periscope (acquired by Twitter beginning of this year) and the popular Meerkat app, which offer their live video streaming services to all and not just celebs. In fact, Facebook announced this month it is piloting a new service which will offer live streaming to the wide public as well.

live_cover_3

While offline photos and videos get uploaded fully and then distributed and made accessible to followers and friends, serving live video streams is much more challenging to implement at scale. And to make things even worse, the viral nature of social media (and of celeb content in particular) often creates spikes where thousands of followers demand the same popular content at the same time, a phenomenon the Facebook team calls the “thundering herd” problem.

An interesting post by Facebook engineering shares information on these challenges and the approaches they took: Facebook’s system uses Content Delivery Network (CDN) architecture with a two-layer caching of the content, with the edge cache closest to the users and serving 98 percent of the content. This design aims to reduce the load from the backend server processing the incoming live feed from the broadcaster. Another useful optimization for further reducing the load on the backend is request coalescing, whereby when many followers (in the case of celebs it could reach millions!) are asking for some content that’s missing in the cache (cache miss), only one instance request will proceed to the backend to fetch the content on behalf of all to avoid a flood.

facebook-live-stream-cache

It’s interesting to note that the celebs’ service and the newer public service show different considerations and trade-offs of throughput and latency which brought Facebook’s engineering team to make changes to adapt the architecture to the new service:

Where building Live for Facebook Mentions was an exercise in making sure the system didn’t get overloaded, building Live for people was an exercise in reducing latency.

The content itself is broken down into tiny segments of multiplexed audio and video for more efficient distribution and lower latency. The new Live service (for the wide public) even called for changing the underlying streaming protocol to enable an even better latency, reduce the lag between broadcaster and viewer by 5x.

This is a fascinating exercise in scalable architecture for live streaming, which is said to effectively scale to millions of broadcasters. Such open discussions can pave the way to smaller companies in the social media, internet of things (IoT) and the ever-more-connected world. You can read the full post here.

1311765722_picons03 Follow Horovits on Twitter!

Leave a comment

Filed under Solution Architecture, Uncategorized

How IBM is using big data to fix Beijing’s pollution crisis

A fascinating way to leverage big data to help the world

Leave a comment

Filed under Uncategorized