Google today announced that it’s releasing its Cloud Dataflow in open beta. This big data analytics service was launched in closed beta version at Google’s annual developer conference last June, with a major update last December when they released an open source Java SDK to make it easier for developers to integrate with the new service.
Just last month Google announced that it was moving its Cloud Pub/Sub into public beta. This service for real-time messaging is yet another layer in the overall big data and analytics suite that Google has been building up.
Google’s strategy aims to cater for the full big data and analytics cycle of Capture->Store->Process->Analyze from within Google Cloud Platform’s organic services (such as Pub/Sub, Dataflow and BigQuery), as well as with plugging in external popular frameworks such as Hadoop, Spark and Kafka, in a modular way.
Google’s offering comes as a response to Amazon’s offering in the big data and analytics area, with services such as Kinesis, RedShift, Elastic MapReduce and Lambda. Interesting to note that last week at the AWS Summit in San Francisco Amazon announced Lambda service is generally available for production use. Amazon also maintains its smart strategy of tightening integration between their services, now enabling to run AWS Lambda functions in response to events in Amazon Cognito.
Amazon also puts emphasis on optimizing the infrastructure services for big data. A couple of weeks ago AWS launched new type of EC2 instances with high density storage optimized for storing and processing multi-terabyte data sets.
Another very interesting announcement from AWS last week was the announcement of Amazon Machine Learning new service, which gives an important dimension of analytics to their suite.
Follow Dotan on Twitter!