One of the toughest decisions CIOs are facing when deciding to migrate their workloads to the cloud is – which cloud provider is best suited for their workloads? pricing is one aspect which has been extensively reviewed, with fierce war around price reductions, led by Amazon’s constant price cuts and innovative pricing schemes with/without prior commitment and auctioned spot instances.
Another important aspect is the performance measures of the cloud provider and its fit to the company’s specific workloads. Google, Amazon and others have been investing extensively in gaining superior performance with their hyperscale computing. But unlike pricing, performance is much more complex to measure and compare, with lack of standardization around the relevant KPIs (key performance indicators) and benchmark definitions.
Now comes Google to the rescue with a new open-source tool called PerfKit Benchmarker. The new tool, released a couple of days ago, will enable measuring various KPIs, from elementary such as CPU and bandwidth and up to end-to-end provisioning times. Another tool, Perfkit Explorer, provides visualization of the benchmark results for easy interpretation. In its blog post, Google Cloud Platform Performance Team says
You’ll now have a way to easily benchmark across cloud platforms, while getting a transparent view of application throughput, latency, variance, and overhead.
The tools are managed as open source projects hosted in GitHub (ASLv2 license). It’s said over 30 leading researchers, companies, and customers have been involved in this project, including big industry names such as Intel, Cisco, Microsoft and RedHat, and academia endorsements from Stanford and MIT.
Currently the kit contains 20 benchmarks from basic ping and iperf through Hadoop, Cassandra and MongoDB NoSQL databases. Current release supports the three major public cloud providers: Amazon’s AWS, Microsoft’s Azure, and of course Google’s GCE.
Open source is the new de facto standard. With a new open source tool backed by industry leaders and the community, we can gain a standard benchmark to compare different vendors not just on price but also on performance. As the acceptance of the tool grows, more cloud providers will be covered by the benchmarks, and more benchmarks will be added to address the needs of more complex workloads. Then we will be able to truely choose best of breed for our needs.
Follow Dotan on Twitter!
horovits 