How to Optimize Apache Spark Cloud Clusters for Cost and Runtime Goals with Sync Computing
Optimizing Apache Spark clusters on EMR and Databricks to hit cost goals is a time-consuming task, but is an increasingly important task in today’s economy. Specifically for production workloads on AWS, optimizing instance types, spark configurations, and cluster sizes is an incredibly complex process. The Sync Autotuner for Apache Spark helps to solve this problem by predicting workload performance on alternative cloud infrastructure and configurations. In this talk developers will learn how to increase productivity, reduce cloud costs, and improve performance of the production Apache Spark workloads. In the second half we’ll walk users through a live hands-on demo of the Autotuner product.