Understanding Azure Databricks

Azure Databricks is a cloud service based on Databricks software, a third-party offering that provides analytics, orchestration, and collaboration tools for creating and operating machine learning applications. Databricks is based on Apache Spark and Apache SQL, along with other components such as Python and R libraries, and notebooks that developers can use to build machine learning models and run those models in production on distributed clusters to create high-performing applications.

Service Overview

At the core of Databricks are workspaces, a secure environment for storing data and processes, where machine learning developers and data engineers create models, collaborate via notebooks, add tools and runtimes for specific requirements, and configure access to Azure data services. Microsoft provides automated scaling and back-end compute and storage resources, as well as integrations to other Azure services that can be used to enhance Databricks and create a complete application environment.