SQL Server 2019 Big Data Clusters

• SQL Server 2019 introduces Big Data Clusters, a new deployment option that allows it to work with data lakes and unstructured data on its own.
• Performance gains are likely for workloads that combine relational data with unstructured data.
• A centrally controlled data lake could help customers share unstructured data more securely for multiple applications.
• The deployment option is not a full replacement for Hadoop and will typically be slower than a native Hadoop environment for many processes.
• Administrators will have to master substantially different management technologies to take advantage of new Big Data Cluster capabilities.

SQL Server 2019, currently in preview, introduces Big Data Clusters, a deployment option that enables SQL Server to create, manage, and query large, unstructured data sources, such as a data lake, on its own. Key changes include the ability to manage and read a Hadoop Distributed File System (HDFS) directly, which removes the need to deploy an external unstructured data management system. Also included are Apache Spark for machine learning workloads and scale-out features optimized for distributed computing and increase performance. However, the deployment option does not simplify managing the HDFS components, and it requires a container orchestration solution.