HDInsight Previews AKS Deployment and New Frameworks

• HDInsight is receiving more investment with new functionality and frameworks to address industry changes in technology.
• A new Kubernetes container deployment option, using Microsoft services, simplifies scaling and introduces new open-source frameworks.
• The existing VM deployment option is also receiving updates with the release of a new back-end version and security features.

Azure HDInsight is receiving a Kubernetes-based deployment option that simplifies management and includes two newer frameworks for high-speed analytics and data warehouses that also improve how the service works with other Azure services. The existing VM deployment option (now called HDInsight Classic) is not being deprecated and is, in fact, also receiving updates. Migrating to the new deployment option is not required, and compatibility between the two will help customers who prefer or need both options. (For an overview, see the sidebar “HDInsight Overview, Nov. 2023.”)

Overall Roadmap

Until recently, there have been relatively few updates or new features to HDInsight other than updates to the back-end platform and regular releases of the open-source frameworks. However, in Oct. 2023, Microsoft announced a public preview of HDInsight for AKS, a new deployment option (with new frameworks) that allows customers to deploy HDInsight inside containers using the Azure Kubernetes Service (AKS).