HDInsight 4.0 Updates Hadoop, Enterprise Features

• Azure HDInsight 4.0 includes the latest versions of Hadoop components, performance improvements, and enterprise-level management and security features.
• Migrating to 4.0 requires considerable effort, as the underlying framework and metastore are updated, and the service is not backward compatible.
• An expanding set of add-on applications allows customers to use their preferred tool sets.

Azure HDInsight is Microsoft’s cloud-based service for Apache’s Hadoop open-source software framework, a popular solution for managing and analyzing Big Data. HDInsight is much like an on-premises Hadoop deployment where the customer has control over the servers and components deployed (such as analytic tools), but Microsoft manages the back-end infrastructure and automates deployments. The service appeals to organizations familiar with Hadoop that want to deploy multiple Hadoop configurations or that have other data and applications already hosted in Azure.

Overview

HDInsight offers multiple native Apache Hadoop deployment configurations (called cluster types) that are optimized for specific workloads, like unstructured data mining, high-speed analytics, and data warehousing. It uses native Hadoop configurations (such as HBase, Hive, Kafka, and Spark) that are based on current enterprise editions. Clusters are deployed using the Hortonworks Data Platform (from a Microsoft partner) that includes enterprise controls, security, and management.