Understanding Microsoft's Big Data Landscape

Big Data such as social media posts and high-speed streaming credit card transactions often contain unprocessed information that can help organizations respond to customer needs in real time, improve efficiency of business processes, and expose trends and insights to help with decision making. Microsoft’s Big Data landscape includes solutions that capture high-speed data, provide scalable petabyte-level data storage, and deliver sophisticated analytic and machine learning solutions. The Azure-based services provide quicker deployment and scalability with less upfront investment than on-premises solutions, which could enable new kinds of Big Data solutions. However, most solutions require specialized skills, and constructing a complete architecture can be complex.

Big Data Overview

Big Data has many definitions and is viewed differently depending on the industry and individual organizations. It is most applied to data sources with high levels of the three “V”s: volume (total data size), variety (lack of predictable structure), and velocity (rate at which new data arrives). A solution is generally considered a Big Data solution if it addresses at least two of these attributes. Big Data can be large unstructured data sources, such as Web logs and streaming media posts that contain a variety of content types and are collected or occur at high speed, but it can also be large structured and semi-structured data sources that also occur at high speed, such as credit card transactions, equipment sensor readings, and weather forecasts.