The Cortana Intelligence Suite is a fully managed data analytics platform that offers a range of services aimed at helping businesses find meaning in all of the data they are collecting. This post will dive into the services that are offered by the platform. Prior to discussing these services in detail, let’s take a quick lap around data analytics specific to goals, what to expect when building and maturing a competency for your business, and challenges businesses experience while on this journey. After laying this groundwork, we’ll discuss the Cortana Intelligence Suite in a bit more detail and provide a list of the services offered by the platform. Additionally, we’ll cover how the Cortana Intelligence Suite can offer the tools needed to implement a robust strategy to address the key challenge areas we’ve described.
Since, the main goal of building a solution using Cortana Intelligence is about achieving tangible benefits and competitive advantages for your business, we’ll wrap up by talking about the strategic impact and potential opportunity your data assets can provide. Essentially, we’ll use this section to answer the question “why should I care about data analytics”? The Cortana Intelligence Suite uses the power of Microsoft’s Azure cloud platform in a fully managed way, so you’ll hear a lot about Azure in this post as we describe the services offered.
Data Analytics
So what is Data Analytics and how does this term relate with or overlap other terms you’ve heard, like data mining? Simply put, data analytics is the practice of inspecting unrefined or untreated data with the goal of finding patterns that lead to conclusions about that information. These conclusions are then used to guide business decisions. The focus on finding conclusions that lead to actionable intelligence is what really differentiates data analytics from other terms you may have heard, such as “data mining”. While data mining focuses on exploring data and finding patterns in the data, analytics takes the process a step (or a few steps) further by focusing on what we can deduce based on the relationships or patterns we’ve found.
For us, data mining can be used as a component of a more comprehensive data analytics strategy, however the disciplines are unique in this way. Data analytics includes an “artistic” component which is concerned with representing conclusions to business users in the most effective way. Data analytics often includes visualization elements that aid in making inferences by speaking to business decision makers and guiding their actions.
Key Goals
- Machine-assisted processes are used for recognizing patterns and generating conclusions about your business data
- Conclusions drawn are relevant, timely, and actionable
- Conclusions are effectively communicated to guide business decisions
The Data Analytics Maturity Model
Let’s take at a typical maturity model for implementing data analytics in an organization. This chart depicts the goals and benefits of mature data analytics processing. As you can see, by growing data analytics capabilities in the enterprise, we go from more traditional reporting and business intelligence capabilities to predictive and prescriptive data analytics.
Through this process we are optimizing the information that decision makers have at their disposal. This takes a business beyond simply reporting on what happened to identifying why an event or set of events happened. Ultimately, as the process matures, capabilities are added to allow prediction with statistical confidence. Predictive analytics allow us to gain foresight and prescribe a set of actions a business can take to gain a desired outcome. Essentially, we advance from basic “what happened?” insights to detailing what can be done to achieve positive future outcomes.
Generally, the value increases as your data analytics capabilities mature, but we should keep in mind that the level of difficulty associated with progressing to later stages of maturity have traditionally been difficult and costly.
Challenges
So, why is it difficult? First off, we should start by recognizing the fact that data is pervasive in the modern business world. According to Gartner, more than 85 percent of the data available to organizations is automatically generated – from every web server, device, sensor, upload, tweet, purchase, shipment and keystroke. However, many organizations find it difficult to draw actionable insights from this data because of the size, scale, velocity, and aggregate complexity that comes with collecting data at every turn. In this environment, insights are elusive because “people-heavy” and manual decision-making procedures simply cannot keep up with the pace that data is collected, and therefore struggle to provide meaningful conclusions as the pace of data collection increases. Automated “machine-assisted” processes can help, but businesses must be aware that building these processes come with real challenges to address.
- Identifying data that should be used and making it available for analytical processing.
As stated, an incredible amount of data can be generated internally, but external data also needs to be used in many cases – for example, data from competitors. How is this data discovered, transformed, and stored so it has the potential to be useful?
- Building the infrastructure.
What is the best way to build or use statistical or analytical models once data is available? How do businesses use or apply these techniques without breaking the bank or attempting to build the tools from scratch? Is an entirely new infrastructure required specific to analytical processing?
- Communicating insights and conclusions.
Once insights are gained through pattern recognition or data analysis, how are these insights communicated to the business to optimize or transform the way the business operates?
The Cortana Intelligence Suite
This is where Cortana Intelligence can help. The goal of the Cortana Intelligence Suite is to deliver an end-to-end platform with integrated and comprehensive tools and services to help build smart applications and optimize business. The platform is fully managed and uses the power of Microsoft Azure to leverage the experience Microsoft has built internally to address the challenges previously discussed. Since many of the services under the Cortana Intelligence umbrella are offered as consumable services, businesses can often avoid a costly infrastructure build-out required to support a data analytics platform.
Information Management
To address the first challenge listed above, Cortana Intelligence provides services to bring data in, so you can analyze it. The data can come from line of business applications directly, third party sources, or a large number of sensors and devices. Information management capabilities, like Azure Data Factory, can pull data from many diverse sources. These sources may include relational DBs (i.e. SQL) or non-relational stores, such as your existing Hadoop cluster. These interactions can be automated and scheduled, and perform any necessary data transformations (Example: setting certain data columns as dates vs. currency etc.). Think ETL (Extract, Transform, Load), but in the cloud, with the ability to scale as needed to suit the size and complexity of your data. Event hubs do the same for IoT scenarios where ingestion of data is needed from many potential endpoints.
Big Data Stores
The data brought in can be persisted in flexible big data storage services like Azure Data Lake Store and Azure SQL DW. Data Lake is an extremely interesting service that traces its roots to Microsoft’s Cosmos system – the internal Microsoft service that manages a massive amount (Exabyte scale) of diverse data and has helped Microsoft fundamentally become a data-driven company.
So these services help businesses get a handle on the first challenge we talked about. How do businesses set the stage and identify data that is vast and potentially unrelated at first glance and make it available for analytical processing?
Machine Learning and Analysis
You can use a number of analytics services ranging from Azure Machine Learning, to Azure HDInsight, to Azure Stream Analytics to analyze the data stored in big data storage services. This means you can create analytics services and models specific to your business need, for example – real-time demand forecasting. These services and tools help businesses with the second challenge by providing a software platform and managed infrastructure on top of the most cutting-edge analytical tools. In many cases, these tools will enable web service endpoints for further integration and relatively simple consumption by your business applications. Imagine the ability to call a machine learning algorithm to answer a specific question about expected growth.
Dashboards, Visualizations, and Communicating Insights
If you recall, the third challenge centered around communicating insights effectively to business users. The analytics services and models created by taking these steps can be surfaced as interactive dashboards and visualizations, via Power BI or custom business applications. In fact, Power BI has the ability to reach back into the analytics layer using the web service endpoints we just discussed. On the other hand, Power BI can be used directly or can be embedded in other applications, so there is a lot of flexibility available. These same analytics services and models created can also be integrated into different user interfaces; web apps, mobile apps, or rich client apps. Additionally, another interesting scenario is analytics-based insights delivered via integrations with the Cortana personal digital assistant. In this case, end users can naturally interact with models via speech, etc. so end users can proactively be notified by Cortana if the analytics model finds a new anomaly. As you can probably tell, there is no “on size fits all” for effectively communicating actionable insights. The actual delivery mechanism will vary based on business culture and specific stakeholder needs. However, the Cortana Intelligence suite provides the flexibility to bring refined visualization tools and services to the table that can be tailored to a business’s unique needs.
It’s probably worth noting the distinction between the Cortana Personal Digital Assistant and the Cortana Intelligence Suite. The digital assistant is what you use on your devices and can interface with the Intelligence Suite. The Cortana Intelligence suite is the broader range of services we’ve just described.
To round things out, the Cortana Intelligence suite offers Cognitive Service APIs (formerly named Perceptual Intelligence) to allow applications to leverage pre-built scientific algorithms for services like computer vision, facial recognition, speech recognition, and natural language processing (to name a few). These APIs can be useful in both interactions with your business users and in generating data for further analytical processing. A great example of this would be to use the optical character recognition API to read text from images and use the output as a data source for further textual analysis and processing. These APIs are offered as consumable services and are built using the same Azure Machine Learning and Azure analytics services in the backend, but do not require you to build your own models directly. You can interact with the Perceptual Intelligence APIs by simply sending your own data in for analysis.
Strategic Impact
We’ve discussed data analytics in a broad context, described the challenges businesses have advancing to a mature data analytics capability, and described the services that the Cortana Intelligence suite offers to help address these challenges. Let’s finish by discussing the strategic impact.
Businesses are finding that data is indeed the key strategic business asset. Every device, every customer, every activity – everything that’s happening in the world around us is producing incredibly rich data that can help us create new experiences, gain insight into how we can be more efficient, or help to shape new business models and new inventions. Leveraging this data can be the differentiator for businesses who can effectively apply this asset to guide business decisions. International Data Corporation reports that companies that take advantage of their data have the potential to raise an additional $1.6 Trillion in revenue over companies that don’t (1). IBM chief executive, Ginni Rometty, who was speaking at the 2016 annual IBM shareholder meeting recently told shareholders that she sees a $2 trillion opportunity in machine learning and advanced analytics over the next 10 years (2).
So how can businesses take advantage of these new opportunities? According to IDC, they must do four things:
- Leverage diverse data sources
- Embrace new forms of analytics
- Empower more people across their organization to ask questions of their data
- Make decisions faster
The services offered by the Cortana Intelligence suite can aid in each of these goals we described, where specific services in the platform fit in the larger picture.
Sources:
- Vesset, Dan, & Morris, Henry D., & Gantz, John F. (May 2014). Capturing the $1.6 Trillion Data Dividend. IDC Document #248459. Retrieved from
http://download.microsoft.com/documents/en-us/making_the_right_analytics_investments_whitepaper.pdf - Bort, Julie (Feb 2016). The IT industry is launching new markets worth more than $2 trillion, IBM CEO says. Business Insider. Retrieved from
http://www.businessinsider.com/ibm-ceo-pursues-a-new-2-trillion-market-2016-2 - Oberoi, Herain (Oct 2015). Cortana Analytics Overview [PowerPoint slides].
