An Azure Machine Learning News Aggregator Solution
Here is an overview of a personalized, machine-curated delivery of news and related significant data in the public domain powered by Azure Machine Learning and Azure Cognitive Services, built by AIS.
A Little Background
Today’s 24-hour news cycle produces a vast amount of publicly available information about events occurring all over the globe. Imagine the data that is produced in a single hour, let alone in a single day. Now imagine having the responsibility of distilling that data into actionable information for a single team or individual. It would quickly become overwhelming! This was the reality for our client. They have offices and teams all over the world, and each team was seeking personalized views of data relevant to their region, mission and preferences. But by implementing machine learning technology, AIS developed a solution to help break through the noise.
The Challenge
The client wanted to deliver continuously updated news and related content to its large user base, which includes stakeholders around the world. In this case, data would need to be consumed from a diverse set of public news sources and then combined and presented to users in a single location. Instead of simply dumping the most up-to-date content for users to sift through, they needed to tailor the content to the individual consumer. For the best experience, the content would need to adapt on the fly to users’ changing interests and highlight the most relevant information.
To accomplish these goals, the implemented solution would need to execute large-scale data ingestion and automatically discover meaning in the content; it could then generate higher-level topics, classify each article based on topic, and rank each article’s relevance to the discovered topics. Additionally, the solution would need to effectively match a user’s preferences and metadata (e.g., location) to the evolving topics in recent news content, and present this functionality in a standard, consumable format.
Solution
To address each of these needs and create a resilient, cost-effective solution, we chose to utilize a combination of services offered by Microsoft Azure, with Azure Machine Learning playing a prominent role in the underlying engine. The solution is delivered as a set of lightweight web services that clients simply call through the secure gateway, so multiple client applications can now integrate with the Intelligent News Aggregator.
Technology Overview
The solution is comprised of the following high-level components:
Data Ingestion
- Azure Web Jobs
- Azure Blob Storage
Topic Discovery, Classification, Relevance Ranking
- Azure Machine Learning
- Azure Blob Storage
Application Service Logic
- Azure Web Apps (Web API)
- Azure Machine Learning Web Services
- External Search News Search (Cognitive Services / Bing News Search API)
API Gateway Services
- Azure API Management
Client Application(s)
Please see the representative diagram below:
