
In this talk, Raghav Matta and Vivek Singh explore a real-world scenario involving real-time social media analysis using Databricks AI Functions in SQL for enrichment powered by large language models (LLMs).
They begin by outlining the business problem, focusing on how organizations monitor sentiment, trends, and reactions on social media platforms.
Next, they walk through the solution architecture, including Azure Event Hubs, Unity Catalog, and Azure Databricks, with a focus on Databricks AI Functions in SQL, such as aianalyzesentiment(), aiextract(), and aiclassify().
They then ingest live or simulated social media content into Databricks, apply Databricks AI Functions to analyze sentiment and topics, store the results in Delta Lake, and create visualizations using Databricks AI/BI.
Finally, they use a Databricks Genie Agent to ask questions about the analyzed data in natural language and derive further insights.