May 8

Burak Güneli’s talk is divided into two main sections. First, he will provide insights into the sustainability of both the web and AI. Drawing on his experience as a Coordinator, Data Analyst, and Author for the Web Almanac, Burak will share calculated statistics, including exclusive sustainability data that is currently unavailable online, as this year’s Sustainability chapter has not yet been released.
In the second section, Burak will present an approach to optimize agent connections in applications. This will feature a demonstration of semantic caching for prompts, a technique that saves prompts as vectors in IndexedDB and applies cosine similarity to prevent sending redundant prompts to the agent. This allows applications to instantly display cached answers, completely bypassing wait times and cost-heavy AI calculations.

In this workshop, Raghav Matta explores a real-world scenario of performing real-time social media analysis using Databricks with SQL AI Functions for LLM‑powered enrichment.

The session begins with the business problem, focusing on how organizations monitor sentiment, trends, and reactions on social platforms.

Next, the talk walks through the Databricks architecture, including Event Hubs, Unity Catalog, and Azure Databricks, with a focus on SQL AI functions such as ai_analyze_sentiment, ai_extract(), and ai_classify().

The session then demonstrates how to ingest live or simulated social content into Databricks, apply SQL AI Functions to analyze sentiment and topics, store results in Delta Lake, and create visual insights using Databricks AI/BI.

Additionally, Raghav shows how to use Databricks Genie to interact with the analyzed data and derive further insights.

Reference: https://www.youtube.com/watch?v=BzL-8iM_YOM

How do you select LLMs to deploy on a GPU cluster for hundreds of engineers?
LLM benchmark results are proudly published with every new model release — but what’s really behind those numbers?

This talk guides you through designing benchmarks that truly measure the real-world problem-solving capabilities of LLMs.

We’ll explore methods for evaluating business-relevant LLM skills such as:

coding performance,

tool usage and API calling,

behavior in agentic workflows.

After uncovering common flaws in popular benchmarks, we’ll demonstrate alternative techniques for LLM evaluation — including:

ground truth-based testing,

LLM-as-a-judge, and

execution-based validation (e.g., testing LLM-generated code).

We’ll also highlight common pitfalls, such as how prompt variations and generation parameters can subtly and silently skew benchmark results.

Finally, we’ll provide a practical decision framework to help you choose the right LLM, balancing latency, cost, and output quality.

By the end of this talk, you’ll be equipped to go beyond the hype and evaluate LLMs like an engineer — not just by the score.