The hookRight now, somewhere in the universe, a star is collapsing. Within seconds, NASA’s General Coordinates Network (GCN) broadcasts alerts over public Apache Kafka streams to which anyone can subscribe, covering gamma-ray bursts, gravitational-wave detections, and neutrino detections. Getting that data is the easy part—the first 5%. The other 95%—cleaning, classifying, enriching, and reasoning over it—is where data engineering actually happens.The 2025 approachIn 2025, turning that stream into something a human could query meant building a retrieval-augmented generation (RAG) application by hand: 370 lines of code across two notebooks just to handle chunking, embeddings, and vector syncing before any real work began.What’s new now?The boilerplate is largely gone, and the whole thing now runs on Databricks Free Edition: a no-cost workspace that requires no credit card or cloud account and has no fixed expiry. This is not a time-limited trial. It provides access to a broad set of Databricks lakehouse and AI capabilities—including foundation models, AI Functions, Genie, Agent Bricks, and Lakebase, a built-in, fully managed Postgres database—subject to Free Edition’s usage and feature limits. In short, it makes many of the same core capabilities used by enterprises available without the invoice.The buildIn this session, Frank Munz builds a live, end-to-end pipeline from scratch using real cosmic-event data. He ingests NASA’s GCN Kafka stream with a few lines of SQL using Lakeflow Spark Declarative Pipelines; classifies each event inline—as a gamma-ray burst, a gravitational-wave detection, or a neutrino detection—with a single ai_classify() call and no model to train or host; and then queries the results in plain English with Genie, showing where Genie Code, Genie Agents, and Model Context Protocol (MCP) servers each fit and where each one reaches its limits. The models, tables, and external connections used in the workflow remain governed through Unity Catalog throughout.The takeawayAttendees leave knowing how to reproduce a modern streaming-and-AI workflow in Databricks Free Edition using live data from space—at no cost and in a single evening.Key takeaways include:How to ingest a live external Kafka stream into the lakehouse with SQL alone, without Spark boilerplate How AI Functions bring large language model (LLM) inference directly into a streaming query, with no model to train or host Where Genie Code, Genie Agents, and MCP servers each fit, along with the honest limits of each How Free Edition can be used to build and prototype applications How Lakebase provides a managed Postgres database alongside streaming data, without additional infrastructure, within Databricks Free Edition How Unity Catalog centralizes governance for the models, tables, and external connections used in the workflow