Michal Pilarski, Mateusz Adamczak

Software Tester | Data QA, Software Engineer
GISKI | Dynatrace
Poland

About

Throughout his career, Michal has always been connected to geospatial data and GIS geoprocessing. He enjoys identifying and overcoming challenges in testing Big Data with geometry attributes. He has experience in preparing testing strategies for ETL (Extract, Transform, Load) systems . His technology stack includes Python, Pytest, ArcGIS, QGIS, FME, Robot Framework, Selenium, Playwright, HP ALM, QTest, ReportPortal, Snowflake, DBT, Airflow, and GeoPandas . Additionally, he teaches young students Python coding in MinecraftWith around 12 years of experience in the software industry, Mateusz Adamczak has covered most of the key roles — tester, developer, DevOps engineer – and also served as a Scrum Master for a period of time.This diverse background gives him a comprehensive overview of the software development lifecycle, which he enjoys sharing with others.
Workshop

Michal Pilarski, Mateusz Adamczak | Data Testing of ETL Pipeline

Pytest, Airflow, Python, Pandas
In the time of data-driven decision-making, the data validation of ETL (Extract, Transform, Load) pipelines is crucial for delivering high quality information. This study explores the design and testing of an ETL data pipeline built with Apache Airflow, Python Pandas, and Pytest. Airflow orchestrates pipeline workflows, ensuring transformation dependencies and scheduling are managed correctly. Pandas handles data manipulation, offering robust tools for efficient transformations. Pytest enables a structured framework for data attributes testing like reliability, accuracy, consistency, completeness and uniqueness. Overall, the presented approach demonstrates how integrating workflow orchestration, data transformation, and automated testing creates a reliable foundation for trustworthy analytics. By embedding validation directly into the ETL process, potential data quality issues can be detected early, reducing downstream risks and improving decision-making confidence. The combination of Airflow, Pandas, and Pytest provides a scalable and maintainable framework that supports continuous monitoring of data quality, highlighting the importance of test-driven practices in modern data engineering pipelines.
2026-11-24
09:00
17:00