Blog
I write about the technologies I work with every day—from Apache Kafka and Apache Flink to distributed systems, software architecture, and AI. Most articles are published on Medium and grouped below by topic.
Hands-On with Flink (7 articles)
Hands-On with Flink — Part 1: Filtering Data from Kafka
Build your first Apache Flink application from scratch. Learn how to read events from Kafka, filter data in real time, write the results back to Kafka, and deploy your first Flink job using Docker Compose.
Read on Medium →
Hands-On with Flink — Part 2: Running on Kubernetes with the Operator
Deploy your first Apache Flink application to Kubernetes. Learn how to use the Flink Kubernetes Operator, understand the roles of the JobManager and TaskManagers, and run Flink in both Application and Session deployment modes.
Read on Medium →
Hands-On with Flink — Part 3: SQL instead of Java
Replace Java code with Flink SQL and simplify your data pipelines. Learn how to define Kafka sources and sinks, write streaming SQL queries, and deploy a complete Flink SQL job with significantly less boilerplate.
Read on Medium →
Hands-On with Flink — Part 4: SQL and Avro
Work with Avro data in Apache Flink SQL. Learn how to use Avro schemas with Kafka topics, integrate Schema Registry, and process strongly typed streaming data without writing Java code.
Read on Medium →
Hands-On with Flink — Part 5: Managing State
Understand how Apache Flink manages state in streaming applications. Learn how state enables stateful computations, how checkpoints provide fault tolerance, and why state management is fundamental to building reliable real-time data pipelines.
Read on Medium →
Hands-On with Flink — Part 6: Calling LLMs from Flink
Integrate Large Language Models directly into Apache Flink pipelines. Learn how to call LLMs from Flink SQL, enrich streaming data with AI-generated insights, and build intelligent real-time applications using practical examples.
Read on Medium →
Hands-On with Flink — Part 7: Exploring Flink Checkpoints
In this part of the series, we explore Apache Flink checkpoints—the foundation of fault-tolerant stream processing. You’ll configure checkpointing on Kubernetes, verify that it works, and see how Flink recovers after a TaskManager failure.
Read on Medium →
Apache Kafka (6 articles)
Why Kafka Consumers Sometimes Show Lag = 1 Forever
Understand why Kafka consumer lag sometimes stays at 1 forever. Learn how consumer lag is calculated, why this behavior is expected in certain scenarios, and how to distinguish real processing delays from misleading metrics.
Understanding Kafka KRaft: How Controllers and Brokers Talk in the Zookeeper-less World
Explore how Kafka controllers and brokers communicate in KRaft mode. Learn how the Raft-based architecture replaces ZooKeeper, how metadata is replicated, and how the cluster stays consistent without an external coordination service.
When Avro References Avro
Learn how Avro schema references help manage complex data models. Explore when and why to use schema references, how they simplify schema evolution, and how to work with them effectively in Kafka and Schema Registry.
Hands-Docker Compose for Running Kafka in KRaft Mode
Set up a complete Apache Kafka KRaft cluster with Docker Compose. Learn how to run Kafka without ZooKeeper, understand the required KRaft configuration, and create a local environment for development and testing.
How does Schema Validation Work on Apache Kafka?
Understand how schema validation protects your Kafka topics from incompatible data. Learn how Schema Registry validates messages, how compatibility rules are enforced, and how to use schema validation to improve data quality and reliability.
An Approach for Rewinding Kafka Consumers
Safely rewind Kafka consumers without losing control of your data processing. Learn different approaches to resetting consumer offsets, when to use each strategy, and how to replay messages reliably in production environments..