PinnedBuilding an Adaptive Anomaly Detection Pipeline for Financial Streams This is not a finished project post. It is a working notes post of what I am building, why I am building it, what is working, what is not yet, and where it is going. I am writing this now, mid-build, Jun 22, 2026·7 min read
How to Handle Tough Data Engineering Interview Questions: Tips from Real ExperiencesJun 30, 2025·7 min read
Identifying Customers with Consecutive Monthly Orders: A Data Engineering ApproachIntroduction In the world of e-commerce and subscription-based businesses, identifying customers who demonstrate consistent purchasing behavior is crucial for customer retention strategies, loyalty programs, and revenue forecasting. One particularly ...Jun 3, 2025·6 min read
Python, Pipelines, and the Rise of the MeshAs organizations double down on becoming data-driven, the data engineering landscape continues to evolve at a rapid pace. In 2025, several transformative trends are redefining how data is collected, processed, and utilized across enterprises. Here's ...May 15, 2025·3 min read
What Went Wrong with Forward: A Postmortem on AI-Powered CarePodsIn November 2024, Forward, a healthcare tech startup once backed by over $650 million in funding, abruptly shut down. This is not just a teardown; it’s a lesson in how data systems can make or break a technically ambitious product. Context: What Forw...May 9, 2025·4 min read
Building Resilient Multi-Source ETL Pipelines: A System Design ApproachIntroduction Building ETL pipelines that can withstand constantly evolving input formats presents a significant system design challenge. This article outlines a resilient ETL architecture that minimizes downstream disruptions and accelerates root-cau...May 5, 2025·5 min read
How Netflix Uses Both Spark and SnowflakeNetflix is one of the largest data-driven companies on the planet — it serves billions of streaming events, personalized recommendations, and real-time analytics every single day. To handle this scale, Netflix doesn’t pick just Spark or just Snowflak...Apr 27, 2025·3 min read
Distributed File Systems (DFS): Architecture, Use Cases & EvolutionIntroduction In today's data-driven world, traditional file systems on a single machine are inadequate for meeting the scale and performance demands of modern data infrastructure. This is where Distributed File Systems (DFS) become essential. In this...Apr 15, 2025·6 min read
