⚡ Optimized supply chain payment workflows by analyzing large-scale vendor-client datasets with PySpark on Databricks, directly reducing outstanding payments by 14% and reconciliation errors by 20%. 🛠️ Engineered scalable data pipelines using Databricks, Apache Spark, and Azure Data Lake Storage. Implemented Medallion Architecture (Bronze, Silver, Gold) to enhance data quality, boosting processing speed by 20% and significantly reducing ETL latency. 📊 Developed automated, real-time dashboards using Python (Dash), SQL, and Airflow, delivering up-to-the-minute insights that accelerated data-driven decision-making for key stakeholders. 🔄 Orchestrated end-to-end data ingestion across diverse cloud and on-premise systems, leveraging Hadoop-based connectors and Apache Airflow for robust scheduling and pipeline orchestration. 🤝 Partnered with product managers and cross-functional teams to deploy KPI-rich, interactive risk-monitoring reports that proactively identified supply chain bottlenecks and enabled targeted mitigation strategies. 💻 Core Tech Stack: PySpark, Databricks, Apache Airflow, Python (Dash), SQL, Azure Data Lake Storage, Hadoop, Medallion Architecture.

Data Science & Analytics

Senior Data Engineer

Gl Bajaj Institute Of Technology And Management, Greater Noida

India

Senior Data Engineer @ 7-Eleven | Building PrepNPlaced | Ex-Target | Data Engineering Mentor | Building AI Powered CareerOS for Resume, Jobs & Interviews

Durgesh Yadav

Social Accounts

Achievements

Experience

Senior Data Engineer