Data Engineer · Gurugram

Roles and Responsibilities

• Architect, build, and optimize scalable data pipelines using Python, SQL, and distributed processing frameworks such as Spark.

• Design and maintain robust ETL/ELT workflows to integrate structured and semi-structured data from multiple internal and external systems.

• Ensure data quality, lineage, validation, and monitoring across all ingestion and transformation layers.

• Partner with analytics and product teams to operationalize analytical models, KPI dashboards, and metric frameworks.

• Build reusable datasets, analytics marts, and query-optimized schemas to support diverse use cases including reporting, forecasting, personalization, and operational automation.

• Translate raw data into accessible, well-governed, analysis-ready formats.

• Develop SOPs, quality checks, and review workflows to ensure accurate, consistent, and reliable data delivery

• Establish coding standards, documentation requirements, and review mechanisms to ensure high engineering rigor

• Implement monitoring and error-handling frameworks to improve system reliability and reduce failures.

• Lead and mentor a team of data engineers, analysts, and technical contributors.

• Define project scope, timelines, and milestones; drive execution using Agile methodologies or equivalent project management approaches.

• Coordinate with cross-functional teams (engineering, product, business) to ensure alignment and timely delivery of data initiatives.

• Oversee vendor partners, outsourced contributors, or external data service providers where relevant.

• Work closely with senior stakeholders to understand business requirements, translate them into scalable technical solutions, and communicate progress effectively.

• Provide data-driven insights and recommendations to support operational efficiency and strategic decision-making.

• Lead development of data-driven systems such as rules-based or ML-powered recommendation engines, user segmentation pipelines, scoring models, or personalization algorithms.

• Implement frameworks that track user behavior, performance metrics, or system outputs to continuously enhance relevance and accuracy.

• Oversee creation of dashboards and automated reporting solutions using tools like Plotly or equivalent BI tools.

• Collaborate with analytics teams to identify trends, anomalies, or optimization opportunities across operational and customer-facing data.

Required Skills & Experience

• 5-8 years of experience in data engineering, data platforms, or large-scale data projects.

• Strong programming skills in Python, SQL, and experience with Pandas, NumPy, Spark or similar distributed systems.

• Proven track record of building end-to-end data pipelines and delivering production-grade data solutions.

• Experience designing data models, quality frameworks, governance standards, and automated monitoring.

• Demonstrated ability to lead technical teams, manage projects, and collaborate across cross-functional units.

• Experience using data to derive insights, optimize systems, and improve business outcomes.

• Ability to work in fast-paced environments and manage multiple priorities simultaneously

• Excellent communication, stakeholder management, and documentation abilities

• Knowledge in using leading cloud platforms preferably Azure.

• Ability to work in a team in an agile setting, familiarity with JIRA and clear understanding of how Git works