<h4 style="text-align:start;">Responsibilities:</h4>

<ul>
<li>Design and implement data pipelines and architectures to support real-time data processing and analytics.</li>
</ul>

<ul>
<li>Collaborate with cross-functional teams to gather requirements and translate them into technical specifications.</li>
</ul>

<ul>
<li>Lead the development and optimization of data models, ETL processes, and data integration workflows.</li>
</ul>

<ul>
<li>Utilize AWS services to build and maintain scalable data solutions.</li>
</ul>

<ul>
<li>Implement best practices for data governance, security, and compliance.</li>
</ul>

<ul>
<li>Mentor and guide junior data engineers in technical skills and project methodologies.</li>
</ul>

<ul>
<li>Stay updated with industry trends and emerging technologies to continuously improve data engineering practices.</li>
</ul>

<ul>
<li>Conduct performance tuning and optimization of data processing jobs.</li>
</ul>

<ul>
<li>Ensure high availability and reliability of data systems through proactive monitoring and maintenance.</li>
</ul>

<h4 style="text-align:start;">Mandatory Skills:</h4>

<ul>
<li>Proficiency in AWS services, including but not limited to S3, Redshift, Lambda, and Glue.</li>
</ul>

<ul>
<li>Strong experience with PySpark for large-scale data processing.</li>
</ul>

<ul>
<li>Hands-on experience with Apache Flink for real-time data streaming.</li>
</ul>

<ul>
<li>Expertise in data architecture and design principles.</li>
</ul>

<ul>
<li>Proven ability to develop and implement data solutions in a cloud environment.</li>
</ul>

<ul>
<li>Strong problem-solving skills and attention to detail.</li>
</ul>

<ul>
<li>Excellent communication and collaboration skills.</li>
</ul>

<ul>
<li>Deal breaker skill: Proficiency in Databricks.</li>
</ul>

<h4 style="text-align:start;">Preferred Skills:</h4>

<ul>
<li>Experience with data warehousing solutions and data lakes.</li>
</ul>

<ul>
<li>Familiarity with machine learning frameworks and tools.</li>
</ul>

<ul>
<li>Knowledge of containerization technologies such as Docker and Kubernetes.</li>
</ul>

<ul>
<li>Experience with CI/CD pipelines and DevOps practices.</li>
</ul>

<ul>
<li>Understanding of data visualization tools and techniques.</li>
</ul>

<h4 style="text-align:start;">Qualifications:</h4>

<ul>
<li>Bachelor's degree in Computer Science, Engineering, or a related field.</li>
</ul>

<ul>
<li>Relevant certifications in data engineering or cloud technologies are a plus.</li>
</ul>

<ul>
<li>Demonstrated ability to lead projects and work effectively in a team-oriented environment.</li>
</ul>

Lead Data Engineer

India

Responsibilities: Design and implement data pipelines and architectures to support real-time data processing and analytics. Collaborate with cross-functional teams to gather requirements and translate them into technical specifications. Lead the development and optimization of data models, ETL processes, and data integration workflows. Utilize AWS services to build and maintain scalable data solutions. Implement best practices for data governance, security, and compliance. Mentor and guide junior data engineers in technical skills and project methodologies. Stay updated with industry trends and emerging technologies to continuously improve data engineering practices. Conduct performance tuning and optimization of data processing jobs. Ensure high availability and reliability of data systems through proactive monitoring and maintenance. Mandatory Skills: Proficiency in AWS services, including but not limited to S3, Redshift, Lambda, and Glue. Strong experience with PySpark for large-scale data processing. Hands-on experience with Apache Flink for real-time data streaming. Expertise in data architecture and design principles. Proven ability to develop and implement data solutions in a cloud environment. Strong problem-solving skills and attention to detail. Excellent communication and collaboration skills. Deal breaker skill: Proficiency in Databricks. Preferred Skills: Experience with data warehousing solutions and data lakes. Familiarity with machine learning frameworks and tools. Knowledge of containerization technologies such as Docker and Kubernetes. Experience with CI/CD pipelines and DevOps practices. Understanding of data visualization tools and techniques. Qualifications: Bachelor's degree in Computer Science, Engineering, or a related field. Relevant certifications in data engineering or cloud technologies are a plus. Demonstrated ability to lead projects and work effectively in a team-oriented environment.

Lead Data Engineer · Bengaluru

Job highlights

Skillset required

Job Description

Job Description for Lead Data Engineer

More at MPHASIS