Key Responsibilities:

<ul>
<li>Design, develop, and maintain data processing pipelines using Apache Spark.</li>
</ul>

<ul>
<li>Optimize Spark jobs for performance and scalability in distributed environments.</li>
</ul>

<ul>
<li>Work with large datasets to perform ETL (Extract, Transform, Load) operations.</li>
</ul>

<ul>
<li>Collaborate with data scientists, analysts, and engineers to deliver robust data solutions.</li>
</ul>

<ul>
<li>Ensure adherence to best practices for data quality, security, and governance.</li>
</ul>

<ul>
<li>Troubleshoot and resolve issues related to data processing and performance tuning.</li>
</ul>

Required Skills &amp; Qualifications:

<ul>
<li>Proficiency in Spark Core, Spark SQL, and Spark Streaming.</li>
</ul>

<ul>
<li>Strong programming skills in Scala (preferred), or Python/Java.</li>
</ul>

<ul>
<li>Experience with distributed computing and parallel processing concepts.</li>
</ul>

<ul>
<li>Familiarity with the Hadoop ecosystem, Hive, or related technologies.</li>
</ul>

<ul>
<li>Solid understanding of data structures, algorithms, and performance optimization.</li>
</ul>

Lead Big Data Engineer

India

Key Responsibilities: Design, develop, and maintain data processing pipelines using Apache Spark. Optimize Spark jobs for performance and scalability in distributed environments. Work with large datasets to perform ETL (Extract, Transform, Load) operations. Collaborate with data scientists, analysts, and engineers to deliver robust data solutions. Ensure adherence to best practices for data quality, security, and governance. Troubleshoot and resolve issues related to data processing and performance tuning. Required Skills &amp; Qualifications: Proficiency in Spark Core, Spark SQL, and Spark Streaming. Strong programming skills in Scala (preferred), or Python/Java. Experience with distributed computing and parallel processing concepts. Familiarity with the Hadoop ecosystem, Hive, or related technologies. Solid understanding of data structures, algorithms, and performance optimization.

Lead Big Data Engineer · Bengaluru

Job highlights

Skillset required

Job Description

Job Description for Lead Big Data Engineer

More at WIPRO