Responsibilities:

<ul>
<li style="margin-left:0px;">Ensuring high quality software development, with complete documentation and traceability</li>
</ul>

<ul>
<li style="margin-left:0px;">Develop and optimize scalable Spark Java-based data pipelines for processing and analyzing large scale financial data</li>
</ul>

<ul>
<li style="margin-left:0px;">Design and implement distributed computing solutions for risk modeling, pricing and regulatory compliance</li>
</ul>

<ul>
<li style="margin-left:0px;">Ensure efficient data storage and retrieval using Big Data</li>
</ul>

<ul>
<li style="margin-left:0px;">Implement best practices for spark performance tuning including partition, caching and memory management</li>
</ul>

<ul>
<li style="margin-left:0px;">Maintain high code quality through testing, CI/CD pipelines and version control (Git, Jenkins)</li>
</ul>

<ul>
<li style="margin-left:0px;">Work on batch processing frameworks for Market risk analytics</li>
</ul>

<ul>
<li style="margin-left:0px;">Promoting unit/functional testing and code inspection processes</li>
</ul>

<ul>
<li style="margin-left:0px;">Work with business stakeholders and Business Analysts to understand the requirements</li>
</ul>

<ul>
<li style="margin-left:0px;">Work with other data scientists to understand and interpret complex datasets</li>
</ul>

Qualifications:

<ul>
<li style="margin-left:0px;">5- 8 Years of experience in working in data eco systems.</li>
</ul>

<ul>
<li style="margin-left:0px;">4-5 years of hands-on experience in Hadoop, Scala, Java, Spark, Hive, Kafka, Impala, Unix Scripting and other Big data frameworks.</li>
</ul>

<ul>
<li style="margin-left:0px;">3+ years of experience with relational SQL and NoSQL databases: Oracle, MongoDB, HBase</li>
</ul>

<ul>
<li style="margin-left:0px;">Strong proficiency in Python and Spark Java with knowledge of core spark concepts (RDDs, Dataframes, Spark Streaming, etc) and Scala and SQL</li>
</ul>

<ul>
<li style="margin-left:0px;">Data Integration, Migration &amp; Large Scale ETL experience (Common ETL platforms such as PySpark/DataStage/AbInitio etc.) - ETL design &amp; build, handling, reconciliation and normalization</li>
</ul>

<ul>
<li style="margin-left:0px;">Data Modeling experience (OLAP, OLTP, Logical/Physical Modeling, Normalization, knowledge on performance tuning)</li>
</ul>

<ul>
<li style="margin-left:0px;">Experienced in working with large and multiple datasets and data warehouses</li>
</ul>

<ul>
<li style="margin-left:0px;">Experience building and optimizing ‘big data’ data pipelines, architectures, and datasets.</li>
</ul>

<ul>
<li style="margin-left:0px;">Strong analytic skills and experience working with unstructured datasets</li>
</ul>

<ul>
<li style="margin-left:0px;">Ability to effectively use complex analytical, interpretive, and problem-solving techniques</li>
</ul>

<ul>
<li style="margin-left:0px;">Experience with Confluent Kafka, Redhat JBPM, CI/CD build pipelines and toolchain – Git, BitBucket, Jira</li>
</ul>

<ul>
<li style="margin-left:0px;">Experience with external cloud platform such as OpenShift, AWS &amp; GCP</li>
</ul>

<ul>
<li style="margin-left:0px;">Experience with container technologies (Docker, Pivotal Cloud Foundry) and supporting frameworks (Kubernetes, OpenShift, Mesos)</li>
</ul>

<ul>
<li style="margin-left:0px;">Experienced in integrating search solution with middleware &amp; distributed messaging - Kafka</li>
</ul>

<ul>
<li style="margin-left:0px;">Highly effective interpersonal and communication skills with tech/non-tech stakeholders.</li>
</ul>

<ul>
<li style="margin-left:0px;">Experienced in software development life cycle and good problem-solving skills.</li>
</ul>

<ul>
<li style="margin-left:0px;">Excellent problem-solving skills and strong mathematical and analytical mindset</li>
</ul>

<ul>
<li style="margin-left:0px;">Ability to work in a fast-paced financial environment</li>
</ul>

Education:

<ul>
<li style="margin-left:0px;">Bachelor’s/University degree or equivalent experience in computer science, engineering, or similar domain </li>
</ul>

Data Engineer

Responsibilities: Ensuring high quality software development, with complete documentation and traceability Develop and optimize scalable Spark Java-based data pipelines for processing and analyzing large scale financial data Design and implement distributed computing solutions for risk modeling, pricing and regulatory compliance Ensure efficient data storage and retrieval using Big Data Implement best practices for spark performance tuning including partition, caching and memory management Maintain high code quality through testing, CI/CD pipelines and version control (Git, Jenkins) Work on batch processing frameworks for Market risk analytics Promoting unit/functional testing and code inspection processes Work with business stakeholders and Business Analysts to understand the requirements Work with other data scientists to understand and interpret complex datasets Qualifications: 5- 8 Years of experience in working in data eco systems. 4-5 years of hands-on experience in Hadoop, Scala, Java, Spark, Hive, Kafka, Impala, Unix Scripting and other Big data frameworks. 3+ years of experience with relational SQL and NoSQL databases: Oracle, MongoDB, HBase Strong proficiency in Python and Spark Java with knowledge of core spark concepts (RDDs, Dataframes, Spark Streaming, etc) and Scala and SQL Data Integration, Migration &amp; Large Scale ETL experience (Common ETL platforms such as PySpark/DataStage/AbInitio etc.) - ETL design &amp; build, handling, reconciliation and normalization Data Modeling experience (OLAP, OLTP, Logical/Physical Modeling, Normalization, knowledge on performance tuning) Experienced in working with large and multiple datasets and data warehouses Experience building and optimizing ‘big data’ data pipelines, architectures, and datasets. Strong analytic skills and experience working with unstructured datasets Ability to effectively use complex analytical, interpretive, and problem-solving techniques Experience with Confluent Kafka, Redhat JBPM, CI/CD build pipelines and toolchain – Git, BitBucket, Jira Experience with external cloud platform such as OpenShift, AWS &amp; GCP Experience with container technologies (Docker, Pivotal Cloud Foundry) and supporting frameworks (Kubernetes, OpenShift, Mesos) Experienced in integrating search solution with middleware &amp; distributed messaging - Kafka Highly effective interpersonal and communication skills with tech/non-tech stakeholders. Experienced in software development life cycle and good problem-solving skills. Excellent problem-solving skills and strong mathematical and analytical mindset Ability to work in a fast-paced financial environment Education: Bachelor’s/University degree or equivalent experience in computer science, engineering, or similar domain

Data Engineer in Mumbai (UID-12613)

Job Description

Job Details

Skills Required for Data Engineer

Job Description for Data Engineer

Explore More Opportunities