Key Responsibilities:

<ul>
<li style="margin-left:0px;">Data Pipeline Development: Design, build, and maintain robust, scalable, and efficient data pipelines to collect, process, and store large volumes of data from various sources.</li>
</ul>

<ul>
<li style="margin-left:0px;">Data Integration: Integrate data from multiple sources, including APIs, databases, and external datasets, ensuring data consistency and reliability.</li>
</ul>

<ul>
<li style="margin-left:0px;">Data Modelling: Develop and maintain data models and schemas that support efficient data</li>
</ul>

<ul>
<li style="margin-left:0px;">storage, retrieval, and analytics.</li>
</ul>

<ul>
<li style="margin-left:0px;">Database Management: Manage and optimize databases, ensuring their performance, availability, and security</li>
</ul>

<ul>
<li style="margin-left:0px;">Data Quality: Implement and monitor data quality checks to ensure the accuracy, completeness, and consistency of data. Perform analysis required to troubleshoot data related</li>
</ul>

<ul>
<li style="margin-left:0px;">issues and assist in the resolution of data issues.</li>
</ul>

<ul>
<li style="margin-left:0px;">Automation: configure data extraction and load jobs and automate repetitive tasks and processes to improve efficiency and reduce errors. Improve performances of jobs and</li>
</ul>

<ul>
<li style="margin-left:0px;">pipelines by applying optimisation and automation techniques.</li>
</ul>

<ul>
<li style="margin-left:0px;">Documentation: Maintain clear and comprehensive documentation of data pipelines, data models, and data integration processes. Maintenance of time allocation reports.</li>
</ul>

<ul>
<li style="margin-left:0px;">Develop understanding of business/ processes and high-level understanding of high-quality digital product delivery</li>
</ul>

<ul>
<li style="margin-left:0px;">Ways of working: Follow Agile and SDLC processes including the creation of data related user stories.</li>
</ul>

<ul>
<li style="margin-left:0px;">Perform code peer reviews and testing as per SDLC or industry standards.</li>
</ul>

<ul>
<li style="margin-left:0px;">Collaboration: Work closely with internal customers, product owners, various core team</li>
</ul>

<ul>
<li style="margin-left:0px;">members, and other stakeholders to understand their data needs and provide appropriate solutions.</li>
</ul>

<ul>
<li style="margin-left:0px;">Technical collaboration and oversight with Vendors as required. Create SOWs and work packets with vendors as required. </li>
</ul>
Education:

<ul>
<li style="margin-left:0px;">Bachelors or Masters degree in computer science, data science, software engineering, information systems or a similar field or equivalent experience. </li>
</ul>
Experience and skills:

<ul>
<li style="margin-left:0px;">Over 9 years experience as a Data Engineer or in a similar role.</li>
</ul>

<ul>
<li style="margin-left:0px;">Experience with data pipeline, ETL and workflow management tools (e.g., Databricks, Data Factory).</li>
</ul>

<ul>
<li style="margin-left:0px;">Proficiency in programming languages such as SQL, Python, R, or Scala.</li>
</ul>

<ul>
<li style="margin-left:0px;">Strong experience with SQL and database management (e.g., MySQL, PostgreSQL, SQL Server).</li>
</ul>

<ul>
<li style="margin-left:0px;">Excellent problem-solving and analytical skills.</li>
</ul>

<ul>
<li style="margin-left:0px;">Strong understanding of data architecture, data modelling, and ETL processes.</li>
</ul>

<ul>
<li style="margin-left:0px;">Ability to work in a fast-paced, dynamic environment and manage multiple tasks simultaneously.</li>
</ul>

<ul>
<li style="margin-left:0px;">Strong communication and collaboration skills.</li>
</ul>

<ul>
<li style="margin-left:0px;">Experience with best practice UX / UI design &amp; development of Business Intelligence data visualisation / exploration tools (Power Bi, Qlik etc)</li>
</ul>

<ul>
<li style="margin-left:0px;">Familiarity with SDLC and Agile ways of working.</li>
</ul>

<ul>
<li style="margin-left:0px;">Familiarity with DevOps tools and practices (e.g. Jenkins, Azure DevOps, etc).</li>
</ul>

<ul>
<li style="margin-left:0px;">Familiarity with Atlassian tools like Bitbucket, JIRA, Confluence. </li>
</ul>
Preferred:

<ul>
<li style="margin-left:0px;">GxP and Non GxP SDLC experience</li>
</ul>

<ul>
<li style="margin-left:0px;">Knowledge of data governance and data security best practices.</li>
</ul>

<ul>
<li style="margin-left:0px;">MS Power Platform / power apps experience</li>
</ul>

<ul>
<li style="margin-left:0px;">Cloud DevOps knowledge (e.g. Terraform)</li>
</ul>

<ul>
<li style="margin-left:0px;">Certifications in cloud data technologies (e.g. Azure Data Fundamentals)</li>
</ul>

<ul>
<li style="margin-left:0px;">Basic data science experience and knowledge.</li>
</ul>

Data Engineer

India

Key Responsibilities: Data Pipeline Development: Design, build, and maintain robust, scalable, and efficient data pipelines to collect, process, and store large volumes of data from various sources. Data Integration: Integrate data from multiple sources, including APIs, databases, and external datasets, ensuring data consistency and reliability. Data Modelling: Develop and maintain data models and schemas that support efficient data storage, retrieval, and analytics. Database Management: Manage and optimize databases, ensuring their performance, availability, and security Data Quality: Implement and monitor data quality checks to ensure the accuracy, completeness, and consistency of data. Perform analysis required to troubleshoot data related issues and assist in the resolution of data issues. Automation: configure data extraction and load jobs and automate repetitive tasks and processes to improve efficiency and reduce errors. Improve performances of jobs and pipelines by applying optimisation and automation techniques. Documentation: Maintain clear and comprehensive documentation of data pipelines, data models, and data integration processes. Maintenance of time allocation reports. Develop understanding of business/ processes and high-level understanding of high-quality digital product delivery Ways of working: Follow Agile and SDLC processes including the creation of data related user stories. Perform code peer reviews and testing as per SDLC or industry standards. Collaboration: Work closely with internal customers, product owners, various core team members, and other stakeholders to understand their data needs and provide appropriate solutions. Technical collaboration and oversight with Vendors as required. Create SOWs and work packets with vendors as required. Education: Bachelors or Masters degree in computer science, data science, software engineering, information systems or a similar field or equivalent experience. Experience and skills: Over 9 years experience as a Data Engineer or in a similar role. Experience with data pipeline, ETL and workflow management tools (e.g., Databricks, Data Factory). Proficiency in programming languages such as SQL, Python, R, or Scala. Strong experience with SQL and database management (e.g., MySQL, PostgreSQL, SQL Server). Excellent problem-solving and analytical skills. Strong understanding of data architecture, data modelling, and ETL processes. Ability to work in a fast-paced, dynamic environment and manage multiple tasks simultaneously. Strong communication and collaboration skills. Experience with best practice UX / UI design &amp; development of Business Intelligence data visualisation / exploration tools (Power Bi, Qlik etc) Familiarity with SDLC and Agile ways of working. Familiarity with DevOps tools and practices (e.g. Jenkins, Azure DevOps, etc). Familiarity with Atlassian tools like Bitbucket, JIRA, Confluence. Preferred: GxP and Non GxP SDLC experience Knowledge of data governance and data security best practices. MS Power Platform / power apps experience Cloud DevOps knowledge (e.g. Terraform) Certifications in cloud data technologies (e.g. Azure Data Fundamentals) Basic data science experience and knowledge.

Data Engineer · New Delhi

Job highlights

Skillset required

Job Description

Job Description for Data Engineer

More at HCL