Data Engineer

Job Purpose

Data Engineers are responsible for the design, development, and operationalization of scalable, secure, and high-performance data platforms that support enterprise-wide analytics and decision-making. They build and maintain the infrastructure that enables efficient data ingestion, transformation, storage, and access across diverse business domains.

These professionals architect and implement robust data solutions that meet business and analytical requirements, including data lakes, data warehouses, and real-time streaming platforms. They develop and optimize ETL/ELT pipelines, manage large-scale datasets, and ensure data quality, consistency, and integrity throughout the data lifecycle.

Data Engineers maintain a secure and compliant data environment by implementing data governance policies, access controls, and encryption standards. They monitor system performance, troubleshoot data pipeline issues, and plan for capacity, scalability, and operational resilience. Their responsibilities may include automating workflows, supporting 24/7 operations, and participating in incident response and recovery planning.

This role requires strong analytical, problem-solving, and communication skills, as well as the ability to collaborate effectively with data analysts, software engineers, business stakeholders, and IT teams. Data Engineers apply a customer-centric mindset to deliver reliable, accessible, and well-documented data services that empower self-service analytics and data-driven decision-making. They also contribute to knowledge sharing, mentor peers, and stay current with emerging technologies in data engineering and cloud infrastructure.

Key Responsibilities

Data Platform Architecture & Development

• Design and implement scalable data platforms (e.g., data lakes, warehouses, streaming systems).
• Ensure systems are secure, high-performance, and aligned with business needs.

Data Pipeline Engineering (ETL/ELT)

• Build and optimize data ingestion, transformation, and loading processes.
• Handle batch and real-time data workflows with tools like Spark, Airflow, or cloud-native services.

Data Quality & Governance

• Implement data validation, cleansing, and monitoring processes to ensure data accuracy and consistency.
• Collaborate with data stewards and governance teams to enforce data standards and policies.
• Maintain metadata, data lineage, and documentation for transparency and traceability.

Platform Operations and Monitoring

• Monitor data infrastructure and pipeline performance using observability tools and alerts.
• Troubleshoot and resolve data-related issues, ensuring minimal disruption to business operations.
• Plan for system capacity, scalability, and disaster recovery.

Security and Compliance

• Apply data security best practices, including encryption, access control, and auditing.
• Ensure compliance with data privacy regulations (e.g., GDPR, HIPAA) and internal policies.
• Collaborate with security teams to assess and mitigate risks.

Cross-functional Collaboration

• Work closely with data analysts, business users, and software engineers to understand data requirements.
• Translate business needs into technical specifications and data solutions.
• Provide support and training to enable self-service analytics and data access.

Continuous Improvement and Innovation

• Identify opportunities to optimize data workflows and reduce technical debt.
• Stay current with emerging data technologies, tools, and best practices.
• Contribute to the development of internal standards, frameworks, and reusable components.

Knowledge/Qualifications for the Role

• Bachelor’s degree in Computer Science, Information Systems, Data Engineering, or a related field.
• Demonstrated proficiency in written and verbal communication in English.
• 7 or more years of professional experience in data engineering or a related field.
• Hands-on experience designing, building, and maintaining scalable data pipelines and ETL/ELT workflows using tools such as Apache Spark, Airflow, or cloud-native services (e.g. AWS Glue, Azure Data Factory).
• Experience with cloud platforms (e.g. AWS, Azure) for data storage, processing.
• Proficiency in programming languages such as Python, SQL with a strong understanding of data structures and algorithms.
• Collaboration experience with cross-functional teams including data scientists, analysts, and software engineers to deliver end-to-end data solutions.
• Experience implementing data quality, validation, and governance practices to ensure data integrity and compliance.
• Experience working with large-scale datasets, including structured, semi-structured, and unstructured data is highly preferred.
• Relevant certification is required.
• Willingness and ability to travel domestically and internationally, as necessary.