Data Engineer

The Data Engineer designs, builds, and maintains scalable data pipelines and custom automation workflows across GCP, BigQuery, Apache Airflow, and Kubernetes.

Key Responsibilities:

• Pipeline Development: Build and maintain a highly performance, robust and scalable ETL/ELT pipelines to ingest data from relational/NoSQL databases and third-party APIs into central data warehouses.

• Workflow Automation: Orchestrate job dependencies, schedules, and containerized tasks.

• Operational Process Automation: Build and maintain custom automated workflows, scripts, and business logic to streamline operations and automate routine administrative tasks.

• Data Quality & Governance: Maintain data dictionaries, data lineage, and automated validation checks to ensure high data integrity and accuracy, while managing access control and ensuring compliance with data privacy regulations.

• System Reliability & Troubleshooting: Implement logging, auditing, error handling, and monitoring to detect, alert, and recover from pipeline failures while ensuring system health.

• Cross-Functional Collaboration: Partner with Product, Data, and DevOps teams to build reliable data infrastructure and maintain clear technical documentation.

Qualifications:

• Experience: 3–5+ years of hands-on experience as a Data Engineer role.

• Education: Bachelor’s degree in Computer Science, Information Systems, Engineering, or a related technical field (or equivalent practical experience).

Hard Skills:

• Data Modelling: Dimensional Modeling, Multi-Layer Architecture, Data Lineage & Governance.

• Cloud & Warehousing: Google Cloud Platform (GCP), BigQuery, Cloud Run, Cloud Storage.

• Orchestration & Containers: Apache Airflow, Celery, Docker, Kubernetes.

• Languages & Databases: Advanced Python, SQL, MongoDB, and REST API/Webhook integrations.

• Engineering Practices: Git/version control, CI/CD pipelines, logging, auditing, error-handling frameworks, and IAM/Access Management basics.

Soft Skills:

• Troubleshooting & Problem Solving: Strong analytical skills with a focus on auditability, issue resolution, and system reliability.

• Detail-Oriented: High attention to detail regarding data accuracy, validation, and process automation.

• Self-Driven & Collaborative: Able to work independently with minimal oversight while collaborating effectively within a team environment.