Data Engineer

Job Description:

• Build and maintain data pipelines from various data sources.

• Develop and maintain web crawling/scraping processes to collect data.

• Perform ETL (Extract, Transform, Load) processes to prepare data for further use.

• Clean, transform, and validate data to ensure good data quality.

• Develop scripts to process documents and generate metadata from PDF files.

• Monitor and troubleshoot data crawling and processing processes.

• Automate data processing workflows using tools such as Airflow, n8n, or similar tools.

• Collaborate with the team to support data needs for applications, analytics, and AI-related projects.

Qualification:

• Bachelor’s degree in computer science, mathematics, statistics, or related fields.

• Minimum 1 Years experience as Data engineer, Data Processing, Web Scarping, or a related field.

• Proficient in SQL and at least one programming language, preferably Python.

• Familiar with web crawling/scraping and data cleaning. • Familiar with ETL/data pipeline processes.

• Familiarity with Airflow, n8n, or similar tools.

• Familiar with relational databases such as PostgreSQL or MySQL.

• Experience with PDF/document processing is a plus.

• Strong analytical and problem-solving skills.

• Able to work independently and within a team.

• Willing to continuously learn and grow.