This role is based in our Menara Skymind office.
We are looking for a Data Engineer to join our Data Engineering team in Penang.
Purpose of role:
Data Engineer is responsible for designing, building, and maintaining scalable data pipelines and infrastructure to support the organization’s data and analytics needs. They play a critical role in the end-to-end data lifecycle by ensuring that data is reliably collected, efficiently processed, and made accessible for analysis across various business functions. This position requires strong technical proficiency in data engineering tools, cloud platforms, and integration frameworks, as well as a deep understanding of data modelling, data quality, and automation best practices. In addition, they are expected to troubleshoot issues, manage risks, and continuously optimize data workflows for performance and reliability.
The Data Engineer also collaborates closely with key stakeholders across the Intelligence & Advisory (I&A) division, including Data Analysts, Research teams, and Technology partners, to translate business requirements into robust technical solutions. They contribute to shaping data strategy, reviewing integration and pipeline designs, and mentoring junior team members to uplift engineering standards across the organization.
Key Responsibilities:
- Design, develop, and maintain scalable ETL/ELT pipelines for data ingestion from diverse sources (APIs, web scraping, cloud systems, files).
- Implement data quality checks, monitoring, and validation processes to ensure accuracy and reliability.
- Collaborate with stakeholders to understand data requirements and translate them into efficient technical solutions.
- Manage and optimize data storage across cloud and on-premises environments (e.g., AWS S3, databases, and warehouses).
- Automate manual processes (e.g., Excel, VBA workflows) into centralized and scalable solutions.
- Work with orchestration tools to schedule and monitor data workflows.
- Maintain documentation for data pipelines, transformations, and system integrations.
- Ensure data security, compliance, and governance standards are applied consistently.
Ethics & Transparency:
- Ensure strict compliance with global data regulations (such as GDPR, CCPA) and internal governance standards to safeguard personal and sensitive data across all data pipelines and storage systems.
- Maintain transparency in data engineering processes by clearly documenting data sources, transformation logic, data quality rules, and pipeline behavior so stakeholders understand how data is processed and delivered.
- Take ownership of assigned engineering tasks and proactively engage with stakeholders, including analysts, researchers, and technology teams to ensure clear, timely, and accurate communication throughout the data lifecycle.
- Demonstrate a strong understanding and commitment to Data Engineering and I&A goals, values, and best practices, adhering to all relevant policies, security guidelines, and ethical data-handling frameworks.
