Data Engineer

Transformation Department

Main Responsibilities‚Äč

  • Design and develop data infrastructure and architecture for large scale data storage and processing.
  • Construct cloud-based data lakes, warehouse and marts in Azure/AWS/Google cloud environment.
  • Manage Azure/AWS/Google environment to enable ingestion of structured and unstructured data.
  • Write scripts or APIs to build data pipelines to automatically acquire different data sources and store them into the data lakes or warehouse on a daily or periodic basis.
  • Perform data cleansing, extraction and transformation. Execute data loading jobs from data lakes to data warehouse or data marts.
  • Model data marts for statistical analysis, visualisation and advanced analytics.
  • Make recommendations to improve data lineage for reliability, quality and security.
  • Administer cloud-based data infrastructure and tools on ingestion, modelling, visualisation and advanced analytics.
  • Administer security and infrastructure expansion for the cloud-based data warehouse.
  • Advise data scientists and users on the limitation and the realistic usage of the data sources.

Job Requirements

  • Degree in Engineering or Computer Science from a good, recognised university.
  • At least 5 years hands-on experience in developing and managing data warehouses.
  • Exposure to financial analytic projects from the buy-side industry is a plus.
  • Proficient in advanced SQL for ETL and data modelling is essential.
  • Good troubleshooting and coding skills in Python or equivalent for API and data processing.
  • Good knowledge of data security and governance is a plus.
  • Strong proficiency in English and Mandarin for effective presentation purposes.
  • Experience with big data management and streaming tools such as Hadoop, Spark, Kafka, Storm.
  • Experience with traditional relational SQL and NoSQL databases, including but not limited to Postgres and MySQL.
  • Columnar-DB experience such as BigQuery, and Schema-flexible DB experience such as MongoDB or Azure Table/Cosmos would be highly advantageous.

Application

We welcome your application.¬† Please send your CV to [email protected], stating your notice period and reasons for leaving your previous roles.¬† We regret that only shortlisted candidates would be notified.