Skip to content

Join our Team

 

Data Engineer

Data Team/Full-time

Amsterdam, Netherlands

Energy Trading Platform Amsterdam
Etpa is a designated European Power Exchange for the short-term electricity market. Our state-of-the-art platform is a key enabler for the transition to renewable energy. We provide access to the spot-trading to many new players in the markets, including battery companies, Energy management systems, asset owners and utility companies. We outperform our competitors in terms of speed, cash management, scalability, cost and IT connectivity. Our current portfolio of exchanges includes Intraday (XBID), Congestion management (GOPACS) and day after trading (EX-Post). We are rapidly expanding our geography and products, introducing XBID to Germany, Austria and Belgium this year.

Job description
We are seeking a highly skilled and motivated Data Engineer to join our data team. As a Data Engineer, you will be responsible for designing, developing, and maintaining our data architecture and infrastructure. The ideal candidate should have a strong background in Python, RDBMS, Apache Spark, Apache Hadoop (or a Hadoop-based tool such as EMR), AWS Athena, and experience with tools like Dagster (or similar), Superset (or similar), Kubernetes, and a proven track record in building robust data pipelines, data modeling, and data integration solutions. Additionally, expertise in AWS, Git, infrastructure management, and a focus on optimizing and monitoring data pipelines for performance are essential for success in this role.


Responsibilities

- Design, develop, and maintain scalable and efficient data architecture and infrastructure.
- Implement and optimize data pipelines for the extraction, transformation, and loading (ETL) of data.
- Collaborate with cross-functional teams and stakeholders to understand data requirements and deliver effective solutions.
- Develop and implement data modeling strategies to support business analytics and reporting needs.
- Utilize Python for scripting and programming tasks related to data engineering.
- Implement and manage data security measures in compliance with industry standards.
- Utilize Apache Spark, Hadoop-based technologies (e.g., EMR), and AWS Athena for distributed data processing.
- Work with workflow management tools like Dagster for orchestrating complex data pipelines.
- Utilize visualization tools like Superset for data exploration and analysis.
- Deploy and manage data pipelines in the AWS cloud environment.
- Work with containerization and orchestration platforms like Kubernetes.
- Monitor and optimize data pipelines for performance and efficiency.
- Provide expertise in infrastructure management related to data engineering processes.
- Version control using Git for codebase management.
- Stay updated on industry trends and technologies, contributing to continuous learning within the team.


Qualifications

- Bachelor's degree in Computer Science, Information Technology, or a related field.
- Proven experience as a Data Engineer or similar role.
- Strong programming skills, particularly in Python.
- Proficiency in relational databases (e.g., MySQL, PostgreSQL) and experience with data modeling.
- Hands-on experience with Apache Spark, Hadoop-based technologies (e.g., EMR), and AWS Athena.
- Expertise in designing and implementing data integration solutions.
- Experience with AWS cloud services.
- Experience with workflow management tools like Dagster or similar.
- Experience with visualization tools like Superset or similar.
- Experience with containerization and orchestration platforms like Kubernetes.
- Ability to optimize and monitor data pipelines for performance.
- Understanding of data security and compliance requirements.
- Familiarity with Git for version control.
- Knowledge of data warehousing concepts and experience with solutions like AWS Redshift is a plus.
- Strong collaboration and communication skills.
- Continuous learning mindset and adaptability to emerging technologies.

If you are passionate about leveraging data to drive business success, we invite you to apply for this exciting opportunity. Join our dynamic team and contribute to the advancement of our data engineering capabilities.

If you would like more information, you can contact us via the contact form or by emailing recruitment@etpa.nl 

Important: Aquisition based on the job opening is not appreciated.