Scala/Spark Data Engineer

Rövid leírás

Our partner is a European airline company and they are looking for an agile & objective minded experienced developer with a pro-active mindset that is eager to learn new things. 

Leírás

• Design, develop and maintain data pipelines and workflows to handle large volumes of data
• Ensure data quality, reliability and scalability
• Optimize query performance
• Develop and maintain data models and schema definitions
• Collaborate with cross-functional teams to identify and prioritize data requirements
• Develop and maintain ETL processes
• Manage and monitor data processing and storage systems
• Troubleshoot and debug data-related issues

Követelmények

• At least 3 years experience with data engineering of batch-oriented data pipelines
• At least 2 years experience with Scala (experience with Spark is a plus)
• Proficiency in SQL and database technologies
• Knowledge of big data processing frameworks such as Spark
• Familiarity with distributed systems and cloud computing environments
• Familiarity with scheduling tools such as Airflow
• Familiarity with Java
• Familiarity with Azure or GCP or AWS (and/or CI/CD: GitHub, GitHub Actions)
• Ability to work in a Distributed Agile/Scrum team with some team members in AMS and some team members in BUD
• Engineering mindset and analytical thinking
• Proficiency in English, both written and spoken