Scala/Spark Data Engineer
Budapest
Állandó 3-5 years of professional experience Hybrid
Our partner is a European airline company and they are looking for an agile & objective minded experienced developer with a pro-active mindset that is eager to learn new things. Responsibilities:
- Design, develop and maintain data pipelines and workflows to handle large volumes of data
- Ensure data quality, reliability and scalability
- Optimize query performance
- Develop and maintain data models and schema definitions
- Collaborate with cross-functional teams to identify and prioritize data requirements
- Develop and maintain ETL processes
- Manage and monitor data processing and storage systems
- Troubleshoot and debug data-related issues
Requirements:
- At least 3 years experience with data engineering of batch-oriented data pipelines
- At least 2 years experience with Scala (experience with Spark is a plus)
- Proficiency in SQL and database technologies
- Knowledge of big data processing frameworks such as Spark
- Familiarity with distributed systems and cloud computing environments
- Familiarity with scheduling tools such as Airflow
- Familiarity with Java
- Familiarity with Azure or GCP or AWS (and/or CI/CD: GitHub, GitHub Actions)
- Ability to work in a Distributed Agile/Scrum team with some team members in AMS and some team members in BUD
- Engineering mindset and analytical thinking
- Proficiency in English, both written and spoken
Bonus Skills:
- Knowledge of machine learning techniques and frameworks
- Familiarity with agile development methodologies
