Descripción de la oferta
Senior Data EngineerAre you a forward-thinking professional with a strong background in Data Engineering and an interest in financial services? Join as a Senior Data Engineer and help build and evolve enterprise data ecosystems for customers.Design scalable data pipelines, ensure high-quality data flows and integrate complex APIs while collaborating with cross-functional teams to deliver robust and reliable data products.This is a hybrid role based in Madrids city center, ideal for those eager to thrive in a dynamic environment and make a significant impact in private banking technology.Join EPAM and contribute to shaping the future of financial services in Spain!ResponsibilitiesDesign, build and optimize scalable Python-based ETL/ELT pipelines using frameworks such as pandas or PolarsOrchestrate data pipelines using Dagster, Airflow or similar toolsDevelop efficient data ingestion pipelines for batch, incremental and streaming scenariosIntegrate with internal and external APIs ensuring robust authentication, error handling and data qualityImplement best-in-class data models including dimensional modelling and domain-driven structuresManage application lifecycle and reduce architecture debtDeploy, operate and monitor data pipelines using Docker, Kubernetes and GitLabDevelop and maintain CI/CD pipelines for data workflows and infrastructure componentsCollaborate with data scientists, software engineers and platform teams to enhance data services and deployment processesSupport troubleshooting, incident response and participate in architecture discussionsRequirementsBachelors or Masters degree in Computer Science, Engineering, Data Science or related fieldMinimum 5 years of experience in data engineering or backend engineering with a strong data focusExpert-level Python skills with frameworks such as pandas or PolarsExperience with data orchestration frameworks (Dagster, Airflow or similar)Strong understanding of data modelling, ETL patterns and performance considerationsProficiency with Docker, Kubernetes and GitLab CI/CDHands-on experience with SQL and relational databases (MS SQL preferred)Practical experience with Azure data and compute services or comparable AWS/GCP experiencePassion for building robust, maintainable and well-tested data systemsStrong focus on scalability, reliability, data quality and monitoringNice to haveExperience in regulated environments such as finance, insurance or healthcareCertifications in cloud, Kubernetes or DevOps (CKA, AWS/GCP/Azure)Familiarity with distributed systems, microservices and API-driven architecturesCommitment to automation, reproducibility and DevOps practicesCreativity and new ideas to enhance the platformAbility to work under pressure and tough deadlinesStrong communication skills and product-thinking mindsetComfortable working in agile environmentsFluency in SpanishPython, pandas, Polars, Dagster, Airflow, Docker, ETL
