The Senior Data Engineer is responsible for preparing data for modeling and predictive analytics. This includes integrating new data sources, performing pre-processing operations such as data cleansing, feature engineering, statistical analysis, and sampling, as well as managing a wide range of ETL-related tasks. The ideal candidate should demonstrate strong software development practices and a continuous learning mindset. Proficiency in Python, machine learning, data storage solutions, and ETL pipeline development is essential.
Job Responsibilities:
- Design and implement scalable data pipelines using SQL, Airflow, Python, Alteryx, and cloud-based technologies
- Lead the integration of data sources and design data models to meet complex business needs
- Ensure data quality by identifying issues and collaborating with source owners to implement preventive measures
- Deploy and optimize machine learning models, statistical methods, and analytics programs
- Research new data acquisition methods and innovative uses of existing data to drive business value
- Integrate advanced data management tools and software engineering practices to enhance system performance
- Build and maintain high-quality datasets to support decision-making and data-driven products
- Develop custom software components and analytics applications aligned with business requirements
- Implement strategies to improve data reliability, efficiency, and scalability
- Drive innovation through R&D initiatives within the data engineering team
- Collaborate cross-functionally to build a flexible, scalable, and high-performing data platform
- Maintain clear and comprehensive technical documentation for ongoing projects
Requirements
- Proficient in Python, with hands-on experience in data analysis and automation
- Skilled in using Dataiku for advanced data workflows and machine learning applications
- Familiarity with cloud services like AWS or Azure.
- Strong background in data visualization and statistical analysis to drive data-driven insights
- Practical experience with machine learning models and managing large-scale (big data) projects
- Solid understanding of programming concepts and data architecture principles
- Capable of defining and tracking meaningful metrics for data and analytics initiatives
- Excellent analytical skills with a balance of quantitative rigor and qualitative insight
- Highly proficient in Microsoft Office and Google Suite for documentation and collaboration
- Bachelor’s and/or Master’s degree in Computer Science or a related technical field
- Agile, innovative, and solutions-oriented thinker with a strong sense of ownership
- Collaborative and dependable team player, effective in multicultural and diverse environments