We are looking for a Data Engineer to join our Clients data platform team and contribute to the development of high-performance, real-time data pipelines and unified analytics infrastructure. The ideal candidate has hands-on experience in data ingestion, transformation, and modeling from complex source systems such as ERPs, CRMs, and cloud platforms—without relying on traditional ETL frameworks.
Key Responsibilities:
- Design, develop, and maintain scalable data pipelines for direct ingestion from enterprise source systems (ERP, CRM, databases, APIs).
- Preserve raw, transaction-level data while enabling real-time access and analytics.
- Leverage in-memory processing and distributed computing (e.g., Apache Spark) to optimize query performance.
- Collaborate with analytics and BI teams to define physical and business data models for downstream consumption.
- Support real-time and incremental data refreshes using lakehouse or modern data architecture principles.
- Implement data access controls including role-based security and row-level permissions.
- Ensure data quality and governance through automation, auditing, and monitoring.
- Integrate the unified analytics platform with external BI tools (Power BI, Tableau, Excel, etc.).
- Contribute to the platform's scalability, reliability, and performance tuning.
Required Qualifications:
- Bachelor’s or Master’s degree in Computer Science, Information Systems, Engineering, or a related field.
- 3–5+ years of experience in data engineering or backend data infrastructure.
- Proficiency in SQL, Python, or Scala for data manipulation and scripting.
- Experience with distributed data processing frameworks like Apache Spark.
- Knowledge of cloud data services (Azure, AWS, or GCP) and storage (e.g., Data Lake, S3, ADLS).
- Understanding of data modeling, real-time ingestion, and schema management.
- Familiarity with business systems like Oracle, SAP, or Salesforce is a plus.
Preferred Skills:
- Experience working in a zero-ETL or direct data mapping environment.
- Exposure to lakehouse architecture and columnar data stores.
- Familiarity with modern data pipeline orchestration tools (Airflow, dbt, etc.).
- Experience integrating with BI tools and building consumable data products.
Job Type: Full-time