Job Title     : Sr. Data Engineer  
Experience : 5-10 Years
Location     : Chennai, Hybrid     

Job Description :  
We are looking for a skilled and motivated Data Engineer to join our team. The ideal candidate will have experience with data processing, database management, and reporting. You will play a crucial role in building and maintaining scalable data pipelines, ensuring the availability and accessibility of data for analytical purposes. Strong communication skills and proficiency in SQL and Tableau are key to this role.

Responsibilities:

• Design, develop, and Maintain Robust ETL/ELT Pipelines using Informatica and PySpark.
• Automate data Workflows and orchestrate complex data pipelines using Apache Airflow.
• Migrate the Informatica ETL to PySpark.
• Collaborate closely with data analysts, scientists, and business stakeholders to transform raw data into actionable insights.
• Ensure high-quality code through best practices, including testing, version control, and CI/CD processes.
• Optimize data processes for performance, scalability, and reliability in a cloud or hybrid environment.
• Collaborate with cross-functional teams to understand data requirements and implement solutions.
• Write complex SQL queries to extract, clean, and transform data.
• Monitor and troubleshoot data issues and provide timely resolutions.
• Work closely with the data team to ensure data integrity and security.
• Document data processes, pipeline designs, and system architecture.

Must have:

• 7+ years of hands-on experience in data engineering roles.
• Strong Proficiency with Informatica for ETL development.
• Strong programming skills in Python, with Experience using PySpark for big data processing.
• Proven Experience with Airflow for Workflow orchestration.
• Familiarity with data warehousing concepts and cloud platforms (e.g., AWS, GCP, or Azure) is a plus.
• Excellent problem-solving skills, attention to detail, and a collaborative mindset.
• Strong communication skills to effectively interact with team members and stakeholders.
• Solid understanding of data engineering principles and data modeling.
• Ability to work independently and manage multiple tasks in a fast-paced environment.
• Attention to detail and a problem-solving mindset.
• Experience in building real-time/streaming data pipelines.
• Initiative and problem-solving skills when working independently.
• Good collaboration and communication skills, highly self-driven, and take ownership.
• Experience in writing well-documented, Clean, and Effective code is a must.

Good to Have:

• Experience with cloud-based data platforms (AWS/GCP/Azure).
• Good knowledge of Kubernetes experience.
• AWS Solution Architect certification.
• Familiarity with data warehousing concepts and tools.

Education:

• BE/B.Tech/MS/M.Tech/ME from a reputed institute.