Ideal Candidate

• The ideal candidate will come with hands-on experience as a Hadoop Administrator preferably from the Banking and Financial services domain. As a Big Data Engineer , you will be a member of a small , agile team of data engineers responsible for developing/maintaining an innovative big data platform as a service for business units that need to manage mission critical data and diverse application stakeholders at scale. The platform manages data ingestion, warehousing , and governance , and enables developers to quickly create complex queries. The platform provides automatic scaling and elasticity to enable efficient use of resources, and provides services such as security , logging , and data provenance.

Requirements

• 5-7 years experience as Cloudera Hadoop Administrator preferably  in a large financial institution
• Manage the tech stack (F1 platform), upgrades, deployments, and support new tenants on the platform
• Experience in Big data stack (Cloudera CDH 5.x, Hadoop, Kafka, Impala, Hive, Docker, Kubernetes)
• Experience managing Cloudera clusters and CI/CD (Jenkins)
•  Knowledge of SQL and shell scripting
• Programming knowledge in Java / Python a huge plus
• Certified as Cloudera and Spark administration a huge plus
• Great at debugging, troubleshooting, designing, and implementing solutions to complex technical issues
• Exposure to the big data tools to be able to perform investigation, root cause analysis and providing a solution
• An understanding of agile development practices and common tools, such as Github, Jira, etc.
• Aptitude to independently learn new technologies, prototype, and propose solutions
• Demonstrated cloud-native focus with relevant cloud experience, preferably using Google Cloud Platform

Responsibilities

• Hadoop command line administration, including use of logs to debug job and systems issues related to Spark, MapReduce, Tez, Kafka, Hive etc.
• Security implementations and checks with Apache Knox, Apache Ranger, Sentry & Kerberos
• Able to help and Guide L1/L2 support engineers to fix day-to-day Operational issues
• Develop bash shell or python scripts, LINUX utilities & LINUX Commands
• Working with data delivery teams to setup new Hadoop users. This job includes setting up Linux users, setting up Kerberos principals and testing HDFS, Hive, Pig, Spark and MapReduce access for the new users.
• Cluster maintenance as well as creation and removal of nodes using tools like Ganglia, Nagios, Cloudera Manager Enterprise and other tools.
• Participates in enterprise strategy development, including environmental analysis, opportunity identification, value cases and business innovation portfolio development