DATA ENGINEER / ARCHITECT (Databricks & Pre Sales)
Jaipur / Gurgaon / Noida / Pune / Preferred office location | Posted: 2023-07-24 18:36:49
Pre- Sales experience is a must. • Hands on experience with distributed computing framework like DataBricks, Spark-Ecosystem (Spark Core, PySpark, Spark Streaming, SparkSQL) • Willing to work with product teams to best optimize product features/functions.
• Experience on Batch workloads and real time streaming with high volume frequency • Performance optimization on Spark workloads • Environment setup, user management, Authentication and cluster management on Databricks • Professional curiosity and the ability to enable yourself in new technologies and tasks. • Good understanding of SQL and a good grasp of relational and analytical database management theory and practice. Good To Have: • Hands on experience with distributed computing framework like DataBricks. • Experience with Databricks migration from On-premise to Cloud OR Cloud to Cloud • Migration of ETL workloads from Apache Spark implementations to Databricks • Experience on Databricks ML will be a plus • Migration from Spark 2.0 to Spark 3.0 • Certifications : ? Databricks Solution Architect Essentials badge ? Databricks Developer Essentials ? Apache Spark Programming with ? Lakehouse with Delta Lake Deep Dive ? Fundamentals of Unified with Databricks Key Skills: • Python, SQL and Pyspark • Big Ecosystem (Hadoop, Hive, Sqoop, HDFS, Hbase) • Spark Ecosystem (Spark Core, Spark Streaming, Spark SQL) / Databricks • Azure (ADF, ADB, Logic Apps, Azure SQL database, Azure Key Vaults, ADLS, Synapse) • AWS (Lambda,AWS Glue, S3, Redshift) • Modelling, ETL Methodology.