DATA ENGINEER / ARCHITECT (Databricks & Pre Sales)

Pre- Sales experience is a must. • Hands on experience with distributed computing framework like DataBricks, Spark-Ecosystem (Spark Core, PySpark, Spark Streaming, SparkSQL) • Willing to work with product teams to best optimize product features/functions.

• Experience on Batch workloads and real time streaming with high volume  frequency • Performance optimization on Spark workloads • Environment setup, user management, Authentication and cluster management on Databricks • Professional curiosity and the ability to enable yourself in new technologies and tasks. • Good understanding of SQL and a good grasp of relational and analytical database management theory and practice. Good To Have: • Hands on experience with distributed computing framework like DataBricks. • Experience with Databricks migration from On-premise to Cloud OR Cloud to Cloud • Migration of ETL workloads from Apache Spark implementations to Databricks • Experience on Databricks ML will be a plus • Migration from Spark 2.0 to Spark 3.0 • Certifications : ? Databricks Solution Architect Essentials badge ? Databricks Developer Essentials ? Apache Spark Programming with ? Lakehouse with Delta Lake Deep Dive ? Fundamentals of Unified with Databricks Key Skills: • Python, SQL and Pyspark • Big Ecosystem (Hadoop, Hive, Sqoop, HDFS, Hbase) • Spark Ecosystem (Spark Core, Spark Streaming, Spark SQL) / Databricks • Azure (ADF, ADB, Logic Apps, Azure SQL database, Azure Key Vaults, ADLS, Synapse) • AWS (Lambda,AWS Glue, S3, Redshift) • Modelling, ETL Methodology.

Posted Date
2023-07-24 18:36:49
Experience
6 -12 years
Primary Skills
Spark structured streaming on Databricks
Required Documents
Resume
Contact
bhawya@lorventech.com,diana@lorventech.com,ashwiny@lorventech.com,geetha@lorventech.com,fathih@lorventech.com,ganeshk@lorventech.com,sarga@lorventech.com,naresh@lorventech.com,jeevitha@lorventech.com
Bootstrap Example