Data Engineer

Design, develop and implement data pipelines to ingest, store, and publish customer data

• Orchestrate and schedule data pipeline jobs to optimize run times and reduce compute

utilization threshholds

• Significant participation in analysis, interpretation, and translation of complex health plan

data, issues, trends and relationships to provide our customers detailed insights into their data

• Analyze and interpret complex data on source and target systems, identify the gaps, and

provide solutions

• Ability to development full end-to-end data pipelines that consist of complex SQL queries,

stored procedures, triggers, and detailed logging

• Optimizes SSIS package execution and SQL execution to minimize load times.

• Ensures data integrity throughout the ETL process and appropriately handle errors

• Work closely with Scrum team members using Agile processes to iteratively develop and

improve ETL pipelines.

• Proactively provide feedback and process improvement recommendations to the team and

management

• Participate in cloud migration development initiatives to move data and technologies to

Azure

• Integrate with and implement detailed logging and status alerts to improve troubleshooting

and investigation of data pipelines

Skills Full mastery of entire MSBI platform stack (SSIS, SSRS, SSAS, T-SQL, MSSQL Server)

Demonstrate knowledge of ETL/ELT technologies (Python, Scala, dbt)

Demonstrate knowledge of cloud orchestration tools (Airflow, ADF)

Demonstrate knowledge of cloud data visualization and report technologies (PowerBI, Tableau,

Looker)

Demonstrate knowledge of cloud compute technologies (Spark, Databricks, Snowpark)

Demonstrate knowledge of federated MPP sql query engine like Trino or Starburst

Posted Date
2022-12-12 15:18:24
Experience
3 -6 years
Primary Skills
Python, Scala, dbt, Spark, Databricks, Snowpark, Airflow, ADF, SSIS, SSRS, SSAS, T-SQL, MSSQL Server
Required Documents
Resume
Contact
bhawya@lorventech.com,diana@lorventech.com,hema@lorventech.com,ashwiny@lorventech.com,latha@lorventech.com,sam@lorventech.com,fathih@lorventech.com,santhana@lorventech.com
Bootstrap Example