Location
Deerfield, IL | United States
Job description
Role: Data Engineer
Location: Deerfield IL (Onsite)
Duration: Long Term
Contract Role
Must Have:
- Spark/Scala Data Engineer Microsoft Azure Azure Databricks Spark Scala Delta Lake
Role Description:
- Work in Agile development projects for batch and real time data ingestion transformation.
- Work closely with product owners understand requirements lead offshore team on Development and QE of data pipelines.
- Work with upstream/downstream teams to resolve any issues/blockers.
- Work with System Integration/Performance engineering team for integration/performance testing of data pipelines.
- Coordinate with DevOps/RunOps teams for deployment in higher environments.
Competencies:
- Digital : Apache Spark Digital : Scala Digital : Databricks.
Experience (Years):
68
Essential Skills:
- Minimum 46 years of experience in Data Warehousing and Analytics
- Minimum 34 years of experience on Spark with Scala / Azure Databricks
- Hands on experience of building data pipelines using Apache Spark (Azure Databricks) Scala
- Hands on experience working on Databricks batch.
- Hands on experience in Azure Data Factory.
- Hands on experience in designing and delivering solutions using the Azure Data Analytics platform including Azure Storage Azure SQL Data Warehouse Azure Data Lake.
- Experience working on Delta Lakes.
- Experience working in a Dev/Ops environment with tools such as Azure DevOps
- Should be able to work independently on assigned projects with offshore team.
Desirable Skills:
- Knowledge of Talend Tool.
- Exposure to Azure PaaS services is highly desirable.
- Problemsolving and good analytical skills
- Communication teamwork and negotiation skills
Job tags
Salary
