***** Direct Client Requirement ****
Title: ETL Databricks with Snowflask
Location: NYC, NY
Duration: Long term
Interview Type: Skype and Phone
Work Status: Successful applicants must be legally authorized to work in the U.S.
Job Type: C2C, C2H ,W2
Experience: 8+ YEARS
The ideal candidate is skilled in data wrangling and comfortable implementing emerging technologies.
Overall experience of 8+ years
Experience in Azure Cloud Platform. (ADLS, Blob).
Excellent understanding of big data file formats like JSON, Avro, Parquet and other prevalent big data file formats
Experience developing big data applications using Spark, HDFS, etc.
Experience in Azure Databricks platform.
Expertise in Python, PySpark, Spark
Demonstrates up-to-date expertise in Data Engineering, complex data pipeline development
Design, develop, implement and tune large-scale distributed systems and pipelines that process large volume of data; focusing on scalability, low-latency, and fault-tolerance in every system built.
Experience with Java, Python, Hive and Spark to write data pipelines and data processing layers
Demonstrates expertise in writing complex, highly-optimized SQL queries across large data sets
Experience with Cloud Technologies (Azure)
Provides and supports the implementation and operations of streaming and batch data pipelines and analytical solutions
Performance tuning experience of systems working with large data sets
Experience with Data Governance ( Data Quality, Metadata Management, Security, etc.)
Good to have development experience on RESTful API data service
Position Keywords: Python , PySpark , Azure Databricks , implement , RESTful , Cloud Technologies , analytical solutions , JSON , SQL queries , big data , Java , Hive , HDFS , Azure Cloud Platform , Spark , data processing , Data Governance , Blob , ADLS
Pay Rate: DOE
Job Duration: 12 Months
% Travel Required: None
Job Posted by: Consulting Services
Job ID: TM343
Work Authorization: Successful applicants must be legally authorized to work in the U.S