Location
McLean, VA | United States
Job description
Title: Spark Developer with ETL
Location: Mclean VA )D Onsite.
Contract role
Job Description:
Must be strong in Spark/Bigdata ETL and Pyspark and Python
- Your expertise in Python PySpark ETL processes CI/CD (Jenkins or GitHub) and experience with both streaming and batch workflows will be essential in ensuring the efficient flow and processing of data to support our clients.
- Collaborate with crossfunctional teams to understand data requirements and design robust data architecture solutions
- Implement ETL processes to extract transform and load data from various sources
- Ensure data quality integrity and consistency throughout the ETL pipeline
- Utilize your expertise in Python and PySpark to develop efficient data processing and analysis scripts
- Optimize code for performance and scalability keeping uptodate with the latest industry best practices
- Integrate data from different systems and sources to provide a unified view for analytical purposes
- Collaborate with data analysts to implement solutions that meet their data integration needs
- Design and implement streaming workflows using PySpark Streaming or other relevant technologies
- Develop batch processing workflows for largescale data processing and analysis
Job tags
Salary
