Location
New York State | United States
Job description
Job Title: Senior Data Engineer
Location: Remote
Job Description:
We are seeking a highly skilled Senior Data Engineer to join our remote team. As a Senior Data Engineer, you will be responsible for designing, building, and maintaining scalable data pipelines and analytics solutions. This is a remote position, offering the flexibility to work from anywhere.
Responsibilities:
- Design and implement robust data pipelines using Apache Airflow for orchestration.
- Develop data processing jobs using Apache Spark for large-scale data analysis and transformation.
- Utilize BigQuery for data warehousing and querying large datasets.
- Collaborate with cross-functional teams to gather requirements and design data solutions that meet business needs.
- Implement and manage infrastructure on Google Cloud Platform (GCP) using Terraform.
- Write clean, efficient, and maintainable code in Scala.
- Optimize data pipelines for performance and scalability.
- Monitor pipeline health and troubleshoot issues as they arise.
- Stay updated on industry best practices and emerging technologies in data engineering.
Requirements:
- Bachelor's degree in Computer Science, Engineering, or a related field.
- Proven experience as a Data Engineer or similar role.
- Strong expertise in Google Cloud Platform (GCP) services and infrastructure.
- Experience in designing and implementing data pipelines for e-commerce platforms.
- Proficiency in Apache Spark using Scala for data processing and analysis.
- Hands-on experience with Apache Airflow for workflow orchestration.
- Familiarity with BigQuery for data warehousing and analytics.
- Experience with infrastructure as code using Terraform.
- Excellent problem-solving and analytical skills.
- Strong communication and collaboration skills.
- Ability to work independently and efficiently in a remote environment.
Required Skills:
- Strong experience with Google Cloud Platform (GCP)
- Experience in e-commerce data pipelines
- Proficiency in Apache Spark using Scala
Must Have:
- Apache Airflow
- Apache Spark
- BigQuery
- GCP/Google Cloud/Terraform
- Scala
Job tags
Salary
