Relic Consultancy Services
Location
Hyderabad | India
Job description
We are seeking an experienced Data Engineer with expertise in both AWS and Azure cloud platforms. As a key member of our team you will be responsible for designing building and maintaining scalable data solutions to support our organisations datadriven initiatives. You will work closely with crossfunctional teams to understand business requirements architect data solutions and implement best practices for data management ETL (Extract Transform Load) data warehousing and analytics.
Contract: 6 Months (Fulltime Onsite)
Key Responsibilities:
Cloud Platform Expertise: Utilise your deep understanding of both AWS and Azure cloud platforms to architect implement and optimise data solutions that leverage the strengths of each platform.
Data Architecture Design: Design and implement robust data architectures that span multiple cloud environments ensuring scalability reliability and performance.
ETL Development: Develop test and optimise ETL processes to extract data from various sources transform it according to business requirements and load it into target systems such as data warehouses or data lakes.
Data Integration: Integrate data from disparate sources and systems across AWS and Azure environments to provide a unified view of data for analytics reporting and other business needs.
Big Data Technologies: Leverage big data technologies and frameworks such as Apache Spark Hadoop Kafka etc. to process and analyze large volumes of data efficiently.
Data Governance and Security: Implement data governance policies security measures and compliance controls to protect sensitive data and ensure regulatory compliance across AWS and Azure environments.
Performance Optimization: Monitor and optimize data pipelines queries and processes to improve performance reduce latency and maximize throughput.
Data Quality Assurance: Implement data quality checks validation rules and monitoring processes to ensure the accuracy completeness and consistency of data across different systems and stages of the data lifecycle.
Collaboration and Communication: Collaborate with business stakeholders data scientists analysts and other IT teams to understand requirements provide technical guidance and deliver solutions that meet business objectives.
Documentation and Best Practices: Document data architectures design decisions and best practices for data engineering on AWS and Azure and provide training and support to other team members to promote knowledge sharing and improve data engineering capabilities across the organization.
Qualifications:
apache spark,data modeling,sql,python,s3,ec2,amazon redshift,glue,athena,electronic medical record (emr),azure,aws,nosql,hadoop,kafka,data integration,etl,data architecture,big data technologies,data quality assurance,data governance,web performance optimization,big data,cloud,data engineering,analytics
Job tags
Salary
