Relic Consultancy Services
Location
Coimbatore | India
Job description
We are seeking an experienced Data Engineer with expertise in both AWS and Azure cloud platforms. As a key member of our team, you will be responsible for designing, building, and maintaining scalable data solutions to support our organisation's data-driven initiatives. You will work closely with cross-functional teams to understand business requirements, architect data solutions, and implement best practices for data management, ETL (Extract, Transform, Load), data warehousing, and analytics.
Contract: 6 Months (Fulltime - Onsite) Key Responsibilities Cloud Platform Expertise: Utilise your deep understanding of both AWS and Azure cloud platforms to architect, implement, and optimise data solutions that leverage the strengths of each platform. Data Architecture Design: Design and implement robust data architectures that span multiple cloud environments, ensuring scalability, reliability, and performance. ETL Development: Develop, test, and optimise ETL processes to extract data from various sources, transform it according to business requirements, and load it into target systems, such as data warehouses or data lakes. Data Integration: Integrate data from disparate sources and systems across AWS and Azure environments to provide a unified view of data for analytics, reporting, and other business needs. Big Data Technologies: Leverage big data technologies and frameworks, such as Apache Spark, Hadoop, Kafka, etc., to process and analyze large volumes of data efficiently. Data Governance and Security: Implement data governance policies, security measures, and compliance controls to protect sensitive data and ensure regulatory compliance across AWS and Azure environments. Performance Optimization: Monitor and optimize data pipelines, queries, and processes to improve performance, reduce latency, and maximize throughput. Data Quality Assurance: Implement data quality checks, validation rules, and monitoring processes to ensure the accuracy, completeness, and consistency of data across different systems and stages of the data lifecycle. Collaboration and Communication: Collaborate with business stakeholders, data scientists, analysts, and other IT teams to understand requirements, provide technical guidance, and deliver solutions that meet business objectives. Documentation and Best Practices: Document data architectures, design decisions, and best practices for data engineering on AWS and Azure, and provide training and support to other team members to promote knowledge sharing and improve data engineering capabilities across the organization. QualificationsJob tags
Salary
