Location
Pune | India
Job description
Position : Cloud Data Architect
Purpose of the Position:
You will be a critical member of the InfoCepts Cloud Data Architect Team. This position requires a sound knowledge on Big Data using Cloud technologies, EMR, Athena, Spark-Scala, PySpark, S3, AWS Lambda etc. Strong foundation on database concepts and SQL is also required.
Location: Pune/Nagpur
Key Result Areas and Activities:
- Study existing technology landscape and understand current data integration framework and do impact assessment for the requirements
- Should be able to design complex Big Data use cases using AWS Services
- Should be able to reason about various architecture choices in terms of: Cost, Performance and Durability
- Should be able to suggest optimizations in existing designs
- Ensure optimal balance between cost and performance
- Project documentation, Adheres to Quality guidelines & Schedules
- Works hand in hand with PM for successful delivery of project and provide Estimation, scoping, scheduling assistance
- Effectively communicate architectural decisions to stakeholders
- Articulate design decisions very clearly with stakeholders
- Perform Proof-of-Concepts and document all observations before proposing a new solution.
- Conduct design review sessions with other teams and identify scope of improvement.
- Suggests automation to improve existing processes
- Assist other Data Engineers by providing expert advice or troubleshooting steps whenever required
- Keep suggesting new ways to improve team productivity
- Create technology focused training plans whenever required
- Deliver technology focused training sessions with team members whenever required
- Conduct Expert Knowledge Sharing sessions with Client Stakeholders whenever required
- Assist in RFPs for other Accounts whenever required
- Assist in designing case study documents whenever required
Work and Technical Experience:
Essential Skills:
- Overall, 9+ years of IT Exp
- 5+ years of AWS related project
- In depth knowledge of following AWS Services required: S3, EC2, EMR, Athena, AWS Glue, Lambda.
- Cloud Databases (Good to have) – AWS Aurora
- MPP Databases (anyone) – AWS Redshift, Snowflake, SingleStore
- Big Data (Must have) – Apache Spark
- Programming Language (Must have) – Scala, Python
- Big Data Table formats (Must have at least one) – Delta Lake (open source), Apache Iceberg
- Operating System (Must have) - Any flavor of Linux, Windows
- Must have very strong SQL Skills
- Orchestration Tools (Must have): Apache Airflow
- Expertise in developing ETL workflows comprising complex transformations like SCD, deduplications, aggregations etc.
- Should have thorough conceptual understanding of AWS VPC, Subnets, Security Groups, & Route Tables
- Should be a quick and self-learner and be ready to adapt to new AWS Services or new Big Data Technologies as and when required.
- Should know Data Fabric and Data Mesh concepts.
- Should know Data Warehousing concepts very well
Desirable Skills:
- Good to have cleared Associate Level or Professional Level AWS Certification
- Big Data (Good to Have) – Hadoop, Hive
- Programming Language (Good to have) – Java, Bash
- Big Data Platform (Good to have) – Databricks
Qualifications:
- Bachelor's degree in computer science, engineering, or related field (Master's degree is a plus)
- Demonstrated continued learning through one or more technical certifications or related methods.
- At least 10 years of IT experience; Minimum 5 years of experience on Cloud related projects
Qualities:
- Hold strong technical knowledge and experience
- Should have the capability to deep dive and research in various technical related fields
- Self-motivated and focused on delivering outcomes for a fast growing team and firm
- Able to communicate persuasively through speaking, writing, and client presentations
- Able to consult, write, and present persuasively
- Able to work in a self-organized and cross-functional team
- Able to iterate based on new information, peer reviews, and feedback
- Prior experience of working in a large media company would be added advantage
Job tags
Salary
