Valiance Analytics
Location
Bangalore | India
Job description
We are seeking a skilled Big Data Engineer with 3+ years of experience to join our dynamic team. The ideal candidate will have expertise in Python, SQL, Hadoop, Hive, and Airflow, and a strong background in working with large datasets. As a Big Data Engineer, you will be responsible for designing, developing, and maintaining our big data infrastructure to ensure optimal performance and scalability. Key Responsibilities: Designing and implementing big data solutions using Hadoop, Hive, and other relevant technologies Developing data pipelines and ETL processes to extract, transform, and load data from various sources Optimizing and tuning big data systems for performance and efficiency Collaborating with data scientists and analysts to understand data requirements and deliver actionable insights Monitoring and troubleshooting data processing jobs to ensure data quality and reliability Staying up-to-date with the latest trends and technologies in big data and analytics Skills and Qualifications: Bachelor's degree in Computer Science, Engineering, or a related field 3+ years of experience in big data engineering roles Proficiency in Python and SQL for data manipulation and analysis Hands-on experience with Hadoop ecosystem tools such as HDFS, MapReduce, and Spark Strong knowledge of Hive for data warehousing and querying large datasets Experience with workflow management tools like Airflow for orchestrating data pipelines Familiarity with cloud platforms such as AWS or Azure is a plus Excellent problem-solving skills and attention to detail Strong communication and teamwork abilities
Job tags
Salary
