Location
Markham, ON | Canada
Job description
Data Engineer
Position Description
We are Canada's largest independent information technology services firm, and after 40 years, we're still growing! Innovation, technology, and service delivery are our focus. Our goal is to ensure our clients remain ahead of the competition. We provide a full spectrum of managed services from IT and business process outsourcing to systems integration and consulting that are transforming our clients’ operations and helping them to succeed.
We have an excellent opportunity for a Data Engineer to be part of a team that advocates the use of advanced Data technologies to solve business problems, deliver modern solutions while building close relationships with both client and CGI team members.
Your future duties and responsibilities
• Build, own and run data engineering pipelines, workflows and standardized practices for delivering new products and capabilities using Data and Cloud technologies, including data transformation, processing and analysis.
• Architect, develop, and maintain real-time data streaming solutions using Apache Kafka.
• Design and implement data flow management systems to ensure smooth and reliable data ingestion and processing.
• Develop and maintain schema registry for ensuring data schema compatibility and evolution.
• Optimize Kafka cluster performance.
• Integrate Kafka streams with Big Query and other data storage systems for analytics and reporting purposes.
• Implement data security measures to protect sensitive data during transit and storage.
• Work closely with the Data Engineering team to develop efficient ETL processes and ensure data quality.
• Troubleshoot production issues.
• Perform optimization and tuning of data workflow processes and applications.
• Love solving complex problems, be up-to-date on the latest technology trends and have a strong desire to learn constantly.
Required qualifications to be successful in this role
• Bachelors’ in computer science, Engineering Management Information Systems, or Computer Information Systems is required.
• Proficiency in Java and other programming languages used for data engineering purposes of at least 5+ years.
• Experience with setting up and managing Kafka clusters in production environments.
• Familiarity with schema registry and schema evolution concepts.
• Deep understanding of techniques used in creating and serving schemas at the time of data consumption.
• Hands-on experience with Google Big Query or other similar cloud-based data warehousing systems.
• Strong understanding of data flow management and ETL principles.
• Strong interpersonal and excellent written and verbal communication skills, combined with comprehensive, practical experience and knowledge in end-to-end delivery of Big Data solutions.
Preferred Qualifications:
• Familiarity with various data storage and processing technologies, such as relational databases, NoSQL, Hadoop.
• Proficient in programming languages and frameworks commonly used in data engineering, such as Python, SQL and Spark.
• Experience in AWS/Azure data platforms
• Familiarity with different development methodologies (e.g. waterfall, agile, XP, scrum)
• Understanding of data governance and data management practices.
• Experience working with multiple clients and projects at a time.
#LI-KM1
Insights you can act on While technology is at the heart of our clients’ digital transformation, we understand that people are at the heart of business success.
When you join CGI, you become a trusted advisor, collaborating with colleagues and clients to bring forward actionable insights that deliver meaningful and sustainable outcomes. We call our employees "members" because they are CGI shareholders and owners and owners who enjoy working and growing together to build a company we are proud of. This has been our Dream since 1976, and it has brought us to where we are today — one of the world’s largest independent providers of IT and business consulting services.
At CGI, we recognize the richness that diversity brings. We strive to create a work culture where all belong and collaborate with clients in building more inclusive communities. As an equal-opportunity employer, we want to empower all our members to succeed and grow. If you require an accommodation at any point during the recruitment process, please let us know. We will be happy to assist.
Ready to become part of our success story? Join CGI — where your ideas and actions make a difference.
Job tags
Salary
