Location
Folsom, CA | United States
Job description
Software Data Engineer
Research, design, and develop computer and network software or specialized utility programs. Analyze user needs and develop software solutions, applying principles and techniques of computer science, engineering, and mathematical analysis. Update software or enhance existing software capabilities. May work with computer hardware engineers to integrate hardware and software systems and develop specifications and performance requirements. May maintain databases within an application area, working individually or coordinating database development as part of a team.
- Design and implement scalable and reliable data architectures on Google Cloud Platform.
- Create and maintain data models, schemas, and structures for efficient storage and retrieval of data.
- Develop data pipelines to ingest, process, and transform data from various sources into usable formats.
- Implement ETL processes to move and integrate data across different systems and platforms.
- Work with big data technologies such as Apache Hadoop, Apache Spark, and Google Cloud's BigQuery to process and analyze large datasets.
- Design and optimize data processing workflows for performance and cost efficiency.
- Implementing and managing data warehouses for storing and retrieving structured and semi-structured data.
- Optimize queries and ensure data consistency and accuracy within the data warehouse.
- Ensure data quality, integrity, and security throughout the data lifecycle.
- Implement access controls, encryption, and other security measures to protect sensitive data.
- Monitor data pipelines and systems to ensure they meet performance and reliability standards.
- Identify and resolve issues related to data processing, ingestion, and storage.
- Collaborate with cross-functional teams, including data scientists, analysts, and business stakeholders, to understand data requirements and deliver solutions. Communicate technical concepts and solutions to non-technical stakeholders.
- Implement automation for routine tasks to improve efficiency and reduce manual efforts.
- Continuously optimize data processes and infrastructure for performance, cost, and scalability.
- Create and maintain documentation for data engineering processes, workflows, and system configurations.
Education: Bachelor s Degree in Engineering, Science, Math, Computer Science, Business, Electrical, Electronics or equivalent.
Report this job
- Dice Id: 10200467
- Position Id: TNSDE24
Job tags
Salary
