HR Aspiration
Location
Provincia di Reggio nell'Emilia | Italy
Job description
Job Title: Logstash Developer
Job Description:
Role Overview:
As a Logstash Developer you will play a pivotal role in the data processing pipeline by designing developing and maintaining Logstash configurations to extract transform and enrich data from diverse sources. Your expertise in Logstash and data parsing will be critical in ensuring that the organization can efficiently ingest and analyse data for various purposes. Key Responsibilities:
1. Logstash Configuration: Develop and maintain Logstash configuration files (Logstash pipelines) to ingest data from sources like log files message queues databases and APIs.
2. Data Parsing: Create Logstash filter configurations to parse raw data into structured formats using techniques such as grok patterns regular expressions and custom filters.
3. Data Transformation: Implement data transformation logic within Logstash to ensure that data is standardized and suitable for downstream processing and analysis.
4. Data Enrichment: Enrich parsed data by adding contextual information geolocation data or other relevant metadata to enhance its value for analytics and reporting.
5. Error Handling: Implement errorhandling mechanisms to identify and address issues related to data ingestion parsing and transformation.
6. Performance Optimization: Continuously optimize Logstash configurations for resource utilization and data processing speed considering scalability and efficiency.
7. Monitoring and Troubleshooting: Monitor Logstash pipelines set up alerts and troubleshoot any performance or data quality issues that may arise.
8. Documentation: Maintain comprehensive and uptodate documentation for Logstash configurations filter patterns and data processing workflows.
9. Collaboration: Collaborate closely with crossfunctional teams including DevOps SysOps Data Engineers and Analysts to ensure seamless data integration and delivery.
10.Security: Implement security best practices to protect sensitive data during ingestion and processing. Requirements:
Proven experience as a Logstash Developer or similar role with a focus on Logstash configuration and data parsing.
Expertise in Logstash filter configuration including grok patterns regex and custom filters.
Strong knowledge of data parsing transformation and enrichment techniques.
Familiarity with Opensearch/Elasticsearch or other data storage and analytics platforms.
Proficiency in scripting and automation for Logstash pipeline management.
Strong analytical and problemsolving skills.
Effective communication skills and the ability to work collaboratively in a team environment.
Knowledge of data security best practices.
Experience with monitoring and troubleshooting Logstash pipelines is a plus.
logstash,pipelines,elasticsearch
Job tags
Salary
