Data Engineer

As a Data Engineer, you will implement data quality controls, design and manage data architectures and automate workflows. Your proficiency in Python, PySpark, and Azure cloud platforms (especially Databricks and Delta Live Tables) will be crucial for building and maintaining robust data pipelines. You'll also be responsible for data quality checks, data modeling, CI/CD implementation, and monitoring, all while adhering to Agile principles. Your proactive, team-oriented approach, along with strong communication skills, will contribute to your success in this fully remote role, a contract for an indefinite time, with benefits statutory and beyond.

Functions

• Implementing data quality controls
• Design and management of data architectures
• Workflow automation and orchestration
• Collaborative work and use of agile methodologies

Requirements

• English Conversational Advanced (C1-C2)
• Have a minimum of 3 years of experience as a Data Engineer
• Programming Languages: Proficient in Python and PySpark with a strong understanding software engineering best practices
• Cloud Computing: Utilize Azure cloud-based data platforms specifically leveraging Databricks and Delta Live Tables for data engineering tasks while effectively utilizing services related to storage compute and security.
• Data Pipelines: Design build and maintain robust and scalable and automated data pipelines for batch and streaming data ingestion of data and processing.
• Data Quality: Implement data quality checks and balances throughout the data pipeline including profiling validation and root cause analysis to ensure data accuracy completeness and consistency.
• Data Architecture and Modeling: Design and implement robust data models and architectures that align with business requirements and support efficient data processing analysis and reporting.
• Orchestration: Utilize workflow orchestration tools to automate data pipeline execution and dependency management.
• CI/CD: Implement continuous integration and continuous delivery (CI/CD) practices for automated testing and deployment of data pipelines.
• Monitoring and Alerting: Integrate monitoring and alerting mechanisms to track pipeline health identify performance bottlenecks and proactively address issues.
• Strong Agile principles: Utilize Agile development methodologies actively participating in sprint planning daily stand-ups sprint reviews and retrospectives. Be flexible and adaptable to changing requirements and priorities throughout the project lifecycle.

Soft skills

• Responsible, organization and planning
• Proactive and teamwork
• Empathetic and Commitment

Labor Sector: IT

Languages: English Conversational (C1-C2)

Modality: Remote

Work schedule Monday to Friday 9am to 6pm

Benefits Company benefits (statutory and beyond)