Middle Data Engineer

As a Data Engineer on the Enterprise Data & Intelligence team, you will design, build, and maintain the data pipelines and models that underpin Fourth’s single source of truth. Reporting to the Senior Data Engineer, you will gather and translate business requirements into reliable, well-tested data solutions on our Azure-based cloud platform. You will collaborate closely with analytics engineers and business stakeholders to scope, prioritise, and deliver data products that serve FP&A, Commercial, Product, and AI teams.

Responsibilities:

1. Data Pipeline Development & Optimisation

• Build, maintain, and optimise scalable ETL/ELT pipelines (batch and near-real-time) on Azure Data Cloud Platform (e.g., Data Lake, Microsoft Fabric, Azure Data Factory).
• Develop and refine data models to support BI reporting, analytics, and ML/AI use cases.
• Write efficient, well-documented T-SQL and PySpark code following team coding standards.

2. Data Quality, Testing & Governance

• Implement automated testing, data validation, and monitoring (SLAs, alerts) to ensure pipeline reliability.
• Contribute to data governance practices, including lineage tracking, metadata management, and quality controls.
• Support CI/CD pipelines for data assets, ensuring version control and reproducibility.

3. Requirements Scoping & Cross-Functional Collaboration

• Partner with analytics engineers to scope, refine, and prioritise data requirements from business stakeholders.
• Work with Analysts, BI Developers, Data Scientists, and business teams to translate requirements into production-ready data solutions.
• Provide input on data readiness for machine learning and analytics projects.

4. Platform Contribution & Continuous Improvement

• Contribute to the evolution of the ED&I data platform, including tooling, standards, and documentation.
• Stay current with emerging data engineering patterns and technologies; propose improvements to team processes.
• Leverage AI-driven development tools (e.g., generative-AI code assistants, automated data profiling) to accelerate delivery.
• Support performance tuning and cost optimisation across the data platform.

Requirements:

Experience: 3+ years in data engineering or a closely related role.

Education: Bachelor’s degree in Computer Science, Data Engineering, or a related field.

Technical Skills:

○Strong T-SQL skills and working proficiency in PySpark or Python for data processing.

○Hands-on experience with MS Azure Storage Explorer and SSMS.

○Hands-on experience with cloud-based data engineering services and orchestration tools (e.g., Azure Data Factory, Microsoft Fabric).

○Practical experience building ETL/ELT pipelines and dimensional or analytical data models.

○Familiarity with CI/CD practices in data engineering, including version control (Git) and automated testing.

Preferred Qualifications

• Experience with real-time or streaming data architectures.
• Experience with PowerShell, Apache Kafka, and/or KQL.
• Exposure to AI/ML workflows (feature engineering, data preparation for model training).
• Familiarity with Power BI or other BI/visualisation tools.
• Experience using AI productivity tools (e.g., ChatGPT, Claude, Copilot, Cursor) in day-to-day and data engineering tasks.
• Understanding of data security, privacy, and compliance considerations.

We offer*:

Flexible working format - remote, office-based or flexible

A competitive salary and good compensation package

Personalized career growth

Professional development tools (mentorship program, tech talks and trainings, centers of excellence, and more)

Active tech communities with regular knowledge sharing

Education reimbursement

Memorable anniversary presents

Corporate events and team buildings

Other location-specific benefits

• not applicable for freelancers