Senior Data Engineer · Available for opportunities
Piotr Koper
Data Engineering · Pipeline Architecture · Cloud Infrastructure
Senior Data Engineer specialising in platform engineering, IaC, and CI/CD, with 4+ years of experience building scalable data pipelines and infrastructure. Currently leading a working group across Terraform, GitLab CI/CD, and Docker image management.
About
I came into data engineering the long way round. Over a decade in logistics and operations management — Halfords, DHL, Aston Martin Lagonda — taught me how supply chains actually work, how to lead cross-functional teams under pressure, and crucially, how to build tooling when the tools you're given aren't good enough.
In 2021 I was selected for JLR's internal Data Engineering conversion programme, run with Infinity Works. It was competitive, it was intense, and it was the best decision I've made professionally. What followed wasn't a clean pivot into writing SQL — it was a gradual move toward the parts of the stack I find most interesting: infrastructure, automation, and the systems that make data teams actually function.
Four years in, I lead working groups across GitLab CI/CD pipelines, Docker image management, and Terraform at JLR. I own the Workload Identity Federation rollout between GitLab and GCP. I spend more time thinking about authentication patterns and Terraform module design than I do writing pipeline code — and I'm completely fine with that.
I'm interested in roles that sit at the intersection of data and platform engineering. Somewhere that values building things properly, documenting them well, and automating the things that shouldn't need a human. Rust is on the learning list. Opinions on tab width are strong.
Based in
West Midlands, UK
Currently
Senior Data Engineer, JLR
Background
Logistics & Operations → Data Engineering → Platform Engineering
Education
MSc Information Systems, Technical University of Łódź
MSc ICT for Engineers, Coventry University
Languages
Polish — native
English — fluent
Skills & Stack
Experience
Senior Data Engineer
Jaguar Land Rover
Part of the core Data Engineering team within JLR’s Data & AI Office (DAIO), focused on infrastructure, developer tooling, and engineering standards. Progressed from pipeline development and data products into platform engineering, taking on technical leadership across multiple working groups.
Data Pipelines & Products
- Designed, developed and delivered ETL and ELT data pipelines for high-volume data applications using Python and SQL, incorporating automated data testing to ensure continuous data quality in deployment
- Evaluated Data Contract standards through hands-on testing, including dbt integration — and made a merged contribution to the Open Data Contract Standard (ODCS) open source project, shipped in v3.1.0
CI/CD Working Group — Lead Own and maintain semantically versioned GitLab CI/CD pipeline templates across the Data & AI Office and wider organisation, acting as one of the primary merge request reviewers and approvers.
- Develop new templates and drive reusability, code quality, security, and consistency — enabling multiple teams across the enterprise to validate and deploy code quickly with enforced standards
- Refactored multiple existing pipelines into a modular structure of reusable components, default templates, and pipeline definitions — covering Docker image builds, container scanning, Cloud Run, Cloud Functions, and other deployment targets
- Led the Workload Identity Federation rollout, designing an authentication template that replaces long-lived credentials in CI/CD variables with short-lived OIDC tokens while remaining backward compatible with legacy Service Account Key projects — materially improving security posture across a high volume of daily pipeline runs
- Designed and delivered bespoke CI/CD pipelines for the Dagster and Astronomer Airflow orchestration platforms, enabling SaaS integration with automated, controlled deployments
Terraform Working Group — Lead Lead the Terraform working group, owning module standards, provisioning patterns, and infrastructure-as-code practices across the data platform.
- Build and maintain reusable Terraform modules that let teams provision cloud resources consistently via CI/CD, without bespoke infrastructure work per project
- Provisioned the Workload Identity Federation infrastructure — Pool, Provider, and bindings — underpinning keyless GitLab to GCP authentication across the organisation
- Refactored the Notifications System Terraform module end to end — Cloud Run services behind API Gateway, private Cloud SQL PostgreSQL with enforced SSL and automated backups, Memorystore Redis, and Secret Manager with least-privilege IAM access, all over private VPC networking
Docker Images Working Group — Lead Lead the working group responsible for maintaining and improving standardised Docker images stored in GCP Artifact Registry, used by the DAIO and other teams.
- Maintain and improve custom runtime images for GitLab runners with the required tooling and dependencies pre-installed — removing runtime install steps from CI jobs, simplifying pipeline definitions, and reducing pipeline execution time
- Keep images current and secure through regular base image version updates and package maintenance
Standards, Documentation & Best Practices Develop and maintain engineering standards, guides, and best practices adopted by technical and non-technical teams across the organisation — written as Docs as Code, versioned and reviewed alongside the code they describe.
- Authored end-to-end documentation for key platform initiatives — including the Workload Identity Federation architecture, implementation and migration guide, the Notifications System, Terraform modules, test projects, Dagster and Astronomer Airflow guides, CI/CD components and pipeline templates and others
- Primary developer of standardised project setup scripts in Bash (Linux) and PowerShell (Windows), paired with pre-commit hooks for linting, formatting, and validation — promoting consistent development practices across teams
Notable Projects
- Python Task Orchestrator Evaluation (Lead) — structured evaluation of two enterprise orchestration platforms, building end-to-end data pipelines covering data validation, CI/CD integration, event-triggered execution, cross-platform authentication, and integrations with GCP, dbt, Dataform, and Snowflake. The formal evaluation report was adopted as the technical recommendation to the Enterprise Technology Board
- Notifications System — core developer of a GCP-based notifications system routing messages to email, Slack, and Jira based on subscription behaviour and API calls during pipeline execution
- GitLab Instance Migration — part of the team that designed and provisioned the Kubernetes cluster, namespace, and Helm module for self-hosted GitLab runners during a full instance migration, managed entirely via Terraform
- Enterprise Data Platform Investigation — contributed to a cross-functional investigation with the Enterprise Architect into the target layout and usage patterns for JLR’s Enterprise Data Platform
Data Engineering Conversion Course
Academy · Infinity Works · Jaguar Land Rover
Successfully selected for a competitive internal conversion programme, transitioning into data engineering from a non-engineering background. Completed the structured curriculum covering data pipelines, ETL and ELT patterns, SQL, and Python — formally completed May 2022.
This programme marked the start of a deliberate career transition into data engineering, building the foundations that underpinned subsequent platform and infrastructure work.
CPAT Lead & Planner
Jaguar Land Rover
Cross-functional programme role managing the Capacity, Parts Availability and Timing assessment process across JLR vehicle programmes. Responsible for critical path planning, risk management, and stakeholder communication across Purchasing, Supply Chain and Programme teams.
Built several internal tools to address existing process gaps — including a redesigned CPAT Input Matrix to improve data control and an automated WIP file creation process saving approximately 12 hours per file. Also developed an eCAR data scrape tool integrated directly into the WIP workflow.
This role developed the stakeholder management, cross-functional leadership and data-driven decision making skills that directly underpin the working group leadership taken on in data engineering.
Planning Manager & Logistics Operations
Halfords · Aston Martin Lagonda (DHL) · DHL
Over a decade of progressive logistics and operations management across major UK distribution and manufacturing environments, including Halfords (1.4M units/week), Aston Martin Lagonda, and DHL.
Consistently built tooling and reports to replace manual processes — a Logistics Performance Tool tracking volumes, hours and costs against budget; automated planning tools; and redesigned operational workflows. The throughput numbers and team sizes involved meant data quality and analytical rigour weren’t optional — they were how the operation ran.
Selected for a competitive DHL Head Start management programme from hundreds of UK and Ireland applicants, one of 28 accepted.
This background directly informs how I approach data engineering — the domain problems, the stakeholder relationships, and the instinct to automate anything done more than twice.
Projects
Octopus Energy Dashboard
Built a personal data pipeline to track and visualise energy costs and consumption using the Octopus Energy API. The project covers the full data lifecycle — ingestion, transformation, storage, and visualisation.
Pipeline Dagster-based pipeline runs multiple times daily, pulling variable tariff rates and consumption data from the Octopus API, performing cleaning and transformation in Python, and writing to a PostgreSQL database hosted on an Oracle Cloud VM.
Infrastructure & Deployment GitHub Actions handles deployment, authenticating to the Oracle VM and deploying the Dagster pipeline automatically. The database is connected to a Grafana dashboard providing day-on-day and week-on-week comparisons, including year-over-year cost tracking.
Why it matters Built entirely outside of work, this project touches every layer of a real data platform — orchestration, transformation, storage, deployment and visualisation — using the same tools evaluated professionally (Dagster) and hosted on real infrastructure.
Solis Battery Manager
A personal data pipeline monitoring solar panel performance and battery status using the Solis Cloud API. Runs on a schedule, validates inverter data, and delivers a daily summary notification via email covering battery charge status, PV generation, and energy exported to the grid.
Pipeline Dagster-based pipeline with scheduled execution — ingests data from the Solis Cloud API using HMAC-signed authentication, validates the response, and triggers a Pingram email notification with the day’s solar metrics.
Infrastructure & Deployment Full production-grade Dagster deployment: separate code server, webserver, and daemon services orchestrated via Docker Compose, with PostgreSQL backing run history and schedule state. Multi-platform Docker images (amd64/arm64) built and pushed to GitHub Container Registry on every merge to main. GitHub Actions handles the complete pipeline — build, publish, SSH to Oracle Cloud VM, pull the new image, create environment variables on the fly, and restart the stack with zero manual intervention.
Documentation The project is backed by comprehensive documentation written for both operational use and future reference — covering architecture decisions, deployment runbook, environment setup, API authentication approach, and local development guide. Written with the same standard applied professionally at JLR: if someone else needs to pick this up, or future me returns after a year away, the documentation should be enough to get back to a running state without guesswork.
Why it matters Built entirely outside of work on real infrastructure, this project demonstrates a complete engineering workflow: API integration with custom authentication, data validation, containerised multi-service deployment, secrets management, multi-platform builds, and automated CI/CD — using the same tools and patterns applied professionally.
Education & Certifications
degree
MSc Management and Production — Systems and Information Technologies
Technical University of Lodz, Poland
degree
MSc Information and Communication Technology for Engineers
Coventry University, United Kingdom
Contact
Open to data and software engineering roles with a focus on infrastructure, platform engineering, and developer tooling. Equally comfortable in CI/CD, IaC, containerisation, and data systems — interested in roles that value engineering depth over narrow specialisation.
piotr.koper.v0@gmail.comDownload CV (PDF)