Senior Data Engineer · Available for opportunities

Piotr Koper

Data Engineering · Pipeline Architecture · Cloud Infrastructure

Senior Data Engineer specialising in platform engineering, IaC, and CI/CD, with 4+ years of experience building scalable data pipelines and infrastructure. Currently leading a working group across Terraform, GitLab CI/CD, and Docker image management.

About

I came into data engineering the long way round. Over a decade in logistics and operations management — Halfords, DHL, Aston Martin Lagonda — taught me how supply chains actually work, how to lead cross-functional teams under pressure, and crucially, how to build tooling when the tools you're given aren't good enough.

In 2021 I was selected for JLR's internal Data Engineering conversion programme, run with Infinity Works. It was competitive, it was intense, and it was the best decision I've made professionally. What followed wasn't a clean pivot into writing SQL — it was a gradual move toward the parts of the stack I find most interesting: infrastructure, automation, and the systems that make data teams actually function.

Four years in, I lead working groups across GitLab CI/CD pipelines, Docker image management, and Terraform at JLR. I own the Workload Identity Federation rollout between GitLab and GCP. I spend more time thinking about authentication patterns and Terraform module design than I do writing pipeline code — and I'm completely fine with that.

I'm interested in roles that sit at the intersection of data and platform engineering. Somewhere that values building things properly, documenting them well, and automating the things that shouldn't need a human. Rust is on the learning list. Opinions on tab width are strong.

Based in

West Midlands, UK

Currently

Senior Data Engineer, JLR

Background

Logistics & Operations → Data Engineering → Platform Engineering

Education

MSc Information Systems, Technical University of Łódź

MSc ICT for Engineers, Coventry University

Languages

Polish — native

English — fluent

Skills & Stack

Programming Languages
PythonSQLBashHCLYAML
Cloud Platforms & Services
GCP — BigQuery · GCS · Cloud Run · Cloud Functions · Cloud SQL · Artifact RegistryAWSSnowflake
Infrastructure & IaC
TerraformDockerKanikoPodmanKubernetesHelm
CI/CD & DevOps
GitLab CI/CDGitHub ActionsGitLab ComponentsPipeline TemplatingContainer Scanning
Data Orchestration & Transformation
DagsterApache Airflow (Cloud Composer / Astronomer)dbtDataform
Pipeline Patterns
ETL / ELTPipeline ArchitectureData ModellingData ContractsAPIs
Security & Identity
Workload Identity FederationIAMContainer ScanningSecret Management
Developer Tooling
GitLinuxPre-commituvGrafanaObsidian

Experience

2022 — Present

Senior Data Engineer

Jaguar Land Rover

Part of the core Data Engineering team within JLR’s Data & AI Office (DAIO), focused on infrastructure, developer tooling, and engineering standards. Progressed from pipeline development and data products into platform engineering, taking on technical leadership across multiple working groups.

Data Pipelines & Products

  • Designed, developed and delivered ETL and ELT data pipelines for high-volume data applications using Python and SQL, incorporating automated data testing to ensure continuous data quality in deployment
  • Evaluated Data Contract standards through hands-on testing, including dbt integration — and made a merged contribution to the Open Data Contract Standard (ODCS) open source project, shipped in v3.1.0

CI/CD Working Group — Lead Own and maintain semantically versioned GitLab CI/CD pipeline templates across the Data & AI Office and wider organisation, acting as one of the primary merge request reviewers and approvers.

  • Develop new templates and drive reusability, code quality, security, and consistency — enabling multiple teams across the enterprise to validate and deploy code quickly with enforced standards
  • Refactored multiple existing pipelines into a modular structure of reusable components, default templates, and pipeline definitions — covering Docker image builds, container scanning, Cloud Run, Cloud Functions, and other deployment targets
  • Led the Workload Identity Federation rollout, designing an authentication template that replaces long-lived credentials in CI/CD variables with short-lived OIDC tokens while remaining backward compatible with legacy Service Account Key projects — materially improving security posture across a high volume of daily pipeline runs
  • Designed and delivered bespoke CI/CD pipelines for the Dagster and Astronomer Airflow orchestration platforms, enabling SaaS integration with automated, controlled deployments

Terraform Working Group — Lead Lead the Terraform working group, owning module standards, provisioning patterns, and infrastructure-as-code practices across the data platform.

  • Build and maintain reusable Terraform modules that let teams provision cloud resources consistently via CI/CD, without bespoke infrastructure work per project
  • Provisioned the Workload Identity Federation infrastructure — Pool, Provider, and bindings — underpinning keyless GitLab to GCP authentication across the organisation
  • Refactored the Notifications System Terraform module end to end — Cloud Run services behind API Gateway, private Cloud SQL PostgreSQL with enforced SSL and automated backups, Memorystore Redis, and Secret Manager with least-privilege IAM access, all over private VPC networking

Docker Images Working Group — Lead Lead the working group responsible for maintaining and improving standardised Docker images stored in GCP Artifact Registry, used by the DAIO and other teams.

  • Maintain and improve custom runtime images for GitLab runners with the required tooling and dependencies pre-installed — removing runtime install steps from CI jobs, simplifying pipeline definitions, and reducing pipeline execution time
  • Keep images current and secure through regular base image version updates and package maintenance

Standards, Documentation & Best Practices Develop and maintain engineering standards, guides, and best practices adopted by technical and non-technical teams across the organisation — written as Docs as Code, versioned and reviewed alongside the code they describe.

  • Authored end-to-end documentation for key platform initiatives — including the Workload Identity Federation architecture, implementation and migration guide, the Notifications System, Terraform modules, test projects, Dagster and Astronomer Airflow guides, CI/CD components and pipeline templates and others
  • Primary developer of standardised project setup scripts in Bash (Linux) and PowerShell (Windows), paired with pre-commit hooks for linting, formatting, and validation — promoting consistent development practices across teams

Notable Projects

  • Python Task Orchestrator Evaluation (Lead) — structured evaluation of two enterprise orchestration platforms, building end-to-end data pipelines covering data validation, CI/CD integration, event-triggered execution, cross-platform authentication, and integrations with GCP, dbt, Dataform, and Snowflake. The formal evaluation report was adopted as the technical recommendation to the Enterprise Technology Board
  • Notifications System — core developer of a GCP-based notifications system routing messages to email, Slack, and Jira based on subscription behaviour and API calls during pipeline execution
  • GitLab Instance Migration — part of the team that designed and provisioned the Kubernetes cluster, namespace, and Helm module for self-hosted GitLab runners during a full instance migration, managed entirely via Terraform
  • Enterprise Data Platform Investigation — contributed to a cross-functional investigation with the Enterprise Architect into the target layout and usage patterns for JLR’s Enterprise Data Platform
2021 — 2022

Data Engineering Conversion Course

Academy · Infinity Works · Jaguar Land Rover

Successfully selected for a competitive internal conversion programme, transitioning into data engineering from a non-engineering background. Completed the structured curriculum covering data pipelines, ETL and ELT patterns, SQL, and Python — formally completed May 2022.

This programme marked the start of a deliberate career transition into data engineering, building the foundations that underpinned subsequent platform and infrastructure work.

PythonSQLData EngineeringETLELT
2018 — 2021

CPAT Lead & Planner

Jaguar Land Rover

Cross-functional programme role managing the Capacity, Parts Availability and Timing assessment process across JLR vehicle programmes. Responsible for critical path planning, risk management, and stakeholder communication across Purchasing, Supply Chain and Programme teams.

Built several internal tools to address existing process gaps — including a redesigned CPAT Input Matrix to improve data control and an automated WIP file creation process saving approximately 12 hours per file. Also developed an eCAR data scrape tool integrated directly into the WIP workflow.

This role developed the stakeholder management, cross-functional leadership and data-driven decision making skills that directly underpin the working group leadership taken on in data engineering.

Excel VBAData AnalysisProcess AutomationStakeholder Management
2007 — 2018

Planning Manager & Logistics Operations

Halfords · Aston Martin Lagonda (DHL) · DHL

Over a decade of progressive logistics and operations management across major UK distribution and manufacturing environments, including Halfords (1.4M units/week), Aston Martin Lagonda, and DHL.

Consistently built tooling and reports to replace manual processes — a Logistics Performance Tool tracking volumes, hours and costs against budget; automated planning tools; and redesigned operational workflows. The throughput numbers and team sizes involved meant data quality and analytical rigour weren’t optional — they were how the operation ran.

Selected for a competitive DHL Head Start management programme from hundreds of UK and Ireland applicants, one of 28 accepted.

This background directly informs how I approach data engineering — the domain problems, the stakeholder relationships, and the instinct to automate anything done more than twice.

ExcelVBAOperations ManagementData Analysis

Projects

Octopus Energy Dashboard

Built a personal data pipeline to track and visualise energy costs and consumption using the Octopus Energy API. The project covers the full data lifecycle — ingestion, transformation, storage, and visualisation.

Pipeline Dagster-based pipeline runs multiple times daily, pulling variable tariff rates and consumption data from the Octopus API, performing cleaning and transformation in Python, and writing to a PostgreSQL database hosted on an Oracle Cloud VM.

Infrastructure & Deployment GitHub Actions handles deployment, authenticating to the Oracle VM and deploying the Dagster pipeline automatically. The database is connected to a Grafana dashboard providing day-on-day and week-on-week comparisons, including year-over-year cost tracking.

Why it matters Built entirely outside of work, this project touches every layer of a real data platform — orchestration, transformation, storage, deployment and visualisation — using the same tools evaluated professionally (Dagster) and hosted on real infrastructure.

DagsterPythonPostgreSQLGrafanaGitHub ActionsDocker

Solis Battery Manager

A personal data pipeline monitoring solar panel performance and battery status using the Solis Cloud API. Runs on a schedule, validates inverter data, and delivers a daily summary notification via email covering battery charge status, PV generation, and energy exported to the grid.

Pipeline Dagster-based pipeline with scheduled execution — ingests data from the Solis Cloud API using HMAC-signed authentication, validates the response, and triggers a Pingram email notification with the day’s solar metrics.

Infrastructure & Deployment Full production-grade Dagster deployment: separate code server, webserver, and daemon services orchestrated via Docker Compose, with PostgreSQL backing run history and schedule state. Multi-platform Docker images (amd64/arm64) built and pushed to GitHub Container Registry on every merge to main. GitHub Actions handles the complete pipeline — build, publish, SSH to Oracle Cloud VM, pull the new image, create environment variables on the fly, and restart the stack with zero manual intervention.

Documentation The project is backed by comprehensive documentation written for both operational use and future reference — covering architecture decisions, deployment runbook, environment setup, API authentication approach, and local development guide. Written with the same standard applied professionally at JLR: if someone else needs to pick this up, or future me returns after a year away, the documentation should be enough to get back to a running state without guesswork.

Why it matters Built entirely outside of work on real infrastructure, this project demonstrates a complete engineering workflow: API integration with custom authentication, data validation, containerised multi-service deployment, secrets management, multi-platform builds, and automated CI/CD — using the same tools and patterns applied professionally.

DagsterPythonDockerPostgreSQLGitHub ActionsGitHub Container RegistryAPIPingram
↗

Education & Certifications

2007

degree

MSc Management and Production — Systems and Information Technologies

Technical University of Lodz, Poland

2006

degree

MSc Information and Communication Technology for Engineers

Coventry University, United Kingdom

Contact

Open to data and software engineering roles with a focus on infrastructure, platform engineering, and developer tooling. Equally comfortable in CI/CD, IaC, containerisation, and data systems — interested in roles that value engineering depth over narrow specialisation.

piotr.koper.v0@gmail.comDownload CV (PDF)