Data Engineer — Data Lakehouse (Hybrid, Kraków, Poland)
- Location
- Not specified
- Employment
- Full-time
- Level
- Specialist
- Category
- Data & Analytics
- Posted
Description
What you'll be doing
Own the lakehouse architecture: bronze/silver/gold layers, Iceberg/Delta tables, schema evolution.
Land operational data via CDC streaming (Kafka, Debezium), handling late and duplicate events.
Design data layout for speed and cost: partitioning, compaction, file sizing, query performance on Trino/Athena/Snowflake.
Own retention and archival: storage tiering, regulatory retention, immutability, GDPR deletion.
Guarantee correctness: freshness SLAs, drift detection, reconciliation against the source wallet and ledger systems.
Own governance: catalog and lineage, row/column access control, PII masking, encryption, audit trails.
Monitor ingestion health, data anomalies, and cloud storage/compute spend.
Must-have
Senior: 5+ years in data engineering, with real ownership of a large-scale data lake or lakehouse.
Lakehouse architecture — bronze/silver/gold layering, an open table format (Iceberg, Delta, or Hudi), schema evolution.
Data layout & query optimization at TB+ scale — partitioning, compaction, file sizing, query performance on Trino/Athena/Snowflake.
Cloud lakehouse/DWH in production — Snowflake, Databricks, or BigQuery.
CDC & streaming ingestion — Kafka + Debezium or equivalent; late, duplicate and out-of-order events.
Strong SQL and data modeling — enough relational grounding to reason about the OLTP systems you capture from. Critical for financial ledgers.
Correctness — freshness SLAs, drift detection, reconciliation against source wallet/ledger systems.
Governance — catalogs, lineage, row/column access control, PII masking, retention, GDPR deletion.
Cloud object storage — S3 or GCS, plus storage tiering and archival.
Python and an orchestrator — Airflow or Dagster, as tools.
Location & work model
Kraków, Poland. Hybrid — 2 days per week from the office.
Apply at the source
This role was published by Commit Offshore and listed via Djinni. Applications are handled there, not on this site.
Original posting: https://djinni.co/jobs/847648-data-engineer-data-lakehouse-hybrid-krakow-po/