Lakehouse Formats 2026 — Iceberg vs Delta Lake vs Hudi

A folder of Parquet files in cloud storage is not a table. It has no transactions, no safe way to change a column, and no way to tell a reader which files belong to "the current version." Lakehouse table formats fix that by adding a metadata layer on top of the files. Apache Iceberg, Delta Lake, and Apache Hudi are the three that matter, and in 2026 the practical question is less "which one is best" than "which one fits my engines, my catalog, and my write pattern."

This guide explains how each format works underneath, where each fits, what the 2024 acquisitions and catalog announcements changed, and how to choose without relying on benchmark folklore.

What a lakehouse table format actually does

Before table formats, a data lake was a directory tree. A "table" was a convention: everything under s3://bucket/sales/ is the sales table, partitioned by folders such as date=2024-03-01/. Query engines listed the folders, opened the files, and hoped nobody was writing at that moment.

That design breaks in predictable ways:

• A job that fails halfway leaves some new files and not others, so readers see partial data.
• Two jobs writing at once can overwrite each other with no warning.
• Renaming or reordering a column means rewriting or reinterpreting every file.
• Finding the files for a query requires listing directories, which is slow on object stores at scale.
• There is no record of what the table looked like yesterday.

A table format replaces the convention with a record. The record says exactly which data files make up the table at each moment, what the schema is, and how the files are partitioned. Readers consult the record rather than the directory. Writers add files first, then publish them by updating the record in one atomic step.

Think of it like a library catalog. The books (data files) sit on shelves in no particular order that matters. The catalog says which books are in the collection today, which were withdrawn, and where each one is. If you want to know what the collection looked like last month, you consult an older version of the catalog, not the shelves.

The capabilities that follow from this record:

Capability	What it means in practice
ACID transactions	A write either appears completely or not at all, and concurrent writers are coordinated
Schema evolution	Add, drop, or rename columns without rewriting existing data files
Time travel	Query or restore the table as of an earlier snapshot or timestamp
Partition evolution or clustering	Change how data is laid out without a full rewrite (details differ by format)
Row-level updates and deletes	Change individual rows without rewriting an entire table
Engine neutrality	Multiple query engines read and write the same table

For where this layer sits among ingestion, transformation, and BI tools, see our modern data stack 2026 reference.

Where the term "lakehouse" came from

The idea of combining warehouse reliability with data lake economics was formalized in a 2021 paper presented at the CIDR conference by authors from Databricks, titled "Lakehouse: A New Generation of Open Platforms that Unify Data Warehousing and Advanced Analytics" (CIDR 2021). The table formats themselves predate the paper. Hudi came out of Uber and was described in a March 2017 Uber Engineering post (Uber Engineering). Iceberg began at Netflix in 2017, was donated to the Apache Software Foundation in 2018, and became a top-level Apache project in May 2020 (Wikipedia). Delta Lake came from Databricks and was described in a 2020 VLDB paper (VLDB).

All three solve the same problem. They differ in how the record is stored, how writes are coordinated, and which workloads they were designed around.

The three formats at a glance

Format	Started at	Governance	Design emphasis	Typical fit
Apache Iceberg	Netflix	Apache Software Foundation	Correctness at scale, engine neutrality	Multi-engine lakehouses, vendor-neutral builds
Delta Lake	Databricks	Open source project (delta-io on GitHub)	Tight integration with Spark and Databricks	Databricks-centered stacks
Apache Hudi	Uber	Apache Software Foundation	Record-level upserts and incremental processing	CDC ingestion, streaming-heavy tables

All three store data in open columnar files, typically Parquet, so the underlying bytes are portable. The formats differ in the metadata around those files, which is where lock-in and compatibility questions live.

How Iceberg works: a tree of metadata

An Iceberg table is described by a small tree. At the top is a table metadata file, which holds the schema, partition specification, and a list of snapshots. Each snapshot points to a manifest list, which points to manifests, and each manifest lists data files together with statistics such as column value ranges. Query engines use those statistics to skip files that cannot match a filter, without opening them.

The Iceberg specification is explicit about how this evolved across format versions (Iceberg spec). Version 2 introduced row-level deletes, along with sequence numbers in table metadata and manifest lists to order changes. Version 3 added deletion vectors stored using the Puffin file format, default values for struct fields, and new types including variant, geometry, geography, unknown, and nanosecond-precision timestamps. In version 3, writers are not allowed to add new position delete files, which pushes row-level deletes toward deletion vectors.

Commits are an atomic pointer swap

Iceberg's concurrency model is optimistic. A writer prepares new metadata assuming no one else is writing, then commits by atomically swapping the pointer to the table metadata file. If another writer got there first, the swap fails, and the loser rebuilds its metadata from the latest state. The reliability documentation describes how a retry validates its assumptions: if a compaction's source file was deleted by a concurrent commit, the compaction must fail rather than apply stale changes (Iceberg reliability docs).

The practical consequence is that the catalog, the service that stores the pointer to the current metadata, is part of the correctness story. It must support an atomic compare-and-swap. That is one reason catalog choice matters as much as format choice, a point covered below.

What Iceberg is known for

The project's own feature list highlights several capabilities (Apache Iceberg):

• Schema evolution. Add, rename, reorder, or change column types without rewriting the table.
• Hidden partitioning. Iceberg generates partition values from column data, so queries do not need to name partition columns to benefit from pruning.
• Time travel and rollback. Query a past snapshot or reset the table to an earlier state.
• Built-in compaction procedures. Strategies such as bin-packing and sorting reorganize file layout.
• Multi-engine safety. Spark, Trino, Flink, Presto, Hive, and Impala can safely read and write the same tables concurrently.

Where Iceberg is harder

Iceberg does less out of the box than a vertically integrated platform. You choose a catalog, you schedule maintenance (compaction, snapshot expiry, orphan file cleanup) unless a managed service does it for you, and engine support for newer spec features varies by version. The multi-engine strength is also a testing burden: a feature available in one engine may not be implemented in another yet, so check the specific engine and version before relying on a spec capability.

How Delta Lake works: an ordered log

Delta Lake takes a different route to the same end. A Delta table has a _delta_log directory under the table root. Each commit is a newline-delimited JSON file named by version number, zero-padded to 20 digits, and the protocol states that these files are "the unit of atomicity for a table" (Delta protocol).

Each commit file contains actions, one per line. The main ones:

• add registers a data file as part of the table.
• remove marks a file as no longer part of the table. It acts as a tombstone until it expires, which is how VACUUM knows what can be physically deleted.
• metaData carries the schema, partition columns, and configuration. The first version of a table must contain one.
• protocol sets the minimum reader and writer versions and the table features required to use the table.

To avoid replaying thousands of small JSON files, Delta periodically writes checkpoints: Parquet files holding the reconciled table state at a version. A reader loads the latest checkpoint and applies only the commits after it.

Delta features worth knowing

• Deletion vectors mark rows as logically deleted without rewriting the data file. Readers must exclude those rows. The table needs reader version 3 and writer version 7 with the deletionVectors feature.
• Column mapping assigns each column a physical name and a 32-bit id, so renames and drops do not require rewriting data. Modes are none, id, and name.
• Time travel reads an earlier version of the log.
• Change data feed exposes row-level changes between versions to Delta clients.

Delta UniForm: one copy of data, several metadata views

Delta Universal Format, or UniForm, lets Iceberg and Hudi clients read a Delta table. Because all three formats use Parquet data files, one copy of the data can serve all of them. UniForm generates Iceberg metadata asynchronously after each Delta commit, so write overhead is small (Delta UniForm docs).

The documented limits are worth reading before treating UniForm as a drop-in bridge:

UniForm detail	What the documentation says
Direction	Read-only from the Iceberg and Hudi side. External writes can cause data loss
Iceberg writes (Delta side)	Requires Delta Lake 3.1 or later
Hudi support	Preview, requires Delta Lake 3.2 or later
Requirements	Column mapping enabled, minimum reader version 2 and writer version 7, the IcebergCompatV2 feature
Deletion vectors	Not supported unless the table is upgraded using a REORG operation
Change data feed, Delta Sharing	Work for Delta clients, with no Iceberg support

So UniForm is a read bridge. It lets an Iceberg-speaking engine query a Delta table, which is valuable. It does not make a Delta table writable by Iceberg writers.

How Hudi works: a timeline of actions

Hudi (pronounced "hoodie") treats a table as a log of actions on a timeline. Each action has a requested instant, which acts as a transaction ID, an inflight state, and a completed instant. The timeline is the source of truth for the table's state (Hudi timeline docs).

The main actions:

• COMMIT: an atomic write of a batch of records into base files.
• DELTA_COMMIT: an atomic write to a merge-on-read table, where some or all data goes to delta log files.
• COMPACTION: a table service that merges delta files into base files.
• CLEANS: a table service that removes older file slices no longer needed.

Uber's original 2017 post describes the problem Hudi was built for. HDFS lacked efficient record-level updates and incremental processing, so teams ran a separate low-latency serving layer next to the batch layer. Hudi added upserts, deletes, and incremental change pulls to HDFS datasets, so workloads tolerating roughly 10 minutes of latency could avoid that second layer (Uber Engineering).

Copy-on-write versus merge-on-read

Hudi's signature design choice is that you pick, per table, how updates are stored. The Hudi documentation lays out the trade-off (Hudi table types):

Trade-off	Copy-on-write	Merge-on-read
Update handling	Rewrites the affected base files	Appends changes to row-based delta log files
Write latency	Higher	Lower
Query latency	Lower	Higher
Read amplification	None	Grows with the number of changed records in queried file groups
Typical fit	Batch ETL, slowly changing tables, fast OLAP reads	CDC pipelines, streaming ingestion, frequent updates and deletes

Merge-on-read queries can run in a read-optimized mode that skips merging, at the cost of seeing data only up to the last compaction. Periodic compaction folds the logs into new base files.

Iceberg and Delta now offer comparable ideas, with delete files or deletion vectors that defer rewriting. Hudi built its design around this trade-off from the start, which is why it remains a reasonable choice for heavy upsert workloads.

Row-level deletes: how the three compare

Changing one row in an immutable Parquet file is the central engineering problem. Each format has an approach.

Format	Approach to row-level changes	Cost shifted to
Iceberg v2	Position and equality delete files that readers apply	Read time, until compaction
Iceberg v3	Deletion vectors stored in Puffin files	Read time, but cheaper to apply
Delta Lake	Deletion vectors alongside data files	Read time, until files are rewritten
Hudi MoR	Delta log files merged with base files	Read time, until compaction
Hudi CoW	Rewrite the base file	Write time

The pattern is the same everywhere: you can pay at write time by rewriting files, or at read time by merging change records, and either way scheduled compaction brings the table back to a clean state. Choosing a format does not remove that choice. It changes which knobs you get.

Where each format wins

Iceberg

Iceberg is a strong default when several engines touch the same tables, or when avoiding single-vendor dependence is a stated goal. Its spec is public, its metadata is engine-independent, and its catalog interface is standardized through a REST specification.

Its position strengthened through 2024 as large platform vendors made commitments to it. Snowflake announced the Polaris Catalog on June 3, 2024, an Iceberg catalog implementing the Iceberg REST API, with a commitment to open source it within 90 days (Snowflake). Amazon announced S3 Tables on December 3, 2024, which stores tabular data in Iceberg format and automates compaction, snapshot expiry, and unreferenced file removal (AWS News Blog). AWS claimed up to three times faster queries and up to ten times more transactions per second compared with self-managed table storage. Treat that as a vendor claim for a specific comparison, not a general result.

Delta Lake

Delta Lake is a strong default when the stack is built around Databricks. The format, the engine, the catalog, and the ML tooling are designed together, and the default experience is simpler because fewer decisions are left to the team. Delta's ordered log and checkpoint design is also easy to reason about, which helps with debugging.

Hudi

Hudi fits tables that receive a steady stream of record-level changes, such as database change data capture, where the timeline and Hudi's indexing features make upserts a first-class operation. Teams with existing Hudi pipelines have little reason to migrate without a specific pain.

An editorial assessment, not a measured statistic: new multi-engine lakehouse projects in 2026 more often start with Iceberg than with Hudi, and engine support for Hudi tends to be narrower. Verify your specific engines before ruling either in or out.

The June 2024 Tabular acquisition and what it changed

On June 4, 2024, Databricks announced it had agreed to acquire Tabular, a company founded by Ryan Blue, Daniel Weeks, and Jason Reid, who were among the original creators of Iceberg. A June 7 update on Databricks' page said the deal had closed (Databricks). Databricks said that, in the short term, interoperability would proceed through Delta UniForm, and that the long-term goal was "a single, open, and common standard of interoperability." It also said the work would take several years across the Iceberg and Delta communities.

For a buyer, the practical reading is that the two largest communities are explicitly working toward convergence, but that convergence is a multi-year effort, not a finished state. Plan for the formats you can use today.

Catalogs: the decision that sits beside the format

A format defines how table state is stored. A catalog decides where the pointer to current state lives, who may read or change it, and how tables are discovered. Because Iceberg commits are an atomic pointer swap, the catalog must support that swap reliably.

Catalog	Typical context	Notes
AWS Glue Data Catalog	AWS-centered stacks	Mature and integrated with AWS services
Hive Metastore	Existing Hadoop-era estates	Widely deployed; fewer modern governance features
Iceberg REST catalog implementations	Multi-engine Iceberg	Standard HTTP interface; several implementations exist
Unity Catalog	Databricks stacks	Built into Databricks; evolving cross-engine support
Apache Polaris	Snowflake-centered multi-engine	Announced by Snowflake in June 2024; donated to Apache and a top-level project since February 2026
Project Nessie	Git-like catalog semantics for Iceberg	Branching and tagging of table state

Questions to settle before picking a catalog:

1. Which engines must read the tables, and which must write to them?
2. Does the catalog support the atomic commit semantics the format needs?
3. Who manages access control, and does it apply consistently across engines?
4. Is the catalog a managed service or something your team must run?
5. If you later leave the catalog vendor, can the table state be exported cleanly?

Catalog sprawl is a real failure mode. Two teams creating tables in two catalogs means data that cannot be joined without copying. Decide the catalog explicitly, and write down who owns it.

Performance: what is and is not knowable

Published format benchmarks are easy to find and hard to generalize. Results depend on engine version, file sizes, partitioning, data skew, and whether tables were compacted. A fair statement is narrower:

• Read performance is driven mostly by file layout and statistics, so well-compacted tables in any of the three formats perform similarly on the same engine.
• Write performance depends on the update pattern. Heavy record-level upserts favor designs that append changes (merge-on-read, deletion vectors) and compact later. Bulk batch loads are less sensitive to format.
• Metadata overhead grows with the number of snapshots, manifests, or log files. Each format provides maintenance operations to bound it, and each requires you to run them.
• Compaction is needed in all three. Without it, small files multiply, metadata grows, and queries slow down.

Run a pilot with your own data and your own engine instead of trusting a published chart. The difference between a tuned and an untuned table in the same format is usually larger than the difference between formats.

File size, partitioning, and maintenance

Three operational habits matter more than the format badge.

File size. Many small files slow every engine, because each file has open and metadata overhead. Target file sizes in the hundreds of megabytes are common, but defaults and recommendations vary by engine and format, so check the documentation of the engine you run.

Partitioning. Over-partitioning creates many tiny files. Iceberg's hidden partitioning and partition evolution reduce the cost of getting this wrong, and Delta offers liquid clustering as an alternative to fixed partition columns. Neither removes the need to understand your query filters.

Scheduled maintenance. Plan recurring jobs for compaction, snapshot or version expiry, and removal of unreferenced files. Some managed services handle this. S3 Tables, for example, performs compaction, snapshot management, and unreferenced file removal automatically.

Migrating between formats

Moving between formats is possible and rarely free.

From	To	Typical path	Cost
Delta	Iceberg (read access)	UniForm exposes Iceberg metadata	Low, but read-only and subject to UniForm limits
Delta	Iceberg (full)	Metadata translation tools such as Apache XTable, or a data rewrite	Moderate to high
Iceberg	Delta	Rewrite or translation tooling	Moderate to high
Hudi	Another format	Typically a rewrite, planned per table	High

Because all three use Parquet, translating metadata without moving data is sometimes feasible, but write paths, catalogs, and table features (deletion vectors, column mapping, change feeds) complicate it. Test on a copy and validate row counts and schema before cutover.

How to choose: a decision guide

Answer in order. Stop at the first decisive answer.

1. Is the platform already Databricks-centered, with no plan to change? Start with Delta Lake. Add UniForm if outside engines need read access.
2. Do multiple independent engines need to read and write the same tables? Start with Iceberg, and choose the catalog before the first table.
3. Is the dominant workload a continuous stream of record-level changes? Evaluate Hudi's merge-on-read alongside Iceberg and Delta with deletion vectors, using your own data.
4. Is the team small, with one warehouse and no lake? You may not need a lakehouse format yet. A warehouse, or DuckDB over Parquet, may be sufficient.
5. Is there a strong existing investment in one format? Stay unless there is a concrete pain that a migration solves.

Persona shortcuts

• A platform team at a mid-size company using Snowflake, Spark, and Trino: Iceberg with a REST catalog.
• A Databricks-first data science group: Delta Lake, with UniForm for external readers.
• A team ingesting database changes into the lake every few minutes: pilot Hudi merge-on-read against alternatives.
• A startup with a few terabytes and one analyst: probably neither yet. Start simple and revisit.

Where formats fit in the stack

Lakehouse formats sit at the storage layer, beneath transformation tools and orchestrators. Transformation frameworks like dbt can run on engines that read these tables (see our dbt data engineering guide). Orchestration choices are covered in our Airflow vs Prefect vs Dagster comparison. The compute layer is discussed in the Spark vs Polars vs Dask comparison and in Apache Spark Explained.

A typical layout:

[Ingestion: Fivetran, Airbyte, or Kafka]
        |
[Storage: S3, GCS, or Azure Blob with Iceberg, Delta, or Hudi tables]
        |   (Catalog: Glue, Unity, Polaris, Nessie, or an Iceberg REST catalog)
[Compute: Spark, Trino, Flink, DuckDB, or warehouse engines]
        |
[Transformation: dbt or SQLMesh]
        |
[Consumption: BI tools, ML training, reverse ETL]

Machine learning teams reading lakehouse tables for training may also want feature stores, which sit on top of the same storage. And because tables with enforced schema evolution still need agreement between producers and consumers, data contracts complement the format's own guarantees.

Common mistakes

1. No catalog strategy. "We will pick one later" leads to tables scattered across catalogs.
2. Skipping maintenance. Without compaction and snapshot expiry, performance and storage costs drift in the wrong direction.
3. Over-partitioning. Many partitions with little data each produce tiny files and slow queries.
4. Format flip-flopping. Switching formats after each vendor announcement burns engineering time. Decide on engines and workloads, not headlines.
5. Assuming UniForm is bidirectional. It is a read bridge from Delta to Iceberg and Hudi clients, and the documentation warns that external writes can cause data loss.
6. Treating spec features as universal. A capability in the Iceberg spec, such as v3 deletion vectors, may not be implemented in every engine you use. Check each one.
7. Benchmarking without tuning. Compare formats only after compacting and tuning each equally.

Frequently asked questions

Q: Is Iceberg replacing Delta Lake? A: There is no published measure showing that it has. Iceberg has broad cross-engine momentum, and Databricks has said it is working toward a common interoperability standard, but Delta remains the native format inside Databricks. Pick based on the engines and catalog you will actually run.

Q: What is the relationship between Tabular and Databricks? A: Databricks announced on June 4, 2024 that it would acquire Tabular, founded by Iceberg's original creators, and a June 7 update said the deal had closed. Databricks said it would pursue interoperability through Delta UniForm in the short term and aim for a common standard in the long term.

Q: Can I use Iceberg without Spark? A: Yes. The Iceberg project lists Spark, Trino, Flink, Presto, Hive, and Impala as engines that can read and write the same tables, and other engines add support over time. Check the specific engine and version for the features you need.

Q: Do I need a lakehouse format at all? A: Not always. If one warehouse holds all your data and you do not need open storage, the warehouse's native tables may be simpler. A table format earns its complexity when you need open storage, multiple engines, or very large tables on object storage. Our data engineering vs data science guide explains who typically owns these decisions.

Q: What is the difference between a table format and a file format? A: A file format such as Parquet defines how rows and columns are laid out inside a single file. A table format defines which files make up a table, how the schema and partitions are tracked, and how commits are made safe. Table formats sit on top of file formats.

Q: What about Apache Paimon and other newer formats? A: Other projects, such as Apache Paimon, target streaming-oriented lakehouse designs. This article covers the three formats with the widest platform support. If your workload is streaming-first, evaluate newer options with a pilot, and verify engine support before committing.

Q: How do I handle GDPR-style row deletions in a lakehouse? A: Each format supports row-level deletes through its mechanism (delete files or deletion vectors in Iceberg and Delta, delta logs in Hudi merge-on-read), but deleted data remains in older snapshots and files until expiry and physical cleanup run. Build snapshot expiry and file removal into the compliance process, and have legal review the retention design.

The takeaway

Lakehouse formats are three answers to one question: how do you make a pile of immutable files behave like a table. Iceberg answers with a tree of metadata and an atomic pointer swap, Delta Lake with an ordered log and checkpoints, and Hudi with a timeline built around record-level updates. The files underneath are mostly Parquet in every case, so the real differences show up in engine support, catalog fit, and how each handles frequent changes.

The 2024 announcements from Snowflake, Amazon, and Databricks moved the industry toward interoperability, but they did not erase the differences, and UniForm remains a read bridge, not a merge. Pick the format that matches your engines and write patterns, choose the catalog deliberately, schedule maintenance from day one, and test with your own data before trusting any published comparison.

---

Read next:

• Airflow vs Prefect vs Dagster 2026 — Orchestration Showdown
• Apache Spark Explained — Big Data Processing 2026
• Data Contracts 2026 — Why They Matter and How to Ship Them
• What Is DuckDB? In-Process Analytics SQL

Last verified: October 2026