Advanced Concepts in Iceberg Table Design

Designing efficient Iceberg tables involves key decisions about partitioning, sorting, and retention to optimize query speed, ingestion latency, and storage costs. These have traditionally required data engineering know-how and expertise to implement and maintain as the number of tables increase and query patterns evolve. 

In particular to optimal performance are the careful adjustments required to manage high-cardinality columns, data skew, and value density. These factors directly impact read and write efficiency, where even small adjustments can drive significant gains in performance and storage reduction.

In this session, we’ll dive into advanced strategies for Iceberg table partitioning and sorting, concluding with an introduction to Upsolver’s Adaptive Clustering – a dynamic solution for table partitioning.

What You’ll Learn:

• • Challenges with current approaches to partitioning, sorting and clustering
  • Performance and cost impacts of high cardinality and skewed data
  • Drawbacks of common, best practice, partitioning approaches
  • How Apache Iceberg improves on these common best practices
  • How Adaptive Clustering solves these challenges by automating table layouts decisions

Presented by: