Hi Joseph, You may find everything about partitioning and bucketing under https://iceberg.apache.org/spec/#partition-transforms. I don't think we can add new bucketing functions now. I'm also curious whether we can have bucketing functions at table definition such that partitioning will be consistent in read/write across engines.
Wikimedia is awesome! Regards, Manu On Tue, Jul 4, 2023 at 9:08 PM Joseph Allemandou <jalleman...@wikimedia.org> wrote: > Hi Iceberg team, > > I'm working at the WikimediaFoundation, and we started using Iceberg for > some of our big-data tables - we love it :) > > One of the needs we'll have in the future would be to partition data using > a specific bucketing function. > How complex would that be to add a new function to the ones already > present in the Iceberg partitioning mechanism? Is there any docs on doing > that? > Bonus points: Are there any plans to make it possible for users to > reference their own bucketing functions at table definition? > > Many thanks for the awesome project<3 > > -- > Joseph Allemandou (joal) (he / him) > Staff Data Engineer > Wikimedia Foundation >
