advancedxy commented on issue #5626: URL: https://github.com/apache/iceberg/issues/5626#issuecomment-1640074625
> @vamen, you can just add multiple bucket partitions to bucket by more than one column. We chose to do this rather than hash the fields together so they can be used together or independently. Hi, @rdblue I think we cannot express the same semantic for bucket transform on multiple columns with multiple bucket partitions. For example `bucket(16, id_col, sec_col)` means that we need to bucket the `(id_col, sec_col)` into 16 buckets. If we are going to replace it with `bucket(16, id_col) + bucket(16, sec_col)`, it would create 16 * 16 = 256 buckets. The most similar bucket spec would be `bucket(4, id_col) + bucket(4, sec_col)`, which creates 4 * 4 = 16 buckets. However that assumes the cardinality of bucket columns are balanced. It might also possible be `bucket(2, id_col) + bucket(8, sec_col)`. Did I miss anything? Therefore, I believe it's still valuable to support bucketing on multiple columns, especially for primary key is made of multiple columns. WDYT? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
