Hi all, As per https://iceberg.apache.org/docs/latest/spark-writes/#writing-to-partitioned-tables, sort is required for Spark writing to a partitioned table. Does anyone know the reason behind it? If this is to avoid creating too many small files, isn't shuffle/repartition sufficient?
Thanks, Manu
