alexeykudinkin commented on code in PR #7396:
URL: https://github.com/apache/hudi/pull/7396#discussion_r1044804504
##########
hudi-client/hudi-spark-client/src/main/java/org/apache/hudi/execution/bulkinsert/NonSortPartitioner.java:
##########
@@ -25,18 +25,37 @@
import org.apache.spark.api.java.JavaRDD;
/**
- * A built-in partitioner that only does coalesce for input records for bulk
insert operation,
- * corresponding to the {@code BulkInsertSortMode.NONE} mode.
+ * A built-in partitioner that avoids expensive sorting for the input records
for bulk insert
+ * operation, by doing either of the following:
+ * <p>
+ * - If respecting the outputSparkPartitions, only does coalesce for input
records;
+ * <p>
+ * - Otherwise, returns input records as is.
+ * <p>
+ * Corresponds to the {@code BulkInsertSortMode.NONE} mode.
*
* @param <T> HoodieRecordPayload type
*/
public class NonSortPartitioner<T extends HoodieRecordPayload>
implements BulkInsertPartitioner<JavaRDD<HoodieRecord<T>>> {
+ private final boolean mustRespectNumOutputPartitions;
Review Comment:
nit: how about `enforceNumOutputPartitions`?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
