codope commented on a change in pull request #3833:
URL: https://github.com/apache/hudi/pull/3833#discussion_r740184018
##########
File path:
hudi-client/hudi-spark-client/src/main/java/org/apache/hudi/execution/bulkinsert/BulkInsertMapFunction.java
##########
@@ -60,8 +63,19 @@ public BulkInsertMapFunction(String instantTime, boolean
areRecordsSorted,
@Override
public Iterator<List<WriteStatus>> call(Integer partition,
Iterator<HoodieRecord<T>> recordItr) {
+ // Use SingleFileHandleCreateFactory when clustering plan is
SparkSingleFileSortPlanStrategy,
+ // and execution strategy is SparkSingleFileSortExecutionStrategy.
+ CreateHandleFactory writeHandleFactory;
+ if
(config.getClusteringPlanStrategyClass().equals(SparkSingleFileSortPlanStrategy.class.getName())
+ &&
config.getClusteringExecutionStrategyClass().equals(SparkSingleFileSortExecutionStrategy.class.getName()))
{
Review comment:
I thought of doing that but the problem is WriteHandleFactory is not
serializable. I will have to make it serializable and change the bulkInsert()
API. Let me try to refactor.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
