danny0405 commented on code in PR #13422:
URL: https://github.com/apache/hudi/pull/13422#discussion_r2141566356
##########
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/async/AsyncClusteringService.java:
##########
@@ -103,6 +103,12 @@ protected Pair<CompletableFuture, ExecutorService>
startService() {
* Update the write client to be used for clustering.
*/
public synchronized void updateWriteClient(BaseHoodieWriteClient
writeClient) {
- this.clusteringClient.updateWriteClient(writeClient);
+ this.clusteringClient.updateWriteClient(writeClient.createNewClient());
Review Comment:
If we also schedules a plan in the async compaction/clustering service, then
this fix might be necessary, but let's also keep caution to the double memory
occupation for the fs view on the driver here.
Is it possible we move the scheduling into the writer itself and the service
is only responsible for execution?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
