michael1991 opened a new issue, #10906:
URL: https://github.com/apache/hudi/issues/10906
**Describe the problem you faced**
I have two jobs:
- JobA use spark3.3.2+hudi0.14.1 append batch log data;
- JobB use spark3.3.2+hudi0.12.3 to read log data produced by JobA.
Then error occurs in JobA like title.
**Expected behavior**
We dont write concurrently, we should not get this error.
**Environment Description**
* Hudi version : 0.14.1(single spark writer), 0.12.3(multi spark reader with
hoodie.metadata.enable to improve read performance)
* Spark version : 3.3.2
* Hive version : no
* Hadoop version : 3.3.6
* Storage (HDFS/S3/GCS..) : GCS
* Running on Docker? (yes/no) : no, Dataproc
**Stacktrace**
```2024-03-22 02:53:58 ERROR CleanActionExecutor:267 - Failed to perform
previous clean operation, instant:
[==>20240322023326699__clean__INFLIGHT__20240322023338718]
org.apache.hudi.exception.HoodieException: Failed to apply clean commit to
metadata
at
org.apache.hudi.table.action.BaseActionExecutor.writeTableMetadata(BaseActionExecutor.java:91)
~[hudi-spark3.3-bundle_2.12-0.14.1.jar:0.14.1]
at
org.apache.hudi.table.action.clean.CleanActionExecutor.runClean(CleanActionExecutor.java:227)
~[hudi-spark3.3-bundle_2.12-0.14.1.jar:0.14.1]
at
org.apache.hudi.table.action.clean.CleanActionExecutor.runPendingClean(CleanActionExecutor.java:193)
~[hudi-spark3.3-bundle_2.12-0.14.1.jar:0.14.1]
at
org.apache.hudi.table.action.clean.CleanActionExecutor.execute(CleanActionExecutor.java:263)
~[hudi-spark3.3-bundle_2.12-0.14.1.jar:0.14.1]
at
org.apache.hudi.table.HoodieSparkCopyOnWriteTable.clean(HoodieSparkCopyOnWriteTable.java:291)
~[hudi-spark3.3-bundle_2.12-0.14.1.jar:0.14.1]
at
org.apache.hudi.client.BaseHoodieTableServiceClient.clean(BaseHoodieTableServiceClient.java:763)
~[hudi-spark3.3-bundle_2.12-0.14.1.jar:0.14.1]
at
org.apache.hudi.client.BaseHoodieWriteClient.clean(BaseHoodieWriteClient.java:862)
~[hudi-spark3.3-bundle_2.12-0.14.1.jar:0.14.1]
at
org.apache.hudi.client.BaseHoodieWriteClient.clean(BaseHoodieWriteClient.java:835)
~[hudi-spark3.3-bundle_2.12-0.14.1.jar:0.14.1]
at
org.apache.hudi.async.AsyncCleanerService.lambda$startService$0(AsyncCleanerService.java:55)
~[hudi-spark3.3-bundle_2.12-0.14.1.jar:0.14.1]
at
java.util.concurrent.CompletableFuture$AsyncSupply.run(CompletableFuture.java:1700)
~[?:?]
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
~[?:?]
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
~[?:?]
at java.lang.Thread.run(Thread.java:829) ~[?:?]
Caused by: java.lang.IllegalArgumentException
at
org.apache.hudi.common.util.ValidationUtils.checkArgument(ValidationUtils.java:33)
~[hudi-spark3.3-bundle_2.12-0.14.1.jar:0.14.1]
at
org.apache.hudi.common.table.timeline.HoodieActiveTimeline.transitionState(HoodieActiveTimeline.java:618)
~[hudi-spark3.3-bundle_2.12-0.14.1.jar:0.14.1]
at
org.apache.hudi.common.table.timeline.HoodieActiveTimeline.transitionState(HoodieActiveTimeline.java:599)
~[hudi-spark3.3-bundle_2.12-0.14.1.jar:0.14.1]
at
org.apache.hudi.common.table.timeline.HoodieActiveTimeline.saveAsComplete(HoodieActiveTimeline.java:224)
~[hudi-spark3.3-bundle_2.12-0.14.1.jar:0.14.1]
at
org.apache.hudi.table.action.commit.BaseSparkCommitActionExecutor.commit(BaseSparkCommitActionExecutor.java:311)
~[hudi-spark3.3-bundle_2.12-0.14.1.jar:0.14.1]
at
org.apache.hudi.table.action.commit.BaseCommitActionExecutor.autoCommit(BaseCommitActionExecutor.java:201)
~[hudi-spark3.3-bundle_2.12-0.14.1.jar:0.14.1]
at
org.apache.hudi.table.action.commit.BaseCommitActionExecutor.commitOnAutoCommit(BaseCommitActionExecutor.java:183)
~[hudi-spark3.3-bundle_2.12-0.14.1.jar:0.14.1]
at
org.apache.hudi.table.action.commit.BaseSparkCommitActionExecutor.updateIndexAndCommitIfNeeded(BaseSparkCommitActionExecutor.java:279)
~[hudi-spark3.3-bundle_2.12-0.14.1.jar:0.14.1]
at
org.apache.hudi.table.action.commit.BaseSparkCommitActionExecutor.execute(BaseSparkCommitActionExecutor.java:184)
~[hudi-spark3.3-bundle_2.12-0.14.1.jar:0.14.1]
at
org.apache.hudi.table.action.deltacommit.SparkUpsertPreppedDeltaCommitActionExecutor.execute(SparkUpsertPreppedDeltaCommitActionExecutor.java:44)
~[hudi-spark3.3-bundle_2.12-0.14.1.jar:0.14.1]
at
org.apache.hudi.table.HoodieSparkMergeOnReadTable.upsertPrepped(HoodieSparkMergeOnReadTable.java:126)
~[hudi-spark3.3-bundle_2.12-0.14.1.jar:0.14.1]
at
org.apache.hudi.table.HoodieSparkMergeOnReadTable.upsertPrepped(HoodieSparkMergeOnReadTable.java:88)
~[hudi-spark3.3-bundle_2.12-0.14.1.jar:0.14.1]
at
org.apache.hudi.client.SparkRDDWriteClient.upsertPreppedRecords(SparkRDDWriteClient.java:156)
~[hudi-spark3.3-bundle_2.12-0.14.1.jar:0.14.1]
at
org.apache.hudi.client.SparkRDDWriteClient.upsertPreppedRecords(SparkRDDWriteClient.java:63)
~[hudi-spark3.3-bundle_2.12-0.14.1.jar:0.14.1]
at
org.apache.hudi.metadata.HoodieBackedTableMetadataWriter.commitInternal(HoodieBackedTableMetadataWriter.java:1099)
~[hudi-spark3.3-bundle_2.12-0.14.1.jar:0.14.1]
at
org.apache.hudi.metadata.SparkHoodieBackedTableMetadataWriter.commit(SparkHoodieBackedTableMetadataWriter.java:117)
~[hudi-spark3.3-bundle_2.12-0.14.1.jar:0.14.1]
at
org.apache.hudi.metadata.HoodieBackedTableMetadataWriter.processAndCommit(HoodieBackedTableMetadataWriter.java:810)
~[hudi-spark3.3-bundle_2.12-0.14.1.jar:0.14.1]
at
org.apache.hudi.metadata.HoodieBackedTableMetadataWriter.update(HoodieBackedTableMetadataWriter.java:899)
~[hudi-spark3.3-bundle_2.12-0.14.1.jar:0.14.1]
at
org.apache.hudi.table.action.BaseActionExecutor.writeTableMetadata(BaseActionExecutor.java:86)
~[hudi-spark3.3-bundle_2.12-0.14.1.jar:0.14.1]
... 12 more
2024-03-22 02:54:03 ERROR HoodieAsyncService:105 - Service shutdown with
error
java.util.concurrent.ExecutionException:
org.apache.hudi.exception.HoodieException: Failed to apply clean commit to
metadata
at
java.util.concurrent.CompletableFuture.reportGet(CompletableFuture.java:395)
~[?:?]
at
java.util.concurrent.CompletableFuture.get(CompletableFuture.java:1999) ~[?:?]
at
org.apache.hudi.async.HoodieAsyncService.waitForShutdown(HoodieAsyncService.java:103)
~[hudi-spark3.3-bundle_2.12-0.14.1.jar:0.14.1]
at
org.apache.hudi.async.AsyncCleanerService.waitForCompletion(AsyncCleanerService.java:75)
~[hudi-spark3.3-bundle_2.12-0.14.1.jar:0.14.1]
at
org.apache.hudi.client.BaseHoodieTableServiceClient.asyncClean(BaseHoodieTableServiceClient.java:133)
~[hudi-spark3.3-bundle_2.12-0.14.1.jar:0.14.1]
at
org.apache.hudi.client.BaseHoodieWriteClient.autoCleanOnCommit(BaseHoodieWriteClient.java:595)
~[hudi-spark3.3-bundle_2.12-0.14.1.jar:0.14.1]
at
org.apache.hudi.client.BaseHoodieWriteClient.mayBeCleanAndArchive(BaseHoodieWriteClient.java:579)
~[hudi-spark3.3-bundle_2.12-0.14.1.jar:0.14.1]
at
org.apache.hudi.client.BaseHoodieWriteClient.commitStats(BaseHoodieWriteClient.java:248)
~[hudi-spark3.3-bundle_2.12-0.14.1.jar:0.14.1]
at
org.apache.hudi.client.SparkRDDWriteClient.commit(SparkRDDWriteClient.java:104)
~[hudi-spark3.3-bundle_2.12-0.14.1.jar:0.14.1]
at
org.apache.hudi.HoodieSparkSqlWriterInternal.commitAndPerformPostOperations(HoodieSparkSqlWriter.scala:1081)
~[hudi-spark3.3-bundle_2.12-0.14.1.jar:0.14.1]
at
org.apache.hudi.HoodieSparkSqlWriterInternal.writeInternal(HoodieSparkSqlWriter.scala:520)
~[hudi-spark3.3-bundle_2.12-0.14.1.jar:0.14.1]
at
org.apache.hudi.HoodieSparkSqlWriterInternal.write(HoodieSparkSqlWriter.scala:204)
~[hudi-spark3.3-bundle_2.12-0.14.1.jar:0.14.1]
at
org.apache.hudi.HoodieSparkSqlWriter$.write(HoodieSparkSqlWriter.scala:121)
~[hudi-spark3.3-bundle_2.12-0.14.1.jar:0.14.1]
at
org.apache.hudi.DefaultSource.createRelation(DefaultSource.scala:150)
~[hudi-spark3.3-bundle_2.12-0.14.1.jar:0.14.1]
at
org.apache.spark.sql.execution.datasources.SaveIntoDataSourceCommand.run(SaveIntoDataSourceCommand.scala:47)
~[spark-sql_2.12-3.3.2.jar:0.14.1]
at
org.apache.spark.sql.execution.command.ExecutedCommandExec.sideEffectResult$lzycompute(commands.scala:75)
~[spark-sql_2.12-3.3.2.jar:3.3.2]
at
org.apache.spark.sql.execution.command.ExecutedCommandExec.sideEffectResult(commands.scala:73)
~[spark-sql_2.12-3.3.2.jar:3.3.2]
at
org.apache.spark.sql.execution.command.ExecutedCommandExec.executeCollect(commands.scala:84)
~[spark-sql_2.12-3.3.2.jar:3.3.2]
at
org.apache.spark.sql.execution.QueryExecution$$anonfun$eagerlyExecuteCommands$1.$anonfun$applyOrElse$1(QueryExecution.scala:98)
~[spark-sql_2.12-3.3.2.jar:0.14.1]
at
org.apache.spark.sql.execution.SQLExecution$.$anonfun$withNewExecutionId$6(SQLExecution.scala:109)
~[spark-sql_2.12-3.3.2.jar:0.14.1]
at
org.apache.spark.sql.execution.SQLExecution$.withSQLConfPropagated(SQLExecution.scala:169)
~[spark-sql_2.12-3.3.2.jar:0.14.1]
at
org.apache.spark.sql.execution.SQLExecution$.$anonfun$withNewExecutionId$1(SQLExecution.scala:95)
~[spark-sql_2.12-3.3.2.jar:0.14.1]
at org.apache.spark.sql.SparkSession.withActive(SparkSession.scala:779)
~[spark-sql_2.12-3.3.2.jar:3.3.2]
at
org.apache.spark.sql.execution.SQLExecution$.withNewExecutionId(SQLExecution.scala:64)
~[spark-sql_2.12-3.3.2.jar:0.14.1]
at
org.apache.spark.sql.execution.QueryExecution$$anonfun$eagerlyExecuteCommands$1.applyOrElse(QueryExecution.scala:98)
~[spark-sql_2.12-3.3.2.jar:0.14.1]
at
org.apache.spark.sql.execution.QueryExecution$$anonfun$eagerlyExecuteCommands$1.applyOrElse(QueryExecution.scala:94)
~[spark-sql_2.12-3.3.2.jar:0.14.1]
at
org.apache.spark.sql.catalyst.trees.TreeNode.$anonfun$transformDownWithPruning$1(TreeNode.scala:584)
~[spark-catalyst_2.12-3.3.2.jar:3.3.2]
at
org.apache.spark.sql.catalyst.trees.CurrentOrigin$.withOrigin(TreeNode.scala:176)
~[spark-catalyst_2.12-3.3.2.jar:3.3.2]
at
org.apache.spark.sql.catalyst.trees.TreeNode.transformDownWithPruning(TreeNode.scala:584)
~[spark-catalyst_2.12-3.3.2.jar:3.3.2]
at
org.apache.spark.sql.catalyst.plans.logical.LogicalPlan.org$apache$spark$sql$catalyst$plans$logical$AnalysisHelper$$super$transformDownWithPruning(LogicalPlan.scala:30)
~[spark-catalyst_2.12-3.3.2.jar:3.3.2]
at
org.apache.spark.sql.catalyst.plans.logical.AnalysisHelper.transformDownWithPruning(AnalysisHelper.scala:267)
~[spark-catalyst_2.12-3.3.2.jar:3.3.2]
at
org.apache.spark.sql.catalyst.plans.logical.AnalysisHelper.transformDownWithPruning$(AnalysisHelper.scala:263)
~[spark-catalyst_2.12-3.3.2.jar:3.3.2]
at
org.apache.spark.sql.catalyst.plans.logical.LogicalPlan.transformDownWithPruning(LogicalPlan.scala:30)
~[spark-catalyst_2.12-3.3.2.jar:3.3.2]
at
org.apache.spark.sql.catalyst.plans.logical.LogicalPlan.transformDownWithPruning(LogicalPlan.scala:30)
~[spark-catalyst_2.12-3.3.2.jar:3.3.2]
at
org.apache.spark.sql.catalyst.trees.TreeNode.transformDown(TreeNode.scala:560)
~[spark-catalyst_2.12-3.3.2.jar:3.3.2]
at
org.apache.spark.sql.execution.QueryExecution.eagerlyExecuteCommands(QueryExecution.scala:94)
~[spark-sql_2.12-3.3.2.jar:0.14.1]
at
org.apache.spark.sql.execution.QueryExecution.commandExecuted$lzycompute(QueryExecution.scala:81)
~[spark-sql_2.12-3.3.2.jar:0.14.1]
at
org.apache.spark.sql.execution.QueryExecution.commandExecuted(QueryExecution.scala:79)
~[spark-sql_2.12-3.3.2.jar:0.14.1]
at
org.apache.spark.sql.execution.QueryExecution.assertCommandExecuted(QueryExecution.scala:116)
~[spark-sql_2.12-3.3.2.jar:0.14.1]
at
org.apache.spark.sql.DataFrameWriter.runCommand(DataFrameWriter.scala:860)
~[spark-sql_2.12-3.3.2.jar:3.3.2]
at
org.apache.spark.sql.DataFrameWriter.saveToV1Source(DataFrameWriter.scala:390)
~[spark-sql_2.12-3.3.2.jar:3.3.2]
at
org.apache.spark.sql.DataFrameWriter.saveInternal(DataFrameWriter.scala:363)
~[spark-sql_2.12-3.3.2.jar:3.3.2]
at org.apache.spark.sql.DataFrameWriter.save(DataFrameWriter.scala:239)
~[spark-sql_2.12-3.3.2.jar:3.3.2]
at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
~[?:?]
at
jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
~[?:?]
at
jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
~[?:?]
at java.lang.reflect.Method.invoke(Method.java:566) ~[?:?]
at
org.apache.spark.deploy.JavaMainApplication.start(SparkApplication.scala:52)
~[spark-core_2.12-3.3.2.jar:3.3.2]
at
org.apache.spark.deploy.SparkSubmit.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:973)
~[spark-core_2.12-3.3.2.jar:3.3.2]
at
org.apache.spark.deploy.SparkSubmit.doRunMain$1(SparkSubmit.scala:180)
~[spark-core_2.12-3.3.2.jar:3.3.2]
at org.apache.spark.deploy.SparkSubmit.submit(SparkSubmit.scala:203)
~[spark-core_2.12-3.3.2.jar:3.3.2]
at org.apache.spark.deploy.SparkSubmit.doSubmit(SparkSubmit.scala:90)
~[spark-core_2.12-3.3.2.jar:3.3.2]
at
org.apache.spark.deploy.SparkSubmit$$anon$2.doSubmit(SparkSubmit.scala:1061)
~[spark-core_2.12-3.3.2.jar:3.3.2]
at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:1070)
~[spark-core_2.12-3.3.2.jar:3.3.2]
at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
~[spark-core_2.12-3.3.2.jar:3.3.2]
Caused by: org.apache.hudi.exception.HoodieException: Failed to apply clean
commit to metadata
at
org.apache.hudi.table.action.BaseActionExecutor.writeTableMetadata(BaseActionExecutor.java:91)
~[hudi-spark3.3-bundle_2.12-0.14.1.jar:0.14.1]
at
org.apache.hudi.table.action.clean.CleanActionExecutor.runClean(CleanActionExecutor.java:227)
~[hudi-spark3.3-bundle_2.12-0.14.1.jar:0.14.1]
at
org.apache.hudi.table.action.clean.CleanActionExecutor.runPendingClean(CleanActionExecutor.java:193)
~[hudi-spark3.3-bundle_2.12-0.14.1.jar:0.14.1]
at
org.apache.hudi.table.action.clean.CleanActionExecutor.execute(CleanActionExecutor.java:263)
~[hudi-spark3.3-bundle_2.12-0.14.1.jar:0.14.1]
at
org.apache.hudi.table.HoodieSparkCopyOnWriteTable.clean(HoodieSparkCopyOnWriteTable.java:291)
~[hudi-spark3.3-bundle_2.12-0.14.1.jar:0.14.1]
at
org.apache.hudi.client.BaseHoodieTableServiceClient.clean(BaseHoodieTableServiceClient.java:763)
~[hudi-spark3.3-bundle_2.12-0.14.1.jar:0.14.1]
at
org.apache.hudi.client.BaseHoodieWriteClient.clean(BaseHoodieWriteClient.java:862)
~[hudi-spark3.3-bundle_2.12-0.14.1.jar:0.14.1]
at
org.apache.hudi.client.BaseHoodieWriteClient.clean(BaseHoodieWriteClient.java:835)
~[hudi-spark3.3-bundle_2.12-0.14.1.jar:0.14.1]
at
org.apache.hudi.async.AsyncCleanerService.lambda$startService$0(AsyncCleanerService.java:55)
~[hudi-spark3.3-bundle_2.12-0.14.1.jar:0.14.1]
at
java.util.concurrent.CompletableFuture$AsyncSupply.run(CompletableFuture.java:1700)
~[?:?]
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
~[?:?]
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
~[?:?]
at java.lang.Thread.run(Thread.java:829) ~[?:?]
Caused by: java.lang.IllegalArgumentException
at
org.apache.hudi.common.util.ValidationUtils.checkArgument(ValidationUtils.java:33)
~[hudi-spark3.3-bundle_2.12-0.14.1.jar:0.14.1]
at
org.apache.hudi.common.table.timeline.HoodieActiveTimeline.transitionState(HoodieActiveTimeline.java:618)
~[hudi-spark3.3-bundle_2.12-0.14.1.jar:0.14.1]
at
org.apache.hudi.common.table.timeline.HoodieActiveTimeline.transitionState(HoodieActiveTimeline.java:599)
~[hudi-spark3.3-bundle_2.12-0.14.1.jar:0.14.1]
at
org.apache.hudi.common.table.timeline.HoodieActiveTimeline.saveAsComplete(HoodieActiveTimeline.java:224)
~[hudi-spark3.3-bundle_2.12-0.14.1.jar:0.14.1]
at
org.apache.hudi.table.action.commit.BaseSparkCommitActionExecutor.commit(BaseSparkCommitActionExecutor.java:311)
~[hudi-spark3.3-bundle_2.12-0.14.1.jar:0.14.1]
at
org.apache.hudi.table.action.commit.BaseCommitActionExecutor.autoCommit(BaseCommitActionExecutor.java:201)
~[hudi-spark3.3-bundle_2.12-0.14.1.jar:0.14.1]
at
org.apache.hudi.table.action.commit.BaseCommitActionExecutor.commitOnAutoCommit(BaseCommitActionExecutor.java:183)
~[hudi-spark3.3-bundle_2.12-0.14.1.jar:0.14.1]
at
org.apache.hudi.table.action.commit.BaseSparkCommitActionExecutor.updateIndexAndCommitIfNeeded(BaseSparkCommitActionExecutor.java:279)
~[hudi-spark3.3-bundle_2.12-0.14.1.jar:0.14.1]
at
org.apache.hudi.table.action.commit.BaseSparkCommitActionExecutor.execute(BaseSparkCommitActionExecutor.java:184)
~[hudi-spark3.3-bundle_2.12-0.14.1.jar:0.14.1]
at
org.apache.hudi.table.action.deltacommit.SparkUpsertPreppedDeltaCommitActionExecutor.execute(SparkUpsertPreppedDeltaCommitActionExecutor.java:44)
~[hudi-spark3.3-bundle_2.12-0.14.1.jar:0.14.1]
at
org.apache.hudi.table.HoodieSparkMergeOnReadTable.upsertPrepped(HoodieSparkMergeOnReadTable.java:126)
~[hudi-spark3.3-bundle_2.12-0.14.1.jar:0.14.1]
at
org.apache.hudi.table.HoodieSparkMergeOnReadTable.upsertPrepped(HoodieSparkMergeOnReadTable.java:88)
~[hudi-spark3.3-bundle_2.12-0.14.1.jar:0.14.1]
at
org.apache.hudi.client.SparkRDDWriteClient.upsertPreppedRecords(SparkRDDWriteClient.java:156)
~[hudi-spark3.3-bundle_2.12-0.14.1.jar:0.14.1]
at
org.apache.hudi.client.SparkRDDWriteClient.upsertPreppedRecords(SparkRDDWriteClient.java:63)
~[hudi-spark3.3-bundle_2.12-0.14.1.jar:0.14.1]
at
org.apache.hudi.metadata.HoodieBackedTableMetadataWriter.commitInternal(HoodieBackedTableMetadataWriter.java:1099)
~[hudi-spark3.3-bundle_2.12-0.14.1.jar:0.14.1]
at
org.apache.hudi.metadata.SparkHoodieBackedTableMetadataWriter.commit(SparkHoodieBackedTableMetadataWriter.java:117)
~[hudi-spark3.3-bundle_2.12-0.14.1.jar:0.14.1]
at
org.apache.hudi.metadata.HoodieBackedTableMetadataWriter.processAndCommit(HoodieBackedTableMetadataWriter.java:810)
~[hudi-spark3.3-bundle_2.12-0.14.1.jar:0.14.1]
at
org.apache.hudi.metadata.HoodieBackedTableMetadataWriter.update(HoodieBackedTableMetadataWriter.java:899)
~[hudi-spark3.3-bundle_2.12-0.14.1.jar:0.14.1]
at
org.apache.hudi.table.action.BaseActionExecutor.writeTableMetadata(BaseActionExecutor.java:86)
~[hudi-spark3.3-bundle_2.12-0.14.1.jar:0.14.1]
at
org.apache.hudi.table.action.clean.CleanActionExecutor.runClean(CleanActionExecutor.java:227)
~[hudi-spark3.3-bundle_2.12-0.14.1.jar:0.14.1]
at
org.apache.hudi.table.action.clean.CleanActionExecutor.runPendingClean(CleanActionExecutor.java:193)
~[hudi-spark3.3-bundle_2.12-0.14.1.jar:0.14.1]
at
org.apache.hudi.table.action.clean.CleanActionExecutor.execute(CleanActionExecutor.java:263)
~[hudi-spark3.3-bundle_2.12-0.14.1.jar:0.14.1]
at
org.apache.hudi.table.HoodieSparkCopyOnWriteTable.clean(HoodieSparkCopyOnWriteTable.java:291)
~[hudi-spark3.3-bundle_2.12-0.14.1.jar:0.14.1]
at
org.apache.hudi.client.BaseHoodieTableServiceClient.clean(BaseHoodieTableServiceClient.java:763)
~[hudi-spark3.3-bundle_2.12-0.14.1.jar:0.14.1]
at
org.apache.hudi.client.BaseHoodieWriteClient.clean(BaseHoodieWriteClient.java:862)
~[hudi-spark3.3-bundle_2.12-0.14.1.jar:0.14.1]
at
org.apache.hudi.client.BaseHoodieWriteClient.clean(BaseHoodieWriteClient.java:835)
~[hudi-spark3.3-bundle_2.12-0.14.1.jar:0.14.1]
at
org.apache.hudi.async.AsyncCleanerService.lambda$startService$0(AsyncCleanerService.java:55)
~[hudi-spark3.3-bundle_2.12-0.14.1.jar:0.14.1]
at
java.util.concurrent.CompletableFuture$AsyncSupply.run(CompletableFuture.java:1700)
~[?:?]
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
~[?:?]
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
~[?:?]
at java.lang.Thread.run(Thread.java:829) ~[?:?]
2024-03-22 02:54:03 WARN HoodieSparkSqlWriterInternal:534 - Closing write
client```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
