umehrot2 commented on issue #1764:
URL: https://github.com/apache/hudi/issues/1764#issuecomment-649928569
Wow, I just started running into this same issue as I was running
performance tests for RFC-12 bootstrap implementation. cc @bvaradar
```
org.apache.hudi.exception.HoodieCommitException: Failed to complete commit
00000000000001 due to finalize errors.
at
org.apache.hudi.table.action.commit.BaseCommitActionExecutor.finalizeWrite(BaseCommitActionExecutor.java:255)
at
org.apache.hudi.table.action.commit.BaseCommitActionExecutor.commit(BaseCommitActionExecutor.java:216)
at
org.apache.hudi.table.action.bootstrap.BootstrapCommitActionExecutor.commit(BootstrapCommitActionExecutor.java:205)
at
org.apache.hudi.table.action.commit.BaseCommitActionExecutor.commitOnAutoCommit(BaseCommitActionExecutor.java:178)
at
org.apache.hudi.table.action.commit.BaseCommitActionExecutor.updateIndexAndCommitIfNeeded(BaseCommitActionExecutor.java:172)
at
org.apache.hudi.table.action.bootstrap.BootstrapCommitActionExecutor.metadataBootstrap(BootstrapCommitActionExecutor.java:178)
at
org.apache.hudi.table.action.bootstrap.BootstrapCommitActionExecutor.execute(BootstrapCommitActionExecutor.java:136)
at
org.apache.hudi.table.HoodieCopyOnWriteTable.bootstrap(HoodieCopyOnWriteTable.java:143)
at
org.apache.hudi.client.HoodieWriteClient.bootstrap(HoodieWriteClient.java:158)
at
org.apache.hudi.HoodieSparkSqlWriter$.bootstrap(HoodieSparkSqlWriter.scala:203)
at org.apache.hudi.DefaultSource.createRelation(DefaultSource.scala:139)
at
org.apache.spark.sql.execution.datasources.SaveIntoDataSourceCommand.run(SaveIntoDataSourceCommand.scala:45)
at
org.apache.spark.sql.execution.command.ExecutedCommandExec.sideEffectResult$lzycompute(commands.scala:70)
at
org.apache.spark.sql.execution.command.ExecutedCommandExec.sideEffectResult(commands.scala:68)
at
org.apache.spark.sql.execution.command.ExecutedCommandExec.doExecute(commands.scala:86)
at
org.apache.spark.sql.execution.SparkPlan$$anonfun$execute$1.apply(SparkPlan.scala:173)
at
org.apache.spark.sql.execution.SparkPlan$$anonfun$execute$1.apply(SparkPlan.scala:169)
at
org.apache.spark.sql.execution.SparkPlan$$anonfun$executeQuery$1.apply(SparkPlan.scala:197)
at
org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:151)
at
org.apache.spark.sql.execution.SparkPlan.executeQuery(SparkPlan.scala:194)
at org.apache.spark.sql.execution.SparkPlan.execute(SparkPlan.scala:169)
at
org.apache.spark.sql.execution.QueryExecution.toRdd$lzycompute(QueryExecution.scala:114)
at
org.apache.spark.sql.execution.QueryExecution.toRdd(QueryExecution.scala:112)
at
org.apache.spark.sql.DataFrameWriter$$anonfun$runCommand$1.apply(DataFrameWriter.scala:676)
at
org.apache.spark.sql.DataFrameWriter$$anonfun$runCommand$1.apply(DataFrameWriter.scala:676)
at
org.apache.spark.sql.execution.SQLExecution$.org$apache$spark$sql$execution$SQLExecution$$executeQuery$1(SQLExecution.scala:83)
at
org.apache.spark.sql.execution.SQLExecution$$anonfun$withNewExecutionId$1$$anonfun$apply$1.apply(SQLExecution.scala:94)
at
org.apache.spark.sql.execution.QueryExecutionMetrics$.withMetrics(QueryExecutionMetrics.scala:141)
at
org.apache.spark.sql.execution.SQLExecution$.org$apache$spark$sql$execution$SQLExecution$$withMetrics(SQLExecution.scala:178)
at
org.apache.spark.sql.execution.SQLExecution$$anonfun$withNewExecutionId$1.apply(SQLExecution.scala:93)
at
org.apache.spark.sql.execution.SQLExecution$.withSQLConfPropagated(SQLExecution.scala:200)
at
org.apache.spark.sql.execution.SQLExecution$.withNewExecutionId(SQLExecution.scala:92)
at
org.apache.spark.sql.DataFrameWriter.runCommand(DataFrameWriter.scala:676)
at
org.apache.spark.sql.DataFrameWriter.saveToV1Source(DataFrameWriter.scala:285)
at org.apache.spark.sql.DataFrameWriter.save(DataFrameWriter.scala:271)
at org.apache.spark.sql.DataFrameWriter.save(DataFrameWriter.scala:229)
... 49 elided
Caused by: org.apache.hudi.exception.HoodieIOException: Consistency check
failed to ensure all files APPEAR
at org.apache.hudi.table.HoodieTable.waitForAllFiles(HoodieTable.java:553)
at
org.apache.hudi.table.HoodieTable.cleanFailedWrites(HoodieTable.java:495)
at org.apache.hudi.table.HoodieTable.finalizeWrite(HoodieTable.java:398)
at
org.apache.hudi.table.action.commit.BaseCommitActionExecutor.finalizeWrite(BaseCommitActionExecutor.java:252)
... 84 more
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
