[
https://issues.apache.org/jira/browse/HUDI-1427?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vinoth Chandar closed HUDI-1427.
--------------------------------
Resolution: Fixed
> Throw a FileAlreadyExistsException when set HOODIE_AUTO_COMMIT_PROP to true
> ---------------------------------------------------------------------------
>
> Key: HUDI-1427
> URL: https://issues.apache.org/jira/browse/HUDI-1427
> Project: Apache Hudi
> Issue Type: Bug
> Components: Spark Integration
> Reporter: pengzhiwei
> Assignee: pengzhiwei
> Priority: Major
> Labels: pull-request-available
> Fix For: 0.7.0
>
>
> I have test write hudi with COW & MOR write type. When set
> HOODIE_AUTO_COMMIT_PROP to true, a FileAlreadyExistsException throw out:
> {code:java}
> Exception in thread "main" org.apache.hudi.exception.HoodieIOException:
> Failed to create file
> /tmp/hudi/tbl_price_cow1/.hoodie/20201202104150.commitException in thread
> "main" org.apache.hudi.exception.HoodieIOException: Failed to create file
> /tmp/hudi/tbl_price_cow1/.hoodie/20201202104150.commit at
> org.apache.hudi.common.table.timeline.HoodieActiveTimeline.createImmutableFileInPath(HoodieActiveTimeline.java:474)
> at
> org.apache.hudi.common.table.timeline.HoodieActiveTimeline.transitionState(HoodieActiveTimeline.java:350)
> at
> org.apache.hudi.common.table.timeline.HoodieActiveTimeline.transitionState(HoodieActiveTimeline.java:325)
> at
> org.apache.hudi.common.table.timeline.HoodieActiveTimeline.saveAsComplete(HoodieActiveTimeline.java:144)
> at
> org.apache.hudi.client.AbstractHoodieWriteClient.commitStats(AbstractHoodieWriteClient.java:181)
> at
> org.apache.hudi.client.SparkRDDWriteClient.commit(SparkRDDWriteClient.java:101)
> at
> org.apache.hudi.HoodieSparkSqlWriter$.commitAndPerformPostOperations(HoodieSparkSqlWriter.scala:413)
> at
> org.apache.hudi.HoodieSparkSqlWriter$.write(HoodieSparkSqlWriter.scala:210)
> at org.apache.hudi.DefaultSource.createRelation(DefaultSource.scala:125) at
> org.apache.spark.sql.execution.datasources.SaveIntoDataSourceCommand.run(SaveIntoDataSourceCommand.scala:46)
> at
> org.apache.spark.sql.execution.command.ExecutedCommandExec.sideEffectResult$lzycompute(commands.scala:70)
> at
> org.apache.spark.sql.execution.command.ExecutedCommandExec.sideEffectResult(commands.scala:68)
> at
> org.apache.spark.sql.execution.command.ExecutedCommandExec.doExecute(commands.scala:86)
> at
> org.apache.spark.sql.execution.SparkPlan$$anonfun$execute$1.apply(SparkPlan.scala:137)
> at
> org.apache.spark.sql.execution.SparkPlan$$anonfun$execute$1.apply(SparkPlan.scala:133)
> at
> org.apache.spark.sql.execution.SparkPlan$$anonfun$executeQuery$1.apply(SparkPlan.scala:161)
> at
> org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:151)
> at
> org.apache.spark.sql.execution.SparkPlan.executeQuery(SparkPlan.scala:158) at
> org.apache.spark.sql.execution.SparkPlan.execute(SparkPlan.scala:133)
> {code}
> It seems that _../.hoodie/20201202104150.commit_ has commit twice and result
> in this exception.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
