Victor Sadkov created HUDI-5150:
-----------------------------------
Summary: False write success status on some errors
Key: HUDI-5150
URL: https://issues.apache.org/jira/browse/HUDI-5150
Project: Apache Hudi
Issue Type: Bug
Components: core
Reporter: Victor SadkovHoodieSparkSqlWriter [returns a success status|https://github.com/apache/hudi/blob/master/hudi-spark-datasource/hudi-spark-common/src/main/scala/org/apache/hudi/HoodieSparkSqlWriter.scala#L337] in its {{write()}} method instead of throwing an exception when something fails (and such an error can legitimately happen, for example [here|https://github.com/apache/hudi/blob/master/hudi-spark-datasource/hudi-spark-common/src/main/scala/org/apache/hudi/HoodieSparkSqlWriter.scala#L767]). And there is a multitude of places where return values of this method are ignored (unit tests were omitted, but there is a lot of such cases over there as well): [https://github.com/apache/hudi/blob/master/rfc/rfc-38/rfc-38.md#L77] [https://github.com/apache/hudi/blob/master/hudi-spark-datasource/hudi-spark/src/main/scala/org/apache/spark/sql/hudi/command/MergeIntoHoodieTableCommand.scala#L351] [https://github.com/apache/hudi/blob/master/hudi-spark-datasource/hudi-spark-common/src/main/scala/org/apache/hudi/DefaultSource.scala#L148] [https://github.com/apache/hudi/blob/master/hudi-spark-datasource/hudi-spark-common/src/main/scala/org/apache/hudi/HoodieStreamingSink.scala#L128] [https://github.com/apache/hudi/blob/master/hudi-spark-datasource/hudi-spark-common/src/main/scala/org/apache/spark/sql/hudi/command/AlterHoodieTableDropPartitionCommand.scala#L62] [https://github.com/apache/hudi/blob/master/hudi-spark-datasource/hudi-spark-common/src/main/scala/org/apache/spark/sql/hudi/command/TruncateHoodieTableCommand.scala#L88] This makes Hudi to swallow an error and report being successful on some write failures. This issue actually happened in our infrastructure, but I don't have a good example to reproduce it, unfortunately. For me it feels like a critical issue, please adjust the priority accordingly if I was not right :) -- This message was sent by Atlassian Jira (v8.20.10#820010)
