Ytimetravel opened a new issue, #11835: URL: https://github.com/apache/hudi/issues/11835
**Describe the problem you faced** Dear community, Recently I discovered a case: a write failure can cause the hoodi.properties file corrupted. Problem site: <img width="1210" alt="image" src="https://github.com/user-attachments/assets/a8301a1a-749e-4a3e-bb46-e1a9cb038097";>It causes other write tasks to fail. The process in which this situation occurs is as follows: 1. Executing the commit will trigger the maybeDeleteMetadataTable process.（If need） <img width="904" alt="image" src="https://github.com/user-attachments/assets/d8a1617b-606b-4a58-9207-725ac09f663e";> 2. An exception occurred during the following process, causing the properties file write to fail. <img width="1319" alt="image" src="https://github.com/user-attachments/assets/b837aafe-4ee0-486f-9a27-ad19cd4fcc58";> <img width="1359" alt="image" src="https://github.com/user-attachments/assets/2b614a2e-7d98-4553-b972-439a98234048";> <img width="1354" alt="image" src="https://github.com/user-attachments/assets/9d9ba22b-4e9c-4c42-a722-f8d82c2c0162";> **File status：properties error（len=0） properties_backup error-free** 3. Then it triggers rollback. <img width="1306" alt="image" src="https://github.com/user-attachments/assets/86e5338c-8083-4f14-a02f-06d8eee9be5e";> <img width="1237" alt="image" src="https://github.com/user-attachments/assets/495cab27-fbb0-4e36-976f-c8c0553db039";> <img width="1292" alt="image" src="https://github.com/user-attachments/assets/9764cbfc-43f9-4a70-98e5-6e3064587cb0";> 4. Since the table version cannot be correctly obtained at this point, it triggers an upgrade from 0 to 6. <img width="1375" alt="image" src="https://github.com/user-attachments/assets/adcc68ca-63f6-463d-af92-6fd355c563e2";> <img width="1323" alt="image" src="https://github.com/user-attachments/assets/3cc8887d-0d18-496f-9005-466bdd6e7193";> <img width="1158" alt="image" src="https://github.com/user-attachments/assets/6c7fb890-fe08-4393-8265-db0bdf558ea3";> **File status：properties error（len=0） properties_backup removed** 5. Attempt to create a properties_backup file <img width="1333" alt="image" src="https://github.com/user-attachments/assets/fd973183-77e4-4e57-b77c-844540070537";> <img width="1289" alt="image" src="https://github.com/user-attachments/assets/809e7283-2ace-4bc9-94fd-371f43ea45b0";> I think that we should not only check if the hoodie.properties file exists when performing recoverIfNeeded, we need more information to ensure that the hoodie.properties file is correct, rather than directly skipping file processing and deleting the backup file. Any suggestion? **Environment Description** * Hudi version : 0.14.0 * Spark version :2.4 * Hadoop version :2.6 * Storage (HDFS/S3/GCS..) :HDFS **Stacktrace** Caused by: org.apache.hudi.exception.HoodieException: Error updating table configs. at org.apache.hudi.internal.DataSourceInternalWriterHelper.commit(DataSourceInternalWriterHelper.java:91) at org.apache.hudi.internal.HoodieDataSourceInternalWriter.commit(HoodieDataSourceInternalWriter.java:91) at org.apache.spark.sql.execution.datasources.v2.WriteToDataSourceV2Exec.doExecute(WriteToDataSourceV2Exec.scala:76) ... 69 more Suppressed: java.lang.IllegalArgumentException: hoodie.table.name property needs to be specified at org.apache.hudi.common.table.HoodieTableConfig.generateChecksum(HoodieTableConfig.java:523) at org.apache.hudi.common.table.HoodieTableConfig.getOrderedPropertiesWithTableChecksum(HoodieTableConfig.java:321) at org.apache.hudi.common.table.HoodieTableConfig.storeProperties(HoodieTableConfig.java:339) at org.apache.hudi.common.table.HoodieTableConfig.modify(HoodieTableConfig.java:438) at org.apache.hudi.common.table.HoodieTableConfig.delete(HoodieTableConfig.java:481) at org.apache.hudi.table.upgrade.UpgradeDowngrade.run(UpgradeDowngrade.java:151) at org.apache.hudi.client.BaseHoodieWriteClient.tryUpgrade(BaseHoodieWriteClient.java:1399) at org.apache.hudi.client.BaseHoodieWriteClient.doInitTable(BaseHoodieWriteClient.java:1255) at org.apache.hudi.client.BaseHoodieWriteClient.initTable(BaseHoodieWriteClient.java:1296) at org.apache.hudi.client.BaseHoodieWriteClient.rollback(BaseHoodieWriteClient.java:769) at org.apache.hudi.internal.DataSourceInternalWriterHelper.abort(DataSourceInternalWriterHelper.java:99) at org.apache.hudi.internal.HoodieDataSourceInternalWriter.abort(HoodieDataSourceInternalWriter.java:96) at org.apache.spark.sql.execution.datasources.v2.WriteToDataSourceV2Exec.doExecute(WriteToDataSourceV2Exec.scala:82) ... 69 more Caused by: org.apache.hudi.exception.HoodieIOException: Error updating table configs. at org.apache.hudi.common.table.HoodieTableConfig.modify(HoodieTableConfig.java:466) at org.apache.hudi.common.table.HoodieTableConfig.update(HoodieTableConfig.java:475) at org.apache.hudi.common.table.HoodieTableConfig.setMetadataPartitionState(HoodieTableConfig.java:816) at org.apache.hudi.common.table.HoodieTableConfig.clearMetadataPartitions(HoodieTableConfig.java:847) at org.apache.hudi.metadata.HoodieTableMetadataUtil.deleteMetadataTable(HoodieTableMetadataUtil.java:1396) at org.apache.hudi.metadata.HoodieTableMetadataUtil.deleteMetadataTable(HoodieTableMetadataUtil.java:275) at org.apache.hudi.table.HoodieTable.maybeDeleteMetadataTable(HoodieTable.java:995) at org.apache.hudi.table.HoodieSparkTable.getMetadataWriter(HoodieSparkTable.java:116) at org.apache.hudi.table.HoodieTable.getMetadataWriter(HoodieTable.java:947) at org.apache.hudi.client.BaseHoodieWriteClient.writeTableMetadata(BaseHoodieWriteClient.java:359) at org.apache.hudi.client.BaseHoodieWriteClient.commit(BaseHoodieWriteClient.java:285) at org.apache.hudi.client.BaseHoodieWriteClient.commitStats(BaseHoodieWriteClient.java:236) at org.apache.hudi.client.BaseHoodieWriteClient.commitStats(BaseHoodieWriteClient.java:211) at org.apache.hudi.internal.DataSourceInternalWriterHelper.commit(DataSourceInternalWriterHelper.java:88) ... 71 more Caused by: java.io.InterruptedIOException: Interrupted while waiting for data to be acknowledged by pipeline at org.apache.hadoop.hdfs.DFSOutputStream.waitForAckedSeqno(DFSOutputStream.java:3520) at org.apache.hadoop.hdfs.DFSOutputStream.flushInternal(DFSOutputStream.java:3498) at org.apache.hadoop.hdfs.DFSOutputStream.closeImpl(DFSOutputStream.java:3690) at org.apache.hadoop.hdfs.DFSOutputStream.close(DFSOutputStream.java:3625) at org.apache.hadoop.fs.FSDataOutputStream$PositionCache.close(FSDataOutputStream.java:80) at org.apache.hadoop.fs.FSDataOutputStream.close(FSDataOutputStream.java:115) at org.apache.hadoop.fs.FSDataOutputStream$PositionCache.close(FSDataOutputStream.java:80) at org.apache.hadoop.fs.FSDataOutputStream.close(FSDataOutputStream.java:115) at org.apache.hudi.common.fs.SizeAwareFSDataOutputStream.close(SizeAwareFSDataOutputStream.java:75) at org.apache.hudi.common.table.HoodieTableConfig.modify(HoodieTableConfig.java:449) ... 84 more -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
