beyond1920 opened a new issue, #10885: URL: https://github.com/apache/hudi/issues/10885
Dear community, I recently encountered an problem: The `.hoodie_partition_metadata` does not exist in leaf partition path. But the writer job completed successfully. It would result in incorrect query results. It does not only happened for mor table, but also for cow table. For example: <img width="2403" alt="image" src="https://github.com/apache/hudi/assets/1525333/08b7faec-0e40-453a-8a2e-75ccc01e9ef4";> It would caused unexpected result of following SQL query because `FileSystemBackedTableMetadata` would skip `p_date=20230219` when get partition paths list. `select * from sourceT where p_date between '20230201' and '20230230 '`. The writer job completed success, but the spark logs are already cleaned, so I could not find more information. I guess something wrong when tasks call `HoodiePartitionMetadata#trySave`. No task ever created the `hoodie_partition_metadata` file successfully, but all exceptions are all ignored here. <img width="1246" alt="image" src="https://github.com/apache/hudi/assets/1525333/e063cda9-c324-403d-91ea-0fc273f0c3be";> Is there anyone encountered the similar issue? Are there any suggestions? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
