pvary commented on a change in pull request #1465:
URL: https://github.com/apache/iceberg/pull/1465#discussion_r490131562
##########
File path:
core/src/main/java/org/apache/iceberg/hadoop/HadoopTableOperations.java
##########
@@ -289,19 +311,24 @@ int readVersionHint() {
return Integer.parseInt(in.readLine().replace("\n", ""));
} catch (Exception e) {
- LOG.warn("Error reading version hint file {}", versionHintFile, e);
+ LOG.debug("Error reading version hint file {}", versionHintFile, e);
try {
- if (getMetadataFile(1) != null) {
- // We just assume corrupted metadata and start to read from the
first version file
- return 1;
+ // List the metadata directory to find the version files, and try to
recover the max available version
+ FileStatus[] files = fs.listStatus(metadataRoot(), name ->
VERSION_PATTERN.matcher(name.getName()).matches());
+ int maxVersion = 0;
+
+ for (FileStatus file : files) {
+ version = version(file.getPath().getName());
Review comment:
I was so focused on not finding a way to replicate the failures on my
side, that I missed this.
Big KUDOS for catching this! :)
I hope fixing this will fix the tests as well (Still not sure why they were
passing on my side 😢)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
