afilipchik commented on a change in pull request #1515: [HUDI-795] Ignoring
missing aux folder
URL: https://github.com/apache/incubator-hudi/pull/1515#discussion_r409208032
##########
File path:
hudi-client/src/main/java/org/apache/hudi/table/HoodieCommitArchiveLog.java
##########
@@ -219,14 +220,23 @@ private boolean
deleteArchivedInstants(List<HoodieInstant> archivedInstants) thr
* @throws IOException in case of error
*/
private boolean deleteAllInstantsOlderorEqualsInAuxMetaFolder(HoodieInstant
thresholdInstant) throws IOException {
- List<HoodieInstant> instants = metaClient.scanHoodieInstantsFromFileSystem(
- new Path(metaClient.getMetaAuxiliaryPath()),
HoodieActiveTimeline.VALID_EXTENSIONS_IN_ACTIVE_TIMELINE, false);
+ List<HoodieInstant> instants = null;
+ boolean success = true;
+ try {
+ instants =
+ metaClient.scanHoodieInstantsFromFileSystem(
+ new Path(metaClient.getMetaAuxiliaryPath()),
+ HoodieActiveTimeline.VALID_EXTENSIONS_IN_ACTIVE_TIMELINE,
+ false);
+ } catch (FileNotFoundException e) {
Review comment:
not sure about it. Did see that folder disappears/reappears sometimes. Can
be related to the way GCS connector treats a situation when all the files from
the folder are removed.
On gcs folders are not real, they are just logical. So, file with name:
gs://bucket/folder/file doesn't live in the folder "folder", it just has a long
"/" separated name. I saw some logic in the GCS FS java implementation that
creates fake file "/" when everything is removed from the logical folder. So,
my speculation is that if this call fails or not enabled, that removing all the
files from the folder will end up if folder being removed as well.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
