duongkame commented on PR #4346: URL: https://github.com/apache/ozone/pull/4346#issuecomment-1467224910
> LGTM, cache TTL is 6 hours as default, as currently observed that location information is not used. I can see below JIRA where block location is populated for integration with Hive as per code comment... > > [HDDS-2188](https://issues.apache.org/jira/browse/HDDS-2188). Implement LocatedFileStatus & getFileBlockLocations to provide node/localization information to Yarn/Mapreduce [HDDS-2914](https://issues.apache.org/jira/browse/HDDS-2914). Certain Hive queries started to fail on generating splits (#563) [HDDS-2914](https://issues.apache.org/jira/browse/HDDS-2914). Certain Hive queries started to fail on generating splits > > Plz check if this integration have some imact Thanks for the deep-dive, @sumitagrawl. As discussed in the community meeting, listFileStatus doesn't calculate block tokens for the return BlockLocationInformation. Thus, clients of the listFileStatus API can't use the result to read data. So, whatever the usage of `LocatedFileStatus` & `getFileBlockLocations ` from the list API is not on the critical path of reading data and therefore no guarantee of strong block location info consistency. I also got through the JIRAs you found. They helped me understand how the `LocatedFileStatus` is brought to OFS API. Also, the JIRAs only concerns with `getFileStatus` API which provide individual file information, and not `listFileStatus`. My best guess is that the `LocatedFileStatus` is reused for both OFS APIs for programming consistency (and not actually intended to `listFileStatus` clients). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
