Ashutosh Chauhan commented on HIVE-21225:

[~vgumashta] Which approach you are thinking here? [~ekoifman]'s of encoding 
identifier in file names or [~gopalv]'s single recursive call. 
If we change names of dirs not sure if that will have any impact on data in 
existing tables.

> ACID: getAcidState() should cache a recursive dir listing locally
> -----------------------------------------------------------------
>                 Key: HIVE-21225
>                 URL: https://issues.apache.org/jira/browse/HIVE-21225
>             Project: Hive
>          Issue Type: Improvement
>          Components: Transactions
>            Reporter: Gopal V
>            Assignee: Vaibhav Gumashta
>            Priority: Major
>         Attachments: async-pid-44-2.svg
> Currently getAcidState() makes 3 calls into the FS api which could be 
> answered by making a single recursive listDir call and reusing the same data 
> to check for isRawFormat() and isValidBase().
> All delta operations for a single partition can go against a single listed 
> directory snapshot instead of interacting with the NameNode or ObjectStore 
> within the inner loop.

This message was sent by Atlassian JIRA

Reply via email to