[jira] [Commented] (HIVE-21225) ACID: getAcidState() should cache a recursive dir listing locally

[ https://issues.apache.org/jira/browse/HIVE-21225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17049879#comment-17049879 ] Aditya Shah commented on HIVE-21225: [~gopalv]  I further noticed Hive-22001. It seems we are

[jira] [Commented] (HIVE-21225) ACID: getAcidState() should cache a recursive dir listing locally

[ https://issues.apache.org/jira/browse/HIVE-21225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17038066#comment-17038066 ] Gopal Vijayaraghavan commented on HIVE-21225: - bq. I have reopened the issue as I couldn't

[jira] [Commented] (HIVE-21225) ACID: getAcidState() should cache a recursive dir listing locally

[ https://issues.apache.org/jira/browse/HIVE-21225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17036891#comment-17036891 ] Aditya Shah commented on HIVE-21225: Still couldn't attach files. Image hosted:  1. 

[jira] [Commented] (HIVE-21225) ACID: getAcidState() should cache a recursive dir listing locally

[ https://issues.apache.org/jira/browse/HIVE-21225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17036870#comment-17036870 ] Aditya Shah commented on HIVE-21225: [~gopalv]  Thanks for your reply. I'm not using S3guard which

[jira] [Commented] (HIVE-21225) ACID: getAcidState() should cache a recursive dir listing locally

[ https://issues.apache.org/jira/browse/HIVE-21225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17036574#comment-17036574 ] Gopal Vijayaraghavan commented on HIVE-21225: - bq. The overhead compared to the benefit where

[jira] [Commented] (HIVE-21225) ACID: getAcidState() should cache a recursive dir listing locally

[ https://issues.apache.org/jira/browse/HIVE-21225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17036142#comment-17036142 ] Aditya Shah commented on HIVE-21225: [~vgumashta] I had some doubts similar to what [~vgarg] raised

[jira] [Commented] (HIVE-21225) ACID: getAcidState() should cache a recursive dir listing locally

[ https://issues.apache.org/jira/browse/HIVE-21225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16891379#comment-16891379 ] Vineet Garg commented on HIVE-21225: +1 pending minor review comments > ACID: getAcidState() should

[jira] [Commented] (HIVE-21225) ACID: getAcidState() should cache a recursive dir listing locally

[ https://issues.apache.org/jira/browse/HIVE-21225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16889613#comment-16889613 ] Hive QA commented on HIVE-21225: Here are the results of testing the latest attachment:

[jira] [Commented] (HIVE-21225) ACID: getAcidState() should cache a recursive dir listing locally

[ https://issues.apache.org/jira/browse/HIVE-21225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16889608#comment-16889608 ] Hive QA commented on HIVE-21225: | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Commented] (HIVE-21225) ACID: getAcidState() should cache a recursive dir listing locally

[ https://issues.apache.org/jira/browse/HIVE-21225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16889267#comment-16889267 ] Hive QA commented on HIVE-21225: Here are the results of testing the latest attachment:

[jira] [Commented] (HIVE-21225) ACID: getAcidState() should cache a recursive dir listing locally

[ https://issues.apache.org/jira/browse/HIVE-21225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16889255#comment-16889255 ] Hive QA commented on HIVE-21225: | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Commented] (HIVE-21225) ACID: getAcidState() should cache a recursive dir listing locally

[ https://issues.apache.org/jira/browse/HIVE-21225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16886324#comment-16886324 ] Jason Dere commented on HIVE-21225: --- [~gopalv] and [~vgumashta] brought this one to my attention as a

[jira] [Commented] (HIVE-21225) ACID: getAcidState() should cache a recursive dir listing locally

[ https://issues.apache.org/jira/browse/HIVE-21225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16885179#comment-16885179 ] Hive QA commented on HIVE-21225: Here are the results of testing the latest attachment:

[jira] [Commented] (HIVE-21225) ACID: getAcidState() should cache a recursive dir listing locally

[ https://issues.apache.org/jira/browse/HIVE-21225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16885176#comment-16885176 ] Hive QA commented on HIVE-21225: Here are the results of testing the latest attachment:

[jira] [Commented] (HIVE-21225) ACID: getAcidState() should cache a recursive dir listing locally

[ https://issues.apache.org/jira/browse/HIVE-21225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16885127#comment-16885127 ] Hive QA commented on HIVE-21225: | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Commented] (HIVE-21225) ACID: getAcidState() should cache a recursive dir listing locally

[ https://issues.apache.org/jira/browse/HIVE-21225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16884823#comment-16884823 ] Hive QA commented on HIVE-21225: Here are the results of testing the latest attachment:

[jira] [Commented] (HIVE-21225) ACID: getAcidState() should cache a recursive dir listing locally

[ https://issues.apache.org/jira/browse/HIVE-21225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16884814#comment-16884814 ] Hive QA commented on HIVE-21225: | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Commented] (HIVE-21225) ACID: getAcidState() should cache a recursive dir listing locally

[ https://issues.apache.org/jira/browse/HIVE-21225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16884775#comment-16884775 ] Hive QA commented on HIVE-21225: Here are the results of testing the latest attachment:

[jira] [Commented] (HIVE-21225) ACID: getAcidState() should cache a recursive dir listing locally

[ https://issues.apache.org/jira/browse/HIVE-21225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16884765#comment-16884765 ] Hive QA commented on HIVE-21225: | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Commented] (HIVE-21225) ACID: getAcidState() should cache a recursive dir listing locally

[ https://issues.apache.org/jira/browse/HIVE-21225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16884632#comment-16884632 ] Hive QA commented on HIVE-21225: Here are the results of testing the latest attachment:

[jira] [Commented] (HIVE-21225) ACID: getAcidState() should cache a recursive dir listing locally

[ https://issues.apache.org/jira/browse/HIVE-21225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16884614#comment-16884614 ] Hive QA commented on HIVE-21225: | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Commented] (HIVE-21225) ACID: getAcidState() should cache a recursive dir listing locally

[ https://issues.apache.org/jira/browse/HIVE-21225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16884608#comment-16884608 ] Vaibhav Gumashta commented on HIVE-21225: - [~vgarg] FYI, with respect to the 3 calls:

[jira] [Commented] (HIVE-21225) ACID: getAcidState() should cache a recursive dir listing locally

[ https://issues.apache.org/jira/browse/HIVE-21225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16884602#comment-16884602 ] Vaibhav Gumashta commented on HIVE-21225: - Looks like the export import integration with

[jira] [Commented] (HIVE-21225) ACID: getAcidState() should cache a recursive dir listing locally

[ https://issues.apache.org/jira/browse/HIVE-21225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16882283#comment-16882283 ] Vineet Garg commented on HIVE-21225: bq. I think MM tables retain these subdirs since its MoveTask

[jira] [Commented] (HIVE-21225) ACID: getAcidState() should cache a recursive dir listing locally

[ https://issues.apache.org/jira/browse/HIVE-21225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16881708#comment-16881708 ] Eugene Koifman commented on HIVE-21225: --- Full Acid uses stmtId for Insert ... as select union

[jira] [Commented] (HIVE-21225) ACID: getAcidState() should cache a recursive dir listing locally

[ https://issues.apache.org/jira/browse/HIVE-21225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16881703#comment-16881703 ] Gopal V commented on HIVE-21225: bq. union_mm/delta_001_001_0001/HIVE_UNION_SUBDIR_1 Are you

[jira] [Commented] (HIVE-21225) ACID: getAcidState() should cache a recursive dir listing locally

[ https://issues.apache.org/jira/browse/HIVE-21225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16881618#comment-16881618 ] Vineet Garg commented on HIVE-21225: bq. While building directory snapshots, check for isValidBase,

[jira] [Commented] (HIVE-21225) ACID: getAcidState() should cache a recursive dir listing locally

[ https://issues.apache.org/jira/browse/HIVE-21225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16881614#comment-16881614 ] Vaibhav Gumashta commented on HIVE-21225: - [~vgarg] Thanks for the review: let me update the

[jira] [Commented] (HIVE-21225) ACID: getAcidState() should cache a recursive dir listing locally

[ https://issues.apache.org/jira/browse/HIVE-21225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16881610#comment-16881610 ] Vineet Garg commented on HIVE-21225: The patch is building directory snapshot (dir path to file

[jira] [Commented] (HIVE-21225) ACID: getAcidState() should cache a recursive dir listing locally

[ https://issues.apache.org/jira/browse/HIVE-21225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16881575#comment-16881575 ] Vineet Garg commented on HIVE-21225: Went over the patch once again and I am struggling to understand

[jira] [Commented] (HIVE-21225) ACID: getAcidState() should cache a recursive dir listing locally

[ https://issues.apache.org/jira/browse/HIVE-21225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16881291#comment-16881291 ] Hive QA commented on HIVE-21225: Here are the results of testing the latest attachment:

[jira] [Commented] (HIVE-21225) ACID: getAcidState() should cache a recursive dir listing locally

[ https://issues.apache.org/jira/browse/HIVE-21225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16881253#comment-16881253 ] Hive QA commented on HIVE-21225: | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Commented] (HIVE-21225) ACID: getAcidState() should cache a recursive dir listing locally

[ https://issues.apache.org/jira/browse/HIVE-21225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16880809#comment-16880809 ] Vineet Garg commented on HIVE-21225: I have removed {{hiddenDirFilter}} in {{HIVE-21225.9.patch}} but

[jira] [Commented] (HIVE-21225) ACID: getAcidState() should cache a recursive dir listing locally

[ https://issues.apache.org/jira/browse/HIVE-21225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16880807#comment-16880807 ] Vineet Garg commented on HIVE-21225: [~vgumashta] {{hiddenDirFilter}} is filtering out directories

[jira] [Commented] (HIVE-21225) ACID: getAcidState() should cache a recursive dir listing locally

[ https://issues.apache.org/jira/browse/HIVE-21225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16880346#comment-16880346 ] Hive QA commented on HIVE-21225: Here are the results of testing the latest attachment:

[jira] [Commented] (HIVE-21225) ACID: getAcidState() should cache a recursive dir listing locally

[ https://issues.apache.org/jira/browse/HIVE-21225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16880183#comment-16880183 ] Hive QA commented on HIVE-21225: Here are the results of testing the latest attachment:

[jira] [Commented] (HIVE-21225) ACID: getAcidState() should cache a recursive dir listing locally

[ https://issues.apache.org/jira/browse/HIVE-21225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16880154#comment-16880154 ] Hive QA commented on HIVE-21225: | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Commented] (HIVE-21225) ACID: getAcidState() should cache a recursive dir listing locally

[ https://issues.apache.org/jira/browse/HIVE-21225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16879827#comment-16879827 ] Hive QA commented on HIVE-21225: Here are the results of testing the latest attachment:

[jira] [Commented] (HIVE-21225) ACID: getAcidState() should cache a recursive dir listing locally

[ https://issues.apache.org/jira/browse/HIVE-21225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16879818#comment-16879818 ] Hive QA commented on HIVE-21225: | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Commented] (HIVE-21225) ACID: getAcidState() should cache a recursive dir listing locally

[ https://issues.apache.org/jira/browse/HIVE-21225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16878650#comment-16878650 ] Hive QA commented on HIVE-21225: Here are the results of testing the latest attachment:

[jira] [Commented] (HIVE-21225) ACID: getAcidState() should cache a recursive dir listing locally

[ https://issues.apache.org/jira/browse/HIVE-21225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16878620#comment-16878620 ] Hive QA commented on HIVE-21225: | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Commented] (HIVE-21225) ACID: getAcidState() should cache a recursive dir listing locally

[ https://issues.apache.org/jira/browse/HIVE-21225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16874707#comment-16874707 ] Hive QA commented on HIVE-21225: Here are the results of testing the latest attachment:

[jira] [Commented] (HIVE-21225) ACID: getAcidState() should cache a recursive dir listing locally

[ https://issues.apache.org/jira/browse/HIVE-21225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16874596#comment-16874596 ] Hive QA commented on HIVE-21225: Here are the results of testing the latest attachment:

[jira] [Commented] (HIVE-21225) ACID: getAcidState() should cache a recursive dir listing locally

[ https://issues.apache.org/jira/browse/HIVE-21225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16872752#comment-16872752 ] Hive QA commented on HIVE-21225: Here are the results of testing the latest attachment:

[jira] [Commented] (HIVE-21225) ACID: getAcidState() should cache a recursive dir listing locally

[ https://issues.apache.org/jira/browse/HIVE-21225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16872724#comment-16872724 ] Hive QA commented on HIVE-21225: | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Commented] (HIVE-21225) ACID: getAcidState() should cache a recursive dir listing locally

[ https://issues.apache.org/jira/browse/HIVE-21225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16872237#comment-16872237 ] Hive QA commented on HIVE-21225: Here are the results of testing the latest attachment:

[jira] [Commented] (HIVE-21225) ACID: getAcidState() should cache a recursive dir listing locally

[ https://issues.apache.org/jira/browse/HIVE-21225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16871097#comment-16871097 ] Hive QA commented on HIVE-21225: Here are the results of testing the latest attachment:

[jira] [Commented] (HIVE-21225) ACID: getAcidState() should cache a recursive dir listing locally

[ https://issues.apache.org/jira/browse/HIVE-21225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16871068#comment-16871068 ] Hive QA commented on HIVE-21225: | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Commented] (HIVE-21225) ACID: getAcidState() should cache a recursive dir listing locally

[ https://issues.apache.org/jira/browse/HIVE-21225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16861948#comment-16861948 ] Hive QA commented on HIVE-21225: Here are the results of testing the latest attachment:

[jira] [Commented] (HIVE-21225) ACID: getAcidState() should cache a recursive dir listing locally

[ https://issues.apache.org/jira/browse/HIVE-21225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16861907#comment-16861907 ] Hive QA commented on HIVE-21225: | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Commented] (HIVE-21225) ACID: getAcidState() should cache a recursive dir listing locally

[ https://issues.apache.org/jira/browse/HIVE-21225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16861234#comment-16861234 ] Vaibhav Gumashta commented on HIVE-21225: - DFSClient, which returns HdfsLocatedFileStatus which

[jira] [Commented] (HIVE-21225) ACID: getAcidState() should cache a recursive dir listing locally

[ https://issues.apache.org/jira/browse/HIVE-21225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16856492#comment-16856492 ] Hive QA commented on HIVE-21225: Here are the results of testing the latest attachment:

[jira] [Commented] (HIVE-21225) ACID: getAcidState() should cache a recursive dir listing locally

[ https://issues.apache.org/jira/browse/HIVE-21225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16856469#comment-16856469 ] Hive QA commented on HIVE-21225: | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Commented] (HIVE-21225) ACID: getAcidState() should cache a recursive dir listing locally

[ https://issues.apache.org/jira/browse/HIVE-21225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16856184#comment-16856184 ] Gopal V commented on HIVE-21225: The DirSnapshot can also implement .exists(), so that you can avoid

[jira] [Commented] (HIVE-21225) ACID: getAcidState() should cache a recursive dir listing locally

[ https://issues.apache.org/jira/browse/HIVE-21225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16856170#comment-16856170 ] Gopal V commented on HIVE-21225: This patch looks good, but it still does 2 things - it generates one FS

[jira] [Commented] (HIVE-21225) ACID: getAcidState() should cache a recursive dir listing locally

[ https://issues.apache.org/jira/browse/HIVE-21225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16845330#comment-16845330 ] Gopal V commented on HIVE-21225: [~vgumashta]: is that the right patch for this JIRA? > ACID:

[jira] [Commented] (HIVE-21225) ACID: getAcidState() should cache a recursive dir listing locally

[ https://issues.apache.org/jira/browse/HIVE-21225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16778983#comment-16778983 ] Gopal V commented on HIVE-21225: The specific improvement I'm looking at is to make the {{List

[jira] [Commented] (HIVE-21225) ACID: getAcidState() should cache a recursive dir listing locally

[ https://issues.apache.org/jira/browse/HIVE-21225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16778979#comment-16778979 ] Ashutosh Chauhan commented on HIVE-21225: - [~vgumashta] Which approach you are thinking here?

[jira] [Commented] (HIVE-21225) ACID: getAcidState() should cache a recursive dir listing locally

[ https://issues.apache.org/jira/browse/HIVE-21225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16763911#comment-16763911 ] Eugene Koifman commented on HIVE-21225: --- perhaps an easier/better solution is to add another suffix