[jira] [Updated] (HIVE-19588) Several invocation of file listing when creating VectorizedOrcAcidRowBatchReader
[ https://issues.apache.org/jira/browse/HIVE-19588?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prasanth Jayachandran updated HIVE-19588: - Resolution: Fixed Fix Version/s: 4.0.0 3.0.1 3.1.0 Status: Resolved (was: Patch Available) Committed to branch-3, branch-3.0 and master. Thanks for the reviews! > Several invocation of file listing when creating > VectorizedOrcAcidRowBatchReader > > > Key: HIVE-19588 > URL: https://issues.apache.org/jira/browse/HIVE-19588 > Project: Hive > Issue Type: Bug > Components: Transactions >Affects Versions: 3.1.0 >Reporter: Nita Dembla >Assignee: Prasanth Jayachandran >Priority: Major > Fix For: 3.1.0, 3.0.1, 4.0.0 > > Attachments: HIVE-19588.1.patch, HIVE-19588.2.patch, > HIVE-19588.3.patch, HIVE-19588.4.patch, Screen Shot 2018-05-16 at 2.23.25 > PM.png > > > Looks like we are doing file listing several times when creating one instance > of VectorizedOrcAcidRowBatchReader > AcidUtils.parseBaseOrDeltaBucketFilename() does full file listing (when > there are files with bucket_* prefix) just to get a single file out of a path > to figure out if it has ACID schema (as part of HIVE-18190). > There is full file listing where we populate > 1) ColumnizedDeleteEventRegistry > 2) SortMergedDeleteEventRegistry > 3) Twice in computeOffsetAndBucket() > > Attaching profiles which [~gopalv] took while debugging. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-19588) Several invocation of file listing when creating VectorizedOrcAcidRowBatchReader
[ https://issues.apache.org/jira/browse/HIVE-19588?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prasanth Jayachandran updated HIVE-19588: - Attachment: HIVE-19588.4.patch > Several invocation of file listing when creating > VectorizedOrcAcidRowBatchReader > > > Key: HIVE-19588 > URL: https://issues.apache.org/jira/browse/HIVE-19588 > Project: Hive > Issue Type: Bug > Components: Transactions >Affects Versions: 3.1.0 >Reporter: Nita Dembla >Assignee: Prasanth Jayachandran >Priority: Major > Attachments: HIVE-19588.1.patch, HIVE-19588.2.patch, > HIVE-19588.3.patch, HIVE-19588.4.patch, Screen Shot 2018-05-16 at 2.23.25 > PM.png > > > Looks like we are doing file listing several times when creating one instance > of VectorizedOrcAcidRowBatchReader > AcidUtils.parseBaseOrDeltaBucketFilename() does full file listing (when > there are files with bucket_* prefix) just to get a single file out of a path > to figure out if it has ACID schema (as part of HIVE-18190). > There is full file listing where we populate > 1) ColumnizedDeleteEventRegistry > 2) SortMergedDeleteEventRegistry > 3) Twice in computeOffsetAndBucket() > > Attaching profiles which [~gopalv] took while debugging. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-19588) Several invocation of file listing when creating VectorizedOrcAcidRowBatchReader
[ https://issues.apache.org/jira/browse/HIVE-19588?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prasanth Jayachandran updated HIVE-19588: - Attachment: HIVE-19588.3.patch > Several invocation of file listing when creating > VectorizedOrcAcidRowBatchReader > > > Key: HIVE-19588 > URL: https://issues.apache.org/jira/browse/HIVE-19588 > Project: Hive > Issue Type: Bug > Components: Transactions >Affects Versions: 3.1.0 >Reporter: Nita Dembla >Assignee: Prasanth Jayachandran >Priority: Major > Attachments: HIVE-19588.1.patch, HIVE-19588.2.patch, > HIVE-19588.3.patch, Screen Shot 2018-05-16 at 2.23.25 PM.png > > > Looks like we are doing file listing several times when creating one instance > of VectorizedOrcAcidRowBatchReader > AcidUtils.parseBaseOrDeltaBucketFilename() does full file listing (when > there are files with bucket_* prefix) just to get a single file out of a path > to figure out if it has ACID schema (as part of HIVE-18190). > There is full file listing where we populate > 1) ColumnizedDeleteEventRegistry > 2) SortMergedDeleteEventRegistry > 3) Twice in computeOffsetAndBucket() > > Attaching profiles which [~gopalv] took while debugging. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-19588) Several invocation of file listing when creating VectorizedOrcAcidRowBatchReader
[ https://issues.apache.org/jira/browse/HIVE-19588?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prasanth Jayachandran updated HIVE-19588: - Attachment: HIVE-19588.2.patch > Several invocation of file listing when creating > VectorizedOrcAcidRowBatchReader > > > Key: HIVE-19588 > URL: https://issues.apache.org/jira/browse/HIVE-19588 > Project: Hive > Issue Type: Bug > Components: Transactions >Affects Versions: 3.1.0 >Reporter: Nita Dembla >Assignee: Prasanth Jayachandran >Priority: Major > Attachments: HIVE-19588.1.patch, HIVE-19588.2.patch, Screen Shot > 2018-05-16 at 2.23.25 PM.png > > > Looks like we are doing file listing several times when creating one instance > of VectorizedOrcAcidRowBatchReader > AcidUtils.parseBaseOrDeltaBucketFilename() does full file listing (when > there are files with bucket_* prefix) just to get a single file out of a path > to figure out if it has ACID schema (as part of HIVE-18190). > There is full file listing where we populate > 1) ColumnizedDeleteEventRegistry > 2) SortMergedDeleteEventRegistry > 3) Twice in computeOffsetAndBucket() > > Attaching profiles which [~gopalv] took while debugging. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-19588) Several invocation of file listing when creating VectorizedOrcAcidRowBatchReader
[ https://issues.apache.org/jira/browse/HIVE-19588?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prasanth Jayachandran updated HIVE-19588: - Status: Patch Available (was: Open) > Several invocation of file listing when creating > VectorizedOrcAcidRowBatchReader > > > Key: HIVE-19588 > URL: https://issues.apache.org/jira/browse/HIVE-19588 > Project: Hive > Issue Type: Bug > Components: Transactions >Affects Versions: 3.1.0 >Reporter: Nita Dembla >Assignee: Prasanth Jayachandran >Priority: Major > Attachments: HIVE-19588.1.patch, Screen Shot 2018-05-16 at 2.23.25 > PM.png > > > Looks like we are doing file listing several times when creating one instance > of VectorizedOrcAcidRowBatchReader > AcidUtils.parseBaseOrDeltaBucketFilename() does full file listing (when > there are files with bucket_* prefix) just to get a single file out of a path > to figure out if it has ACID schema (as part of HIVE-18190). > There is full file listing where we populate > 1) ColumnizedDeleteEventRegistry > 2) SortMergedDeleteEventRegistry > 3) Twice in computeOffsetAndBucket() > > Attaching profiles which [~gopalv] took while debugging. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-19588) Several invocation of file listing when creating VectorizedOrcAcidRowBatchReader
[ https://issues.apache.org/jira/browse/HIVE-19588?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prasanth Jayachandran updated HIVE-19588: - Attachment: HIVE-19588.1.patch > Several invocation of file listing when creating > VectorizedOrcAcidRowBatchReader > > > Key: HIVE-19588 > URL: https://issues.apache.org/jira/browse/HIVE-19588 > Project: Hive > Issue Type: Bug > Components: Transactions >Affects Versions: 3.1.0 >Reporter: Nita Dembla >Assignee: Prasanth Jayachandran >Priority: Major > Attachments: HIVE-19588.1.patch, Screen Shot 2018-05-16 at 2.23.25 > PM.png > > > Looks like we are doing file listing several times when creating one instance > of VectorizedOrcAcidRowBatchReader > AcidUtils.parseBaseOrDeltaBucketFilename() does full file listing (when > there are files with bucket_* prefix) just to get a single file out of a path > to figure out if it has ACID schema (as part of HIVE-18190). > There is full file listing where we populate > 1) ColumnizedDeleteEventRegistry > 2) SortMergedDeleteEventRegistry > 3) Twice in computeOffsetAndBucket() > > Attaching profiles which [~gopalv] took while debugging. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-19588) Several invocation of file listing when creating VectorizedOrcAcidRowBatchReader
[ https://issues.apache.org/jira/browse/HIVE-19588?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prasanth Jayachandran updated HIVE-19588: - Description: Looks like we are doing file listing several times when creating one instance of VectorizedOrcAcidRowBatchReader AcidUtils.parseBaseOrDeltaBucketFilename() does full file listing (when there are files with bucket_* prefix) just to get a single file out of a path to figure out if it has ACID schema (as part of HIVE-18190). There is full file listing where we populate 1) ColumnizedDeleteEventRegistry 2) SortMergedDeleteEventRegistry 3) Twice in computeOffsetAndBucket() Attaching profiles which [~gopalv] took while debugging. was: Looks like we are doing file listing several times when creating one instance of VectorizedOrcAcidRowBatchReader AcidUtils.parseBaseOrDeltaBucketFilename() does full file listing (when there are files with bucket_* prefix) just to get a single file out of a path to figure out if it has ACID schema (as part of https://issues.apache.org/jira/browse/HIVE-18190). There is full file listing where we populate 1) ColumnizedDeleteEventRegistry 2) SortMergedDeleteEventRegistry 3) Twice in computeOffsetAndBucket() Attaching profiles which [~gopalv] took while debugging. > Several invocation of file listing when creating > VectorizedOrcAcidRowBatchReader > > > Key: HIVE-19588 > URL: https://issues.apache.org/jira/browse/HIVE-19588 > Project: Hive > Issue Type: Bug > Components: Transactions >Affects Versions: 3.1.0 >Reporter: Nita Dembla >Assignee: Prasanth Jayachandran >Priority: Major > Attachments: Screen Shot 2018-05-16 at 2.23.25 PM.png > > > Looks like we are doing file listing several times when creating one instance > of VectorizedOrcAcidRowBatchReader > AcidUtils.parseBaseOrDeltaBucketFilename() does full file listing (when > there are files with bucket_* prefix) just to get a single file out of a path > to figure out if it has ACID schema (as part of HIVE-18190). > There is full file listing where we populate > 1) ColumnizedDeleteEventRegistry > 2) SortMergedDeleteEventRegistry > 3) Twice in computeOffsetAndBucket() > > Attaching profiles which [~gopalv] took while debugging. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-19588) Several invocation of file listing when creating VectorizedOrcAcidRowBatchReader
[ https://issues.apache.org/jira/browse/HIVE-19588?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prasanth Jayachandran updated HIVE-19588: - Attachment: Screen Shot 2018-05-16 at 2.23.25 PM.png > Several invocation of file listing when creating > VectorizedOrcAcidRowBatchReader > > > Key: HIVE-19588 > URL: https://issues.apache.org/jira/browse/HIVE-19588 > Project: Hive > Issue Type: Bug > Components: Transactions >Affects Versions: 3.1.0 >Reporter: Nita Dembla >Assignee: Prasanth Jayachandran >Priority: Major > Attachments: Screen Shot 2018-05-16 at 2.23.25 PM.png > > > Looks like we are doing file listing several times when creating one instance > of VectorizedOrcAcidRowBatchReader > AcidUtils.parseBaseOrDeltaBucketFilename() does full file listing (when there > are files with bucket_* prefix) just to get a single file out of a path to > figure out if it has ACID schema (as part of > https://issues.apache.org/jira/browse/HIVE-18190). > There is full file listing where we populate > 1) ColumnizedDeleteEventRegistry > 2) SortMergedDeleteEventRegistry > 3) Twice in computeOffsetAndBucket() > > Attaching profiles which [~gopalv] took while debugging. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
