[jira] [Commented] (HIVE-21165) ACID: pass query hint to the writers to write hive.acid.key.index
[ https://issues.apache.org/jira/browse/HIVE-21165?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16795663#comment-16795663 ] Abhishek Somani commented on HIVE-21165: Got it. I was confused between hive.acid.key.index and hive.acid.stats. Is it perhaps then that hive.acid.stats is not required anymore as it just counts types of events? > ACID: pass query hint to the writers to write hive.acid.key.index > - > > Key: HIVE-21165 > URL: https://issues.apache.org/jira/browse/HIVE-21165 > Project: Hive > Issue Type: Bug > Components: Transactions >Affects Versions: 3.1.1 >Reporter: Vaibhav Gumashta >Assignee: Vaibhav Gumashta >Priority: Major > > For the query based compactor from HIVE-20699, the compaction runs as a sql > query. However, this mechanism skips over writing hive.acid.key.index for > each stripe, which is used to skip over stripes that are not supposed to be > read. We need a way to pass a query hint to the writer so that it can write > this index data, when invoked from a sql query. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-21165) ACID: pass query hint to the writers to write hive.acid.key.index
[ https://issues.apache.org/jira/browse/HIVE-21165?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16795644#comment-16795644 ] Eugene Koifman commented on HIVE-21165: --- The logic in HIVE-20738 relies on this index. > ACID: pass query hint to the writers to write hive.acid.key.index > - > > Key: HIVE-21165 > URL: https://issues.apache.org/jira/browse/HIVE-21165 > Project: Hive > Issue Type: Bug > Components: Transactions >Affects Versions: 3.1.1 >Reporter: Vaibhav Gumashta >Assignee: Vaibhav Gumashta >Priority: Major > > For the query based compactor from HIVE-20699, the compaction runs as a sql > query. However, this mechanism skips over writing hive.acid.key.index for > each stripe, which is used to skip over stripes that are not supposed to be > read. We need a way to pass a query hint to the writer so that it can write > this index data, when invoked from a sql query. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-21165) ACID: pass query hint to the writers to write hive.acid.key.index
[ https://issues.apache.org/jira/browse/HIVE-21165?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16794812#comment-16794812 ] Abhishek Somani commented on HIVE-21165: In ACID v2, as every file has only one type of event (insert or delete), isn't hive.acid.key.index not required anymore (as also mentioned in HIVE-20580)? > ACID: pass query hint to the writers to write hive.acid.key.index > - > > Key: HIVE-21165 > URL: https://issues.apache.org/jira/browse/HIVE-21165 > Project: Hive > Issue Type: Bug > Components: Transactions >Affects Versions: 3.1.1 >Reporter: Vaibhav Gumashta >Assignee: Vaibhav Gumashta >Priority: Major > > For the query based compactor from HIVE-20699, the compaction runs as a sql > query. However, this mechanism skips over writing hive.acid.key.index for > each stripe, which is used to skip over stripes that are not supposed to be > read. We need a way to pass a query hint to the writer so that it can write > this index data, when invoked from a sql query. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
