[jira] [Created] (HIVE-24434) Filter out materialized views for rewriting if plan pattern is not allowed
Krisztian Kasa created HIVE-24434: - Summary: Filter out materialized views for rewriting if plan pattern is not allowed Key: HIVE-24434 URL: https://issues.apache.org/jira/browse/HIVE-24434 Project: Hive Issue Type: Improvement Affects Versions: 4.0.0 Reporter: Krisztian Kasa Assignee: Krisztian Kasa Some materialized views are not enabled for Calcite based rewriting. Rules for validating materialized views are implemented by HIVE-20748. Since text based materialized view query rewrite doesn't have such limitations some logic must be implemented to flag materialized view whether they are enabled to text based rewrite only or both. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (HIVE-24433) AutoCompaction is not getting triggered for CamelCase Partition Values
Naresh P R created HIVE-24433:
-
Summary: AutoCompaction is not getting triggered for CamelCase
Partition Values
Key: HIVE-24433
URL: https://issues.apache.org/jira/browse/HIVE-24433
Project: Hive
Issue Type: Bug
Reporter: Naresh P R
Assignee: Naresh P R
partionKey=paritionValue is getting converted into lowerCase in below 2 places.
[https://github.com/apache/hive/blob/master/standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/txn/TxnHandler.java#L2728]
https://github.com/apache/hive/blob/master/standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/txn/TxnHandler.java#L2851
Because of which TXN_COMPONENTS & HIVE_LOCKS tables are not having entries from
proper partition values.
When query completes, the entry moves from TXN_COMPONENTS to
COMPLETED_TXN_COMPONENTS. Hive AutoCompaction will not recognize the partition
& considers it as invalid partition
create table abc(name string) partitioned by(city string) stored as orc
tblproperties('transactional'='true');
insert into abc partition(city='Bangalore') values('aaa');
Example entry in COMPLETED_TXN_COMPONENTS
{noformat}
+---+--++---+-+-+---+
| CTC_TXNID | CTC_DATABASE | CTC_TABLE | CTC_PARTITION |
CTC_TIMESTAMP | CTC_WRITEID | CTC_UPDATE_DELETE |
+---+--++---+-+-+---+
| 2 | default | abc | city=bangalore | 2020-11-25 09:26:59 |
1 | N |
+---+--++---+-+-+---+
{noformat}
AutoCompaction fails to get triggered with below error
2020-11-25T09:35:10,364 INFO [Thread-9]: compactor.Initiator
(Initiator.java:run(98)) - Checking to see if we should compact
default.abc.city=bangalore
2020-11-25T09:35:10,380 INFO [Thread-9]: compactor.Initiator
(Initiator.java:run(155)) - Can't find partition
default.compaction_test.city=bhubaneshwar, assuming it has been dropped and
moving on
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (HIVE-24432) Delete Notification Events in Batches
David Mollitor created HIVE-24432: - Summary: Delete Notification Events in Batches Key: HIVE-24432 URL: https://issues.apache.org/jira/browse/HIVE-24432 Project: Hive Issue Type: Improvement Affects Versions: 3.2.0 Reporter: David Mollitor Assignee: David Mollitor Notification events are loaded in batches (reduces memory pressure on the HMS), but all of the deletes happen under a single transactions and, when deleting many records, can put a lot of pressure on the backend database. Instead, delete events in batches (in different transactions) as well. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (HIVE-24431) Null Pointer exception while sending data to jdbc
Fabien Carrion created HIVE-24431: - Summary: Null Pointer exception while sending data to jdbc Key: HIVE-24431 URL: https://issues.apache.org/jira/browse/HIVE-24431 Project: Hive Issue Type: Bug Components: JDBC storage handler Affects Versions: All Versions Reporter: Fabien Carrion I was receiving some null pointer while writing in db. I just add a check. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (HIVE-24430) DiskRangeInfo should make use of DiskRangeList
Panagiotis Garefalakis created HIVE-24430: - Summary: DiskRangeInfo should make use of DiskRangeList Key: HIVE-24430 URL: https://issues.apache.org/jira/browse/HIVE-24430 Project: Hive Issue Type: Sub-task Reporter: Panagiotis Garefalakis Assignee: Panagiotis Garefalakis DiskRangeInfo should make user of DiskRangeList instead of List – this will help us transition to ORC 1.6. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (HIVE-24429) Figure out a better way to test failed compactions
Karen Coppage created HIVE-24429:
Summary: Figure out a better way to test failed compactions
Key: HIVE-24429
URL: https://issues.apache.org/jira/browse/HIVE-24429
Project: Hive
Issue Type: Improvement
Reporter: Karen Coppage
This block is executed during compaction:
{code:java}
if(conf.getBoolVar(HiveConf.ConfVars.HIVE_IN_TEST) &&
conf.getBoolVar(HiveConf.ConfVars.HIVETESTMODEFAILCOMPACTION)) {
throw new RuntimeException(HiveConf.ConfVars.HIVETESTMODEFAILCOMPACTION.name()
+ "=true");
}{code}
We should figure out a better way to test failed compaction than including test
code in the source.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (HIVE-24428) Concurrent add_partitions requests may lead to data loss
Zoltan Haindrich created HIVE-24428: --- Summary: Concurrent add_partitions requests may lead to data loss Key: HIVE-24428 URL: https://issues.apache.org/jira/browse/HIVE-24428 Project: Hive Issue Type: Bug Reporter: Zoltan Haindrich Assignee: Zoltan Haindrich in case multiple clients are adding partitions to the same table - when the same partition is being added there is a chance that the data dir is removed after the other client have already written its data https://github.com/apache/hive/blob/5e96b14a2357c66a0640254d5414bc706d8be852/standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java#L3958 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (HIVE-24427) HPL/SQL improvements
Attila Magyar created HIVE-24427: Summary: HPL/SQL improvements Key: HIVE-24427 URL: https://issues.apache.org/jira/browse/HIVE-24427 Project: Hive Issue Type: Improvement Components: hpl/sql Reporter: Attila Magyar Assignee: Attila Magyar -- This message was sent by Atlassian Jira (v8.3.4#803005)
