[jira] [Created] (HIVE-22632) Improve estimateRowSizeFromSchema
Vineet Garg created HIVE-22632: -- Summary: Improve estimateRowSizeFromSchema Key: HIVE-22632 URL: https://issues.apache.org/jira/browse/HIVE-22632 Project: Hive Issue Type: Improvement Components: Statistics Reporter: Vineet Garg Assignee: Vineet Garg estimateRowSizeFromSchema un-necessarily iterate and do look-up. This could be avoided. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (HIVE-22631) Avoid deep copying partition list in listPartitionsByExpr
Vineet Garg created HIVE-22631: -- Summary: Avoid deep copying partition list in listPartitionsByExpr Key: HIVE-22631 URL: https://issues.apache.org/jira/browse/HIVE-22631 Project: Hive Issue Type: Improvement Components: Hive Reporter: Vineet Garg Assignee: Vineet Garg This is an expensive call, I am not sure why deepCopy is required. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (HIVE-22630) Do not retrieve Materialized View definition for rebuild if query is test SQL
Rajkumar Singh created HIVE-22630: - Summary: Do not retrieve Materialized View definition for rebuild if query is test SQL Key: HIVE-22630 URL: https://issues.apache.org/jira/browse/HIVE-22630 Project: Hive Issue Type: Bug Environment: Hive-3.1.2 Reporter: Rajkumar Singh Assignee: Rajkumar Singh for the query like select 1, select current_timestamp, select current_date hive retrieve all the Materialized view from megastore, if the no of databases are too large then this call take lots of time, the situation becomes worse if there are too frequent if hive server receives frequent "select 1" query ( connection pool uses it to check if the connection is valid or not). -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (HIVE-22629) AST Node Children can be quite expensive to build due to List resizing
Slim Bouguerra created HIVE-22629: - Summary: AST Node Children can be quite expensive to build due to List resizing Key: HIVE-22629 URL: https://issues.apache.org/jira/browse/HIVE-22629 Project: Hive Issue Type: Improvement Reporter: Slim Bouguerra Assignee: Slim Bouguerra As per the attached profile, The AST Node can be a major source of CPU and memory churn, due to the ArrayList resizing and copy. In my Opinion this can be amortized by providing the actual size. [~jcamachorodriguez] / [~vgarg] -- This message was sent by Atlassian Jira (v8.3.4#803005)
HIVE-2.4 release plans
Hi all. Are there any plans for Hive-2.4 release? -- Oleksiy
[jira] [Created] (HIVE-22628) Add locks and transactions tables from sys db to information_schema
Zoltan Chovan created HIVE-22628: Summary: Add locks and transactions tables from sys db to information_schema Key: HIVE-22628 URL: https://issues.apache.org/jira/browse/HIVE-22628 Project: Hive Issue Type: Improvement Affects Versions: 4.0.0 Reporter: Zoltan Chovan Assignee: Zoltan Chovan Fix For: 4.0.0 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (HIVE-22627) Add schema changes introduced in HIVE-21443 to the schema upgrade scripts
Zoltan Chovan created HIVE-22627: Summary: Add schema changes introduced in HIVE-21443 to the schema upgrade scripts Key: HIVE-22627 URL: https://issues.apache.org/jira/browse/HIVE-22627 Project: Hive Issue Type: Improvement Reporter: Zoltan Chovan Assignee: Zoltan Chovan -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (HIVE-22626) Fix TestStatsReplicationScenariosACIDNoAutogather
Zoltan Haindrich created HIVE-22626: --- Summary: Fix TestStatsReplicationScenariosACIDNoAutogather Key: HIVE-22626 URL: https://issues.apache.org/jira/browse/HIVE-22626 Project: Hive Issue Type: Bug Components: Test Reporter: Zoltan Haindrich this test is running "alone" because but still; it sometimes runs more than 40m which results in a timeout a jira search reveals that was pretty common: https://issues.apache.org/jira/issues/?jql=text%20~%20%22TestStatsReplicationScenariosACIDNoAutogather%22%20order%20by%20updated%20desc -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (HIVE-22625) Syntax Error in findPotentialCompactions SQL query for MySql/Postgres
Denys Kuzmenko created HIVE-22625: - Summary: Syntax Error in findPotentialCompactions SQL query for MySql/Postgres Key: HIVE-22625 URL: https://issues.apache.org/jira/browse/HIVE-22625 Project: Hive Issue Type: Bug Reporter: Denys Kuzmenko -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (HIVE-22624) Fix results_cache_invalidation and results_cache_lifetime
Zoltan Haindrich created HIVE-22624:
---
Summary: Fix results_cache_invalidation and results_cache_lifetime
Key: HIVE-22624
URL: https://issues.apache.org/jira/browse/HIVE-22624
Project: Hive
Issue Type: Sub-task
Reporter: Zoltan Haindrich
these tests are falling together
* they both run on hive-ptest-slaves-8e2.c.gcp-hive-upstream.internal
* I suspect that they are fighting for the ownership /tmp/hive/_resultscache_
exception#1
{code}
2019-12-11T00:16:02,564 INFO [22d96536-983a-417d-b10c-aa1766ba0994 main]
results.QueryResultsCache: Initializing query results cache at
/tmp/hive/_resultscache_
2019-12-11T00:16:02,569 DEBUG [22d96536-983a-417d-b10c-aa1766ba0994 main]
exec.Utilities: HDFS dir: /tmp/hive/_resultscache_ with schema null,
permission: rwxr-xr-x
2019-12-11T00:16:02,571 ERROR [22d96536-983a-417d-b10c-aa1766ba0994 main]
ql.Driver: FAILED: IllegalStateException java.lang.RuntimeException: The dir:
/tmp/hive/_resultscache_ on HDFS should be writable. Curren
t permissions are: rwxr-xr-x
java.lang.IllegalStateException: java.lang.RuntimeException: The dir:
/tmp/hive/_resultscache_ on HDFS should be writable. Current permissions are:
rwxr-xr-x
at
org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.checkResultsCache(SemanticAnalyzer.java:15230)
at
org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(SemanticAnalyzer.java:12467)
at
org.apache.hadoop.hive.ql.parse.CalcitePlanner.analyzeInternal(CalcitePlanner.java:359)
at
org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:285)
at
org.apache.hadoop.hive.ql.parse.ExplainSemanticAnalyzer.analyzeInternal(ExplainSemanticAnalyzer.java:171)
at
org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:285)
{code}
exception#2
{code}
2019-12-11T00:15:06,156 ERROR [5f435468-f02b-4587-ae48-51bfdbed main]
ql.Driver: FAILED: IllegalStateException java.lang.RuntimeException: The dir:
/tmp/hive/_resultscache_ on HDFS should be writable. Curren
t permissions are: rwxr-xr-x
java.lang.IllegalStateException: java.lang.RuntimeException: The dir:
/tmp/hive/_resultscache_ on HDFS should be writable. Current permissions are:
rwxr-xr-x
at
org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.checkResultsCache(SemanticAnalyzer.java:15230)
at
org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(SemanticAnalyzer.java:12467)
at
org.apache.hadoop.hive.ql.parse.CalcitePlanner.analyzeInternal(CalcitePlanner.java:359)
{code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
Unstable tests ruin our days!
Hey all! Recently the number of unstable tests have rised above the "usual" level... These are usually hard to fix - and in most cases need a deep dive in the area where the test operates. Because of that I tend to just reattach the patch to the jira to get another run in a day or so... The downside of the above approach is that reattaching on unrelated failures has a positive hivqqa queuesize coefficient. There is another downside which might not be obvious first: it reduces the trust in the system and as a result there were cases when I did reattach the patch; but it was a genuine failure...it seemed unrelated; but actually it was. Instead of continuing to reattach patches every day; I would like to propose a way to handle them: * check that the falling test has nothing to do with the actual patch * it's important to be able to run test on our machines - but the most important is to maintain that HiveQA is able to run them successfully; for this reason I think having 2 HiveQA runs for the same changeset where in one of them the unstable test fails is the best * you can search the jira for the testcase and look if other patches have also bumped into it * ? * add a comment about that you are about the disable the test in HIVE-22621 and commit it * I think it would be ok to skip the regular code change process * create a new subtask under HIVE-22619 with the details you know about the falling testcase * (resubmit your patch) What do you think? cheers, Zoltan
[jira] [Created] (HIVE-22623) Fix TestLlapSignerImpl.testSigning
Zoltan Haindrich created HIVE-22623:
---
Summary: Fix TestLlapSignerImpl.testSigning
Key: HIVE-22623
URL: https://issues.apache.org/jira/browse/HIVE-22623
Project: Hive
Issue Type: Sub-task
Components: Test
Reporter: Zoltan Haindrich
sometimes it fails with the exception:
{code}
Error Message
The ID didn't change but we couldn't get the key 2
Stacktrace
java.lang.AssertionError: The ID didn't change but we couldn't get the key 2
at
org.apache.hadoop.hive.llap.security.TestLlapSignerImpl$FakeSecretManager.getCurrentKey(TestLlapSignerImpl.java:196)
at
org.apache.hadoop.hive.llap.security.LlapSignerImpl.serializeAndSign(LlapSignerImpl.java:50)
at
org.apache.hadoop.hive.llap.security.TestLlapSignerImpl.testSigning(TestLlapSignerImpl.java:50)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
at
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
at
org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
at
org.junit.internal.runners.statements.FailOnTimeout$StatementThread.run(FailOnTimeout.java:74)
Standard Error
{code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (HIVE-22622) Hive allows to create a struct with duplicate attribute names
Denys Kuzmenko created HIVE-22622:
-
Summary: Hive allows to create a struct with duplicate attribute
names
Key: HIVE-22622
URL: https://issues.apache.org/jira/browse/HIVE-22622
Project: Hive
Issue Type: Bug
Reporter: Denys Kuzmenko
When you create at table with a struct with twice the same attribute name, hive
allow you to create it.
create table test_struct( duplicateColumn struct);
You can insert data into it :
insert into test_struct select named_struct("id",1,"id",1);
But you can not read it :
select * from test_struct;
Return : java.io.IOException: java.io.IOException: Error reading file:
hdfs://.../test_struct/delta_001_001_/bucket_0 ,
We can create and insert. but fail on read the Struct part of the tables. We
can still read all other columns (if we have more than one) but not the struct
anymore.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (HIVE-22621) Disable unstable tests
Zoltan Haindrich created HIVE-22621: --- Summary: Disable unstable tests Key: HIVE-22621 URL: https://issues.apache.org/jira/browse/HIVE-22621 Project: Hive Issue Type: Sub-task Components: Tests Reporter: Zoltan Haindrich -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (HIVE-22620) Fix TestSSL
Zoltan Haindrich created HIVE-22620: --- Summary: Fix TestSSL Key: HIVE-22620 URL: https://issues.apache.org/jira/browse/HIVE-22620 Project: Hive Issue Type: Sub-task Reporter: Zoltan Haindrich -- This message was sent by Atlassian Jira (v8.3.4#803005)
