[jira] [Created] (HIVE-20836) Fix TestJdbcDriver2.testYarnATSGuid flakiness
Peter Vary created HIVE-20836:
-
Summary: Fix TestJdbcDriver2.testYarnATSGuid flakiness
Key: HIVE-20836
URL: https://issues.apache.org/jira/browse/HIVE-20836
Project: Hive
Issue Type: Test
Components: JDBC
Reporter: Peter Vary
Assignee: Peter Vary
Seen flakiness in internal test.
{code:java}
Error Message
Failed to set the YARN ATS Guid
Stacktrace
java.lang.AssertionError: Failed to set the YARN ATS Guid
at org.junit.Assert.fail(Assert.java:88)
at
org.apache.hive.jdbc.TestJdbcDriver2.testYarnATSGuid(TestJdbcDriver2.java:2434){code}
The query finished too fast, and the GUID thread did not try to check the value.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (HIVE-20837) Tez AM container do not get killed after killing hive jvm when HS2 is not up
mrunmayi Sahasrabudhe created HIVE-20837: Summary: Tez AM container do not get killed after killing hive jvm when HS2 is not up Key: HIVE-20837 URL: https://issues.apache.org/jira/browse/HIVE-20837 Project: Hive Issue Type: Bug Components: Hive Affects Versions: 2.3.2 Reporter: mrunmayi Sahasrabudhe Reproduction steps : 1. Start hive by setting execution engine to tez 3. Run any command 4. yarn application -list 5. Kill the command before it finishes. 6. check in yarn application -list for the same application id. It is visible till upto 10mins and gets shutdown due to timeout. Ideally it should kill all the non default sessions. In case of Non HS2 all the session are non default. Other details: 1. This issue is reproducible with hive-2.1.1 and not in hive1.2 3. Does status of application should be shown as killed or succeed should be decided. for that refer QTEZ-137 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (HIVE-20839) "Cannot find field" error during dynamically partitioned hash join
Jason Dere created HIVE-20839:
-
Summary: "Cannot find field" error during dynamically partitioned
hash join
Key: HIVE-20839
URL: https://issues.apache.org/jira/browse/HIVE-20839
Project: Hive
Issue Type: Bug
Components: Query Processor
Reporter: Jason Dere
Assignee: Jason Dere
{noformat}
2018-10-11T04:40:22,724 ERROR [TezTR-85144_8944_1085_28_996_2
(1539092085144_8944_1085_28_000996_2)] tez.ReduceRecordProcessor: Hit error
while closing operators - failing tree
2018-10-11T04:40:22,724 ERROR [TezTR-85144_8944_1085_28_996_2
(1539092085144_8944_1085_28_000996_2)] tez.TezProcessor:
java.lang.RuntimeException: cannot find field _col304 from [0:_col0, 1:_col1,
2:_col2, 3:_col3, 4:_col4, 5:_col5, 6:_col6, 7:_col7, 8:_col8, 9:_col9,
10:_col10, 11:_col11, 12:_col12, 13:_col13, 14:_col15, 15:_col16, 16:_col17,
17:_col18, 18:_col19, 19:_col20, 20:_col21, 21:_col22, 22:_col23, 23:_col24,
24:_col25, 25:_col26, 26:_col27, 27:_col28, 28:_col29, 29:_col30, 30:_col31,
31:_col32, 32:_col33, 33:_col34, 34:_col35, 35:_col36, 36:_col37, 37:_col38,
38:_col39, 39:_col40, 40:_col41, 41:_col42, 42:_col43, 43:_col44, 44:_col45,
45:_col46, 46:_col47, 47:_col48, 48:_col49, 49:_col50, 50:_col51, 51:_col52,
52:_col53, 53:_col54, 54:_col55, 55:_col56, 56:_col57, 57:_col58, 58:_col59,
59:_col60, 60:_col61, 61:_col62, 62:_col63, 63:_col64, 64:_col65, 65:_col66,
66:_col67, 67:_col68, 68:_col70, 69:_col72, 70:_col73, 71:_col74, 72:_col75,
73:_col76, 74:_col77, 75:_col78, 76:_col79, 77:_col80, 78:_col81, 79:_col82,
80:_col83, 81:_col84, 82:_col85, 83:_col86, 84:_col87, 85:_col88, 86:_col89,
87:_col90, 88:_col91, 89:_col92, 90:_col93, 91:_col94, 92:_col95, 93:_col96,
94:_col97, 95:_col98, 96:_col99, 97:_col100, 98:_col101, 99:_col102,
100:_col103, 101:_col104, 102:_col105, 103:_col106, 104:_col107, 105:_col108,
106:_col109, 107:_col110, 108:_col111, 109:_col112, 110:_col113, 111:_col114,
112:_col115, 113:_col116, 114:_col117, 115:_col118, 116:_col119, 117:_col120,
118:_col121, 119:_col122, 120:_col123, 121:_col124, 122:_col125, 123:_col126,
124:_col127, 125:_col128, 126:_col129, 127:_col130, 128:_col131, 129:_col132,
130:_col133, 131:_col134, 132:_col135, 133:_col136, 134:_col137, 135:_col138,
136:_col139, 137:_col140, 138:_col141, 139:_col142, 140:_col143, 141:_col144,
142:_col145, 143:_col146, 144:_col147, 145:_col148, 146:_col149, 147:_col150,
148:_col151, 149:_col152, 150:_col153, 151:_col154, 152:_col155, 153:_col156,
154:_col157, 155:_col158, 156:_col159, 157:_col160, 158:_col161, 159:_col162,
160:_col163, 161:_col164, 162:_col165, 163:_col166, 164:_col167, 165:_col168,
166:_col169, 167:_col170, 168:_col171, 169:_col318]
at
org.apache.hadoop.hive.serde2.objectinspector.ObjectInspectorUtils.getStandardStructFieldRef(ObjectInspectorUtils.java:485)
at
org.apache.hadoop.hive.serde2.objectinspector.StandardStructObjectInspector.getStructFieldRef(StandardStructObjectInspector.java:153)
at
org.apache.hadoop.hive.ql.exec.ExprNodeColumnEvaluator.initialize(ExprNodeColumnEvaluator.java:80)
at
org.apache.hadoop.hive.ql.exec.JoinUtil.getObjectInspectorsFromEvaluators(JoinUtil.java:91)
at
org.apache.hadoop.hive.ql.exec.AbstractMapJoinOperator.initializeOp(AbstractMapJoinOperator.java:74)
at
org.apache.hadoop.hive.ql.exec.MapJoinOperator.initializeOp(MapJoinOperator.java:144)
at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:374)
at
org.apache.hadoop.hive.ql.exec.tez.ReduceRecordProcessor.init(ReduceRecordProcessor.java:195)
at
org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:188)
at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.run(TezProcessor.java:172)
at
org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:370)
at
org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:73)
at
org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:61)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1869)
{noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (HIVE-20838) Timestamps with timezone are set to null when using the streaming API
Jaume M created HIVE-20838:
--
Summary: Timestamps with timezone are set to null when using the
streaming API
Key: HIVE-20838
URL: https://issues.apache.org/jira/browse/HIVE-20838
Project: Hive
Issue Type: Bug
Components: Hive
Affects Versions: 3.1.0
Reporter: Jaume M
For example:
{code}
beeline> create table default.timest (a TIMESTAMP) stored as orc " +
"TBLPROPERTIES('transactional'='true')
# And then:
connection.write("2018-10-19 10:35:00 America/Los_Angeles".getBytes());
{code}
inserts NULL.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
Review Request 69215: HIVE-20778 Join reordering may not be triggered if all joins in plan are created by decorrelation logic
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/69215/ --- Review request for hive, Ashutosh Chauhan and Jesús Camacho Rodríguez. Bugs: HIVE-20778 https://issues.apache.org/jira/browse/HIVE-20778 Repository: hive-git Description --- Trigger join reordering for subqueries Diffs - ql/src/java/org/apache/hadoop/hive/ql/parse/CalcitePlanner.java 82e975a50d ql/src/test/queries/clientpositive/subquery_notin.q a2d93dfd86 ql/src/test/queries/clientpositive/subquery_select.q 80618c777b ql/src/test/results/clientpositive/constant_prop_3.q.out d106cd325a ql/src/test/results/clientpositive/llap/dynamic_semijoin_reduction_2.q.out 801416d563 ql/src/test/results/clientpositive/llap/explainuser_1.q.out f240468558 ql/src/test/results/clientpositive/llap/lineage3.q.out 9bec309c9c ql/src/test/results/clientpositive/llap/mapjoin_hint.q.out ac505a5c1e ql/src/test/results/clientpositive/llap/optimize_join_ptp.q.out a68275b9e9 ql/src/test/results/clientpositive/llap/subquery_in.q.out 12e8c54480 ql/src/test/results/clientpositive/llap/subquery_in_having.q.out 4f41f246f1 ql/src/test/results/clientpositive/llap/subquery_multi.q.out d8d42ee10b ql/src/test/results/clientpositive/llap/subquery_notin.q.out d981733bd3 ql/src/test/results/clientpositive/llap/subquery_null_agg.q.out fab963b1fd ql/src/test/results/clientpositive/llap/subquery_scalar.q.out 166bd60ce2 ql/src/test/results/clientpositive/llap/subquery_select.q.out 854d215a23 ql/src/test/results/clientpositive/llap/subquery_views.q.out a0bc64ebf1 ql/src/test/results/clientpositive/llap/vector_groupby_mapjoin.q.out a9180bcdc6 ql/src/test/results/clientpositive/llap/vector_mapjoin_reduce.q.out 3b8ee96a40 ql/src/test/results/clientpositive/semijoin5.q.out c047c6d265 ql/src/test/results/clientpositive/spark/spark_dynamic_partition_pruning_3.q.out c2d821d653 ql/src/test/results/clientpositive/spark/spark_explainuser_1.q.out f5a4c9ad86 ql/src/test/results/clientpositive/spark/subquery_in.q.out 535f01b1fd ql/src/test/results/clientpositive/spark/subquery_multi.q.out 5f7b2661fa ql/src/test/results/clientpositive/spark/subquery_notin.q.out 39080da5c9 ql/src/test/results/clientpositive/spark/subquery_null_agg.q.out 0fefcbe408 ql/src/test/results/clientpositive/spark/subquery_scalar.q.out b3252f5415 ql/src/test/results/clientpositive/spark/subquery_select.q.out ad7c8a3fc7 ql/src/test/results/clientpositive/spark/subquery_views.q.out 8a7e4c7d18 ql/src/test/results/clientpositive/spark/vector_mapjoin_reduce.q.out 9f80711c65 ql/src/test/results/clientpositive/subquery_notin_having.q.out dd31d4ab6a ql/src/test/results/clientpositive/vector_groupby_mapjoin.q.out 70cdd7ae5b Diff: https://reviews.apache.org/r/69215/diff/1/ Testing --- Thanks, Vineet Garg
[jira] [Created] (HIVE-20841) LLAP: Make dynamic ports configurable
Prasanth Jayachandran created HIVE-20841: Summary: LLAP: Make dynamic ports configurable Key: HIVE-20841 URL: https://issues.apache.org/jira/browse/HIVE-20841 Project: Hive Issue Type: Bug Components: llap Affects Versions: 4.0.0, 3.2.0 Reporter: Prasanth Jayachandran Assignee: Prasanth Jayachandran Fix For: 4.0.0, 3.2.0 Some ports in llap -> tez interaction code uses dynamic ports, provide an option to make them configurable to facilitate adding them to iptable rules in some environment. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (HIVE-20840) sysdb test is not updated and fails on update
Sergey Shelukhin created HIVE-20840: --- Summary: sysdb test is not updated and fails on update Key: HIVE-20840 URL: https://issues.apache.org/jira/browse/HIVE-20840 Project: Hive Issue Type: Bug Reporter: Sergey Shelukhin Attachments: HIVE-20840.test.patch -- This message was sent by Atlassian JIRA (v7.6.3#76005)
Re: [VOTE] Apache Hive 3.1.1 Release Candidate 0
* Verified signatures and checksums. * Reviewed git rc tag contents * Build src tar.gz * Untarred bin.tar.gz and ran queries +1 to the release On Wed, Oct 24, 2018 at 4:59 PM Daniel Dai wrote: > > Apache Hive 3.1.1 Release Candidate 0 is available here: > > http://people.apache.org/~daijy/apache-hive-3.1.1-rc-0/ > > Maven artifacts are available here: > > https://repository.apache.org/content/repositories/orgapachehive-1092 > > Source tag for RCN is at: > > https://github.com/apache/hive/tree/release-3.1.1-rc0 > > Voting will conclude in 72 hours. > > Hive PMC Members: Please test and vote. > > Thanks. >
[jira] [Created] (HIVE-20843) RELY constraints on primary keys and foreign keys are not recognized
Anurag Mantripragada created HIVE-20843: --- Summary: RELY constraints on primary keys and foreign keys are not recognized Key: HIVE-20843 URL: https://issues.apache.org/jira/browse/HIVE-20843 Project: Hive Issue Type: Bug Affects Versions: 3.0.0, 2.1.1 Reporter: Anurag Mantripragada Assignee: Anurag Mantripragada Fix For: 3.0.0 Hive doesn't recognize RELY constraints after https://issues.apache.org/jira/browse/HIVE-13076. The issue is in BaseSemanticAnalyzer.java where we assign RELY. An unrelated patch fixed this issue in later versions. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
Re: Review Request 69019: HIVE-20617 Fix type of constants in IN expressions to have correct type
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/69019/ --- (Updated Oct. 30, 2018, 11:32 p.m.) Review request for hive, Ashutosh Chauhan, Jesús Camacho Rodríguez, and Vineet Garg. Changes --- patch#14 opens only safe expressions close supports subtrees pointlookoptimizer also works on projections probably some pending q.out changes Bugs: HIVE-20617 https://issues.apache.org/jira/browse/HIVE-20617 Repository: hive-git Description --- For IN expressions the types were never corrected; and pointlookupoptimizer was probably leaving behind fields already which were uncomparable; HIVE-20296 exposed it further by changing the minimal number from 32 to 2. This change generalizes the retyping of constants to also run it for the IN operator ; and also for struct-s. Diffs (updated) - common/src/java/org/apache/hadoop/hive/common/type/HiveChar.java 29dc06dca1a ql/src/java/org/apache/hadoop/hive/ql/ErrorMsg.java e7d71595c7a ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/rules/HivePointLookupOptimizerRule.java 04800cca91b ql/src/java/org/apache/hadoop/hive/ql/parse/CalcitePlanner.java 82e975a50de ql/src/java/org/apache/hadoop/hive/ql/parse/TypeCheckCtx.java 1b56ecd0440 ql/src/java/org/apache/hadoop/hive/ql/parse/TypeCheckProcFactory.java 4968d16876c ql/src/java/org/apache/hadoop/hive/ql/plan/ExprNodeDescUtils.java c274fd7cc9b ql/src/test/queries/clientpositive/groupby_multi_single_reducer3.q 94aea35c422 ql/src/test/queries/clientpositive/in_typecheck_char.q PRE-CREATION ql/src/test/queries/clientpositive/in_typecheck_pointlook.q PRE-CREATION ql/src/test/queries/clientpositive/in_typecheck_varchar.q PRE-CREATION ql/src/test/queries/clientpositive/in_typecheck_x1.q PRE-CREATION ql/src/test/queries/clientpositive/join45X.q PRE-CREATION ql/src/test/queries/clientpositive/pcs.q 2e7eff472b5 ql/src/test/queries/clientpositive/vector_struct_in.q 49a9374bd13 ql/src/test/results/clientpositive/alter_partition_coltype.q.out f6c3c5642ea ql/src/test/results/clientpositive/in_typecheck_char.q.out PRE-CREATION ql/src/test/results/clientpositive/in_typecheck_pointlook.q.out PRE-CREATION ql/src/test/results/clientpositive/in_typecheck_varchar.q.out PRE-CREATION ql/src/test/results/clientpositive/in_typecheck_x1.q.out PRE-CREATION ql/src/test/results/clientpositive/infer_const_type.q.out e1d7de54225 ql/src/test/results/clientpositive/join45.q.out 47aaf7d0abc ql/src/test/results/clientpositive/join45X.q.out PRE-CREATION ql/src/test/results/clientpositive/join47.q.out 4d9e937815e ql/src/test/results/clientpositive/llap/dec_str.q.out 554031e952e ql/src/test/results/clientpositive/llap/explainuser_1.q.out f240468558e ql/src/test/results/clientpositive/llap/lineage3.q.out 9bec309c9c9 ql/src/test/results/clientpositive/llap/vectorization_13.q.out 4ce654f960b ql/src/test/results/clientpositive/llap/vectorization_6.q.out a2f730beca2 ql/src/test/results/clientpositive/llap/vectorization_8.q.out 21ce7b8ebd0 ql/src/test/results/clientpositive/llap/vectorization_short_regress.q.out 7f1c6a295e1 ql/src/test/results/clientpositive/mapjoin47.q.out 294dd69de5c ql/src/test/results/clientpositive/parquet_vectorization_13.q.out 0efce98b555 ql/src/test/results/clientpositive/parquet_vectorization_6.q.out 0bb68883640 ql/src/test/results/clientpositive/parquet_vectorization_8.q.out 957bd7b264c ql/src/test/results/clientpositive/pcs.q.out 4a89fb0787e ql/src/test/results/clientpositive/ppd_udf_col.q.out 814fb5afcfd ql/src/test/results/clientpositive/spark/cbo_simple_select.q.out acf91bf1787 ql/src/test/results/clientpositive/spark/parquet_vectorization_13.q.out 38122393439 ql/src/test/results/clientpositive/spark/parquet_vectorization_6.q.out 6108457aad3 ql/src/test/results/clientpositive/spark/parquet_vectorization_8.q.out 3352dedc584 ql/src/test/results/clientpositive/spark/spark_explainuser_1.q.out f5a4c9ad861 ql/src/test/results/clientpositive/spark/subquery_scalar.q.out b3252f54158 ql/src/test/results/clientpositive/spark/vectorization_13.q.out 34ec9c42dd3 ql/src/test/results/clientpositive/spark/vectorization_6.q.out 5679bb8cfa4 ql/src/test/results/clientpositive/spark/vectorization_short_regress.q.out 231dea6de3e ql/src/test/results/clientpositive/vector_struct_in.q.out 546ef02fa42 ql/src/test/results/clientpositive/vectorization_13.q.out 8897f8427ff ql/src/test/results/clientpositive/vectorization_6.q.out 8dedb63e7da ql/src/test/results/clientpositive/vectorization_8.q.out d81df76a2f6 Diff: https://reviews.apache.org/r/69019/diff/4/ Changes: https://reviews.apache.org/r/69019/diff/3-4/ Testing --- Thanks, Zoltan Haindrich
Ptests not working
It seems we are not getting executors… Is there any known (infra) issue? Thanks, Jesús
[jira] [Created] (HIVE-20842) Fix logic introduced in HIVE-20660 to estimate statistics for group by
Vineet Garg created HIVE-20842: -- Summary: Fix logic introduced in HIVE-20660 to estimate statistics for group by Key: HIVE-20842 URL: https://issues.apache.org/jira/browse/HIVE-20842 Project: Hive Issue Type: Bug Components: Statistics Affects Versions: 4.0.0 Reporter: Vineet Garg Assignee: Vineet Garg HIVE-20660 introduced better estimation for group by operator. But the logic did not account for Partial and Full group by separately. For partial group by parallelism (i.e. number of tasks) should be taken into account. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
Re: [ANNOUNCE] New PMC Member : Zoltan
Congrats! On 10/30/18, 10:16 PM, "Prasanth Jayachandran" wrote: Congratulations! > On Oct 30, 2018, at 10:15 PM, Deepak Jaiswal wrote: > > Congratulations Zoltan! > > On 10/30/18, 10:08 PM, "Ashutosh Chauhan" wrote: > > Hello Hive community, > >I'm pleased to announce that Zoltan Haindrich has accepted the Apache >Hive PMC's invitation, and is our newest PMC member. Many thanks to >Zoltan for all of his hard work. > >Please join me in congratulating Zoltan! > >Thanks, >Ashutosh > >
Re: [ANNOUNCE] New PMC Member : Zoltan
Congratulations Zoltan! On 10/30/18, 10:08 PM, "Ashutosh Chauhan" wrote: Hello Hive community, I'm pleased to announce that Zoltan Haindrich has accepted the Apache Hive PMC's invitation, and is our newest PMC member. Many thanks to Zoltan for all of his hard work. Please join me in congratulating Zoltan! Thanks, Ashutosh
[ANNOUNCE] New PMC Member : Zoltan
Hello Hive community, I'm pleased to announce that Zoltan Haindrich has accepted the Apache Hive PMC's invitation, and is our newest PMC member. Many thanks to Zoltan for all of his hard work. Please join me in congratulating Zoltan! Thanks, Ashutosh
Re: [ANNOUNCE] New PMC Member : Zoltan
Congratulations! > On Oct 30, 2018, at 10:15 PM, Deepak Jaiswal wrote: > > Congratulations Zoltan! > > On 10/30/18, 10:08 PM, "Ashutosh Chauhan" wrote: > > Hello Hive community, > >I'm pleased to announce that Zoltan Haindrich has accepted the Apache >Hive PMC's invitation, and is our newest PMC member. Many thanks to >Zoltan for all of his hard work. > >Please join me in congratulating Zoltan! > >Thanks, >Ashutosh > >
