[jira] [Commented] (HIVE-12181) Change hive.stats.fetch.column.stats value to true for MiniTezCliDriver
[
https://issues.apache.org/jira/browse/HIVE-12181?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15411245#comment-15411245
]
Hive QA commented on HIVE-12181:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12822495/HIVE-12181.12.patch
{color:green}SUCCESS:{color} +1 due to 12 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 8 failed/errored test(s), 10441 tests
executed
*Failed tests:*
{noformat}
TestMsgBusConnection - did not produce a TEST-*.xml file
TestQueryLifeTimeHook - did not produce a TEST-*.xml file
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_explainuser_1
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_tez_dynpart_hashjoin_1
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_tez_join_hash
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_transform_ppr1
org.apache.hadoop.hive.llap.tezplugins.TestLlapTaskSchedulerService.testDelayedLocalityNodeCommErrorImmediateAllocation
org.apache.hive.jdbc.TestJdbcWithMiniHS2.testAddJarConstructorUnCaching
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-MASTER-Build/809/testReport
Console output:
https://builds.apache.org/job/PreCommit-HIVE-MASTER-Build/809/console
Test logs:
http://ec2-204-236-174-241.us-west-1.compute.amazonaws.com/logs/PreCommit-HIVE-MASTER-Build-809/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 8 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12822495 - PreCommit-HIVE-MASTER-Build
> Change hive.stats.fetch.column.stats value to true for MiniTezCliDriver
> ---
>
> Key: HIVE-12181
> URL: https://issues.apache.org/jira/browse/HIVE-12181
> Project: Hive
> Issue Type: Improvement
> Components: Statistics
>Reporter: Ashutosh Chauhan
>Assignee: Ashutosh Chauhan
> Attachments: HIVE-12181.1.patch, HIVE-12181.10.patch,
> HIVE-12181.12.patch, HIVE-12181.2.patch, HIVE-12181.3.patch,
> HIVE-12181.4.patch, HIVE-12181.7.patch, HIVE-12181.8.patch,
> HIVE-12181.9.patch, HIVE-12181.patch, HIVE-12181.patch
>
>
> There was a performance concern earlier, but HIVE-7587 has fixed that. We can
> change the default to true now.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-14412) Add a timezone-aware timestamp
[ https://issues.apache.org/jira/browse/HIVE-14412?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15411204#comment-15411204 ] Rui Li commented on HIVE-14412: --- Here's the [console output|https://builds.apache.org/view/H-L/view/Hive/job/PreCommit-HIVE-MASTER-Build/785/console]. Seems it failed when building hive-ptest. I googled about the error and found some related discussions: [link1|http://markmail.org/message/c7icqksbxcdxcots#query:+page:1+mid:uto57imdxbp2no2f+state:results], [link2|https://github.com/google/gson/issues/819]. However I can't reproduce this locally with maven 3.3.9, and not sure why other tests don't hit this error. [~spena] do you know how to fix this? Thanks. > Add a timezone-aware timestamp > -- > > Key: HIVE-14412 > URL: https://issues.apache.org/jira/browse/HIVE-14412 > Project: Hive > Issue Type: Sub-task > Components: Hive >Reporter: Rui Li >Assignee: Rui Li > Attachments: HIVE-14412.1.patch, HIVE-14412.1.patch > > > Java's Timestamp stores the time elapsed since the epoch. While it's by > itself unambiguous, ambiguity comes when we parse a string into timestamp, or > convert a timestamp to string, causing problems like HIVE-14305. > To solve the issue, I think we should make timestamp aware of timezone. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-14444) Upgrade qtest execution framework to junit4 - migrate most of them
[
https://issues.apache.org/jira/browse/HIVE-1?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15411166#comment-15411166
]
Hive QA commented on HIVE-1:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12822486/HIVE-1.2.patch
{color:green}SUCCESS:{color} +1 due to 21 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 45 failed/errored test(s), 10389 tests
executed
*Failed tests:*
{noformat}
TestMinimrCliDriver-bucket_num_reducers.q-schemeAuthority2.q-scriptfile1.q-and-1-more
- did not produce a TEST-*.xml file
TestMinimrCliDriver-external_table_with_space_in_location_path.q-bucketmapjoin6.q-temp_table_external.q-and-1-more
- did not produce a TEST-*.xml file
TestMinimrCliDriver-groupby2.q-bucket6.q-input16_cc.q-and-1-more - did not
produce a TEST-*.xml file
TestMinimrCliDriver-import_exported_table.q-file_with_header_footer.q-orc_mr_pathalias.q-and-1-more
- did not produce a TEST-*.xml file
TestMinimrCliDriver-infer_bucket_sort_num_buckets.q-index_bitmap3.q-join1.q-and-1-more
- did not produce a TEST-*.xml file
TestMinimrCliDriver-infer_bucket_sort_reducers_power_two.q-scriptfile1_win.q-bucketmapjoin7.q-and-1-more
- did not produce a TEST-*.xml file
TestMinimrCliDriver-insert_dir_distcp.q-non_native_window_udf.q-infer_bucket_sort_merge.q-and-1-more
- did not produce a TEST-*.xml file
TestMinimrCliDriver-leftsemijoin_mr.q-infer_bucket_sort_map_operators.q-bucket4.q-and-1-more
- did not produce a TEST-*.xml file
TestMinimrCliDriver-list_bucket_dml_10.q-orc_merge_diff_fs.q-skewjoin_onesideskew.q
- did not produce a TEST-*.xml file
TestMinimrCliDriver-load_fs2.q-root_dir_external_table.q-join_acid_non_acid.q-and-1-more
- did not produce a TEST-*.xml file
TestMinimrCliDriver-parallel_orderby.q-bucketizedhiveinputformat.q-index_bitmap_auto.q-and-1-more
- did not produce a TEST-*.xml file
TestMinimrCliDriver-reduce_deduplicate.q-disable_merge_for_bucketing.q-load_hdfs_file_with_space_in_the_name.q-and-1-more
- did not produce a TEST-*.xml file
TestMinimrCliDriver-table_nonprintable.q-infer_bucket_sort_bucketed_table.q-uber_reduce.q-and-1-more
- did not produce a TEST-*.xml file
TestMsgBusConnection - did not produce a TEST-*.xml file
TestQueryLifeTimeHook - did not produce a TEST-*.xml file
org.apache.hadoop.hive.cli.TestContribCliDriver.testCliDriver[dboutput]
org.apache.hadoop.hive.cli.TestContribCliDriver.testCliDriver[serde_typedbytes5]
org.apache.hadoop.hive.ql.parse.TestParseNegativeDriver.testCliDriver[ambiguous_join_col]
org.apache.hadoop.hive.ql.parse.TestParseNegativeDriver.testCliDriver[duplicate_alias]
org.apache.hadoop.hive.ql.parse.TestParseNegativeDriver.testCliDriver[garbage]
org.apache.hadoop.hive.ql.parse.TestParseNegativeDriver.testCliDriver[insert_wrong_number_columns]
org.apache.hadoop.hive.ql.parse.TestParseNegativeDriver.testCliDriver[invalid_create_table]
org.apache.hadoop.hive.ql.parse.TestParseNegativeDriver.testCliDriver[invalid_dot]
org.apache.hadoop.hive.ql.parse.TestParseNegativeDriver.testCliDriver[invalid_function_param2]
org.apache.hadoop.hive.ql.parse.TestParseNegativeDriver.testCliDriver[invalid_index]
org.apache.hadoop.hive.ql.parse.TestParseNegativeDriver.testCliDriver[invalid_select]
org.apache.hadoop.hive.ql.parse.TestParseNegativeDriver.testCliDriver[macro_reserved_word]
org.apache.hadoop.hive.ql.parse.TestParseNegativeDriver.testCliDriver[missing_overwrite]
org.apache.hadoop.hive.ql.parse.TestParseNegativeDriver.testCliDriver[nonkey_groupby]
org.apache.hadoop.hive.ql.parse.TestParseNegativeDriver.testCliDriver[quoted_string]
org.apache.hadoop.hive.ql.parse.TestParseNegativeDriver.testCliDriver[unknown_column1]
org.apache.hadoop.hive.ql.parse.TestParseNegativeDriver.testCliDriver[unknown_column2]
org.apache.hadoop.hive.ql.parse.TestParseNegativeDriver.testCliDriver[unknown_column3]
org.apache.hadoop.hive.ql.parse.TestParseNegativeDriver.testCliDriver[unknown_column4]
org.apache.hadoop.hive.ql.parse.TestParseNegativeDriver.testCliDriver[unknown_column5]
org.apache.hadoop.hive.ql.parse.TestParseNegativeDriver.testCliDriver[unknown_column6]
org.apache.hadoop.hive.ql.parse.TestParseNegativeDriver.testCliDriver[unknown_function1]
org.apache.hadoop.hive.ql.parse.TestParseNegativeDriver.testCliDriver[unknown_function2]
org.apache.hadoop.hive.ql.parse.TestParseNegativeDriver.testCliDriver[unknown_function3]
org.apache.hadoop.hive.ql.parse.TestParseNegativeDriver.testCliDriver[unknown_function4]
org.apache.hadoop.hive.ql.parse.TestParseNegativeDriver.testCliDriver[unknown_table1]
org.apache.hadoop.hive.ql.parse.TestParseNegativeDriver.testCliDriver[unknown_table2]
org.apache.hadoop.hive.ql.parse.TestParseNegativeDriver.testCliDriver[wrong_distinct1]
org.apache.hadoop.hive.ql.parse.TestParseNegativeDriver.testCliDriver[wrong_distinct2]
org.apache.hive.jdbc.TestJdbcWithMiniHS2.testAddJarConstructorUnCaching
{noformat}
[jira] [Updated] (HIVE-12181) Change hive.stats.fetch.column.stats value to true for MiniTezCliDriver
[ https://issues.apache.org/jira/browse/HIVE-12181?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashutosh Chauhan updated HIVE-12181: Attachment: HIVE-12181.12.patch > Change hive.stats.fetch.column.stats value to true for MiniTezCliDriver > --- > > Key: HIVE-12181 > URL: https://issues.apache.org/jira/browse/HIVE-12181 > Project: Hive > Issue Type: Improvement > Components: Statistics >Reporter: Ashutosh Chauhan >Assignee: Ashutosh Chauhan > Attachments: HIVE-12181.1.patch, HIVE-12181.10.patch, > HIVE-12181.12.patch, HIVE-12181.2.patch, HIVE-12181.3.patch, > HIVE-12181.4.patch, HIVE-12181.7.patch, HIVE-12181.8.patch, > HIVE-12181.9.patch, HIVE-12181.patch, HIVE-12181.patch > > > There was a performance concern earlier, but HIVE-7587 has fixed that. We can > change the default to true now. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-12181) Change hive.stats.fetch.column.stats value to true for MiniTezCliDriver
[ https://issues.apache.org/jira/browse/HIVE-12181?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashutosh Chauhan updated HIVE-12181: Status: Patch Available (was: Open) > Change hive.stats.fetch.column.stats value to true for MiniTezCliDriver > --- > > Key: HIVE-12181 > URL: https://issues.apache.org/jira/browse/HIVE-12181 > Project: Hive > Issue Type: Improvement > Components: Statistics >Reporter: Ashutosh Chauhan >Assignee: Ashutosh Chauhan > Attachments: HIVE-12181.1.patch, HIVE-12181.10.patch, > HIVE-12181.12.patch, HIVE-12181.2.patch, HIVE-12181.3.patch, > HIVE-12181.4.patch, HIVE-12181.7.patch, HIVE-12181.8.patch, > HIVE-12181.9.patch, HIVE-12181.patch, HIVE-12181.patch > > > There was a performance concern earlier, but HIVE-7587 has fixed that. We can > change the default to true now. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-12181) Change hive.stats.fetch.column.stats value to true for MiniTezCliDriver
[ https://issues.apache.org/jira/browse/HIVE-12181?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashutosh Chauhan updated HIVE-12181: Status: Open (was: Patch Available) > Change hive.stats.fetch.column.stats value to true for MiniTezCliDriver > --- > > Key: HIVE-12181 > URL: https://issues.apache.org/jira/browse/HIVE-12181 > Project: Hive > Issue Type: Improvement > Components: Statistics >Reporter: Ashutosh Chauhan >Assignee: Ashutosh Chauhan > Attachments: HIVE-12181.1.patch, HIVE-12181.10.patch, > HIVE-12181.2.patch, HIVE-12181.3.patch, HIVE-12181.4.patch, > HIVE-12181.7.patch, HIVE-12181.8.patch, HIVE-12181.9.patch, HIVE-12181.patch, > HIVE-12181.patch > > > There was a performance concern earlier, but HIVE-7587 has fixed that. We can > change the default to true now. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-14440) Fix default value of USE_DEPRECATED_CLI in cli.cmd
[ https://issues.apache.org/jira/browse/HIVE-14440?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15411158#comment-15411158 ] Ferdinand Xu commented on HIVE-14440: - Thanks [~vihangk1] for the contribution. LGTM +1 > Fix default value of USE_DEPRECATED_CLI in cli.cmd > -- > > Key: HIVE-14440 > URL: https://issues.apache.org/jira/browse/HIVE-14440 > Project: Hive > Issue Type: Sub-task > Components: CLI >Reporter: Vihang Karajgaonkar >Assignee: Vihang Karajgaonkar >Priority: Minor > Attachments: HIVE-14440.01.patch > > > cli.cmd script sets the default value of USE_DEPRECATED_CLI to false when it > is not set which is different than cli.sh which sets it to true. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (HIVE-13180) WindowingTableFunction fails with ClassCastException: org.apache.hadoop.io.DoubleWritable
[
https://issues.apache.org/jira/browse/HIVE-13180?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rajesh Balamohan resolved HIVE-13180.
-
Resolution: Fixed
> WindowingTableFunction fails with ClassCastException:
> org.apache.hadoop.io.DoubleWritable
> -
>
> Key: HIVE-13180
> URL: https://issues.apache.org/jira/browse/HIVE-13180
> Project: Hive
> Issue Type: Bug
> Components: Hive
>Reporter: Rajesh Balamohan
>
> This is with hive-master branch with tpcds-51 query.
> {noformat}
> ], TaskAttempt 3 failed, info=[Error: Failure while running task:
> attempt_1455662455106_2317_27_02_000284_3:java.lang.RuntimeException:
> java.lang.RuntimeException: org.apache.hadoop.hive.ql.metadata.HiveException:
> Hive Runtime Error while processing row (tag=0)
> {"key":{"_col0":63443,"_col1":"2000-01-04"},"value":{"_col0":10.75}}
> at
> org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:195)
> at
> org.apache.hadoop.hive.ql.exec.tez.TezProcessor.run(TezProcessor.java:160)
> at
> org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:354)
> at
> org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:71)
> at
> org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:59)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
> at
> org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:59)
> at
> org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:36)
> at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: java.lang.RuntimeException:
> org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while
> processing row (tag=0)
> {"key":{"_col0":63443,"_col1":"2000-01-04"},"value":{"_col0":10.75}}
> at
> org.apache.hadoop.hive.ql.exec.tez.ReduceRecordSource.pushRecord(ReduceRecordSource.java:288)
> at
> org.apache.hadoop.hive.ql.exec.tez.ReduceRecordProcessor.run(ReduceRecordProcessor.java:263)
> at
> org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:172)
> ... 14 more
> Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime
> Error while processing row (tag=0)
> {"key":{"_col0":63443,"_col1":"2000-01-04"},"value":{"_col0":10.75}}
> at
> org.apache.hadoop.hive.ql.exec.tez.ReduceRecordSource$GroupIterator.next(ReduceRecordSource.java:356)
> at
> org.apache.hadoop.hive.ql.exec.tez.ReduceRecordSource.pushRecord(ReduceRecordSource.java:278)
> ... 16 more
> Caused by: org.apache.hadoop.hive.ql.metadata.HiveException:
> java.lang.ClassCastException: java.lang.Object cannot be cast to
> org.apache.hadoop.io.DoubleWritable
> at
> org.apache.hadoop.hive.ql.exec.GroupByOperator.process(GroupByOperator.java:775)
> at
> org.apache.hadoop.hive.ql.exec.tez.ReduceRecordSource$GroupIterator.next(ReduceRecordSource.java:347)
> ... 17 more
> Caused by: java.lang.ClassCastException: java.lang.Object cannot be cast to
> org.apache.hadoop.io.DoubleWritable
> at
> org.apache.hadoop.hive.serde2.objectinspector.primitive.WritableDoubleObjectInspector.get(WritableDoubleObjectInspector.java:36)
> at
> org.apache.hadoop.hive.serde2.objectinspector.primitive.WritableDoubleObjectInspector.copyObject(WritableDoubleObjectInspector.java:41)
> at
> org.apache.hadoop.hive.serde2.objectinspector.ObjectInspectorUtils.copyToStandardObject(ObjectInspectorUtils.java:380)
> at
> org.apache.hadoop.hive.serde2.objectinspector.ObjectInspectorUtils.copyToStandardObject(ObjectInspectorUtils.java:324)
> at
> org.apache.hadoop.hive.ql.udf.ptf.WindowingTableFunction$WindowingIterator.next(WindowingTableFunction.java:1416)
> at
> org.apache.hadoop.hive.ql.exec.PTFOperator$PTFInvocation.finishPartition(PTFOperator.java:374)
> at
> org.apache.hadoop.hive.ql.exec.PTFOperator.process(PTFOperator.java:123)
> at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:837)
> at
>
[jira] [Commented] (HIVE-13180) WindowingTableFunction fails with ClassCastException: org.apache.hadoop.io.DoubleWritable
[
https://issues.apache.org/jira/browse/HIVE-13180?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15411154#comment-15411154
]
Rajesh Balamohan commented on HIVE-13180:
-
Sure [~cartershanklin]. Checked with recent builds and this is no longer an
issue. Will close this jira.
> WindowingTableFunction fails with ClassCastException:
> org.apache.hadoop.io.DoubleWritable
> -
>
> Key: HIVE-13180
> URL: https://issues.apache.org/jira/browse/HIVE-13180
> Project: Hive
> Issue Type: Bug
> Components: Hive
>Reporter: Rajesh Balamohan
>
> This is with hive-master branch with tpcds-51 query.
> {noformat}
> ], TaskAttempt 3 failed, info=[Error: Failure while running task:
> attempt_1455662455106_2317_27_02_000284_3:java.lang.RuntimeException:
> java.lang.RuntimeException: org.apache.hadoop.hive.ql.metadata.HiveException:
> Hive Runtime Error while processing row (tag=0)
> {"key":{"_col0":63443,"_col1":"2000-01-04"},"value":{"_col0":10.75}}
> at
> org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:195)
> at
> org.apache.hadoop.hive.ql.exec.tez.TezProcessor.run(TezProcessor.java:160)
> at
> org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:354)
> at
> org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:71)
> at
> org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:59)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
> at
> org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:59)
> at
> org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:36)
> at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: java.lang.RuntimeException:
> org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while
> processing row (tag=0)
> {"key":{"_col0":63443,"_col1":"2000-01-04"},"value":{"_col0":10.75}}
> at
> org.apache.hadoop.hive.ql.exec.tez.ReduceRecordSource.pushRecord(ReduceRecordSource.java:288)
> at
> org.apache.hadoop.hive.ql.exec.tez.ReduceRecordProcessor.run(ReduceRecordProcessor.java:263)
> at
> org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:172)
> ... 14 more
> Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime
> Error while processing row (tag=0)
> {"key":{"_col0":63443,"_col1":"2000-01-04"},"value":{"_col0":10.75}}
> at
> org.apache.hadoop.hive.ql.exec.tez.ReduceRecordSource$GroupIterator.next(ReduceRecordSource.java:356)
> at
> org.apache.hadoop.hive.ql.exec.tez.ReduceRecordSource.pushRecord(ReduceRecordSource.java:278)
> ... 16 more
> Caused by: org.apache.hadoop.hive.ql.metadata.HiveException:
> java.lang.ClassCastException: java.lang.Object cannot be cast to
> org.apache.hadoop.io.DoubleWritable
> at
> org.apache.hadoop.hive.ql.exec.GroupByOperator.process(GroupByOperator.java:775)
> at
> org.apache.hadoop.hive.ql.exec.tez.ReduceRecordSource$GroupIterator.next(ReduceRecordSource.java:347)
> ... 17 more
> Caused by: java.lang.ClassCastException: java.lang.Object cannot be cast to
> org.apache.hadoop.io.DoubleWritable
> at
> org.apache.hadoop.hive.serde2.objectinspector.primitive.WritableDoubleObjectInspector.get(WritableDoubleObjectInspector.java:36)
> at
> org.apache.hadoop.hive.serde2.objectinspector.primitive.WritableDoubleObjectInspector.copyObject(WritableDoubleObjectInspector.java:41)
> at
> org.apache.hadoop.hive.serde2.objectinspector.ObjectInspectorUtils.copyToStandardObject(ObjectInspectorUtils.java:380)
> at
> org.apache.hadoop.hive.serde2.objectinspector.ObjectInspectorUtils.copyToStandardObject(ObjectInspectorUtils.java:324)
> at
> org.apache.hadoop.hive.ql.udf.ptf.WindowingTableFunction$WindowingIterator.next(WindowingTableFunction.java:1416)
> at
> org.apache.hadoop.hive.ql.exec.PTFOperator$PTFInvocation.finishPartition(PTFOperator.java:374)
> at
> org.apache.hadoop.hive.ql.exec.PTFOperator.process(PTFOperator.java:123)
> at
[jira] [Commented] (HIVE-11555) Beeline sends password in clear text if we miss -ssl=true flag in the connect string
[
https://issues.apache.org/jira/browse/HIVE-11555?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15411128#comment-15411128
]
Junjie Chen commented on HIVE-11555:
Hi [~thejas][~bharathv]
Not sure what need to be done here? But I tried to connect mysql without useSSL
option, and it shows following:
WARN: Establishing SSL connection without server's identity verification is not
recommended. According to MySQL 5.5.45+, 5.6.26+ and 5.7.6+ requirements SSL
connection must be established by default if explicit option isn't set. For
compliance with existing applications not using SSL the verifyServerCertificate
property is set to 'false'. You need either to explicitly disable SSL by
setting useSSL=false, or set useSSL=true and provide truststore for server
certificate verification.
So I would propose to employ same policy like mysql. Are you OK with this?
Or you were asking to build an secure way on http? like SASL?
> Beeline sends password in clear text if we miss -ssl=true flag in the connect
> string
>
>
> Key: HIVE-11555
> URL: https://issues.apache.org/jira/browse/HIVE-11555
> Project: Hive
> Issue Type: Bug
> Components: Beeline
>Affects Versions: 1.2.0
>Reporter: bharath v
>Assignee: Junjie Chen
>
> {code}
> I used tcpdump to display the network traffic:
> [root@fe01 ~]# beeline
> Beeline version 0.13.1-cdh5.3.2 by Apache Hive
> beeline> !connect jdbc:hive2://fe01.sectest.poc:1/default
> Connecting to jdbc:hive2://fe01.sectest.poc:1/default
> Enter username for jdbc:hive2://fe01.sectest.poc:1/default: tdaranyi
> Enter password for jdbc:hive2://fe01.sectest.poc:1/default: *
> (I entered "cleartext" as the password)
> The tcpdump in a different window
> tdara...@fe01.sectest.poc:~$ sudo tcpdump -n -X -i lo port 1
> tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
> listening on lo, link-type EN10MB (Ethernet), capture size 65535 bytes
> (...)
> 10:25:16.329974 IP 192.168.32.102.54322 > 192.168.32.102.ndmp: Flags [P.],
> seq 11:35, ack 1, win 512, options [nop,nop,TS val 2412851969 ecr
> 2412851969], length 24
> 0x: 4500 004c 3dd3 4000 4006 3abc c0a8 2066 E..L=.@.@.:f
> 0x0010: c0a8 2066 d432 2710 714c 0edc b45c 9268 ...f.2'.qL...\.h
> 0x0020: 8018 0200 c25b 0101 080a 8fd1 3301 .[3.
> 0x0030: 8fd1 3301 0500 1300 7464 6172 616e ..3...tdaran
> 0x0040: 7969 0063 6c65 6172 7465 7874 yi.cleartext
> (...)
> {code}
> We rely on the user supplied configuration to decide whether to open an SSL
> socket or a Plain one. Instead we can negotiate this information from the HS2
> and connect accordingly.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-14442) CBO: Calcite Operator To Hive Operator(Calcite Return Path): Wrong result/plan in group by with hive.map.aggr=false
[
https://issues.apache.org/jira/browse/HIVE-14442?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=1541#comment-1541
]
Hive QA commented on HIVE-14442:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12822483/HIVE-14442.2.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 6 failed/errored test(s), 10441 tests
executed
*Failed tests:*
{noformat}
TestMsgBusConnection - did not produce a TEST-*.xml file
TestQueryLifeTimeHook - did not produce a TEST-*.xml file
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_orc_llap_counters
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_count
org.apache.hadoop.hive.llap.tezplugins.TestLlapTaskSchedulerService.testDelayedLocalityNodeCommErrorImmediateAllocation
org.apache.hive.jdbc.TestJdbcWithMiniHS2.testAddJarConstructorUnCaching
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-MASTER-Build/807/testReport
Console output:
https://builds.apache.org/job/PreCommit-HIVE-MASTER-Build/807/console
Test logs:
http://ec2-204-236-174-241.us-west-1.compute.amazonaws.com/logs/PreCommit-HIVE-MASTER-Build-807/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 6 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12822483 - PreCommit-HIVE-MASTER-Build
> CBO: Calcite Operator To Hive Operator(Calcite Return Path): Wrong
> result/plan in group by with hive.map.aggr=false
> ---
>
> Key: HIVE-14442
> URL: https://issues.apache.org/jira/browse/HIVE-14442
> Project: Hive
> Issue Type: Sub-task
> Components: CBO
>Reporter: Vineet Garg
>Assignee: Vineet Garg
> Attachments: HIVE-14442.1.patch, HIVE-14442.2.patch
>
>
> Reproducer
> {code} set hive.cbo.returnpath.hiveop=true
> set hive.map.aggr=false
> create table abcd (a int, b int, c int, d int);
> LOAD DATA LOCAL INPATH '../../data/files/in4.txt' INTO TABLE abcd;
> {code}
> {code} explain select count(distinct a) from abcd group by b; {code}
> {code}
> STAGE PLANS:
> Stage: Stage-1
> Map Reduce
> Map Operator Tree:
> TableScan
> alias: abcd
> Statistics: Num rows: 19 Data size: 78 Basic stats: COMPLETE
> Column stats: NONE
> Select Operator
> expressions: a (type: int)
> outputColumnNames: a
> Statistics: Num rows: 19 Data size: 78 Basic stats: COMPLETE
> Column stats: NONE
> Reduce Output Operator
> key expressions: a (type: int), a (type: int)
> sort order: ++
> Map-reduce partition columns: a (type: int)
> Statistics: Num rows: 19 Data size: 78 Basic stats: COMPLETE
> Column stats: NONE
> Reduce Operator Tree:
> Group By Operator
> aggregations: count(DISTINCT KEY._col1:0._col0)
> keys: KEY._col0 (type: int)
> mode: complete
> outputColumnNames: b, $f1
> Statistics: Num rows: 9 Data size: 36 Basic stats: COMPLETE Column
> stats: NONE
> Select Operator
> expressions: $f1 (type: bigint)
> outputColumnNames: _o__c0
> Statistics: Num rows: 9 Data size: 36 Basic stats: COMPLETE
> Column stats: NONE
> File Output Operator
> compressed: false
> Statistics: Num rows: 9 Data size: 36 Basic stats: COMPLETE
> Column stats: NONE
> table:
> input format:
> org.apache.hadoop.mapred.SequenceFileInputFormat
> output format:
> org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
> serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
> {code}
> {code} explain select count(distinct a) from abcd group by c; {code}
> {code}
> STAGE PLANS:
> Stage: Stage-1
> Map Reduce
> Map Operator Tree:
> TableScan
> alias: abcd
> Statistics: Num rows: 19 Data size: 78 Basic stats: COMPLETE
> Column stats: NONE
> Select Operator
> expressions: a (type: int)
> outputColumnNames: a
> Statistics: Num rows: 19 Data size: 78 Basic stats: COMPLETE
> Column stats: NONE
> Reduce Output Operator
> key expressions: a (type: int), a (type: int)
> sort order: ++
> Map-reduce partition columns: a (type: int)
>
[jira] [Updated] (HIVE-14444) Upgrade qtest execution framework to junit4 - migrate most of them
[ https://issues.apache.org/jira/browse/HIVE-1?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zoltan Haindrich updated HIVE-1: Attachment: HIVE-1.2.patch there were 3 issues: * some typo in the negative test executor * initScript prop was always set to a default; and have interpreted as usersetting * a missing testclass for tez giving it another try ;) > Upgrade qtest execution framework to junit4 - migrate most of them > -- > > Key: HIVE-1 > URL: https://issues.apache.org/jira/browse/HIVE-1 > Project: Hive > Issue Type: Sub-task > Components: Tests >Reporter: Zoltan Haindrich >Assignee: Zoltan Haindrich > Attachments: HIVE-1.1.patch, HIVE-1.2.patch > > > this is the second step..migrating all exiting qtestgen generated tests to > junit4 > it might be possible that not all will get migrated in this ticket...I will > leave out the problematic ones... -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-12181) Change hive.stats.fetch.column.stats value to true for MiniTezCliDriver
[
https://issues.apache.org/jira/browse/HIVE-12181?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15411072#comment-15411072
]
Hive QA commented on HIVE-12181:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12822476/HIVE-12181.10.patch
{color:green}SUCCESS:{color} +1 due to 12 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 15 failed/errored test(s), 10441 tests
executed
*Failed tests:*
{noformat}
TestMsgBusConnection - did not produce a TEST-*.xml file
TestQueryLifeTimeHook - did not produce a TEST-*.xml file
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_mapjoin_mapjoin
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_unionDistinct_1
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_dynamic_partition_pruning
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_tez_dynpart_hashjoin_1
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_vectorized_dynamic_partition_pruning
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_explainuser_1
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_mapjoin_mapjoin
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_tez_dynpart_hashjoin_1
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_tez_join_hash
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_transform_ppr1
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_mapjoin_mapjoin
org.apache.hadoop.hive.llap.tezplugins.TestLlapTaskSchedulerService.testForcedLocalityMultiplePreemptionsSameHost2
org.apache.hive.jdbc.TestJdbcWithMiniHS2.testAddJarConstructorUnCaching
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-MASTER-Build/806/testReport
Console output:
https://builds.apache.org/job/PreCommit-HIVE-MASTER-Build/806/console
Test logs:
http://ec2-204-236-174-241.us-west-1.compute.amazonaws.com/logs/PreCommit-HIVE-MASTER-Build-806/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 15 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12822476 - PreCommit-HIVE-MASTER-Build
> Change hive.stats.fetch.column.stats value to true for MiniTezCliDriver
> ---
>
> Key: HIVE-12181
> URL: https://issues.apache.org/jira/browse/HIVE-12181
> Project: Hive
> Issue Type: Improvement
> Components: Statistics
>Reporter: Ashutosh Chauhan
>Assignee: Ashutosh Chauhan
> Attachments: HIVE-12181.1.patch, HIVE-12181.10.patch,
> HIVE-12181.2.patch, HIVE-12181.3.patch, HIVE-12181.4.patch,
> HIVE-12181.7.patch, HIVE-12181.8.patch, HIVE-12181.9.patch, HIVE-12181.patch,
> HIVE-12181.patch
>
>
> There was a performance concern earlier, but HIVE-7587 has fixed that. We can
> change the default to true now.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-14442) CBO: Calcite Operator To Hive Operator(Calcite Return Path): Wrong result/plan in group by with hive.map.aggr=false
[
https://issues.apache.org/jira/browse/HIVE-14442?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vineet Garg updated HIVE-14442:
---
Attachment: HIVE-14442.2.patch
> CBO: Calcite Operator To Hive Operator(Calcite Return Path): Wrong
> result/plan in group by with hive.map.aggr=false
> ---
>
> Key: HIVE-14442
> URL: https://issues.apache.org/jira/browse/HIVE-14442
> Project: Hive
> Issue Type: Sub-task
> Components: CBO
>Reporter: Vineet Garg
>Assignee: Vineet Garg
> Attachments: HIVE-14442.1.patch, HIVE-14442.2.patch
>
>
> Reproducer
> {code} set hive.cbo.returnpath.hiveop=true
> set hive.map.aggr=false
> create table abcd (a int, b int, c int, d int);
> LOAD DATA LOCAL INPATH '../../data/files/in4.txt' INTO TABLE abcd;
> {code}
> {code} explain select count(distinct a) from abcd group by b; {code}
> {code}
> STAGE PLANS:
> Stage: Stage-1
> Map Reduce
> Map Operator Tree:
> TableScan
> alias: abcd
> Statistics: Num rows: 19 Data size: 78 Basic stats: COMPLETE
> Column stats: NONE
> Select Operator
> expressions: a (type: int)
> outputColumnNames: a
> Statistics: Num rows: 19 Data size: 78 Basic stats: COMPLETE
> Column stats: NONE
> Reduce Output Operator
> key expressions: a (type: int), a (type: int)
> sort order: ++
> Map-reduce partition columns: a (type: int)
> Statistics: Num rows: 19 Data size: 78 Basic stats: COMPLETE
> Column stats: NONE
> Reduce Operator Tree:
> Group By Operator
> aggregations: count(DISTINCT KEY._col1:0._col0)
> keys: KEY._col0 (type: int)
> mode: complete
> outputColumnNames: b, $f1
> Statistics: Num rows: 9 Data size: 36 Basic stats: COMPLETE Column
> stats: NONE
> Select Operator
> expressions: $f1 (type: bigint)
> outputColumnNames: _o__c0
> Statistics: Num rows: 9 Data size: 36 Basic stats: COMPLETE
> Column stats: NONE
> File Output Operator
> compressed: false
> Statistics: Num rows: 9 Data size: 36 Basic stats: COMPLETE
> Column stats: NONE
> table:
> input format:
> org.apache.hadoop.mapred.SequenceFileInputFormat
> output format:
> org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
> serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
> {code}
> {code} explain select count(distinct a) from abcd group by c; {code}
> {code}
> STAGE PLANS:
> Stage: Stage-1
> Map Reduce
> Map Operator Tree:
> TableScan
> alias: abcd
> Statistics: Num rows: 19 Data size: 78 Basic stats: COMPLETE
> Column stats: NONE
> Select Operator
> expressions: a (type: int)
> outputColumnNames: a
> Statistics: Num rows: 19 Data size: 78 Basic stats: COMPLETE
> Column stats: NONE
> Reduce Output Operator
> key expressions: a (type: int), a (type: int)
> sort order: ++
> Map-reduce partition columns: a (type: int)
> Statistics: Num rows: 19 Data size: 78 Basic stats: COMPLETE
> Column stats: NONE
> Reduce Operator Tree:
> Group By Operator
> aggregations: count(DISTINCT KEY._col1:0._col0)
> keys: KEY._col0 (type: int)
> mode: complete
> outputColumnNames: c, $f1
> Statistics: Num rows: 9 Data size: 36 Basic stats: COMPLETE Column
> stats: NONE
> Select Operator
> expressions: $f1 (type: bigint)
> outputColumnNames: _o__c0
> Statistics: Num rows: 9 Data size: 36 Basic stats: COMPLETE
> Column stats: NONE
> File Output Operator
> compressed: false
> Statistics: Num rows: 9 Data size: 36 Basic stats: COMPLETE
> Column stats: NONE
> table:
> input format:
> org.apache.hadoop.mapred.SequenceFileInputFormat
> output format:
> org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
> serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
> {code}
> Above two cases has wrong keys in Map side Reduce Output Operator (both has
> a, a instead of b,a and c,a respectively
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-12924) CBO: Calcite Operator To Hive Operator (Calcite Return Path): TestCliDriver groupby_ppr_multi_distinct.q failure
[
https://issues.apache.org/jira/browse/HIVE-12924?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15411065#comment-15411065
]
Vineet Garg commented on HIVE-12924:
This has same issue as HIVE-14396's 3) case i.e. bug in lookup for column info
for non-distinct parameter. Translate groupby end up creating wrong column name
for one of the parameter and later execution engine bails out on not finding
that particular column
> CBO: Calcite Operator To Hive Operator (Calcite Return Path): TestCliDriver
> groupby_ppr_multi_distinct.q failure
>
>
> Key: HIVE-12924
> URL: https://issues.apache.org/jira/browse/HIVE-12924
> Project: Hive
> Issue Type: Sub-task
> Components: CBO
>Reporter: Hari Sankar Sivarama Subramaniyan
>Assignee: Vineet Garg
> Attachments: HIVE-12924.1.patch, HIVE-12924.2.patch,
> HIVE-12924.3.patch
>
>
> {code}
> EXPLAIN EXTENDED
> FROM srcpart src
> INSERT OVERWRITE TABLE dest1
> SELECT substr(src.key,1,1), count(DISTINCT substr(src.value,5)),
> concat(substr(src.key,1,1),sum(substr(src.value,5))), sum(DISTINCT
> substr(src.value, 5)), count(DISTINCT src.value)
> WHERE src.ds = '2008-04-08'
> GROUP BY substr(src.key,1,1)
> {code}
> Ended Job = job_local968043618_0742 with errors
> FAILED: Execution Error, return code 2 from
> org.apache.hadoop.hive.ql.exec.mr.MapRedTask
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-14442) CBO: Calcite Operator To Hive Operator(Calcite Return Path): Wrong result/plan in group by with hive.map.aggr=false
[
https://issues.apache.org/jira/browse/HIVE-14442?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vineet Garg updated HIVE-14442:
---
Attachment: (was: HIVE-14442.2.patch)
> CBO: Calcite Operator To Hive Operator(Calcite Return Path): Wrong
> result/plan in group by with hive.map.aggr=false
> ---
>
> Key: HIVE-14442
> URL: https://issues.apache.org/jira/browse/HIVE-14442
> Project: Hive
> Issue Type: Sub-task
> Components: CBO
>Reporter: Vineet Garg
>Assignee: Vineet Garg
> Attachments: HIVE-14442.1.patch
>
>
> Reproducer
> {code} set hive.cbo.returnpath.hiveop=true
> set hive.map.aggr=false
> create table abcd (a int, b int, c int, d int);
> LOAD DATA LOCAL INPATH '../../data/files/in4.txt' INTO TABLE abcd;
> {code}
> {code} explain select count(distinct a) from abcd group by b; {code}
> {code}
> STAGE PLANS:
> Stage: Stage-1
> Map Reduce
> Map Operator Tree:
> TableScan
> alias: abcd
> Statistics: Num rows: 19 Data size: 78 Basic stats: COMPLETE
> Column stats: NONE
> Select Operator
> expressions: a (type: int)
> outputColumnNames: a
> Statistics: Num rows: 19 Data size: 78 Basic stats: COMPLETE
> Column stats: NONE
> Reduce Output Operator
> key expressions: a (type: int), a (type: int)
> sort order: ++
> Map-reduce partition columns: a (type: int)
> Statistics: Num rows: 19 Data size: 78 Basic stats: COMPLETE
> Column stats: NONE
> Reduce Operator Tree:
> Group By Operator
> aggregations: count(DISTINCT KEY._col1:0._col0)
> keys: KEY._col0 (type: int)
> mode: complete
> outputColumnNames: b, $f1
> Statistics: Num rows: 9 Data size: 36 Basic stats: COMPLETE Column
> stats: NONE
> Select Operator
> expressions: $f1 (type: bigint)
> outputColumnNames: _o__c0
> Statistics: Num rows: 9 Data size: 36 Basic stats: COMPLETE
> Column stats: NONE
> File Output Operator
> compressed: false
> Statistics: Num rows: 9 Data size: 36 Basic stats: COMPLETE
> Column stats: NONE
> table:
> input format:
> org.apache.hadoop.mapred.SequenceFileInputFormat
> output format:
> org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
> serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
> {code}
> {code} explain select count(distinct a) from abcd group by c; {code}
> {code}
> STAGE PLANS:
> Stage: Stage-1
> Map Reduce
> Map Operator Tree:
> TableScan
> alias: abcd
> Statistics: Num rows: 19 Data size: 78 Basic stats: COMPLETE
> Column stats: NONE
> Select Operator
> expressions: a (type: int)
> outputColumnNames: a
> Statistics: Num rows: 19 Data size: 78 Basic stats: COMPLETE
> Column stats: NONE
> Reduce Output Operator
> key expressions: a (type: int), a (type: int)
> sort order: ++
> Map-reduce partition columns: a (type: int)
> Statistics: Num rows: 19 Data size: 78 Basic stats: COMPLETE
> Column stats: NONE
> Reduce Operator Tree:
> Group By Operator
> aggregations: count(DISTINCT KEY._col1:0._col0)
> keys: KEY._col0 (type: int)
> mode: complete
> outputColumnNames: c, $f1
> Statistics: Num rows: 9 Data size: 36 Basic stats: COMPLETE Column
> stats: NONE
> Select Operator
> expressions: $f1 (type: bigint)
> outputColumnNames: _o__c0
> Statistics: Num rows: 9 Data size: 36 Basic stats: COMPLETE
> Column stats: NONE
> File Output Operator
> compressed: false
> Statistics: Num rows: 9 Data size: 36 Basic stats: COMPLETE
> Column stats: NONE
> table:
> input format:
> org.apache.hadoop.mapred.SequenceFileInputFormat
> output format:
> org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
> serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
> {code}
> Above two cases has wrong keys in Map side Reduce Output Operator (both has
> a, a instead of b,a and c,a respectively
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-14442) CBO: Calcite Operator To Hive Operator(Calcite Return Path): Wrong result/plan in group by with hive.map.aggr=false
[
https://issues.apache.org/jira/browse/HIVE-14442?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vineet Garg updated HIVE-14442:
---
Attachment: HIVE-14442.2.patch
> CBO: Calcite Operator To Hive Operator(Calcite Return Path): Wrong
> result/plan in group by with hive.map.aggr=false
> ---
>
> Key: HIVE-14442
> URL: https://issues.apache.org/jira/browse/HIVE-14442
> Project: Hive
> Issue Type: Sub-task
> Components: CBO
>Reporter: Vineet Garg
>Assignee: Vineet Garg
> Attachments: HIVE-14442.1.patch, HIVE-14442.2.patch
>
>
> Reproducer
> {code} set hive.cbo.returnpath.hiveop=true
> set hive.map.aggr=false
> create table abcd (a int, b int, c int, d int);
> LOAD DATA LOCAL INPATH '../../data/files/in4.txt' INTO TABLE abcd;
> {code}
> {code} explain select count(distinct a) from abcd group by b; {code}
> {code}
> STAGE PLANS:
> Stage: Stage-1
> Map Reduce
> Map Operator Tree:
> TableScan
> alias: abcd
> Statistics: Num rows: 19 Data size: 78 Basic stats: COMPLETE
> Column stats: NONE
> Select Operator
> expressions: a (type: int)
> outputColumnNames: a
> Statistics: Num rows: 19 Data size: 78 Basic stats: COMPLETE
> Column stats: NONE
> Reduce Output Operator
> key expressions: a (type: int), a (type: int)
> sort order: ++
> Map-reduce partition columns: a (type: int)
> Statistics: Num rows: 19 Data size: 78 Basic stats: COMPLETE
> Column stats: NONE
> Reduce Operator Tree:
> Group By Operator
> aggregations: count(DISTINCT KEY._col1:0._col0)
> keys: KEY._col0 (type: int)
> mode: complete
> outputColumnNames: b, $f1
> Statistics: Num rows: 9 Data size: 36 Basic stats: COMPLETE Column
> stats: NONE
> Select Operator
> expressions: $f1 (type: bigint)
> outputColumnNames: _o__c0
> Statistics: Num rows: 9 Data size: 36 Basic stats: COMPLETE
> Column stats: NONE
> File Output Operator
> compressed: false
> Statistics: Num rows: 9 Data size: 36 Basic stats: COMPLETE
> Column stats: NONE
> table:
> input format:
> org.apache.hadoop.mapred.SequenceFileInputFormat
> output format:
> org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
> serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
> {code}
> {code} explain select count(distinct a) from abcd group by c; {code}
> {code}
> STAGE PLANS:
> Stage: Stage-1
> Map Reduce
> Map Operator Tree:
> TableScan
> alias: abcd
> Statistics: Num rows: 19 Data size: 78 Basic stats: COMPLETE
> Column stats: NONE
> Select Operator
> expressions: a (type: int)
> outputColumnNames: a
> Statistics: Num rows: 19 Data size: 78 Basic stats: COMPLETE
> Column stats: NONE
> Reduce Output Operator
> key expressions: a (type: int), a (type: int)
> sort order: ++
> Map-reduce partition columns: a (type: int)
> Statistics: Num rows: 19 Data size: 78 Basic stats: COMPLETE
> Column stats: NONE
> Reduce Operator Tree:
> Group By Operator
> aggregations: count(DISTINCT KEY._col1:0._col0)
> keys: KEY._col0 (type: int)
> mode: complete
> outputColumnNames: c, $f1
> Statistics: Num rows: 9 Data size: 36 Basic stats: COMPLETE Column
> stats: NONE
> Select Operator
> expressions: $f1 (type: bigint)
> outputColumnNames: _o__c0
> Statistics: Num rows: 9 Data size: 36 Basic stats: COMPLETE
> Column stats: NONE
> File Output Operator
> compressed: false
> Statistics: Num rows: 9 Data size: 36 Basic stats: COMPLETE
> Column stats: NONE
> table:
> input format:
> org.apache.hadoop.mapred.SequenceFileInputFormat
> output format:
> org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
> serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
> {code}
> Above two cases has wrong keys in Map side Reduce Output Operator (both has
> a, a instead of b,a and c,a respectively
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-14442) CBO: Calcite Operator To Hive Operator(Calcite Return Path): Wrong result/plan in group by with hive.map.aggr=false
[
https://issues.apache.org/jira/browse/HIVE-14442?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vineet Garg updated HIVE-14442:
---
Status: Open (was: Patch Available)
> CBO: Calcite Operator To Hive Operator(Calcite Return Path): Wrong
> result/plan in group by with hive.map.aggr=false
> ---
>
> Key: HIVE-14442
> URL: https://issues.apache.org/jira/browse/HIVE-14442
> Project: Hive
> Issue Type: Sub-task
> Components: CBO
>Reporter: Vineet Garg
>Assignee: Vineet Garg
> Attachments: HIVE-14442.1.patch, HIVE-14442.2.patch
>
>
> Reproducer
> {code} set hive.cbo.returnpath.hiveop=true
> set hive.map.aggr=false
> create table abcd (a int, b int, c int, d int);
> LOAD DATA LOCAL INPATH '../../data/files/in4.txt' INTO TABLE abcd;
> {code}
> {code} explain select count(distinct a) from abcd group by b; {code}
> {code}
> STAGE PLANS:
> Stage: Stage-1
> Map Reduce
> Map Operator Tree:
> TableScan
> alias: abcd
> Statistics: Num rows: 19 Data size: 78 Basic stats: COMPLETE
> Column stats: NONE
> Select Operator
> expressions: a (type: int)
> outputColumnNames: a
> Statistics: Num rows: 19 Data size: 78 Basic stats: COMPLETE
> Column stats: NONE
> Reduce Output Operator
> key expressions: a (type: int), a (type: int)
> sort order: ++
> Map-reduce partition columns: a (type: int)
> Statistics: Num rows: 19 Data size: 78 Basic stats: COMPLETE
> Column stats: NONE
> Reduce Operator Tree:
> Group By Operator
> aggregations: count(DISTINCT KEY._col1:0._col0)
> keys: KEY._col0 (type: int)
> mode: complete
> outputColumnNames: b, $f1
> Statistics: Num rows: 9 Data size: 36 Basic stats: COMPLETE Column
> stats: NONE
> Select Operator
> expressions: $f1 (type: bigint)
> outputColumnNames: _o__c0
> Statistics: Num rows: 9 Data size: 36 Basic stats: COMPLETE
> Column stats: NONE
> File Output Operator
> compressed: false
> Statistics: Num rows: 9 Data size: 36 Basic stats: COMPLETE
> Column stats: NONE
> table:
> input format:
> org.apache.hadoop.mapred.SequenceFileInputFormat
> output format:
> org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
> serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
> {code}
> {code} explain select count(distinct a) from abcd group by c; {code}
> {code}
> STAGE PLANS:
> Stage: Stage-1
> Map Reduce
> Map Operator Tree:
> TableScan
> alias: abcd
> Statistics: Num rows: 19 Data size: 78 Basic stats: COMPLETE
> Column stats: NONE
> Select Operator
> expressions: a (type: int)
> outputColumnNames: a
> Statistics: Num rows: 19 Data size: 78 Basic stats: COMPLETE
> Column stats: NONE
> Reduce Output Operator
> key expressions: a (type: int), a (type: int)
> sort order: ++
> Map-reduce partition columns: a (type: int)
> Statistics: Num rows: 19 Data size: 78 Basic stats: COMPLETE
> Column stats: NONE
> Reduce Operator Tree:
> Group By Operator
> aggregations: count(DISTINCT KEY._col1:0._col0)
> keys: KEY._col0 (type: int)
> mode: complete
> outputColumnNames: c, $f1
> Statistics: Num rows: 9 Data size: 36 Basic stats: COMPLETE Column
> stats: NONE
> Select Operator
> expressions: $f1 (type: bigint)
> outputColumnNames: _o__c0
> Statistics: Num rows: 9 Data size: 36 Basic stats: COMPLETE
> Column stats: NONE
> File Output Operator
> compressed: false
> Statistics: Num rows: 9 Data size: 36 Basic stats: COMPLETE
> Column stats: NONE
> table:
> input format:
> org.apache.hadoop.mapred.SequenceFileInputFormat
> output format:
> org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
> serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
> {code}
> Above two cases has wrong keys in Map side Reduce Output Operator (both has
> a, a instead of b,a and c,a respectively
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-12181) Change hive.stats.fetch.column.stats value to true for MiniTezCliDriver
[ https://issues.apache.org/jira/browse/HIVE-12181?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashutosh Chauhan updated HIVE-12181: Attachment: HIVE-12181.10.patch > Change hive.stats.fetch.column.stats value to true for MiniTezCliDriver > --- > > Key: HIVE-12181 > URL: https://issues.apache.org/jira/browse/HIVE-12181 > Project: Hive > Issue Type: Improvement > Components: Statistics >Reporter: Ashutosh Chauhan >Assignee: Ashutosh Chauhan > Attachments: HIVE-12181.1.patch, HIVE-12181.10.patch, > HIVE-12181.2.patch, HIVE-12181.3.patch, HIVE-12181.4.patch, > HIVE-12181.7.patch, HIVE-12181.8.patch, HIVE-12181.9.patch, HIVE-12181.patch, > HIVE-12181.patch > > > There was a performance concern earlier, but HIVE-7587 has fixed that. We can > change the default to true now. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-12181) Change hive.stats.fetch.column.stats value to true for MiniTezCliDriver
[ https://issues.apache.org/jira/browse/HIVE-12181?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashutosh Chauhan updated HIVE-12181: Status: Patch Available (was: Open) > Change hive.stats.fetch.column.stats value to true for MiniTezCliDriver > --- > > Key: HIVE-12181 > URL: https://issues.apache.org/jira/browse/HIVE-12181 > Project: Hive > Issue Type: Improvement > Components: Statistics >Reporter: Ashutosh Chauhan >Assignee: Ashutosh Chauhan > Attachments: HIVE-12181.1.patch, HIVE-12181.10.patch, > HIVE-12181.2.patch, HIVE-12181.3.patch, HIVE-12181.4.patch, > HIVE-12181.7.patch, HIVE-12181.8.patch, HIVE-12181.9.patch, HIVE-12181.patch, > HIVE-12181.patch > > > There was a performance concern earlier, but HIVE-7587 has fixed that. We can > change the default to true now. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-12181) Change hive.stats.fetch.column.stats value to true for MiniTezCliDriver
[ https://issues.apache.org/jira/browse/HIVE-12181?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashutosh Chauhan updated HIVE-12181: Status: Open (was: Patch Available) > Change hive.stats.fetch.column.stats value to true for MiniTezCliDriver > --- > > Key: HIVE-12181 > URL: https://issues.apache.org/jira/browse/HIVE-12181 > Project: Hive > Issue Type: Improvement > Components: Statistics >Reporter: Ashutosh Chauhan >Assignee: Ashutosh Chauhan > Attachments: HIVE-12181.1.patch, HIVE-12181.2.patch, > HIVE-12181.3.patch, HIVE-12181.4.patch, HIVE-12181.7.patch, > HIVE-12181.8.patch, HIVE-12181.9.patch, HIVE-12181.patch, HIVE-12181.patch > > > There was a performance concern earlier, but HIVE-7587 has fixed that. We can > change the default to true now. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-14444) Upgrade qtest execution framework to junit4 - migrate most of them
[
https://issues.apache.org/jira/browse/HIVE-1?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15410932#comment-15410932
]
Hive QA commented on HIVE-1:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12822466/HIVE-1.1.patch
{color:green}SUCCESS:{color} +1 due to 20 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 832 failed/errored test(s), 10386 tests
executed
*Failed tests:*
{noformat}
TestMinimrCliDriver-bucket_num_reducers.q-schemeAuthority2.q-scriptfile1.q-and-1-more
- did not produce a TEST-*.xml file
TestMinimrCliDriver-external_table_with_space_in_location_path.q-bucketmapjoin6.q-temp_table_external.q-and-1-more
- did not produce a TEST-*.xml file
TestMinimrCliDriver-groupby2.q-bucket6.q-input16_cc.q-and-1-more - did not
produce a TEST-*.xml file
TestMinimrCliDriver-import_exported_table.q-file_with_header_footer.q-orc_mr_pathalias.q-and-1-more
- did not produce a TEST-*.xml file
TestMinimrCliDriver-infer_bucket_sort_num_buckets.q-index_bitmap3.q-join1.q-and-1-more
- did not produce a TEST-*.xml file
TestMinimrCliDriver-infer_bucket_sort_reducers_power_two.q-scriptfile1_win.q-bucketmapjoin7.q-and-1-more
- did not produce a TEST-*.xml file
TestMinimrCliDriver-insert_dir_distcp.q-non_native_window_udf.q-infer_bucket_sort_merge.q-and-1-more
- did not produce a TEST-*.xml file
TestMinimrCliDriver-leftsemijoin_mr.q-infer_bucket_sort_map_operators.q-bucket4.q-and-1-more
- did not produce a TEST-*.xml file
TestMinimrCliDriver-list_bucket_dml_10.q-orc_merge_diff_fs.q-skewjoin_onesideskew.q
- did not produce a TEST-*.xml file
TestMinimrCliDriver-load_fs2.q-root_dir_external_table.q-join_acid_non_acid.q-and-1-more
- did not produce a TEST-*.xml file
TestMinimrCliDriver-parallel_orderby.q-bucketizedhiveinputformat.q-index_bitmap_auto.q-and-1-more
- did not produce a TEST-*.xml file
TestMinimrCliDriver-reduce_deduplicate.q-disable_merge_for_bucketing.q-load_hdfs_file_with_space_in_the_name.q-and-1-more
- did not produce a TEST-*.xml file
TestMinimrCliDriver-table_nonprintable.q-infer_bucket_sort_bucketed_table.q-uber_reduce.q-and-1-more
- did not produce a TEST-*.xml file
TestMsgBusConnection - did not produce a TEST-*.xml file
TestQueryLifeTimeHook - did not produce a TEST-*.xml file
org.apache.hadoop.hive.cli.TestBeeLineDriver.initializationError
org.apache.hadoop.hive.cli.TestContribCliDriver.testCliDriver[dboutput]
org.apache.hadoop.hive.cli.TestContribCliDriver.testCliDriver[serde_typedbytes5]
org.apache.hadoop.hive.cli.TestEncryptedHDFSCliDriver.testCliDriver[encryption_drop_table]
org.apache.hadoop.hive.cli.TestEncryptedHDFSCliDriver.testCliDriver[encryption_join_unencrypted_tbl]
org.apache.hadoop.hive.cli.TestEncryptedHDFSCliDriver.testCliDriver[encryption_move_tbl]
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[acid_overwrite]
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[add_partition_with_whitelist]
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[addpart1]
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[alter_concatenate_indexed_table]
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[alter_external_acid]
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[alter_file_format]
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[alter_non_native]
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[alter_partition_change_col_dup_col]
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[alter_partition_change_col_nonexist]
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[alter_partition_coltype_2columns]
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[alter_partition_coltype_invalidcolname]
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[alter_partition_coltype_invalidtype]
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[alter_partition_partial_spec_dyndisabled]
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[alter_partition_with_whitelist]
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[alter_rename_partition_failure2]
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[alter_rename_partition_failure3]
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[alter_rename_partition_failure]
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[alter_table_add_partition]
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[alter_table_constraint_duplicate_pk]
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[alter_table_constraint_invalid_fk_col1]
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[alter_table_constraint_invalid_fk_col2]
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[alter_table_constraint_invalid_fk_tbl1]
[jira] [Commented] (HIVE-14428) HadoopMetrics2Reporter leaks memory if the metrics sink is not configured correctly
[
https://issues.apache.org/jira/browse/HIVE-14428?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15410888#comment-15410888
]
Hive QA commented on HIVE-14428:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12822463/HIVE-14428.1.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 5 failed/errored test(s), 10441 tests
executed
*Failed tests:*
{noformat}
TestMsgBusConnection - did not produce a TEST-*.xml file
TestQueryLifeTimeHook - did not produce a TEST-*.xml file
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_orc_llap_counters
org.apache.hadoop.hive.llap.tezplugins.TestLlapTaskSchedulerService.testDelayedLocalityNodeCommErrorImmediateAllocation
org.apache.hadoop.hive.llap.tezplugins.TestLlapTaskSchedulerService.testForcedLocalityMultiplePreemptionsSameHost2
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-MASTER-Build/804/testReport
Console output:
https://builds.apache.org/job/PreCommit-HIVE-MASTER-Build/804/console
Test logs:
http://ec2-204-236-174-241.us-west-1.compute.amazonaws.com/logs/PreCommit-HIVE-MASTER-Build-804/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 5 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12822463 - PreCommit-HIVE-MASTER-Build
> HadoopMetrics2Reporter leaks memory if the metrics sink is not configured
> correctly
> ---
>
> Key: HIVE-14428
> URL: https://issues.apache.org/jira/browse/HIVE-14428
> Project: Hive
> Issue Type: Sub-task
> Components: HiveServer2
>Reporter: Siddharth Seth
>Assignee: Thejas M Nair
>Priority: Critical
> Attachments: HIVE-14428.1.patch
>
>
> About 80MB held after 7 hours of running. Metrics2Collector aggregates only
> when it's invoked by the Hadoop sink.
> Options - the first one is better IMO.
> 1. Fix Metrics2Collector to aggregate more often, and fix the dependency in
> Hive accordingly
> 2. Don't setup the metrics sub-system if a sink is not configured.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-14444) Upgrade qtest execution framework to junit4 - migrate most of them
[ https://issues.apache.org/jira/browse/HIVE-1?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zoltan Haindrich updated HIVE-1: Status: Patch Available (was: Open) > Upgrade qtest execution framework to junit4 - migrate most of them > -- > > Key: HIVE-1 > URL: https://issues.apache.org/jira/browse/HIVE-1 > Project: Hive > Issue Type: Sub-task > Components: Tests >Reporter: Zoltan Haindrich >Assignee: Zoltan Haindrich > Attachments: HIVE-1.1.patch > > > this is the second step..migrating all exiting qtestgen generated tests to > junit4 > it might be possible that not all will get migrated in this ticket...I will > leave out the problematic ones... -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-14444) Upgrade qtest execution framework to junit4 - migrate most of them
[ https://issues.apache.org/jira/browse/HIVE-1?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zoltan Haindrich updated HIVE-1: Attachment: HIVE-1.1.patch i'm getting lost in the different test executions...but it looks ok to me now ;) so this patch might work ;) ...all i wanted to change is to move from ant/etc to java for all qfile related tests - migrating just a few might lead to a situation where it becomes hard to migrate the remaining ones... there are some things i've left behind like: * BeeLine tests are updated...but somehow they caused compilation errors for me...so i've commented the whole class; AFAIK they are not executed in precommit tests... * Accumulo tests are also not executed...i've update it to junit4 earlier (i wanted to start with that instead perfcli) - i'll add it later...since it haven't been executed lately... * refactoring of these cli classes have been postoned after they are working as expected as java classes * there are a few intresting possible problems i've found; but because this is refactor which tries to keep every functionality as-is - I haven't touched them... * I think all existing features should not change after this patch (qfile,qfile_regex,run_disabled,clustermode,initScript) > Upgrade qtest execution framework to junit4 - migrate most of them > -- > > Key: HIVE-1 > URL: https://issues.apache.org/jira/browse/HIVE-1 > Project: Hive > Issue Type: Sub-task > Components: Tests >Reporter: Zoltan Haindrich >Assignee: Zoltan Haindrich > Attachments: HIVE-1.1.patch > > > this is the second step..migrating all exiting qtestgen generated tests to > junit4 > it might be possible that not all will get migrated in this ticket...I will > leave out the problematic ones... -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-14346) Change the default value for hive.mapred.mode to null
[
https://issues.apache.org/jira/browse/HIVE-14346?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15410851#comment-15410851
]
Lefty Leverenz commented on HIVE-14346:
---
Doc note: The wiki will need to be updated with the new default value for
*hive.mapred.mode* in release 2.2.0. (It also needs to be updated with
HIVE-12727's changes to the default and description.)
* [Configuration Properties -- hive.mapred.mode |
https://cwiki.apache.org/confluence/display/Hive/Configuration+Properties#ConfigurationProperties-hive.mapred.mode]
Added a TODOC2.2 label.
> Change the default value for hive.mapred.mode to null
> -
>
> Key: HIVE-14346
> URL: https://issues.apache.org/jira/browse/HIVE-14346
> Project: Hive
> Issue Type: Bug
> Components: Configuration
>Affects Versions: 2.2.0
>Reporter: Chao Sun
>Assignee: Chao Sun
> Labels: TODOC2.2
> Fix For: 2.2.0
>
> Attachments: HIVE-14346.0.patch, HIVE-14346.1.patch,
> HIVE-14346.2.patch, HIVE-14346.3.patch
>
>
> HIVE-12727 introduces three new configurations to replace the existing
> {{hive.mapred.mode}}, which is deprecated. However, the default value for the
> latter is 'nonstrict', which prevent the new configurations from being used
> (see comments in that JIRA for more details).
> This proposes to change the default value for {{hive.mapred.mode}} to null.
> Users can then set the three new configurations to get more fine-grained
> control over the strict checking. If user want to use the old configuration,
> they can set {{hive.mapred.mode}} to strict/nonstrict.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-14456) HS2 memory leak if hadoop2 metrics sink is not configured properly
[
https://issues.apache.org/jira/browse/HIVE-14456?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15410850#comment-15410850
]
Hive QA commented on HIVE-14456:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12822462/HIVE-14456.1.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 6 failed/errored test(s), 10440 tests
executed
*Failed tests:*
{noformat}
TestMsgBusConnection - did not produce a TEST-*.xml file
TestQueryLifeTimeHook - did not produce a TEST-*.xml file
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_orc_llap_counters
org.apache.hadoop.hive.llap.tezplugins.TestLlapTaskSchedulerService.testDelayedLocalityNodeCommErrorImmediateAllocation
org.apache.hadoop.hive.llap.tezplugins.TestLlapTaskSchedulerService.testForcedLocalityMultiplePreemptionsSameHost2
org.apache.hive.jdbc.TestJdbcWithMiniHS2.testAddJarConstructorUnCaching
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-MASTER-Build/803/testReport
Console output:
https://builds.apache.org/job/PreCommit-HIVE-MASTER-Build/803/console
Test logs:
http://ec2-204-236-174-241.us-west-1.compute.amazonaws.com/logs/PreCommit-HIVE-MASTER-Build-803/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 6 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12822462 - PreCommit-HIVE-MASTER-Build
> HS2 memory leak if hadoop2 metrics sink is not configured properly
> --
>
> Key: HIVE-14456
> URL: https://issues.apache.org/jira/browse/HIVE-14456
> Project: Hive
> Issue Type: Bug
> Components: HiveServer2, Metastore
>Reporter: Thejas M Nair
>Assignee: Thejas M Nair
>Priority: Critical
> Attachments: HIVE-14456.1.patch
>
>
> The dropwizard-metrics-hadoop-metrics2-reporter version needs to be updated
> to pick the fix for this in
> https://github.com/joshelser/dropwizard-hadoop-metrics2/issues/4
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-14346) Change the default value for hive.mapred.mode to null
[
https://issues.apache.org/jira/browse/HIVE-14346?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Lefty Leverenz updated HIVE-14346:
--
Labels: TODOC2.2 (was: )
> Change the default value for hive.mapred.mode to null
> -
>
> Key: HIVE-14346
> URL: https://issues.apache.org/jira/browse/HIVE-14346
> Project: Hive
> Issue Type: Bug
> Components: Configuration
>Affects Versions: 2.2.0
>Reporter: Chao Sun
>Assignee: Chao Sun
> Labels: TODOC2.2
> Fix For: 2.2.0
>
> Attachments: HIVE-14346.0.patch, HIVE-14346.1.patch,
> HIVE-14346.2.patch, HIVE-14346.3.patch
>
>
> HIVE-12727 introduces three new configurations to replace the existing
> {{hive.mapred.mode}}, which is deprecated. However, the default value for the
> latter is 'nonstrict', which prevent the new configurations from being used
> (see comments in that JIRA for more details).
> This proposes to change the default value for {{hive.mapred.mode}} to null.
> Users can then set the three new configurations to get more fine-grained
> control over the strict checking. If user want to use the old configuration,
> they can set {{hive.mapred.mode}} to strict/nonstrict.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-12413) Default mode for hive.mapred.mode should be strict
[ https://issues.apache.org/jira/browse/HIVE-12413?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15410849#comment-15410849 ] Lefty Leverenz commented on HIVE-12413: --- HIVE-12727 changed the default for *hive.mapred.mode* back to nonstrict and deprecated it in release 2.0.0, so the doc needs to be changed again. (Then HIVE-14346 will change the default to null in release 2.2.0.) > Default mode for hive.mapred.mode should be strict > -- > > Key: HIVE-12413 > URL: https://issues.apache.org/jira/browse/HIVE-12413 > Project: Hive > Issue Type: Task > Components: Configuration >Reporter: Ashutosh Chauhan >Assignee: Ashutosh Chauhan > Fix For: 2.0.0 > > Attachments: HIVE-12413.1.patch, HIVE-12413.2.patch, > HIVE-12413.3.patch, HIVE-12413.4.patch, HIVE-12413.patch > > > Non-strict mode allows some questionable semantics and questionable > operations. Its better that user makes a conscious choice to enable such a > behavior. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-14382) Improve the Functionality of Reverse FOR Statement
[
https://issues.apache.org/jira/browse/HIVE-14382?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15410841#comment-15410841
]
Lefty Leverenz commented on HIVE-14382:
---
[~dmtolpeko], the second example in the doc ([http://hplsql.org/for-range])
still shows upper_bound before lower_bound:
{code}
FOR i IN REVERSE 10..1 LOOP
-- i will have values: 10, 9, 8, 7, 6, 5, 4, 3, 1, 1
END LOOP;
{code}
> Improve the Functionality of Reverse FOR Statement
> ---
>
> Key: HIVE-14382
> URL: https://issues.apache.org/jira/browse/HIVE-14382
> Project: Hive
> Issue Type: Improvement
> Components: hpl/sql
>Reporter: Akash Sethi
>Assignee: Akash Sethi
>Priority: Minor
> Fix For: 2.2.0
>
> Attachments: HIVE-14382.1-branch-2.1.patch, HIVE-14382.1.patch
>
> Original Estimate: 0.5h
> Remaining Estimate: 0.5h
>
> According to SQL Standards, Reverse FOR Statement should be like this:-
> FOR index IN Optional[Reverse] Lower_Bound Upper_Bound
> but in hive it is like this :-
> FOR index IN Optional[Reverse] Upper_Bound Lower_Bound
> so i m just trying to improve the functionality for Reverse FOR Statement
> REFERNCES :-
> https://docs.oracle.com/cloud/latest/db112/LNPLS/for_loop_statement.htm#LNPLS1536
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-14447) Set HIVE_TRANSACTIONAL_TABLE_SCAN to the correct job conf for FetchOperator
[ https://issues.apache.org/jira/browse/HIVE-14447?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prasanth Jayachandran updated HIVE-14447: - Resolution: Fixed Fix Version/s: 2.1.1 2.2.0 1.3.0 Status: Resolved (was: Patch Available) Test failures are unrelated to this patch. Committed to branch-1, branch-2.1 and master. > Set HIVE_TRANSACTIONAL_TABLE_SCAN to the correct job conf for FetchOperator > --- > > Key: HIVE-14447 > URL: https://issues.apache.org/jira/browse/HIVE-14447 > Project: Hive > Issue Type: Bug > Components: Hive, Transactions >Affects Versions: 1.3.0, 2.2.0, 2.1.1 >Reporter: Wei Zheng >Assignee: Prasanth Jayachandran > Fix For: 1.3.0, 2.2.0, 2.1.1 > > Attachments: HIVE-14447.1.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
