[jira] [Updated] (HIVE-4570) More information to user on GetOperationStatus in Hive Server2 when query is still executing
[ https://issues.apache.org/jira/browse/HIVE-4570?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajat Khandelwal updated HIVE-4570: --- Attachment: HIVE-4570.04.patch > More information to user on GetOperationStatus in Hive Server2 when query is > still executing > > > Key: HIVE-4570 > URL: https://issues.apache.org/jira/browse/HIVE-4570 > Project: Hive > Issue Type: Improvement > Components: HiveServer2 >Reporter: Amareshwari Sriramadasu >Assignee: Rajat Khandelwal > Attachments: HIVE-4570.01.patch, HIVE-4570.02.patch, > HIVE-4570.03.patch, HIVE-4570.03.patch, HIVE-4570.04.patch, HIVE-4570.04.patch > > > Currently in Hive Server2, when the query is still executing only the status > is set as STILL_EXECUTING. > This issue is to give more information to the user such as progress and > running job handles, if possible. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-4570) More information to user on GetOperationStatus in Hive Server2 when query is still executing
[ https://issues.apache.org/jira/browse/HIVE-4570?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15188877#comment-15188877 ] Rajat Khandelwal commented on HIVE-4570: Taking patch from reviewboard and attaching > More information to user on GetOperationStatus in Hive Server2 when query is > still executing > > > Key: HIVE-4570 > URL: https://issues.apache.org/jira/browse/HIVE-4570 > Project: Hive > Issue Type: Improvement > Components: HiveServer2 >Reporter: Amareshwari Sriramadasu >Assignee: Rajat Khandelwal > Attachments: HIVE-4570.01.patch, HIVE-4570.02.patch, > HIVE-4570.03.patch, HIVE-4570.03.patch, HIVE-4570.04.patch, HIVE-4570.04.patch > > > Currently in Hive Server2, when the query is still executing only the status > is set as STILL_EXECUTING. > This issue is to give more information to the user such as progress and > running job handles, if possible. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-11484) Fix ObjectInspector for Char and VarChar
[ https://issues.apache.org/jira/browse/HIVE-11484?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Deepak Barr updated HIVE-11484: --- Status: Patch Available (was: Open) > Fix ObjectInspector for Char and VarChar > > > Key: HIVE-11484 > URL: https://issues.apache.org/jira/browse/HIVE-11484 > Project: Hive > Issue Type: Bug > Components: Serializers/Deserializers >Reporter: Amareshwari Sriramadasu >Assignee: Deepak Barr > Attachments: HIVE-11484.master.patch > > > The creation of HiveChar and Varchar is not happening through ObjectInspector. > Here is fix we pushed internally : > https://github.com/InMobi/hive/commit/fe95c7850e7130448209141155f28b25d3504216 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-11484) Fix ObjectInspector for Char and VarChar
[ https://issues.apache.org/jira/browse/HIVE-11484?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Deepak Barr updated HIVE-11484: --- Status: Open (was: Patch Available) > Fix ObjectInspector for Char and VarChar > > > Key: HIVE-11484 > URL: https://issues.apache.org/jira/browse/HIVE-11484 > Project: Hive > Issue Type: Bug > Components: Serializers/Deserializers >Reporter: Amareshwari Sriramadasu >Assignee: Deepak Barr > Attachments: HIVE-11484.master.patch > > > The creation of HiveChar and Varchar is not happening through ObjectInspector. > Here is fix we pushed internally : > https://github.com/InMobi/hive/commit/fe95c7850e7130448209141155f28b25d3504216 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-4570) More information to user on GetOperationStatus in Hive Server2 when query is still executing
[ https://issues.apache.org/jira/browse/HIVE-4570?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15188842#comment-15188842 ] Rajat Khandelwal commented on HIVE-4570: Created by mistake, deleted > More information to user on GetOperationStatus in Hive Server2 when query is > still executing > > > Key: HIVE-4570 > URL: https://issues.apache.org/jira/browse/HIVE-4570 > Project: Hive > Issue Type: Improvement > Components: HiveServer2 >Reporter: Amareshwari Sriramadasu >Assignee: Rajat Khandelwal > Attachments: HIVE-4570.01.patch, HIVE-4570.02.patch, > HIVE-4570.03.patch, HIVE-4570.03.patch, HIVE-4570.04.patch > > > Currently in Hive Server2, when the query is still executing only the status > is set as STILL_EXECUTING. > This issue is to give more information to the user such as progress and > running job handles, if possible. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-12439) CompactionTxnHandler.markCleaned() and TxnHandler.openTxns() misc improvements
[
https://issues.apache.org/jira/browse/HIVE-12439?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15188796#comment-15188796
]
Hive QA commented on HIVE-12439:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12792320/HIVE-12439.1.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 8 failed/errored test(s), 9762 tests executed
*Failed tests:*
{noformat}
TestMiniTezCliDriver-schema_evol_text_nonvec_mapwork_table.q-orc_vectorization_ppd.q-vector_left_outer_join2.q-and-12-more
- did not produce a TEST-*.xml file
TestSparkCliDriver-groupby3_map.q-sample2.q-auto_join14.q-and-12-more - did not
produce a TEST-*.xml file
TestSparkCliDriver-groupby_map_ppr_multi_distinct.q-table_access_keys_stats.q-groupby4_noskew.q-and-12-more
- did not produce a TEST-*.xml file
TestSparkCliDriver-join_rc.q-insert1.q-vectorized_rcfile_columnar.q-and-12-more
- did not produce a TEST-*.xml file
TestSparkCliDriver-ppd_join4.q-join9.q-ppd_join3.q-and-12-more - did not
produce a TEST-*.xml file
TestSparkCliDriver-timestamp_lazy.q-bucketsortoptimize_insert_4.q-date_udf.q-and-12-more
- did not produce a TEST-*.xml file
TestTxnCommands2 - did not produce a TEST-*.xml file
org.apache.hadoop.hive.metastore.TestHiveMetaStorePartitionSpecs.testGetPartitionSpecs_WithAndWithoutPartitionGrouping
{noformat}
Test results:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/7208/testReport
Console output:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/7208/console
Test logs:
http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-7208/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 8 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12792320 - PreCommit-HIVE-TRUNK-Build
> CompactionTxnHandler.markCleaned() and TxnHandler.openTxns() misc improvements
> --
>
> Key: HIVE-12439
> URL: https://issues.apache.org/jira/browse/HIVE-12439
> Project: Hive
> Issue Type: Improvement
> Components: Metastore, Transactions
>Affects Versions: 1.0.0
>Reporter: Eugene Koifman
>Assignee: Wei Zheng
> Attachments: HIVE-12439.1.patch
>
>
> # add a safeguard to make sure IN clause is not too large; break up by txn id
> to delete from TXN_COMPONENTS where tc_txnid in ...
> # TxnHandler. openTxns() - use 1 insert with many rows in values() clause,
> rather than 1 DB roundtrip per row
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-4570) More information to user on GetOperationStatus in Hive Server2 when query is still executing
[ https://issues.apache.org/jira/browse/HIVE-4570?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15188719#comment-15188719 ] Rajat Khandelwal commented on HIVE-4570: Created https://reviews.apache.org/r/44608/ > More information to user on GetOperationStatus in Hive Server2 when query is > still executing > > > Key: HIVE-4570 > URL: https://issues.apache.org/jira/browse/HIVE-4570 > Project: Hive > Issue Type: Improvement > Components: HiveServer2 >Reporter: Amareshwari Sriramadasu >Assignee: Rajat Khandelwal > Attachments: HIVE-4570.01.patch, HIVE-4570.02.patch, > HIVE-4570.03.patch, HIVE-4570.03.patch, HIVE-4570.04.patch > > > Currently in Hive Server2, when the query is still executing only the status > is set as STILL_EXECUTING. > This issue is to give more information to the user such as progress and > running job handles, if possible. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-13198) Authorization issues with cascading views
[ https://issues.apache.org/jira/browse/HIVE-13198?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pengcheng Xiong updated HIVE-13198: --- Attachment: HIVE-13198.02.patch > Authorization issues with cascading views > - > > Key: HIVE-13198 > URL: https://issues.apache.org/jira/browse/HIVE-13198 > Project: Hive > Issue Type: Sub-task >Reporter: Pengcheng Xiong >Assignee: Pengcheng Xiong > Fix For: 2.1.0 > > Attachments: HIVE-13198.01.patch, HIVE-13198.02.patch > > > Here is a use case. They have a base table t1, from which they create a view > v1. They further create a view v2 from v1 by applying a filter. User has > access to only view v2, not view v1 or table t1. When user tries to access > v2, they are denied access. > Steps to recreate: > There is a base table t1 that exists in the default database with primary key > id and some employee data (name, ssn etc) > Create view v1 - “create view v1 as select * from default.t1;” > Created v2 - “create view v2 as select * from v1 where id =1;” > Permissions provided for user to select all columns from view v2. When user > runs select * from v2, hive throws an error “user does not have permissions > to select view v1". > Apparently Hive is converting the query to underlying views. > SELECT * FROM v2 LIMIT 100 > To > select `v1`.`id`, `v1`.`name`, `v1`.`ssn`, `v1`.`join_date`, `v1`.`location` > from `hr`.`v1` where `v1`.`id`=1 > Hive should only check for permissions for the view being run in the query, > not any parent views. (This is consistent with ORACLE). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-13198) Authorization issues with cascading views
[ https://issues.apache.org/jira/browse/HIVE-13198?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pengcheng Xiong updated HIVE-13198: --- Status: Patch Available (was: Open) > Authorization issues with cascading views > - > > Key: HIVE-13198 > URL: https://issues.apache.org/jira/browse/HIVE-13198 > Project: Hive > Issue Type: Sub-task >Reporter: Pengcheng Xiong >Assignee: Pengcheng Xiong > Fix For: 2.1.0 > > Attachments: HIVE-13198.01.patch, HIVE-13198.02.patch > > > Here is a use case. They have a base table t1, from which they create a view > v1. They further create a view v2 from v1 by applying a filter. User has > access to only view v2, not view v1 or table t1. When user tries to access > v2, they are denied access. > Steps to recreate: > There is a base table t1 that exists in the default database with primary key > id and some employee data (name, ssn etc) > Create view v1 - “create view v1 as select * from default.t1;” > Created v2 - “create view v2 as select * from v1 where id =1;” > Permissions provided for user to select all columns from view v2. When user > runs select * from v2, hive throws an error “user does not have permissions > to select view v1". > Apparently Hive is converting the query to underlying views. > SELECT * FROM v2 LIMIT 100 > To > select `v1`.`id`, `v1`.`name`, `v1`.`ssn`, `v1`.`join_date`, `v1`.`location` > from `hr`.`v1` where `v1`.`id`=1 > Hive should only check for permissions for the view being run in the query, > not any parent views. (This is consistent with ORACLE). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-13198) Authorization issues with cascading views
[ https://issues.apache.org/jira/browse/HIVE-13198?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pengcheng Xiong updated HIVE-13198: --- Status: Open (was: Patch Available) > Authorization issues with cascading views > - > > Key: HIVE-13198 > URL: https://issues.apache.org/jira/browse/HIVE-13198 > Project: Hive > Issue Type: Sub-task >Reporter: Pengcheng Xiong >Assignee: Pengcheng Xiong > Fix For: 2.1.0 > > Attachments: HIVE-13198.01.patch, HIVE-13198.02.patch > > > Here is a use case. They have a base table t1, from which they create a view > v1. They further create a view v2 from v1 by applying a filter. User has > access to only view v2, not view v1 or table t1. When user tries to access > v2, they are denied access. > Steps to recreate: > There is a base table t1 that exists in the default database with primary key > id and some employee data (name, ssn etc) > Create view v1 - “create view v1 as select * from default.t1;” > Created v2 - “create view v2 as select * from v1 where id =1;” > Permissions provided for user to select all columns from view v2. When user > runs select * from v2, hive throws an error “user does not have permissions > to select view v1". > Apparently Hive is converting the query to underlying views. > SELECT * FROM v2 LIMIT 100 > To > select `v1`.`id`, `v1`.`name`, `v1`.`ssn`, `v1`.`join_date`, `v1`.`location` > from `hr`.`v1` where `v1`.`id`=1 > Hive should only check for permissions for the view being run in the query, > not any parent views. (This is consistent with ORACLE). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-13236) LLAP: token renewal interval needs to be set
[ https://issues.apache.org/jira/browse/HIVE-13236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15188691#comment-15188691 ] Siddharth Seth commented on HIVE-13236: --- +1 for the new patch. > LLAP: token renewal interval needs to be set > > > Key: HIVE-13236 > URL: https://issues.apache.org/jira/browse/HIVE-13236 > Project: Hive > Issue Type: Bug >Affects Versions: 2.0.0 >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin > Attachments: HIVE-13236.01.patch, HIVE-13236.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-13236) LLAP: token renewal interval needs to be set
[ https://issues.apache.org/jira/browse/HIVE-13236?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Shelukhin updated HIVE-13236: Attachment: HIVE-13236.01.patch Workaround for HADOOP-12659 > LLAP: token renewal interval needs to be set > > > Key: HIVE-13236 > URL: https://issues.apache.org/jira/browse/HIVE-13236 > Project: Hive > Issue Type: Bug >Affects Versions: 2.0.0 >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin > Attachments: HIVE-13236.01.patch, HIVE-13236.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-4570) More information to user on GetOperationStatus in Hive Server2 when query is still executing
[ https://issues.apache.org/jira/browse/HIVE-4570?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15188623#comment-15188623 ] Rajat Khandelwal commented on HIVE-4570: Hey, so sorry about the confusion, I have a script that takes patch and revision number from the reviewboard and names the patch file accordingly before posting it on jira. So I'm building once with thriftif profile and would post the complete review request on v4 and also upload a patch named v4. At this moment, the pre-commit job is 14th in the queue. I'm hoping we'll be able to finalize before it starts running. > More information to user on GetOperationStatus in Hive Server2 when query is > still executing > > > Key: HIVE-4570 > URL: https://issues.apache.org/jira/browse/HIVE-4570 > Project: Hive > Issue Type: Improvement > Components: HiveServer2 >Reporter: Amareshwari Sriramadasu >Assignee: Rajat Khandelwal > Attachments: HIVE-4570.01.patch, HIVE-4570.02.patch, > HIVE-4570.03.patch, HIVE-4570.03.patch, HIVE-4570.04.patch > > > Currently in Hive Server2, when the query is still executing only the status > is set as STILL_EXECUTING. > This issue is to give more information to the user such as progress and > running job handles, if possible. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-9660) store end offset of compressed data for RG in RowIndex in ORC
[ https://issues.apache.org/jira/browse/HIVE-9660?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15188622#comment-15188622 ] Sergey Shelukhin commented on HIVE-9660: [~gopalv] fyi. I am working on this. > store end offset of compressed data for RG in RowIndex in ORC > - > > Key: HIVE-9660 > URL: https://issues.apache.org/jira/browse/HIVE-9660 > Project: Hive > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin > > Right now the end offset is estimated, which in some cases results in tons of > extra data being read. > We can add a separate array to RowIndex (positions_v2?) that stores number of > compressed buffers for each RG, or end offset, or something, to remove this > estimation magic -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-13231) Show helpful error message on failure to create table in nested directory
[
https://issues.apache.org/jira/browse/HIVE-13231?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15188617#comment-15188617
]
Hive QA commented on HIVE-13231:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12792047/HIVE-13231.01.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 6 failed/errored test(s), 9786 tests executed
*Failed tests:*
{noformat}
TestMiniTezCliDriver-explainuser_4.q-mapreduce1.q-vectorization_14.q-and-12-more
- did not produce a TEST-*.xml file
TestSparkCliDriver-groupby3_map.q-sample2.q-auto_join14.q-and-12-more - did not
produce a TEST-*.xml file
TestSparkCliDriver-groupby_map_ppr_multi_distinct.q-table_access_keys_stats.q-groupby4_noskew.q-and-12-more
- did not produce a TEST-*.xml file
TestSparkCliDriver-join_rc.q-insert1.q-vectorized_rcfile_columnar.q-and-12-more
- did not produce a TEST-*.xml file
TestSparkCliDriver-ppd_join4.q-join9.q-ppd_join3.q-and-12-more - did not
produce a TEST-*.xml file
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_sortmerge_join_2
{noformat}
Test results:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/7207/testReport
Console output:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/7207/console
Test logs:
http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-7207/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 6 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12792047 - PreCommit-HIVE-TRUNK-Build
> Show helpful error message on failure to create table in nested directory
> -
>
> Key: HIVE-13231
> URL: https://issues.apache.org/jira/browse/HIVE-13231
> Project: Hive
> Issue Type: Bug
> Components: Hive
>Reporter: Reuben Kuhnert
>Assignee: Reuben Kuhnert
>Priority: Minor
> Attachments: HIVE-13231.01.patch
>
>
> cannot store data in a directory whose parent doesn't exist, even though the
> target dir does have an existing ancestor on HDFS. This occurs when trying to
> perform {{create table }}.
> {code}
> 0: jdbc:hive2://10.17.81.192:1/default> create table test3 location
> '/user/hive/data/yshi/nonexisting/test3' as select * from sample_07;
> Error: Error while processing statement: FAILED: Execution Error, return code
> 1 from org.apache.hadoop.hive.ql.exec.MoveTask (state=08S01,code=1)
> Error message:
> 2015-10-29 19:04:46,323 ERROR org.apache.hadoop.hive.ql.exec.Task: Failed
> with exception Unable to rename:
> hdfs://host-10-17-81-192.coe.cloudera.com:8020/user/hive/warehouse/.hive-staging_hive_2015-10-29_19-04-08_375_5385987873542863570-3/-ext-10001
> to: /user/hive/data/yshi/nonexisting/test3
> org.apache.hadoop.hive.ql.metadata.HiveException: Unable to rename:
> hdfs://host-10-17-81-192.coe.cloudera.com:8020/user/hive/warehouse/.hive-staging_hive_2015-10-29_19-04-08_375_5385987873542863570-3/-ext-10001
> to: /user/hive/data/yshi/nonexisting/test3
> at org.apache.hadoop.hive.ql.exec.MoveTask.moveFile(MoveTask.java:101)
> at org.apache.hadoop.hive.ql.exec.MoveTask.execute(MoveTask.java:209)
> at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:153)
> at
> org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:85)
> at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1554)
> at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1321)
> at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1139)
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:962)
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:957)
> at
> org.apache.hive.service.cli.operation.SQLOperation.runInternal(SQLOperation.java:144)
> at
> org.apache.hive.service.cli.operation.SQLOperation.access$000(SQLOperation.java:68)
> at
> org.apache.hive.service.cli.operation.SQLOperation$1$1.run(SQLOperation.java:199)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:415)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1642)
> at
> org.apache.hadoop.hive.shims.HadoopShimsSecure.doAs(HadoopShimsSecure.java:502)
> at
> org.apache.hive.service.cli.operation.SQLOperation$1.run(SQLOperation.java:212)
> at
>
[jira] [Updated] (HIVE-6925) show query progress in Beeline
[ https://issues.apache.org/jira/browse/HIVE-6925?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] wangwenli updated HIVE-6925: External issue ID: hive-7615 > show query progress in Beeline > -- > > Key: HIVE-6925 > URL: https://issues.apache.org/jira/browse/HIVE-6925 > Project: Hive > Issue Type: Improvement > Components: CLI >Affects Versions: 0.12.0 >Reporter: Gwen Shapira >Assignee: Chinna Rao Lalam > > In the "old" Hive CLI, the MR output was written to screen. Making it easy to > watch the progress - map and reduce % done. > In Beeline, there is no output until the query is done (or fails). Showing > some kind of progress indicator will be nice. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-13257) GroupBy with column alias does not support AVG
[
https://issues.apache.org/jira/browse/HIVE-13257?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15188548#comment-15188548
]
Prasanth Jayachandran commented on HIVE-13257:
--
[~ashutoshc] I haven't digged deeply into it. But can this be supported?
> GroupBy with column alias does not support AVG
> --
>
> Key: HIVE-13257
> URL: https://issues.apache.org/jira/browse/HIVE-13257
> Project: Hive
> Issue Type: Bug
>Affects Versions: 2.1.0
>Reporter: Prasanth Jayachandran
>
> For the following query, with hive.groupby.orderby.position.alias set to true
> {code:title=Query}
> SELECT Avg(`t0`.`x_measure__0`) AS `avg_calculation_270497503505567749_ok`
> FROM (SELECT `store_sales`.`ss_ticket_number` AS `ss_ticket_number`,
>Sum(`store_sales`.`ss_net_paid`) AS `x_measure__0`
> FROM `store_sales` `store_sales`
>JOIN `item` `item`
> ON ( `store_sales`.`ss_item_sk` = `item`.`i_item_sk` )
> GROUP BY `store_sales`.`ss_ticket_number`) `t0`
> GROUP BY 1
> HAVING ( Count(1) > 0 );
> {code}
> it throws the following exception
> {code:title=Exception}
> FAILED: SemanticException [Error 10128]: Line 2:7 Not yet supported place for
> UDAF 'Avg’
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-13203) STRUCT type not allowing more than 266 fields. Table get created with more fields but throws error while querying against the table. SHOW CREATE TABLE skips having some f
[ https://issues.apache.org/jira/browse/HIVE-13203?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Satyajit varma updated HIVE-13203: -- Priority: Trivial (was: Major) > STRUCT type not allowing more than 266 fields. Table get created with more > fields but throws error while querying against the table. SHOW CREATE TABLE > skips having some fields in STRUCT. > -- > > Key: HIVE-13203 > URL: https://issues.apache.org/jira/browse/HIVE-13203 > Project: Hive > Issue Type: Bug > Components: Database/Schema >Affects Versions: 1.2.0 >Reporter: Satyajit varma >Assignee: Satyajit varma >Priority: Trivial > > I am trying to create a table with multiple STRUCT column types, one of which > having more than 300 fields. > I get the table created with no problem, but when i load i face few errors in > loading up the data.when i check SHOW CREATE TABLE TABLENAME, i get fields > only till 266th column and rest of them do not appear in the table DDL.(check > the issue at StateOrProvinc, search for StateOrProvinc in the second DDL). > Would like to know, if there is any restriction on the number of fields to be > using in STRUCT datatype. > EXAMPLE: > BELOW IS THE DDL i run in hive and the second below DLL is the TABLE DDL i > get from SHOW CREATE TABLE. > CREATE TABLE property_feature.property_feature_new( > `_id` STRUCT<`mls_sysid`:String,`type`:String>, > all_attrs STRUCT<`1`: String ,`10`: String ,`100`: String ,`105`: String > ,`113`: String ,`118`: String ,`126`: String ,`1299`: String ,`1301`: String > ,`1305`: String ,`1307`: String ,`1319`: String ,`1325`: String ,`1327`: > String ,`1329`: String ,`1339`: String ,`1341`: String ,`1345`: String > ,`1351`: String ,`1353`: String ,`1359`: String ,`1363`: String ,`1365`: > String ,`1367`: String ,`1373`: String ,`1377`: String ,`1397`: String > ,`1401`: String ,`1403`: String ,`1405`: String ,`1409`: String ,`1411`: > String ,`1413`: String ,`1417`: String ,`1419`: String ,`1421`: String > ,`1423`: String ,`1431`: String ,`1437`: String ,`1439`: String ,`144`: > String ,`1441`: String ,`145`: String ,`1451`: String ,`1453`: String ,`146`: > String ,`1467`: String ,`1469`: String ,`1483`: String ,`1485`: String > ,`1491`: String ,`1493`: String ,`15`: String ,`1505`: String ,`1519`: String > ,`1521`: String ,`1523`: String ,`155`: String ,`156`: String ,`158`: String > ,`1586`: String ,`1678`: String ,`169`: String ,`17`: String ,`177`: String > ,`1783`: String ,`1785`: String ,`1787`: String ,`1789`: String ,`1793`: > String ,`1795`: String ,`1797`: String ,`1799`: String ,`1801`: String > ,`1803`: String ,`1805`: String ,`1807`: String ,`1809`: String ,`1811`: > String ,`1815`: String ,`1818`: String ,`1826`: String ,`1829`: String > ,`1874`: String ,`1876`: String ,`1878`: String ,`1880`: String ,`1882`: > String ,`207`: String ,`215`: String ,`226`: String ,`2293`: String ,`232`: > String ,`239`: String ,`241`: String ,`245`: String ,`2456`: String ,`248`: > String ,`2482`: String ,`2490`: String ,`2491`: String ,`2492`: String > ,`2493`: String ,`2495`: String ,`25`: String ,`2502`: String ,`2503`: String > ,`2517`: String ,`2522`: String ,`255`: String ,`256`: String ,`2573`: String > ,`2574`: String ,`260`: String ,`263`: String ,`265`: String ,`276`: String > ,`278`: String ,`286`: String ,`293`: String ,`296`: String ,`299`: String > ,`3`: String ,`30`: String ,`307`: String ,`312`: String ,`320`: String > ,`328`: String ,`340`: String ,`348`: String ,`354`: String ,`363`: String > ,`364`: String ,`369`: String ,`375`: String ,`376`: String ,`383`: String > ,`401`: String ,`405`: String ,`409`: String ,`412`: String ,`42`: String > ,`423`: String ,`444`: String ,`445`: String ,`446`: String ,`447`: String > ,`450`: String ,`451`: String ,`452`: String ,`453`: String ,`455`: String > ,`458`: String ,`461`: String ,`463`: String ,`464`: String ,`51`: String > ,`543472980`: String ,`57`: String ,`75`: String ,`90`: String ,`92`: String > ,`AVMYN`: String ,`AccessibilityAmenities`: String ,`AddressPublic`: String > ,`Amenities`: String ,`AmountofWaterFront`: String ,`AppliancesIncluded`: > String ,`AppraiserInitials`: String ,`AppraiserName`: String ,`Basement`: > String ,`BathsFull`: String ,`BathsHalf`: String ,`BathsTotal`: String > ,`BedsTotal`: String ,`Block`: String ,`Book`: String ,`BuildingNumber`: > String ,`City`: String ,`CloseDate`: String ,`ClosePrice`: String , > `CoListAgentDirectWorkPhone`: String ,`CoListAgentEmail`: String > ,`CoListAgentFullName`: String ,`CoListAgentMLSID`: String > ,`CoListAgent_MUI`: String ,`CoListOfficeMLSID`: String
[jira] [Commented] (HIVE-4570) More information to user on GetOperationStatus in Hive Server2 when query is still executing
[ https://issues.apache.org/jira/browse/HIVE-4570?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15188465#comment-15188465 ] Szehon Ho commented on HIVE-4570: - On closer look, I see you uploaded v3 patch. Can you upload to a new name and also include the thrift stuff for completeness, when running the HiveQA? Thanks > More information to user on GetOperationStatus in Hive Server2 when query is > still executing > > > Key: HIVE-4570 > URL: https://issues.apache.org/jira/browse/HIVE-4570 > Project: Hive > Issue Type: Improvement > Components: HiveServer2 >Reporter: Amareshwari Sriramadasu >Assignee: Rajat Khandelwal > Attachments: HIVE-4570.01.patch, HIVE-4570.02.patch, > HIVE-4570.03.patch, HIVE-4570.03.patch, HIVE-4570.04.patch > > > Currently in Hive Server2, when the query is still executing only the status > is set as STILL_EXECUTING. > This issue is to give more information to the user such as progress and > running job handles, if possible. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-4570) More information to user on GetOperationStatus in Hive Server2 when query is still executing
[ https://issues.apache.org/jira/browse/HIVE-4570?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15188461#comment-15188461 ] Szehon Ho commented on HIVE-4570: - Latest patch on RB https://reviews.apache.org/r/44453/ looked great to me, but I think the code is a little different than v4 patch, for example Driver class, can you double check? > More information to user on GetOperationStatus in Hive Server2 when query is > still executing > > > Key: HIVE-4570 > URL: https://issues.apache.org/jira/browse/HIVE-4570 > Project: Hive > Issue Type: Improvement > Components: HiveServer2 >Reporter: Amareshwari Sriramadasu >Assignee: Rajat Khandelwal > Attachments: HIVE-4570.01.patch, HIVE-4570.02.patch, > HIVE-4570.03.patch, HIVE-4570.03.patch, HIVE-4570.04.patch > > > Currently in Hive Server2, when the query is still executing only the status > is set as STILL_EXECUTING. > This issue is to give more information to the user such as progress and > running job handles, if possible. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-13254) GBY cardinality estimation is wrong partition columns is involved
[
https://issues.apache.org/jira/browse/HIVE-13254?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Prasanth Jayachandran updated HIVE-13254:
-
Attachment: q3_ef_transpose_aggr.svg
> GBY cardinality estimation is wrong partition columns is involved
> -
>
> Key: HIVE-13254
> URL: https://issues.apache.org/jira/browse/HIVE-13254
> Project: Hive
> Issue Type: Bug
> Components: Hive
>Affects Versions: 1.3.0, 2.0.0, 2.1.0
>Reporter: Prasanth Jayachandran
> Attachments: q3_ef_transpose_aggr.svg
>
>
> When running the following query on TPCDS-1000 scale, setting
> hive.transpose.aggr.join=true is expected to generate optimal plan but it was
> not generating.
> {code:title=Query}
> SELECT `date_dim`.`d_day_name` AS `d_day_name`,
>`item`.`i_category` AS `i_category`
> FROM `store_sales` `store_sales`
>INNER JOIN `item` `item`
>ON ( `store_sales`.`ss_item_sk` = `item`.`i_item_sk` )
>INNER JOIN `date_dim` `date_dim`
>ON ( `store_sales`.`ss_sold_date_sk` = `date_dim`.`d_date_sk`
> )
> GROUP BY `d_day_name`,
> `i_category`;
> {code}
> The reason for that is stats annotation rule for GROUP BY is not considering
> partition column into account. For the above query, the generated plan is
> attached. As we can see from the plan, GBY is pushed to fact table
> (store_sales) but that output of GBY shuffled to perform join instead of
> MapJoin conversion. This is because of wrong estimation of cardinality/data
> size of GBY on store_sales (Map 1).
> What's happening internally is, GBY computes estimated cardinality which in
> this case is NDV(ss_item_sk) * NDV(ss_sold_date_sk) = 338901 * 1823 ~= 617M.
> This estimate is wrong as ss_sold_date_sk is partition column and estimator
> assumes its non-partition column. In this case, not every tasks reads data
> from all partitions. We need to take estimated task parallelism into account.
> For example: If task parallelism is determined to be 100 the estimate from
> GBY should be ~6M which should convert this vertex into map join vertex.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-13185) orc.ReaderImp.ensureOrcFooter() method fails on small text files with IndexOutOfBoundsException
[
https://issues.apache.org/jira/browse/HIVE-13185?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15188388#comment-15188388
]
Hive QA commented on HIVE-13185:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12792045/HIVE-13185.1.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 7 failed/errored test(s), 9792 tests executed
*Failed tests:*
{noformat}
TestSparkCliDriver-groupby3_map.q-sample2.q-auto_join14.q-and-12-more - did not
produce a TEST-*.xml file
TestSparkCliDriver-groupby_map_ppr_multi_distinct.q-table_access_keys_stats.q-groupby4_noskew.q-and-12-more
- did not produce a TEST-*.xml file
TestSparkCliDriver-join_rc.q-insert1.q-vectorized_rcfile_columnar.q-and-12-more
- did not produce a TEST-*.xml file
TestSparkCliDriver-ppd_join4.q-join9.q-ppd_join3.q-and-12-more - did not
produce a TEST-*.xml file
TestSparkCliDriver-timestamp_lazy.q-bucketsortoptimize_insert_4.q-date_udf.q-and-12-more
- did not produce a TEST-*.xml file
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_authorization_uri_import
org.apache.hive.jdbc.TestSSL.testSSLVersion
{noformat}
Test results:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/7206/testReport
Console output:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/7206/console
Test logs:
http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-7206/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 7 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12792045 - PreCommit-HIVE-TRUNK-Build
> orc.ReaderImp.ensureOrcFooter() method fails on small text files with
> IndexOutOfBoundsException
> ---
>
> Key: HIVE-13185
> URL: https://issues.apache.org/jira/browse/HIVE-13185
> Project: Hive
> Issue Type: Bug
> Components: ORC
>Affects Versions: 2.1.0
>Reporter: Illya Yalovyy
>Assignee: Illya Yalovyy
> Attachments: HIVE-13185.1.patch
>
>
> Steps to reproduce:
> 1. Create a Text source table with one line of data:
> {code}
> create table src (id int);
> insert overwrite table src values (1);
> {code}
> 2. Create a target table:
> {code}
> create table trg (id int);
> {code}
> 3. Try to load small text file to the target table:
> {code}
> load data inpath 'user/hive/warehouse/src/00_0' into table trg;
> {code}
> *Error message:*
> {quote}
> FAILED: SemanticException Unable to load data to destination table. Error:
> java.lang.IndexOutOfBoundsException
> {quote}
> *Stack trace:*
> {noformat}
> org.apache.hadoop.hive.ql.parse.SemanticException: Unable to load data to
> destination table. Error: java.lang.IndexOutOfBoundsException
> at
> org.apache.hadoop.hive.ql.parse.LoadSemanticAnalyzer.ensureFileFormatsMatch(LoadSemanticAnalyzer.java:340)
> at
> org.apache.hadoop.hive.ql.parse.LoadSemanticAnalyzer.analyzeInternal(LoadSemanticAnalyzer.java:224)
> at
> org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:242)
> at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:481)
> at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:317)
> at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:1190)
> at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1285)
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1116)
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1104)
> ...
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-12039) Temporarily disable TestSSL#testSSLVersion
[ https://issues.apache.org/jira/browse/HIVE-12039?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15188379#comment-15188379 ] Ashutosh Chauhan commented on HIVE-12039: - Created HIVE-13253 > Temporarily disable TestSSL#testSSLVersion > --- > > Key: HIVE-12039 > URL: https://issues.apache.org/jira/browse/HIVE-12039 > Project: Hive > Issue Type: Bug > Components: HiveServer2 >Affects Versions: 1.3.0, 2.0.0 >Reporter: Vaibhav Gumashta >Assignee: Ashutosh Chauhan > Fix For: 2.1.0 > > Attachments: HIVE-12039.1.patch, HIVE-12039.2.patch, HIVE-12039.patch > > > Looks like it's only run on Linux and failing after HIVE-11720. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-12039) Temporarily disable TestSSL#testSSLVersion
[ https://issues.apache.org/jira/browse/HIVE-12039?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashutosh Chauhan updated HIVE-12039: Resolution: Fixed Assignee: Ashutosh Chauhan (was: Vaibhav Gumashta) Fix Version/s: 2.1.0 Status: Resolved (was: Patch Available) Pushed to master. > Temporarily disable TestSSL#testSSLVersion > --- > > Key: HIVE-12039 > URL: https://issues.apache.org/jira/browse/HIVE-12039 > Project: Hive > Issue Type: Bug > Components: HiveServer2 >Affects Versions: 1.3.0, 2.0.0 >Reporter: Vaibhav Gumashta >Assignee: Ashutosh Chauhan > Fix For: 2.1.0 > > Attachments: HIVE-12039.1.patch, HIVE-12039.2.patch, HIVE-12039.patch > > > Looks like it's only run on Linux and failing after HIVE-11720. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-12039) Temporarily disable TestSSL#testSSLVersion
[ https://issues.apache.org/jira/browse/HIVE-12039?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashutosh Chauhan updated HIVE-12039: Summary: Temporarily disable TestSSL#testSSLVersion (was: Fix TestSSL#testSSLVersion ) > Temporarily disable TestSSL#testSSLVersion > --- > > Key: HIVE-12039 > URL: https://issues.apache.org/jira/browse/HIVE-12039 > Project: Hive > Issue Type: Bug > Components: HiveServer2 >Affects Versions: 1.3.0, 2.0.0 >Reporter: Vaibhav Gumashta >Assignee: Vaibhav Gumashta > Attachments: HIVE-12039.1.patch, HIVE-12039.2.patch, HIVE-12039.patch > > > Looks like it's only run on Linux and failing after HIVE-11720. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-12039) Fix TestSSL#testSSLVersion
[ https://issues.apache.org/jira/browse/HIVE-12039?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15188363#comment-15188363 ] Thejas M Nair commented on HIVE-12039: -- +1 Can you also please open a jira to re-enable after fixing ? > Fix TestSSL#testSSLVersion > --- > > Key: HIVE-12039 > URL: https://issues.apache.org/jira/browse/HIVE-12039 > Project: Hive > Issue Type: Bug > Components: HiveServer2 >Affects Versions: 1.3.0, 2.0.0 >Reporter: Vaibhav Gumashta >Assignee: Vaibhav Gumashta > Attachments: HIVE-12039.1.patch, HIVE-12039.2.patch, HIVE-12039.patch > > > Looks like it's only run on Linux and failing after HIVE-11720. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-12781) Temporarily disable authorization tests that always fail on Jenkins
[
https://issues.apache.org/jira/browse/HIVE-12781?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15188352#comment-15188352
]
Ashutosh Chauhan commented on HIVE-12781:
-
Created HIVE-13252 to reenable.
> Temporarily disable authorization tests that always fail on Jenkins
> ---
>
> Key: HIVE-12781
> URL: https://issues.apache.org/jira/browse/HIVE-12781
> Project: Hive
> Issue Type: Sub-task
>Reporter: Pengcheng Xiong
>Assignee: Ashutosh Chauhan
> Fix For: 2.1.0
>
> Attachments: HIVE-12781.patch
>
>
> This includes
> {code}
> org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_authorization_uri_import
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-12781) Temporarily disable authorization tests that always fail on Jenkins
[
https://issues.apache.org/jira/browse/HIVE-12781?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ashutosh Chauhan updated HIVE-12781:
Resolution: Fixed
Assignee: Ashutosh Chauhan (was: Sushanth Sowmyan)
Fix Version/s: 2.1.0
Status: Resolved (was: Patch Available)
Pushed to master.
> Temporarily disable authorization tests that always fail on Jenkins
> ---
>
> Key: HIVE-12781
> URL: https://issues.apache.org/jira/browse/HIVE-12781
> Project: Hive
> Issue Type: Sub-task
>Reporter: Pengcheng Xiong
>Assignee: Ashutosh Chauhan
> Fix For: 2.1.0
>
> Attachments: HIVE-12781.patch
>
>
> This includes
> {code}
> org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_authorization_uri_import
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-12781) Temporarily disable authorization tests that always fail on Jenkins
[
https://issues.apache.org/jira/browse/HIVE-12781?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ashutosh Chauhan updated HIVE-12781:
Summary: Temporarily disable authorization tests that always fail on
Jenkins (was: Fix authorization tests that always fail on Jenkins)
> Temporarily disable authorization tests that always fail on Jenkins
> ---
>
> Key: HIVE-12781
> URL: https://issues.apache.org/jira/browse/HIVE-12781
> Project: Hive
> Issue Type: Sub-task
>Reporter: Pengcheng Xiong
>Assignee: Sushanth Sowmyan
> Attachments: HIVE-12781.patch
>
>
> This includes
> {code}
> org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_authorization_uri_import
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-13175) Disallow making external tables transactional
[ https://issues.apache.org/jira/browse/HIVE-13175?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15188338#comment-15188338 ] Eugene Koifman commented on HIVE-13175: --- +1 > Disallow making external tables transactional > - > > Key: HIVE-13175 > URL: https://issues.apache.org/jira/browse/HIVE-13175 > Project: Hive > Issue Type: Bug > Components: Transactions >Affects Versions: 2.0.0 >Reporter: Wei Zheng >Assignee: Wei Zheng > Attachments: HIVE-13175.1.patch, HIVE-13175.2.patch, > HIVE-13175.3.patch, HIVE-13175.4.patch > > > The fact that compactor rewrites contents of ACID tables is in conflict with > what is expected of external tables. > Conversely, end user can write to External table which certainly not what is > expected of ACID table. > So we should explicitly disallow making an external table ACID. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-12781) Fix authorization tests that always fail on Jenkins
[
https://issues.apache.org/jira/browse/HIVE-12781?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15188334#comment-15188334
]
Sushanth Sowmyan commented on HIVE-12781:
-
+1.
Could you create another jira and cc us to figure out why this test fails in
the ptest env and eventually re-enable it?
> Fix authorization tests that always fail on Jenkins
> ---
>
> Key: HIVE-12781
> URL: https://issues.apache.org/jira/browse/HIVE-12781
> Project: Hive
> Issue Type: Sub-task
>Reporter: Pengcheng Xiong
>Assignee: Sushanth Sowmyan
> Attachments: HIVE-12781.patch
>
>
> This includes
> {code}
> org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_authorization_uri_import
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-12781) Fix authorization tests that always fail on Jenkins
[
https://issues.apache.org/jira/browse/HIVE-12781?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ashutosh Chauhan updated HIVE-12781:
Status: Patch Available (was: Open)
> Fix authorization tests that always fail on Jenkins
> ---
>
> Key: HIVE-12781
> URL: https://issues.apache.org/jira/browse/HIVE-12781
> Project: Hive
> Issue Type: Sub-task
>Reporter: Pengcheng Xiong
>Assignee: Sushanth Sowmyan
> Attachments: HIVE-12781.patch
>
>
> This includes
> {code}
> org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_authorization_uri_import
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-12781) Fix authorization tests that always fail on Jenkins
[
https://issues.apache.org/jira/browse/HIVE-12781?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ashutosh Chauhan updated HIVE-12781:
Attachment: HIVE-12781.patch
Patch to temporarily disable test. [~thejas] Can you take a look?
> Fix authorization tests that always fail on Jenkins
> ---
>
> Key: HIVE-12781
> URL: https://issues.apache.org/jira/browse/HIVE-12781
> Project: Hive
> Issue Type: Sub-task
>Reporter: Pengcheng Xiong
>Assignee: Sushanth Sowmyan
> Attachments: HIVE-12781.patch
>
>
> This includes
> {code}
> org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_authorization_uri_import
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-12781) Fix authorization tests that always fail on Jenkins
[
https://issues.apache.org/jira/browse/HIVE-12781?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15188324#comment-15188324
]
Sushanth Sowmyan commented on HIVE-12781:
-
Yeah, not a bad idea to try to disable this temporarily, I think. The problem
with this test is that it fails in a ptest env, but not in a lone run. And in a
lone run, it makes sense that it succeeds, this is the expected correct output.
> Fix authorization tests that always fail on Jenkins
> ---
>
> Key: HIVE-12781
> URL: https://issues.apache.org/jira/browse/HIVE-12781
> Project: Hive
> Issue Type: Sub-task
>Reporter: Pengcheng Xiong
>Assignee: Sushanth Sowmyan
>
> This includes
> {code}
> org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_authorization_uri_import
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-12039) Fix TestSSL#testSSLVersion
[ https://issues.apache.org/jira/browse/HIVE-12039?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashutosh Chauhan updated HIVE-12039: Attachment: HIVE-12039.patch Patch to disable test. [~thejas] Can you take a quick look ? > Fix TestSSL#testSSLVersion > --- > > Key: HIVE-12039 > URL: https://issues.apache.org/jira/browse/HIVE-12039 > Project: Hive > Issue Type: Bug > Components: HiveServer2 >Affects Versions: 1.3.0, 2.0.0 >Reporter: Vaibhav Gumashta >Assignee: Vaibhav Gumashta > Attachments: HIVE-12039.1.patch, HIVE-12039.2.patch, HIVE-12039.patch > > > Looks like it's only run on Linux and failing after HIVE-11720. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-12039) Fix TestSSL#testSSLVersion
[ https://issues.apache.org/jira/browse/HIVE-12039?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashutosh Chauhan updated HIVE-12039: Status: Open (was: Patch Available) > Fix TestSSL#testSSLVersion > --- > > Key: HIVE-12039 > URL: https://issues.apache.org/jira/browse/HIVE-12039 > Project: Hive > Issue Type: Bug > Components: HiveServer2 >Affects Versions: 2.0.0, 1.3.0 >Reporter: Vaibhav Gumashta >Assignee: Vaibhav Gumashta > Attachments: HIVE-12039.1.patch, HIVE-12039.2.patch, HIVE-12039.patch > > > Looks like it's only run on Linux and failing after HIVE-11720. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-12039) Fix TestSSL#testSSLVersion
[ https://issues.apache.org/jira/browse/HIVE-12039?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashutosh Chauhan updated HIVE-12039: Status: Patch Available (was: Open) > Fix TestSSL#testSSLVersion > --- > > Key: HIVE-12039 > URL: https://issues.apache.org/jira/browse/HIVE-12039 > Project: Hive > Issue Type: Bug > Components: HiveServer2 >Affects Versions: 2.0.0, 1.3.0 >Reporter: Vaibhav Gumashta >Assignee: Vaibhav Gumashta > Attachments: HIVE-12039.1.patch, HIVE-12039.2.patch, HIVE-12039.patch > > > Looks like it's only run on Linux and failing after HIVE-11720. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-13236) LLAP: token renewal interval needs to be set
[ https://issues.apache.org/jira/browse/HIVE-13236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15188292#comment-15188292 ] Siddharth Seth commented on HIVE-13236: --- Testing revealed an issue in ZKDelegationTokenSecretManager. > LLAP: token renewal interval needs to be set > > > Key: HIVE-13236 > URL: https://issues.apache.org/jira/browse/HIVE-13236 > Project: Hive > Issue Type: Bug >Affects Versions: 2.0.0 >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin > Attachments: HIVE-13236.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-13251) hive can't read the decimal in AVRO file generated from previous version
[ https://issues.apache.org/jira/browse/HIVE-13251?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15188286#comment-15188286 ] Aihua Xu commented on HIVE-13251: - OK. Seems we are trying to read based on file schema and then convert based on record schema since record schema may not match the file. > hive can't read the decimal in AVRO file generated from previous version > > > Key: HIVE-13251 > URL: https://issues.apache.org/jira/browse/HIVE-13251 > Project: Hive > Issue Type: Bug > Components: Hive >Affects Versions: 2.1.0 >Reporter: Aihua Xu >Assignee: Aihua Xu > > HIVE-7174 makes the avro schema change to match avro definition, while it > breaks the compatibility if the file is generated from the previous Hive. We > should allow to read old file format "precision" : "4", "scale": "8", but > write in the new format. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-12977) Pass credentials in the current UGI while creating Tez session
[
https://issues.apache.org/jira/browse/HIVE-12977?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vinoth Sathappan updated HIVE-12977:
Attachment: HIVE-12977.1.patch
> Pass credentials in the current UGI while creating Tez session
> --
>
> Key: HIVE-12977
> URL: https://issues.apache.org/jira/browse/HIVE-12977
> Project: Hive
> Issue Type: Bug
> Components: Tez
>Reporter: Vinoth Sathappan
>Assignee: Vinoth Sathappan
> Attachments: HIVE-12977.1.patch
>
>
> The credentials present in the current UGI i.e.
> UserGroupInformation.getCurrentUser().getCredentials() isn't passed to the
> Tez session. It is instantiated with null credentials
> session = TezClient.create("HIVE-" + sessionId, tezConfig, true,
> commonLocalResources, null);
> In this case, Tez fails to access resources even if the tokens are available
> in memory.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-13251) hive can't read the decimal in AVRO file generated from previous version
[ https://issues.apache.org/jira/browse/HIVE-13251?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15188242#comment-15188242 ] Aihua Xu commented on HIVE-13251: - [~xuefuz] I'm wondering why we try to get schema info from the file rather than from the record in HIVE-5823 for decimal? > hive can't read the decimal in AVRO file generated from previous version > > > Key: HIVE-13251 > URL: https://issues.apache.org/jira/browse/HIVE-13251 > Project: Hive > Issue Type: Bug > Components: Hive >Affects Versions: 2.1.0 >Reporter: Aihua Xu >Assignee: Aihua Xu > > HIVE-7174 makes the avro schema change to match avro definition, while it > breaks the compatibility if the file is generated from the previous Hive. We > should allow to read old file format "precision" : "4", "scale": "8", but > write in the new format. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-13251) hive can't read the decimal in AVRO file generated from previous version
[ https://issues.apache.org/jira/browse/HIVE-13251?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15188222#comment-15188222 ] Aihua Xu commented on HIVE-13251: - [~jarcec] FYI. Seems we need to handle such backward compatibility although the file format/schema is incorrect to avro since some customers generated such files and see such issues after upgrading. > hive can't read the decimal in AVRO file generated from previous version > > > Key: HIVE-13251 > URL: https://issues.apache.org/jira/browse/HIVE-13251 > Project: Hive > Issue Type: Bug > Components: Hive >Affects Versions: 2.1.0 >Reporter: Aihua Xu >Assignee: Aihua Xu > > HIVE-7174 makes the avro schema change to match avro definition, while it > breaks the compatibility if the file is generated from the previous Hive. We > should allow to read old file format "precision" : "4", "scale": "8", but > write in the new format. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-13251) hive can't be read the decimal in AVRO file generated from previous version
[ https://issues.apache.org/jira/browse/HIVE-13251?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aihua Xu updated HIVE-13251: Summary: hive can't be read the decimal in AVRO file generated from previous version (was: Decimal in AVRO file generated from previous hive can't be read ) > hive can't be read the decimal in AVRO file generated from previous version > --- > > Key: HIVE-13251 > URL: https://issues.apache.org/jira/browse/HIVE-13251 > Project: Hive > Issue Type: Bug > Components: Hive >Affects Versions: 2.1.0 >Reporter: Aihua Xu >Assignee: Aihua Xu > > HIVE-7174 makes the avro schema change to match avro definition, while it > breaks the compatibility if the file is generated from the previous Hive. We > should allow to read old file format "precision" : "4", "scale": "8", but > write in the new format. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-13251) hive can't read the decimal in AVRO file generated from previous version
[ https://issues.apache.org/jira/browse/HIVE-13251?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aihua Xu updated HIVE-13251: Summary: hive can't read the decimal in AVRO file generated from previous version (was: hive can't be read the decimal in AVRO file generated from previous version) > hive can't read the decimal in AVRO file generated from previous version > > > Key: HIVE-13251 > URL: https://issues.apache.org/jira/browse/HIVE-13251 > Project: Hive > Issue Type: Bug > Components: Hive >Affects Versions: 2.1.0 >Reporter: Aihua Xu >Assignee: Aihua Xu > > HIVE-7174 makes the avro schema change to match avro definition, while it > breaks the compatibility if the file is generated from the previous Hive. We > should allow to read old file format "precision" : "4", "scale": "8", but > write in the new format. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-13223) HoS may hang for queries that run on 0 splits
[ https://issues.apache.org/jira/browse/HIVE-13223?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashutosh Chauhan updated HIVE-13223: Attachment: HIVE-13223.1.patch Addressed Sergey's comment. > HoS may hang for queries that run on 0 splits > --- > > Key: HIVE-13223 > URL: https://issues.apache.org/jira/browse/HIVE-13223 > Project: Hive > Issue Type: Bug > Components: Spark >Affects Versions: 2.1.0 >Reporter: Ashutosh Chauhan >Assignee: Ashutosh Chauhan > Attachments: HIVE-13223.1.patch, HIVE-13223.patch > > > Can be seen on all timed out tests after HIVE-13040 went in -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-13223) HoS may hang for queries that run on 0 splits
[ https://issues.apache.org/jira/browse/HIVE-13223?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15188095#comment-15188095 ] Sergey Shelukhin commented on HIVE-13223: - nit: both equals could be done once outside of the loop. > HoS may hang for queries that run on 0 splits > --- > > Key: HIVE-13223 > URL: https://issues.apache.org/jira/browse/HIVE-13223 > Project: Hive > Issue Type: Bug > Components: Spark >Affects Versions: 2.1.0 >Reporter: Ashutosh Chauhan >Assignee: Ashutosh Chauhan > Attachments: HIVE-13223.patch > > > Can be seen on all timed out tests after HIVE-13040 went in -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (HIVE-13247) HIVE-13040 broke spark tests
[ https://issues.apache.org/jira/browse/HIVE-13247?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashutosh Chauhan resolved HIVE-13247. - Resolution: Duplicate > HIVE-13040 broke spark tests > > > Key: HIVE-13247 > URL: https://issues.apache.org/jira/browse/HIVE-13247 > Project: Hive > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Ashutosh Chauhan > > I confirmed that spark tests are getting stuck due to HIVE-13040. join_empty > is an example test; it gets stuck on master presently, presumably because 0 > splits are generated. When I reverted HIVE-13040 locally, it passed for me. > We should fix this or revert HIVE-13040 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-12988) Improve dynamic partition loading IV
[ https://issues.apache.org/jira/browse/HIVE-12988?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15188080#comment-15188080 ] Ashutosh Chauhan commented on HIVE-12988: - Failures are unrelated. This patch addresses Prasanth's concerns as well. > Improve dynamic partition loading IV > > > Key: HIVE-12988 > URL: https://issues.apache.org/jira/browse/HIVE-12988 > Project: Hive > Issue Type: Improvement > Components: Query Processor >Affects Versions: 1.2.0, 2.0.0 >Reporter: Ashutosh Chauhan >Assignee: Ashutosh Chauhan > Attachments: HIVE-12988.2.patch, HIVE-12988.2.patch, > HIVE-12988.3.patch, HIVE-12988.4.patch, HIVE-12988.5.patch, > HIVE-12988.6.patch, HIVE-12988.7.patch, HIVE-12988.patch > > > Parallelize copyFiles() -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-13112) Expose Lineage information in case of CTAS
[ https://issues.apache.org/jira/browse/HIVE-13112?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashutosh Chauhan updated HIVE-13112: Component/s: lineage > Expose Lineage information in case of CTAS > -- > > Key: HIVE-13112 > URL: https://issues.apache.org/jira/browse/HIVE-13112 > Project: Hive > Issue Type: Bug > Components: lineage >Reporter: Harish Butani >Assignee: Harish Butani > Fix For: 2.1.0 > > Attachments: HIVE-13112.01.patch > > > This is not happening because Lineage is being exposed by MoveTask which > checks for a LoadTableDesc. In case of CTAS the table is created after the > MoveTask. > Proposed solution is to add a flag to CreateTableDesc to track a CTAS > operation, and have the DDLTask expose the lineage if this flag is set. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-13112) Expose Lineage information in case of CTAS
[ https://issues.apache.org/jira/browse/HIVE-13112?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashutosh Chauhan updated HIVE-13112: Resolution: Fixed Fix Version/s: 2.1.0 Status: Resolved (was: Patch Available) Pushed to master with updated golden files. > Expose Lineage information in case of CTAS > -- > > Key: HIVE-13112 > URL: https://issues.apache.org/jira/browse/HIVE-13112 > Project: Hive > Issue Type: Bug > Components: lineage >Reporter: Harish Butani >Assignee: Harish Butani > Fix For: 2.1.0 > > Attachments: HIVE-13112.01.patch > > > This is not happening because Lineage is being exposed by MoveTask which > checks for a LoadTableDesc. In case of CTAS the table is created after the > MoveTask. > Proposed solution is to add a flag to CreateTableDesc to track a CTAS > operation, and have the DDLTask expose the lineage if this flag is set. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-12988) Improve dynamic partition loading IV
[
https://issues.apache.org/jira/browse/HIVE-12988?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15188076#comment-15188076
]
Hive QA commented on HIVE-12988:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12791924/HIVE-12988.7.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 7 failed/errored test(s), 9803 tests executed
*Failed tests:*
{noformat}
TestSparkCliDriver-groupby3_map.q-sample2.q-auto_join14.q-and-12-more - did not
produce a TEST-*.xml file
TestSparkCliDriver-groupby_map_ppr_multi_distinct.q-table_access_keys_stats.q-groupby4_noskew.q-and-12-more
- did not produce a TEST-*.xml file
TestSparkCliDriver-join_rc.q-insert1.q-vectorized_rcfile_columnar.q-and-12-more
- did not produce a TEST-*.xml file
TestSparkCliDriver-ppd_join4.q-join9.q-ppd_join3.q-and-12-more - did not
produce a TEST-*.xml file
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver_index_bitmap3
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_authorization_uri_import
org.apache.hive.jdbc.TestSSL.testSSLVersion
{noformat}
Test results:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/7205/testReport
Console output:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/7205/console
Test logs:
http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-7205/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 7 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12791924 - PreCommit-HIVE-TRUNK-Build
> Improve dynamic partition loading IV
>
>
> Key: HIVE-12988
> URL: https://issues.apache.org/jira/browse/HIVE-12988
> Project: Hive
> Issue Type: Improvement
> Components: Query Processor
>Affects Versions: 1.2.0, 2.0.0
>Reporter: Ashutosh Chauhan
>Assignee: Ashutosh Chauhan
> Attachments: HIVE-12988.2.patch, HIVE-12988.2.patch,
> HIVE-12988.3.patch, HIVE-12988.4.patch, HIVE-12988.5.patch,
> HIVE-12988.6.patch, HIVE-12988.7.patch, HIVE-12988.patch
>
>
> Parallelize copyFiles()
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-13248) Change date_add/date_sub/to_date functions to return Date type rather than String
[ https://issues.apache.org/jira/browse/HIVE-13248?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gopal V updated HIVE-13248: --- Affects Version/s: 2.1.0 2.0.0 > Change date_add/date_sub/to_date functions to return Date type rather than > String > - > > Key: HIVE-13248 > URL: https://issues.apache.org/jira/browse/HIVE-13248 > Project: Hive > Issue Type: Improvement > Components: UDF >Affects Versions: 2.0.0, 2.1.0 >Reporter: Jason Dere >Assignee: Jason Dere > > Some of the original "date" related functions return string values rather > than Date values, because they were created before the Date type existed in > Hive. We can try to change these to return Date in the 2.x line. > Date values should be implicitly convertible to String. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-13248) Change date_add/date_sub/to_date functions to return Date type rather than String
[ https://issues.apache.org/jira/browse/HIVE-13248?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gopal V updated HIVE-13248: --- Issue Type: Improvement (was: Bug) > Change date_add/date_sub/to_date functions to return Date type rather than > String > - > > Key: HIVE-13248 > URL: https://issues.apache.org/jira/browse/HIVE-13248 > Project: Hive > Issue Type: Improvement > Components: UDF >Reporter: Jason Dere >Assignee: Jason Dere > > Some of the original "date" related functions return string values rather > than Date values, because they were created before the Date type existed in > Hive. We can try to change these to return Date in the 2.x line. > Date values should be implicitly convertible to String. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-13040) Handle empty bucket creations more efficiently
[ https://issues.apache.org/jira/browse/HIVE-13040?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15187849#comment-15187849 ] Sergey Shelukhin commented on HIVE-13040: - I filed a JIRA for this (HIVE-13247) > Handle empty bucket creations more efficiently > --- > > Key: HIVE-13040 > URL: https://issues.apache.org/jira/browse/HIVE-13040 > Project: Hive > Issue Type: Improvement > Components: Query Processor >Affects Versions: 1.0.0, 1.2.0, 1.1.0, 2.0.0 >Reporter: Ashutosh Chauhan >Assignee: Ashutosh Chauhan > Fix For: 2.1.0 > > Attachments: HIVE-13040.2.patch, HIVE-13040.3.patch, > HIVE-13040.4.patch, HIVE-13040.5.patch, HIVE-13040.6.patch, > HIVE-13040.7.patch, HIVE-13040.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (HIVE-9660) store end offset of compressed data for RG in RowIndex in ORC
[ https://issues.apache.org/jira/browse/HIVE-9660?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Shelukhin reassigned HIVE-9660: -- Assignee: Sergey Shelukhin > store end offset of compressed data for RG in RowIndex in ORC > - > > Key: HIVE-9660 > URL: https://issues.apache.org/jira/browse/HIVE-9660 > Project: Hive > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin > > Right now the end offset is estimated, which in some cases results in tons of > extra data being read. > We can add a separate array to RowIndex (positions_v2?) that stores number of > compressed buffers for each RG, or end offset, or something, to remove this > estimation magic -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-13051) Deadline class has numerous issues
[ https://issues.apache.org/jira/browse/HIVE-13051?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15187811#comment-15187811 ] Sergey Shelukhin commented on HIVE-13051: - Backported to branch-2.0 and branch-1 > Deadline class has numerous issues > -- > > Key: HIVE-13051 > URL: https://issues.apache.org/jira/browse/HIVE-13051 > Project: Hive > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin > Fix For: 1.3.0, 2.1.0, 2.0.1 > > Attachments: HIVE-13051.01.patch, HIVE-13051.patch > > > currentTimeMillis is not a correct way to measure intervals of time; it can > easily be adjusted e.g. by ntpd. System.nanoTime should be used. > It's also unsafe for failure cases, and doesn't appear to update from config > updates correctly. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-13051) Deadline class has numerous issues
[ https://issues.apache.org/jira/browse/HIVE-13051?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Shelukhin updated HIVE-13051: Fix Version/s: 2.0.1 1.3.0 > Deadline class has numerous issues > -- > > Key: HIVE-13051 > URL: https://issues.apache.org/jira/browse/HIVE-13051 > Project: Hive > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin > Fix For: 1.3.0, 2.1.0, 2.0.1 > > Attachments: HIVE-13051.01.patch, HIVE-13051.patch > > > currentTimeMillis is not a correct way to measure intervals of time; it can > easily be adjusted e.g. by ntpd. System.nanoTime should be used. > It's also unsafe for failure cases, and doesn't appear to update from config > updates correctly. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-13245) VectorDeserializeRow throws IndexOutOfBoundsException
[
https://issues.apache.org/jira/browse/HIVE-13245?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15187772#comment-15187772
]
Prasanth Jayachandran commented on HIVE-13245:
--
With hive.vectorized.execution.mapjoin.native.fast.hashtable.enabled=false the
query runs fine.
> VectorDeserializeRow throws IndexOutOfBoundsException
> -
>
> Key: HIVE-13245
> URL: https://issues.apache.org/jira/browse/HIVE-13245
> Project: Hive
> Issue Type: Bug
>Affects Versions: 2.1.0
>Reporter: Prasanth Jayachandran
>
> When running following query on TPCDS 1000 scale, VectorDeserializeRow threw
> ArrayIndexOutOfBoundsException
> {code:title=Query}
> SELECT `customer_address`.`ca_zip` AS `ca_zip`,
>`customer_demographics`.`cd_education_status` AS
> `cd_education_status`,
>Sum(`store_sales`.`ss_net_paid`) AS `SUM:SS_NET_PAID:ok`
> FROM `store_sales` `store_sales`
>INNER JOIN `customer` `customer`
>ON ( `store_sales`.`ss_customer_sk` =
> `customer`.`c_customer_sk` )
>INNER JOIN `customer_address` `customer_address`
>ON ( `customer`.`c_current_addr_sk` =
> `customer_address`.`ca_address_sk` )
>INNER JOIN `customer_demographics` `customer_demographics`
>ON ( `customer`.`c_current_cdemo_sk` =
> `customer_demographics`.`cd_demo_sk` )
> WHERE ( `customer`.`c_first_sales_date_sk` > 2452300
> AND `customer_demographics`.`cd_gender` = 'F'
> AND `customer`.`c_current_addr_sk` IS NOT NULL
> AND `store_sales`.`ss_sold_date_sk` IS NOT NULL
> AND `customer`.`c_current_cdemo_sk` IS NOT NULL )
> GROUP BY `ca_zip`,
> `cd_education_status`;
> {code}
> {code:title=Exception}
> java.lang.RuntimeException: org.apache.hadoop.hive.ql.metadata.HiveException:
> Hive Runtime Error while processing row
> at
> org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:195)
> at
> org.apache.hadoop.hive.ql.exec.tez.TezProcessor.run(TezProcessor.java:160)
> at
> org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:354)
> at
> org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:71)
> at
> org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:59)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
> at
> org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:59)
> at
> org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:36)
> at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: java.lang.RuntimeException:
> org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while
> processing row
> at
> org.apache.hadoop.hive.ql.exec.tez.MapRecordSource.processRow(MapRecordSource.java:95)
> at
> org.apache.hadoop.hive.ql.exec.tez.MapRecordSource.pushRecord(MapRecordSource.java:70)
> at
> org.apache.hadoop.hive.ql.exec.tez.MapRecordProcessor.run(MapRecordProcessor.java:356)
> at
> org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:172)
> ... 14 more
> Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime
> Error while processing row
> at
> org.apache.hadoop.hive.ql.exec.vector.VectorMapOperator.process(VectorMapOperator.java:62)
> at
> org.apache.hadoop.hive.ql.exec.tez.MapRecordSource.processRow(MapRecordSource.java:86)
> ... 17 more
> Caused by: org.apache.hadoop.hive.ql.metadata.HiveException:
> java.lang.ArrayIndexOutOfBoundsException
> at
> org.apache.hadoop.hive.ql.exec.vector.mapjoin.VectorMapJoinInnerLongOperator.process(VectorMapJoinInnerLongOperator.java:392)
> at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:837)
> at
> org.apache.hadoop.hive.ql.exec.vector.VectorSelectOperator.process(VectorSelectOperator.java:143)
> at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:837)
> at
> org.apache.hadoop.hive.ql.exec.vector.VectorFilterOperator.process(VectorFilterOperator.java:121)
> at
[jira] [Commented] (HIVE-13245) VectorDeserializeRow throws IndexOutOfBoundsException
[
https://issues.apache.org/jira/browse/HIVE-13245?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15187769#comment-15187769
]
Prasanth Jayachandran commented on HIVE-13245:
--
It is ON in my tests.
> VectorDeserializeRow throws IndexOutOfBoundsException
> -
>
> Key: HIVE-13245
> URL: https://issues.apache.org/jira/browse/HIVE-13245
> Project: Hive
> Issue Type: Bug
>Affects Versions: 2.1.0
>Reporter: Prasanth Jayachandran
>
> When running following query on TPCDS 1000 scale, VectorDeserializeRow threw
> ArrayIndexOutOfBoundsException
> {code:title=Query}
> SELECT `customer_address`.`ca_zip` AS `ca_zip`,
>`customer_demographics`.`cd_education_status` AS
> `cd_education_status`,
>Sum(`store_sales`.`ss_net_paid`) AS `SUM:SS_NET_PAID:ok`
> FROM `store_sales` `store_sales`
>INNER JOIN `customer` `customer`
>ON ( `store_sales`.`ss_customer_sk` =
> `customer`.`c_customer_sk` )
>INNER JOIN `customer_address` `customer_address`
>ON ( `customer`.`c_current_addr_sk` =
> `customer_address`.`ca_address_sk` )
>INNER JOIN `customer_demographics` `customer_demographics`
>ON ( `customer`.`c_current_cdemo_sk` =
> `customer_demographics`.`cd_demo_sk` )
> WHERE ( `customer`.`c_first_sales_date_sk` > 2452300
> AND `customer_demographics`.`cd_gender` = 'F'
> AND `customer`.`c_current_addr_sk` IS NOT NULL
> AND `store_sales`.`ss_sold_date_sk` IS NOT NULL
> AND `customer`.`c_current_cdemo_sk` IS NOT NULL )
> GROUP BY `ca_zip`,
> `cd_education_status`;
> {code}
> {code:title=Exception}
> java.lang.RuntimeException: org.apache.hadoop.hive.ql.metadata.HiveException:
> Hive Runtime Error while processing row
> at
> org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:195)
> at
> org.apache.hadoop.hive.ql.exec.tez.TezProcessor.run(TezProcessor.java:160)
> at
> org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:354)
> at
> org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:71)
> at
> org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:59)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
> at
> org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:59)
> at
> org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:36)
> at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: java.lang.RuntimeException:
> org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while
> processing row
> at
> org.apache.hadoop.hive.ql.exec.tez.MapRecordSource.processRow(MapRecordSource.java:95)
> at
> org.apache.hadoop.hive.ql.exec.tez.MapRecordSource.pushRecord(MapRecordSource.java:70)
> at
> org.apache.hadoop.hive.ql.exec.tez.MapRecordProcessor.run(MapRecordProcessor.java:356)
> at
> org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:172)
> ... 14 more
> Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime
> Error while processing row
> at
> org.apache.hadoop.hive.ql.exec.vector.VectorMapOperator.process(VectorMapOperator.java:62)
> at
> org.apache.hadoop.hive.ql.exec.tez.MapRecordSource.processRow(MapRecordSource.java:86)
> ... 17 more
> Caused by: org.apache.hadoop.hive.ql.metadata.HiveException:
> java.lang.ArrayIndexOutOfBoundsException
> at
> org.apache.hadoop.hive.ql.exec.vector.mapjoin.VectorMapJoinInnerLongOperator.process(VectorMapJoinInnerLongOperator.java:392)
> at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:837)
> at
> org.apache.hadoop.hive.ql.exec.vector.VectorSelectOperator.process(VectorSelectOperator.java:143)
> at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:837)
> at
> org.apache.hadoop.hive.ql.exec.vector.VectorFilterOperator.process(VectorFilterOperator.java:121)
> at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:837)
>
[jira] [Commented] (HIVE-13245) VectorDeserializeRow throws IndexOutOfBoundsException
[
https://issues.apache.org/jira/browse/HIVE-13245?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15187767#comment-15187767
]
Matt McCline commented on HIVE-13245:
-
[~prasanth_j] No, I have not seen this before. And, it could be what Gopal
said because it is going through Native Vector MapJoin but it isn't clear if
the new Fast hash table is being used (HIVE-12896) or just the regular
BytesBytesMultiHashMap is being used.
Is hive.vectorized.execution.mapjoin.native.fast.hashtable.enabled ON or OFF?
> VectorDeserializeRow throws IndexOutOfBoundsException
> -
>
> Key: HIVE-13245
> URL: https://issues.apache.org/jira/browse/HIVE-13245
> Project: Hive
> Issue Type: Bug
>Affects Versions: 2.1.0
>Reporter: Prasanth Jayachandran
>
> When running following query on TPCDS 1000 scale, VectorDeserializeRow threw
> ArrayIndexOutOfBoundsException
> {code:title=Query}
> SELECT `customer_address`.`ca_zip` AS `ca_zip`,
>`customer_demographics`.`cd_education_status` AS
> `cd_education_status`,
>Sum(`store_sales`.`ss_net_paid`) AS `SUM:SS_NET_PAID:ok`
> FROM `store_sales` `store_sales`
>INNER JOIN `customer` `customer`
>ON ( `store_sales`.`ss_customer_sk` =
> `customer`.`c_customer_sk` )
>INNER JOIN `customer_address` `customer_address`
>ON ( `customer`.`c_current_addr_sk` =
> `customer_address`.`ca_address_sk` )
>INNER JOIN `customer_demographics` `customer_demographics`
>ON ( `customer`.`c_current_cdemo_sk` =
> `customer_demographics`.`cd_demo_sk` )
> WHERE ( `customer`.`c_first_sales_date_sk` > 2452300
> AND `customer_demographics`.`cd_gender` = 'F'
> AND `customer`.`c_current_addr_sk` IS NOT NULL
> AND `store_sales`.`ss_sold_date_sk` IS NOT NULL
> AND `customer`.`c_current_cdemo_sk` IS NOT NULL )
> GROUP BY `ca_zip`,
> `cd_education_status`;
> {code}
> {code:title=Exception}
> java.lang.RuntimeException: org.apache.hadoop.hive.ql.metadata.HiveException:
> Hive Runtime Error while processing row
> at
> org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:195)
> at
> org.apache.hadoop.hive.ql.exec.tez.TezProcessor.run(TezProcessor.java:160)
> at
> org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:354)
> at
> org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:71)
> at
> org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:59)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
> at
> org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:59)
> at
> org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:36)
> at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: java.lang.RuntimeException:
> org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while
> processing row
> at
> org.apache.hadoop.hive.ql.exec.tez.MapRecordSource.processRow(MapRecordSource.java:95)
> at
> org.apache.hadoop.hive.ql.exec.tez.MapRecordSource.pushRecord(MapRecordSource.java:70)
> at
> org.apache.hadoop.hive.ql.exec.tez.MapRecordProcessor.run(MapRecordProcessor.java:356)
> at
> org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:172)
> ... 14 more
> Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime
> Error while processing row
> at
> org.apache.hadoop.hive.ql.exec.vector.VectorMapOperator.process(VectorMapOperator.java:62)
> at
> org.apache.hadoop.hive.ql.exec.tez.MapRecordSource.processRow(MapRecordSource.java:86)
> ... 17 more
> Caused by: org.apache.hadoop.hive.ql.metadata.HiveException:
> java.lang.ArrayIndexOutOfBoundsException
> at
> org.apache.hadoop.hive.ql.exec.vector.mapjoin.VectorMapJoinInnerLongOperator.process(VectorMapJoinInnerLongOperator.java:392)
> at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:837)
> at
>
[jira] [Commented] (HIVE-13245) VectorDeserializeRow throws IndexOutOfBoundsException
[
https://issues.apache.org/jira/browse/HIVE-13245?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15187768#comment-15187768
]
Prasanth Jayachandran commented on HIVE-13245:
--
Yeah. The stacktrace looks similar. Possibly the same issue.
> VectorDeserializeRow throws IndexOutOfBoundsException
> -
>
> Key: HIVE-13245
> URL: https://issues.apache.org/jira/browse/HIVE-13245
> Project: Hive
> Issue Type: Bug
>Affects Versions: 2.1.0
>Reporter: Prasanth Jayachandran
>
> When running following query on TPCDS 1000 scale, VectorDeserializeRow threw
> ArrayIndexOutOfBoundsException
> {code:title=Query}
> SELECT `customer_address`.`ca_zip` AS `ca_zip`,
>`customer_demographics`.`cd_education_status` AS
> `cd_education_status`,
>Sum(`store_sales`.`ss_net_paid`) AS `SUM:SS_NET_PAID:ok`
> FROM `store_sales` `store_sales`
>INNER JOIN `customer` `customer`
>ON ( `store_sales`.`ss_customer_sk` =
> `customer`.`c_customer_sk` )
>INNER JOIN `customer_address` `customer_address`
>ON ( `customer`.`c_current_addr_sk` =
> `customer_address`.`ca_address_sk` )
>INNER JOIN `customer_demographics` `customer_demographics`
>ON ( `customer`.`c_current_cdemo_sk` =
> `customer_demographics`.`cd_demo_sk` )
> WHERE ( `customer`.`c_first_sales_date_sk` > 2452300
> AND `customer_demographics`.`cd_gender` = 'F'
> AND `customer`.`c_current_addr_sk` IS NOT NULL
> AND `store_sales`.`ss_sold_date_sk` IS NOT NULL
> AND `customer`.`c_current_cdemo_sk` IS NOT NULL )
> GROUP BY `ca_zip`,
> `cd_education_status`;
> {code}
> {code:title=Exception}
> java.lang.RuntimeException: org.apache.hadoop.hive.ql.metadata.HiveException:
> Hive Runtime Error while processing row
> at
> org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:195)
> at
> org.apache.hadoop.hive.ql.exec.tez.TezProcessor.run(TezProcessor.java:160)
> at
> org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:354)
> at
> org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:71)
> at
> org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:59)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
> at
> org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:59)
> at
> org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:36)
> at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: java.lang.RuntimeException:
> org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while
> processing row
> at
> org.apache.hadoop.hive.ql.exec.tez.MapRecordSource.processRow(MapRecordSource.java:95)
> at
> org.apache.hadoop.hive.ql.exec.tez.MapRecordSource.pushRecord(MapRecordSource.java:70)
> at
> org.apache.hadoop.hive.ql.exec.tez.MapRecordProcessor.run(MapRecordProcessor.java:356)
> at
> org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:172)
> ... 14 more
> Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime
> Error while processing row
> at
> org.apache.hadoop.hive.ql.exec.vector.VectorMapOperator.process(VectorMapOperator.java:62)
> at
> org.apache.hadoop.hive.ql.exec.tez.MapRecordSource.processRow(MapRecordSource.java:86)
> ... 17 more
> Caused by: org.apache.hadoop.hive.ql.metadata.HiveException:
> java.lang.ArrayIndexOutOfBoundsException
> at
> org.apache.hadoop.hive.ql.exec.vector.mapjoin.VectorMapJoinInnerLongOperator.process(VectorMapJoinInnerLongOperator.java:392)
> at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:837)
> at
> org.apache.hadoop.hive.ql.exec.vector.VectorSelectOperator.process(VectorSelectOperator.java:143)
> at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:837)
> at
> org.apache.hadoop.hive.ql.exec.vector.VectorFilterOperator.process(VectorFilterOperator.java:121)
> at
[jira] [Commented] (HIVE-13221) expose metastore APIs from HS2
[ https://issues.apache.org/jira/browse/HIVE-13221?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15187751#comment-15187751 ] Sergey Shelukhin commented on HIVE-13221: - [~thejas] [~ashutoshc] [~sushanth] can you take a look? > expose metastore APIs from HS2 > -- > > Key: HIVE-13221 > URL: https://issues.apache.org/jira/browse/HIVE-13221 > Project: Hive > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin > Attachments: HIVE-13221.01.patch, HIVE-13221.patch > > > I was always wondering why we don't do that, for the people who run HS2 and > also need metastore due to it being used externally; they don't need to run a > standalone metastore. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-13221) expose metastore APIs from HS2
[ https://issues.apache.org/jira/browse/HIVE-13221?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Shelukhin updated HIVE-13221: Attachment: HIVE-13221.01.patch > expose metastore APIs from HS2 > -- > > Key: HIVE-13221 > URL: https://issues.apache.org/jira/browse/HIVE-13221 > Project: Hive > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin > Attachments: HIVE-13221.01.patch, HIVE-13221.patch > > > I was always wondering why we don't do that, for the people who run HS2 and > also need metastore due to it being used externally; they don't need to run a > standalone metastore. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-13245) VectorDeserializeRow throws IndexOutOfBoundsException
[
https://issues.apache.org/jira/browse/HIVE-13245?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15187722#comment-15187722
]
Gopal V commented on HIVE-13245:
Possibly the same as the off-by-1 error in HIVE-12896 (smaller test-case in
there)
> VectorDeserializeRow throws IndexOutOfBoundsException
> -
>
> Key: HIVE-13245
> URL: https://issues.apache.org/jira/browse/HIVE-13245
> Project: Hive
> Issue Type: Bug
>Affects Versions: 2.1.0
>Reporter: Prasanth Jayachandran
>
> When running following query on TPCDS 1000 scale, VectorDeserializeRow threw
> ArrayIndexOutOfBoundsException
> {code:title=Query}
> SELECT `customer_address`.`ca_zip` AS `ca_zip`,
>`customer_demographics`.`cd_education_status` AS
> `cd_education_status`,
>Sum(`store_sales`.`ss_net_paid`) AS `SUM:SS_NET_PAID:ok`
> FROM `store_sales` `store_sales`
>INNER JOIN `customer` `customer`
>ON ( `store_sales`.`ss_customer_sk` =
> `customer`.`c_customer_sk` )
>INNER JOIN `customer_address` `customer_address`
>ON ( `customer`.`c_current_addr_sk` =
> `customer_address`.`ca_address_sk` )
>INNER JOIN `customer_demographics` `customer_demographics`
>ON ( `customer`.`c_current_cdemo_sk` =
> `customer_demographics`.`cd_demo_sk` )
> WHERE ( `customer`.`c_first_sales_date_sk` > 2452300
> AND `customer_demographics`.`cd_gender` = 'F'
> AND `customer`.`c_current_addr_sk` IS NOT NULL
> AND `store_sales`.`ss_sold_date_sk` IS NOT NULL
> AND `customer`.`c_current_cdemo_sk` IS NOT NULL )
> GROUP BY `ca_zip`,
> `cd_education_status`;
> {code}
> {code:title=Exception}
> java.lang.RuntimeException: org.apache.hadoop.hive.ql.metadata.HiveException:
> Hive Runtime Error while processing row
> at
> org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:195)
> at
> org.apache.hadoop.hive.ql.exec.tez.TezProcessor.run(TezProcessor.java:160)
> at
> org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:354)
> at
> org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:71)
> at
> org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:59)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
> at
> org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:59)
> at
> org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:36)
> at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: java.lang.RuntimeException:
> org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while
> processing row
> at
> org.apache.hadoop.hive.ql.exec.tez.MapRecordSource.processRow(MapRecordSource.java:95)
> at
> org.apache.hadoop.hive.ql.exec.tez.MapRecordSource.pushRecord(MapRecordSource.java:70)
> at
> org.apache.hadoop.hive.ql.exec.tez.MapRecordProcessor.run(MapRecordProcessor.java:356)
> at
> org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:172)
> ... 14 more
> Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime
> Error while processing row
> at
> org.apache.hadoop.hive.ql.exec.vector.VectorMapOperator.process(VectorMapOperator.java:62)
> at
> org.apache.hadoop.hive.ql.exec.tez.MapRecordSource.processRow(MapRecordSource.java:86)
> ... 17 more
> Caused by: org.apache.hadoop.hive.ql.metadata.HiveException:
> java.lang.ArrayIndexOutOfBoundsException
> at
> org.apache.hadoop.hive.ql.exec.vector.mapjoin.VectorMapJoinInnerLongOperator.process(VectorMapJoinInnerLongOperator.java:392)
> at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:837)
> at
> org.apache.hadoop.hive.ql.exec.vector.VectorSelectOperator.process(VectorSelectOperator.java:143)
> at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:837)
> at
> org.apache.hadoop.hive.ql.exec.vector.VectorFilterOperator.process(VectorFilterOperator.java:121)
> at
[jira] [Updated] (HIVE-12439) CompactionTxnHandler.markCleaned() and TxnHandler.openTxns() misc improvements
[ https://issues.apache.org/jira/browse/HIVE-12439?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei Zheng updated HIVE-12439: - Attachment: (was: HIVE-12439.1.patch) > CompactionTxnHandler.markCleaned() and TxnHandler.openTxns() misc improvements > -- > > Key: HIVE-12439 > URL: https://issues.apache.org/jira/browse/HIVE-12439 > Project: Hive > Issue Type: Improvement > Components: Metastore, Transactions >Affects Versions: 1.0.0 >Reporter: Eugene Koifman >Assignee: Wei Zheng > Attachments: HIVE-12439.1.patch > > > # add a safeguard to make sure IN clause is not too large; break up by txn id > to delete from TXN_COMPONENTS where tc_txnid in ... > # TxnHandler. openTxns() - use 1 insert with many rows in values() clause, > rather than 1 DB roundtrip per row -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-12439) CompactionTxnHandler.markCleaned() and TxnHandler.openTxns() misc improvements
[ https://issues.apache.org/jira/browse/HIVE-12439?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei Zheng updated HIVE-12439: - Attachment: HIVE-12439.1.patch > CompactionTxnHandler.markCleaned() and TxnHandler.openTxns() misc improvements > -- > > Key: HIVE-12439 > URL: https://issues.apache.org/jira/browse/HIVE-12439 > Project: Hive > Issue Type: Improvement > Components: Metastore, Transactions >Affects Versions: 1.0.0 >Reporter: Eugene Koifman >Assignee: Wei Zheng > Attachments: HIVE-12439.1.patch > > > # add a safeguard to make sure IN clause is not too large; break up by txn id > to delete from TXN_COMPONENTS where tc_txnid in ... > # TxnHandler. openTxns() - use 1 insert with many rows in values() clause, > rather than 1 DB roundtrip per row -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-13216) ORC Reader will leave file open until GC when opening a malformed ORC file
[
https://issues.apache.org/jira/browse/HIVE-13216?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sergey Shelukhin updated HIVE-13216:
Resolution: Fixed
Fix Version/s: 2.1.0
1.3.0
Status: Resolved (was: Patch Available)
Committed to master and branch-1. Thanks for the review!
> ORC Reader will leave file open until GC when opening a malformed ORC file
> --
>
> Key: HIVE-13216
> URL: https://issues.apache.org/jira/browse/HIVE-13216
> Project: Hive
> Issue Type: Bug
> Components: File Formats, ORC
>Affects Versions: 1.2.0, 1.2.1, 2.0.0
>Reporter: Yuxing Yao
>Assignee: Sergey Shelukhin
>Priority: Minor
> Fix For: 1.3.0, 2.1.0
>
> Attachments: HIVE-13216.patch
>
>
> In ORC extractMetaInfoFromFooter method of ReaderImpl.java:
> A new input stream is open without try-catch-finally to enforce closing.
> Once the footer parse has some exception, the stream close will miss.
> Until GC happen to close the stream.
> private static FileMetaInfo extractMetaInfoFromFooter(FileSystem fs,
> Path path,
> long maxFileLength
> ) throws IOException {
> FSDataInputStream file = fs.open(path);
> ...
> file.close();
> return new FileMetaInfo(
> ps.getCompression().toString(),
> (int) ps.getCompressionBlockSize(),
> (int) ps.getMetadataLength(),
> buffer,
> ps.getVersionList(),
> writerVersion
> );
> }
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-13211) normalize Hive.get overloads to go thru one path
[ https://issues.apache.org/jira/browse/HIVE-13211?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Shelukhin updated HIVE-13211: Resolution: Fixed Fix Version/s: 2.1.0 Status: Resolved (was: Patch Available) Committed to master. Thanks for the review! > normalize Hive.get overloads to go thru one path > > > Key: HIVE-13211 > URL: https://issues.apache.org/jira/browse/HIVE-13211 > Project: Hive > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin > Fix For: 2.1.0 > > Attachments: HIVE-13211.01.patch, HIVE-13211.patch > > > There are multiple subtly different paths in Hive.get(...) methods. Some > close the old db on refresh, some don't. Some check if the client is > compatible with config, some don't. Also there were some parameters (don't > register functions, disallow embedded metastore) that were added recently. > Need to make this stuff go thru one path. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-13211) normalize Hive.get overloads to go thru one path
[ https://issues.apache.org/jira/browse/HIVE-13211?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15187671#comment-15187671 ] Sergey Shelukhin commented on HIVE-13211: - The new spark failure failed to write some file, looks unrelated. The rest are known issues. > normalize Hive.get overloads to go thru one path > > > Key: HIVE-13211 > URL: https://issues.apache.org/jira/browse/HIVE-13211 > Project: Hive > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin > Attachments: HIVE-13211.01.patch, HIVE-13211.patch > > > There are multiple subtly different paths in Hive.get(...) methods. Some > close the old db on refresh, some don't. Some check if the client is > compatible with config, some don't. Also there were some parameters (don't > register functions, disallow embedded metastore) that were added recently. > Need to make this stuff go thru one path. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-10632) Make sure TXN_COMPONENTS gets cleaned up if table is dropped before compaction.
[ https://issues.apache.org/jira/browse/HIVE-10632?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15187666#comment-15187666 ] Wei Zheng commented on HIVE-10632: -- [~alangates] Ping.. > Make sure TXN_COMPONENTS gets cleaned up if table is dropped before > compaction. > --- > > Key: HIVE-10632 > URL: https://issues.apache.org/jira/browse/HIVE-10632 > Project: Hive > Issue Type: Bug > Components: Metastore, Transactions >Affects Versions: 1.0.0 >Reporter: Eugene Koifman >Assignee: Wei Zheng >Priority: Critical > Attachments: HIVE-10632.1.patch, HIVE-10632.2.patch, > HIVE-10632.3.patch, HIVE-10632.4.patch, HIVE-10632.5.patch, > HIVE-10632.6.patch, HIVE-10632.7.patch > > > The compaction process will clean up entries in TXNS, > COMPLETED_TXN_COMPONENTS, TXN_COMPONENTS. If the table/partition is dropped > before compaction is complete there will be data left in these tables. Need > to investigate if there are other situations where this may happen and > address it. > see HIVE-10595 for additional info -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-13175) Disallow making external tables transactional
[ https://issues.apache.org/jira/browse/HIVE-13175?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15187642#comment-15187642 ] Wei Zheng commented on HIVE-13175: -- [~ekoifman] Can you take another look? > Disallow making external tables transactional > - > > Key: HIVE-13175 > URL: https://issues.apache.org/jira/browse/HIVE-13175 > Project: Hive > Issue Type: Bug > Components: Transactions >Affects Versions: 2.0.0 >Reporter: Wei Zheng >Assignee: Wei Zheng > Attachments: HIVE-13175.1.patch, HIVE-13175.2.patch, > HIVE-13175.3.patch, HIVE-13175.4.patch > > > The fact that compactor rewrites contents of ACID tables is in conflict with > what is expected of external tables. > Conversely, end user can write to External table which certainly not what is > expected of ACID table. > So we should explicitly disallow making an external table ACID. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-13216) ORC Reader will leave file open until GC when opening a malformed ORC file
[
https://issues.apache.org/jira/browse/HIVE-13216?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15187637#comment-15187637
]
Sergey Shelukhin commented on HIVE-13216:
-
Tez tests failed due to safe mode on the NN. The rest are unrelated.
> ORC Reader will leave file open until GC when opening a malformed ORC file
> --
>
> Key: HIVE-13216
> URL: https://issues.apache.org/jira/browse/HIVE-13216
> Project: Hive
> Issue Type: Bug
> Components: File Formats, ORC
>Affects Versions: 1.2.0, 1.2.1, 2.0.0
>Reporter: Yuxing Yao
>Assignee: Sergey Shelukhin
>Priority: Minor
> Attachments: HIVE-13216.patch
>
>
> In ORC extractMetaInfoFromFooter method of ReaderImpl.java:
> A new input stream is open without try-catch-finally to enforce closing.
> Once the footer parse has some exception, the stream close will miss.
> Until GC happen to close the stream.
> private static FileMetaInfo extractMetaInfoFromFooter(FileSystem fs,
> Path path,
> long maxFileLength
> ) throws IOException {
> FSDataInputStream file = fs.open(path);
> ...
> file.close();
> return new FileMetaInfo(
> ps.getCompression().toString(),
> (int) ps.getCompressionBlockSize(),
> (int) ps.getMetadataLength(),
> buffer,
> ps.getVersionList(),
> writerVersion
> );
> }
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-13201) Compaction shouldn't be allowed on non-ACID table
[ https://issues.apache.org/jira/browse/HIVE-13201?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15187624#comment-15187624 ] Wei Zheng commented on HIVE-13201: -- [~alangates] Can you confirm? I'm planning to remove those 3 tests. > Compaction shouldn't be allowed on non-ACID table > - > > Key: HIVE-13201 > URL: https://issues.apache.org/jira/browse/HIVE-13201 > Project: Hive > Issue Type: Bug > Components: Transactions >Affects Versions: 2.0.0 >Reporter: Wei Zheng >Assignee: Wei Zheng > Attachments: HIVE-13201.1.patch > > > Looks like compaction is allowed on non-ACID table, although that's of no > sense and does nothing. Moreover the compaction request will be enqueued into > COMPACTION_QUEUE metastore table, which brings unnecessary overhead. > We should prevent compaction commands being allowed on non-ACID tables. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-13234) Remove dead ODBC driver from Hive
[ https://issues.apache.org/jira/browse/HIVE-13234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15187595#comment-15187595 ] Thejas M Nair commented on HIVE-13234: -- +1 > Remove dead ODBC driver from Hive > - > > Key: HIVE-13234 > URL: https://issues.apache.org/jira/browse/HIVE-13234 > Project: Hive > Issue Type: Task > Components: ODBC >Affects Versions: 2.1.0 >Reporter: Gopal V >Assignee: Gopal V > Labels: TODOC1.0 > Attachments: HIVE-13234.1.patch > > > Remove the HiveServer1 ODBC driver which has not been updated for several > years. > https://github.com/apache/hive/tree/master/odbc -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-12049) Provide an option to write serialized thrift objects in final tasks
[
https://issues.apache.org/jira/browse/HIVE-12049?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15187591#comment-15187591
]
Hive QA commented on HIVE-12049:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12791890/HIVE-12049.12.patch
{color:green}SUCCESS:{color} +1 due to 2 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 31 failed/errored test(s), 9786 tests
executed
*Failed tests:*
{noformat}
TestSparkCliDriver-groupby3_map.q-sample2.q-auto_join14.q-and-12-more - did not
produce a TEST-*.xml file
TestSparkCliDriver-groupby_map_ppr_multi_distinct.q-table_access_keys_stats.q-groupby4_noskew.q-and-12-more
- did not produce a TEST-*.xml file
TestSparkCliDriver-join_rc.q-insert1.q-vectorized_rcfile_columnar.q-and-12-more
- did not produce a TEST-*.xml file
TestSparkCliDriver-ppd_join4.q-join9.q-ppd_join3.q-and-12-more - did not
produce a TEST-*.xml file
TestSparkCliDriver-timestamp_lazy.q-bucketsortoptimize_insert_4.q-date_udf.q-and-12-more
- did not produce a TEST-*.xml file
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_columnStatsUpdateForStatsOptimizer_1
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_columnStatsUpdateForStatsOptimizer_2
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_insert_into1
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_insert_into2
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_metadata_only_queries
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_metadata_only_queries_with_filters
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_stats_only_null
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_truncate_table
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union_remove_26
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_insert_into1
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_insert_into2
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_metadata_only_queries
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_metadata_only_queries_with_filters
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_stats_only_null
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_authorization_uri_import
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_insert_into1
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_metadata_only_queries
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_metadata_only_queries_with_filters
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_stats_only_null
org.apache.hive.beeline.TestBeeLineWithArgs.testEmbeddedBeelineOutputs
org.apache.hive.beeline.TestBeeLineWithArgs.testQueryProgress
org.apache.hive.beeline.TestBeeLineWithArgs.testQueryProgressParallel
org.apache.hive.hcatalog.hbase.TestPigHBaseStorageHandler.org.apache.hive.hcatalog.hbase.TestPigHBaseStorageHandler
org.apache.hive.jdbc.TestJdbcDriver2.testExplainStmt
org.apache.hive.jdbc.TestJdbcDriver2.testGetQueryLog
org.apache.hive.jdbc.TestSSL.testSSLVersion
{noformat}
Test results:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/7204/testReport
Console output:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/7204/console
Test logs:
http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-7204/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 31 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12791890 - PreCommit-HIVE-TRUNK-Build
> Provide an option to write serialized thrift objects in final tasks
> ---
>
> Key: HIVE-12049
> URL: https://issues.apache.org/jira/browse/HIVE-12049
> Project: Hive
> Issue Type: Sub-task
> Components: HiveServer2
>Reporter: Rohit Dholakia
>Assignee: Rohit Dholakia
> Attachments: HIVE-12049.1.patch, HIVE-12049.11.patch,
> HIVE-12049.12.patch, HIVE-12049.2.patch, HIVE-12049.3.patch,
> HIVE-12049.4.patch, HIVE-12049.5.patch, HIVE-12049.6.patch,
> HIVE-12049.7.patch, HIVE-12049.9.patch
>
>
> For each fetch request to HiveServer2, we pay the penalty of deserializing
> the row objects and translating them into a different representation suitable
> for the RPC transfer. In a moderate to high concurrency scenarios, this can
> result in significant CPU and memory wastage. By having each task write the
> appropriate thrift objects to the output files, HiveServer2 can simply stream
> a batch of rows on the wire without
[jira] [Updated] (HIVE-10176) skip.header.line.count causes values to be skipped when performing insert values
[
https://issues.apache.org/jira/browse/HIVE-10176?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vladyslav Pavlenko updated HIVE-10176:
--
Attachment: HIVE-10176.1.patch
Regex used as filter for data inserted by sql query.
> skip.header.line.count causes values to be skipped when performing insert
> values
>
>
> Key: HIVE-10176
> URL: https://issues.apache.org/jira/browse/HIVE-10176
> Project: Hive
> Issue Type: Bug
>Affects Versions: 1.0.0
>Reporter: Wenbo Wang
> Attachments: HIVE-10176.1.patch, data
>
>
> When inserting values in to tables with TBLPROPERTIES
> ("skip.header.line.count"="1") the first value listed is also skipped.
> create table test (row int, name string) TBLPROPERTIES
> ("skip.header.line.count"="1");
> load data local inpath '/root/data' into table test;
> insert into table test values (1, 'a'), (2, 'b'), (3, 'c');
> (1, 'a') isn't inserted into the table.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-13244) Get 'test-query-redactor.json' file on TestQueryRedactor.java from the resources classpath
[
https://issues.apache.org/jira/browse/HIVE-13244?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sergio Peña updated HIVE-13244:
---
Attachment: HIVE-13244.1.patch
> Get 'test-query-redactor.json' file on TestQueryRedactor.java from the
> resources classpath
> --
>
> Key: HIVE-13244
> URL: https://issues.apache.org/jira/browse/HIVE-13244
> Project: Hive
> Issue Type: Improvement
>Affects Versions: 2.0.0
>Reporter: Sergio Peña
>Assignee: Sergio Peña
>Priority: Trivial
> Attachments: HIVE-13244.1.patch
>
>
> The 'test-query-redactor.json' file is used on TestQueryRedactor.java for
> doing redactor tests. This file is obtained using the following line:
> {code}
> conf.set("hive.query.redaction.rules", hiveRoot +
> "/ql/src/test/resources/test-query-redactor.json");
> {code}
> The location of the file is correct, but when running tests using the
> test-sources.jar file, the above line does not work. We should instead obtain
> the file using the {{ClassLoader.getSystemResource()}} to make it work.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-13244) Get 'test-query-redactor.json' file on TestQueryRedactor.java from the resources classpath
[
https://issues.apache.org/jira/browse/HIVE-13244?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sergio Peña updated HIVE-13244:
---
Priority: Trivial (was: Major)
> Get 'test-query-redactor.json' file on TestQueryRedactor.java from the
> resources classpath
> --
>
> Key: HIVE-13244
> URL: https://issues.apache.org/jira/browse/HIVE-13244
> Project: Hive
> Issue Type: Improvement
>Affects Versions: 2.0.0
>Reporter: Sergio Peña
>Assignee: Sergio Peña
>Priority: Trivial
> Attachments: HIVE-13244.1.patch
>
>
> The 'test-query-redactor.json' file is used on TestQueryRedactor.java for
> doing redactor tests. This file is obtained using the following line:
> {code}
> conf.set("hive.query.redaction.rules", hiveRoot +
> "/ql/src/test/resources/test-query-redactor.json");
> {code}
> The location of the file is correct, but when running tests using the
> test-sources.jar file, the above line does not work. We should instead obtain
> the file using the {{ClassLoader.getSystemResource()}} to make it work.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-13244) Get 'test-query-redactor.json' file on TestQueryRedactor.java from the resources classpath
[
https://issues.apache.org/jira/browse/HIVE-13244?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sergio Peña updated HIVE-13244:
---
Status: Patch Available (was: Open)
> Get 'test-query-redactor.json' file on TestQueryRedactor.java from the
> resources classpath
> --
>
> Key: HIVE-13244
> URL: https://issues.apache.org/jira/browse/HIVE-13244
> Project: Hive
> Issue Type: Improvement
>Affects Versions: 2.0.0
>Reporter: Sergio Peña
>Assignee: Sergio Peña
>Priority: Trivial
> Attachments: HIVE-13244.1.patch
>
>
> The 'test-query-redactor.json' file is used on TestQueryRedactor.java for
> doing redactor tests. This file is obtained using the following line:
> {code}
> conf.set("hive.query.redaction.rules", hiveRoot +
> "/ql/src/test/resources/test-query-redactor.json");
> {code}
> The location of the file is correct, but when running tests using the
> test-sources.jar file, the above line does not work. We should instead obtain
> the file using the {{ClassLoader.getSystemResource()}} to make it work.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-12330) Fix precommit Spark test part2
[ https://issues.apache.org/jira/browse/HIVE-12330?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergio Peña updated HIVE-12330: --- Priority: Major (was: Trivial) > Fix precommit Spark test part2 > -- > > Key: HIVE-12330 > URL: https://issues.apache.org/jira/browse/HIVE-12330 > Project: Hive > Issue Type: Bug > Components: Testing Infrastructure >Affects Versions: 2.0.0 >Reporter: Szehon Ho >Assignee: Sergio Peña > Fix For: 2.0.0 > > Attachments: HIVE-12229.3-spark.patch, HIVE-12330.4-spark.patch, > HIVE-12330.5-spark.patch, HIVE-12330.6-spark.patch, HIVE-12330.7-spark.patch, > HIVE-12330.patch > > > Regression because of HIVE-11489 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-12330) Fix precommit Spark test part2
[ https://issues.apache.org/jira/browse/HIVE-12330?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergio Peña updated HIVE-12330: --- Priority: Trivial (was: Major) > Fix precommit Spark test part2 > -- > > Key: HIVE-12330 > URL: https://issues.apache.org/jira/browse/HIVE-12330 > Project: Hive > Issue Type: Bug > Components: Testing Infrastructure >Affects Versions: 2.0.0 >Reporter: Szehon Ho >Assignee: Sergio Peña >Priority: Trivial > Fix For: 2.0.0 > > Attachments: HIVE-12229.3-spark.patch, HIVE-12330.4-spark.patch, > HIVE-12330.5-spark.patch, HIVE-12330.6-spark.patch, HIVE-12330.7-spark.patch, > HIVE-12330.patch > > > Regression because of HIVE-11489 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-12781) Fix authorization tests that always fail on Jenkins
[
https://issues.apache.org/jira/browse/HIVE-12781?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15187515#comment-15187515
]
Ashutosh Chauhan commented on HIVE-12781:
-
Failure from recent run:
{code}
Running: diff -a
/home/hiveptest/54.224.7.202-hiveptest-1/apache-github-source-source/itests/qtest/../../itests/qtest/target/qfile-results/clientnegative/authorization_uri_import.q.out
/home/hiveptest/54.224.7.202-hiveptest-1/apache-github-source-source/itests/qtest/../../ql/src/test/results/clientnegative/authorization_uri_import.q.out
11a12,40
> PREHOOK: query: export table import_auth_uri to
> 'ql/test/data/exports/import_auth_uri'
> PREHOOK: type: EXPORT
> PREHOOK: Input: default@import_auth_uri
> A masked pattern was here
> POSTHOOK: query: export table import_auth_uri to
> 'ql/test/data/exports/import_auth_uri'
> POSTHOOK: type: EXPORT
> POSTHOOK: Input: default@import_auth_uri
> A masked pattern was here
> PREHOOK: query: drop table import_auth_uri
> PREHOOK: type: DROPTABLE
> PREHOOK: Input: default@import_auth_uri
> PREHOOK: Output: default@import_auth_uri
> POSTHOOK: query: drop table import_auth_uri
> POSTHOOK: type: DROPTABLE
> POSTHOOK: Input: default@import_auth_uri
> POSTHOOK: Output: default@import_auth_uri
> PREHOOK: query: create database importer
> PREHOOK: type: CREATEDATABASE
> PREHOOK: Output: database:importer
> POSTHOOK: query: create database importer
> POSTHOOK: type: CREATEDATABASE
> POSTHOOK: Output: database:importer
> PREHOOK: query: use importer
> PREHOOK: type: SWITCHDATABASE
> PREHOOK: Input: database:importer
> POSTHOOK: query: use importer
> POSTHOOK: type: SWITCHDATABASE
> POSTHOOK: Input: database:importer
{code}
Shall we disable this test till we find a fix for this?
> Fix authorization tests that always fail on Jenkins
> ---
>
> Key: HIVE-12781
> URL: https://issues.apache.org/jira/browse/HIVE-12781
> Project: Hive
> Issue Type: Sub-task
>Reporter: Pengcheng Xiong
>Assignee: Sushanth Sowmyan
>
> This includes
> {code}
> org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_authorization_uri_import
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-4662) first_value can't have more than one order by column
[
https://issues.apache.org/jira/browse/HIVE-4662?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15187353#comment-15187353
]
Jesus Camacho Rodriguez commented on HIVE-4662:
---
[~ashutoshc], you are right, I focused on the multiple order by columns and I
even did not realize about the DISTINCT.
I removed the DISTINCT keyword for the example query. That way this patch can
go in.
I will follow up on the issue with DISTINCT in HIVE-13242. I explored a bit and
it seems that DISTINCT keyword in select clause is completely ignored for
windowing functions.
> first_value can't have more than one order by column
>
>
> Key: HIVE-4662
> URL: https://issues.apache.org/jira/browse/HIVE-4662
> Project: Hive
> Issue Type: Bug
> Components: SQL
>Affects Versions: 0.11.0
>Reporter: Frans Drijver
>Assignee: Jesus Camacho Rodriguez
> Attachments: HIVE-4662.01.patch, HIVE-4662.patch
>
>
> In the current implementation of the first_value function, it's not allowed
> to have more than one (1) order by column, as so:
> {quote}
> select distinct
> first_value(kastr.DEWNKNR) over ( partition by kastr.DEKTRNR order by
> kastr.DETRADT, kastr.DEVPDNR )
> from RTAVP_DRKASTR kastr
> ;
> {quote}
> Error given:
> {quote}
> FAILED: SemanticException Range based Window Frame can have only 1 Sort Key
> {quote}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-4662) first_value can't have more than one order by column
[
https://issues.apache.org/jira/browse/HIVE-4662?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jesus Camacho Rodriguez updated HIVE-4662:
--
Status: Patch Available (was: In Progress)
> first_value can't have more than one order by column
>
>
> Key: HIVE-4662
> URL: https://issues.apache.org/jira/browse/HIVE-4662
> Project: Hive
> Issue Type: Bug
> Components: SQL
>Affects Versions: 0.11.0
>Reporter: Frans Drijver
>Assignee: Jesus Camacho Rodriguez
> Attachments: HIVE-4662.01.patch, HIVE-4662.patch
>
>
> In the current implementation of the first_value function, it's not allowed
> to have more than one (1) order by column, as so:
> {quote}
> select distinct
> first_value(kastr.DEWNKNR) over ( partition by kastr.DEKTRNR order by
> kastr.DETRADT, kastr.DEVPDNR )
> from RTAVP_DRKASTR kastr
> ;
> {quote}
> Error given:
> {quote}
> FAILED: SemanticException Range based Window Frame can have only 1 Sort Key
> {quote}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-4662) first_value can't have more than one order by column
[
https://issues.apache.org/jira/browse/HIVE-4662?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jesus Camacho Rodriguez updated HIVE-4662:
--
Attachment: HIVE-4662.01.patch
> first_value can't have more than one order by column
>
>
> Key: HIVE-4662
> URL: https://issues.apache.org/jira/browse/HIVE-4662
> Project: Hive
> Issue Type: Bug
> Components: SQL
>Affects Versions: 0.11.0
>Reporter: Frans Drijver
>Assignee: Jesus Camacho Rodriguez
> Attachments: HIVE-4662.01.patch, HIVE-4662.patch
>
>
> In the current implementation of the first_value function, it's not allowed
> to have more than one (1) order by column, as so:
> {quote}
> select distinct
> first_value(kastr.DEWNKNR) over ( partition by kastr.DEKTRNR order by
> kastr.DETRADT, kastr.DEVPDNR )
> from RTAVP_DRKASTR kastr
> ;
> {quote}
> Error given:
> {quote}
> FAILED: SemanticException Range based Window Frame can have only 1 Sort Key
> {quote}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Work started] (HIVE-4662) first_value can't have more than one order by column
[
https://issues.apache.org/jira/browse/HIVE-4662?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Work on HIVE-4662 started by Jesus Camacho Rodriguez.
-
> first_value can't have more than one order by column
>
>
> Key: HIVE-4662
> URL: https://issues.apache.org/jira/browse/HIVE-4662
> Project: Hive
> Issue Type: Bug
> Components: SQL
>Affects Versions: 0.11.0
>Reporter: Frans Drijver
>Assignee: Jesus Camacho Rodriguez
> Attachments: HIVE-4662.patch
>
>
> In the current implementation of the first_value function, it's not allowed
> to have more than one (1) order by column, as so:
> {quote}
> select distinct
> first_value(kastr.DEWNKNR) over ( partition by kastr.DEKTRNR order by
> kastr.DETRADT, kastr.DEVPDNR )
> from RTAVP_DRKASTR kastr
> ;
> {quote}
> Error given:
> {quote}
> FAILED: SemanticException Range based Window Frame can have only 1 Sort Key
> {quote}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-4662) first_value can't have more than one order by column
[
https://issues.apache.org/jira/browse/HIVE-4662?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jesus Camacho Rodriguez updated HIVE-4662:
--
Status: Open (was: Patch Available)
> first_value can't have more than one order by column
>
>
> Key: HIVE-4662
> URL: https://issues.apache.org/jira/browse/HIVE-4662
> Project: Hive
> Issue Type: Bug
> Components: SQL
>Affects Versions: 0.11.0
>Reporter: Frans Drijver
>Assignee: Jesus Camacho Rodriguez
> Attachments: HIVE-4662.patch
>
>
> In the current implementation of the first_value function, it's not allowed
> to have more than one (1) order by column, as so:
> {quote}
> select distinct
> first_value(kastr.DEWNKNR) over ( partition by kastr.DEKTRNR order by
> kastr.DETRADT, kastr.DEVPDNR )
> from RTAVP_DRKASTR kastr
> ;
> {quote}
> Error given:
> {quote}
> FAILED: SemanticException Range based Window Frame can have only 1 Sort Key
> {quote}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-13222) Move rc-file-v0.rc used on TestRCFile.java to src/test/resources
[
https://issues.apache.org/jira/browse/HIVE-13222?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sergio Peña updated HIVE-13222:
---
Resolution: Fixed
Fix Version/s: 2.1.0
1.3.0
Status: Resolved (was: Patch Available)
> Move rc-file-v0.rc used on TestRCFile.java to src/test/resources
>
>
> Key: HIVE-13222
> URL: https://issues.apache.org/jira/browse/HIVE-13222
> Project: Hive
> Issue Type: Improvement
>Affects Versions: 2.0.0
>Reporter: Sergio Peña
>Assignee: Sergio Peña
>Priority: Minor
> Fix For: 1.3.0, 2.1.0
>
> Attachments: HIVE-13222.1.patch
>
>
> The file 'rc-file-v0.rc' is used by TestRCFile to validate RC files. This
> file is on {{src/test/data}}, and it should be good to move it to
> {{src/test/resources}} so the file is packaged into the test-sources.jar when
> -Psources is invoked.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-13221) expose metastore APIs from HS2
[
https://issues.apache.org/jira/browse/HIVE-13221?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15187242#comment-15187242
]
Hive QA commented on HIVE-13221:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12791889/HIVE-13221.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 9 failed/errored test(s), 9788 tests executed
*Failed tests:*
{noformat}
TestSparkCliDriver-groupby3_map.q-sample2.q-auto_join14.q-and-12-more - did not
produce a TEST-*.xml file
TestSparkCliDriver-groupby_map_ppr_multi_distinct.q-table_access_keys_stats.q-groupby4_noskew.q-and-12-more
- did not produce a TEST-*.xml file
TestSparkCliDriver-join_rc.q-insert1.q-vectorized_rcfile_columnar.q-and-12-more
- did not produce a TEST-*.xml file
TestSparkCliDriver-ppd_join4.q-join9.q-ppd_join3.q-and-12-more - did not
produce a TEST-*.xml file
TestSparkCliDriver-timestamp_lazy.q-bucketsortoptimize_insert_4.q-date_udf.q-and-12-more
- did not produce a TEST-*.xml file
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_authorization_uri_import
org.apache.hadoop.hive.metastore.TestMetaStoreListenersError.testEventListenerException
org.apache.hadoop.hive.metastore.TestMetaStoreListenersError.testInitListenerException
org.apache.hive.jdbc.TestSSL.testSSLVersion
{noformat}
Test results:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/7203/testReport
Console output:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/7203/console
Test logs:
http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-7203/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 9 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12791889 - PreCommit-HIVE-TRUNK-Build
> expose metastore APIs from HS2
> --
>
> Key: HIVE-13221
> URL: https://issues.apache.org/jira/browse/HIVE-13221
> Project: Hive
> Issue Type: Bug
>Reporter: Sergey Shelukhin
>Assignee: Sergey Shelukhin
> Attachments: HIVE-13221.patch
>
>
> I was always wondering why we don't do that, for the people who run HS2 and
> also need metastore due to it being used externally; they don't need to run a
> standalone metastore.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-4570) More information to user on GetOperationStatus in Hive Server2 when query is still executing
[ https://issues.apache.org/jira/browse/HIVE-4570?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajat Khandelwal updated HIVE-4570: --- Status: Patch Available (was: Open) > More information to user on GetOperationStatus in Hive Server2 when query is > still executing > > > Key: HIVE-4570 > URL: https://issues.apache.org/jira/browse/HIVE-4570 > Project: Hive > Issue Type: Improvement > Components: HiveServer2 >Reporter: Amareshwari Sriramadasu >Assignee: Rajat Khandelwal > Attachments: HIVE-4570.01.patch, HIVE-4570.02.patch, > HIVE-4570.03.patch, HIVE-4570.03.patch, HIVE-4570.04.patch > > > Currently in Hive Server2, when the query is still executing only the status > is set as STILL_EXECUTING. > This issue is to give more information to the user such as progress and > running job handles, if possible. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-4570) More information to user on GetOperationStatus in Hive Server2 when query is still executing
[ https://issues.apache.org/jira/browse/HIVE-4570?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajat Khandelwal updated HIVE-4570: --- Status: Open (was: Patch Available) > More information to user on GetOperationStatus in Hive Server2 when query is > still executing > > > Key: HIVE-4570 > URL: https://issues.apache.org/jira/browse/HIVE-4570 > Project: Hive > Issue Type: Improvement > Components: HiveServer2 >Reporter: Amareshwari Sriramadasu >Assignee: Rajat Khandelwal > Attachments: HIVE-4570.01.patch, HIVE-4570.02.patch, > HIVE-4570.03.patch, HIVE-4570.03.patch, HIVE-4570.04.patch > > > Currently in Hive Server2, when the query is still executing only the status > is set as STILL_EXECUTING. > This issue is to give more information to the user such as progress and > running job handles, if possible. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-4570) More information to user on GetOperationStatus in Hive Server2 when query is still executing
[ https://issues.apache.org/jira/browse/HIVE-4570?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15187207#comment-15187207 ] Rajat Khandelwal commented on HIVE-4570: Taking patch from reviewboard and attaching > More information to user on GetOperationStatus in Hive Server2 when query is > still executing > > > Key: HIVE-4570 > URL: https://issues.apache.org/jira/browse/HIVE-4570 > Project: Hive > Issue Type: Improvement > Components: HiveServer2 >Reporter: Amareshwari Sriramadasu >Assignee: Rajat Khandelwal > Attachments: HIVE-4570.01.patch, HIVE-4570.02.patch, > HIVE-4570.03.patch, HIVE-4570.03.patch, HIVE-4570.04.patch > > > Currently in Hive Server2, when the query is still executing only the status > is set as STILL_EXECUTING. > This issue is to give more information to the user such as progress and > running job handles, if possible. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-4570) More information to user on GetOperationStatus in Hive Server2 when query is still executing
[ https://issues.apache.org/jira/browse/HIVE-4570?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajat Khandelwal updated HIVE-4570: --- Attachment: HIVE-4570.03.patch > More information to user on GetOperationStatus in Hive Server2 when query is > still executing > > > Key: HIVE-4570 > URL: https://issues.apache.org/jira/browse/HIVE-4570 > Project: Hive > Issue Type: Improvement > Components: HiveServer2 >Reporter: Amareshwari Sriramadasu >Assignee: Rajat Khandelwal > Attachments: HIVE-4570.01.patch, HIVE-4570.02.patch, > HIVE-4570.03.patch, HIVE-4570.03.patch, HIVE-4570.04.patch > > > Currently in Hive Server2, when the query is still executing only the status > is set as STILL_EXECUTING. > This issue is to give more information to the user such as progress and > running job handles, if possible. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-12678) BETWEEN relational operator sometimes returns incorrect results against PARQUET tables
[
https://issues.apache.org/jira/browse/HIVE-12678?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15187146#comment-15187146
]
Chaoyu Tang commented on HIVE-12678:
It might be related to HIVE-13039. [~nbrenwald], could you try Hive 1.3.0 or
2.1.0 to see if you can reproduce the issue?
> BETWEEN relational operator sometimes returns incorrect results against
> PARQUET tables
> --
>
> Key: HIVE-12678
> URL: https://issues.apache.org/jira/browse/HIVE-12678
> Project: Hive
> Issue Type: Bug
>Affects Versions: 1.1.0, 1.2.1
>Reporter: Nicholas Brenwald
>Assignee: Chaoyu Tang
>
> When querying a parquet table, the BETWEEN relational operator returns
> incorrect results when hive.optimize.index.filter and
> hive.optimize.ppd.storage are enabled
> Create a parquet table:
> {code}
> create table t(c string) stored as parquet;
> {code}
> Insert some strings representing dates
> {code}
> insert into t select '2015-12-09' from default.dual limit 1;
> insert into t select '2015-12-10' from default.dual limit 1;
> insert into t select '2015-12-11' from default.dual limit 1;
> {code}
> h3. Example 1
> This query correctly returns 3:
> {code}
> set hive.optimize.index.filter=true;
> set hive.optimize.ppd.storage=true;
> select count(*) from t where c >= '2015-12-09' and c <= '2015-12-11';
> +--+--+
> | _c0 |
> +--+--+
> | 3|
> +--+--+
> {code}
> This query incorrectly returns 1:
> {code}
> set hive.optimize.index.filter=true;
> set hive.optimize.ppd.storage=true;
> select count(*) from t where c between '2015-12-09' and '2015-12-11';
> +--+--+
> | _c0 |
> +--+--+
> | 1|
> +--+--+
> {code}
> Disabling hive.optimize.findex.filter resolves the problem. This query now
> correctly returns 3:
> {code}
> set hive.optimize.index.filter=false;
> set hive.optimize.ppd.storage=true;
> select count(*) from t where c between '2015-12-09' and '2015-12-11';
> +--+--+
> | _c0 |
> +--+--+
> | 3|
> +--+--+
> {code}
> Disabling hive.optimize.ppd.storage resolves the problem. This query now
> correctly returns 3:
> {code}
> set hive.optimize.index.filter=true;
> set hive.optimize.ppd.storage=false;
> select count(*) from t where c between '2015-12-09' and '2015-12-11';
> +--+--+
> | _c0 |
> +--+--+
> | 3|
> +--+--+
> {code}
> h3. Example 2
> This query correctly returns 1:
> {code}
> set hive.optimize.index.filter=true;
> set hive.optimize.ppd.storage=true;
> select count(*) from t where c >= '2015-12-10' and c <= '2015-12-10';
> +--+--+
> | _c0 |
> +--+--+
> | 1|
> +--+--+
> {code}
> This query incorrectly returns 0:
> {code}
> set hive.optimize.index.filter=true;
> set hive.optimize.ppd.storage=true;
> select count(*) from t where c between '2015-12-10' and '2015-12-10';
> +--+--+
> | _c0 |
> +--+--+
> | 0|
> +--+--+
> {code}
> Disabling hive.optimize.findex.filter resolves the problem. This query now
> correctly returns 1:
> {code}
> set hive.optimize.index.filter=false;
> set hive.optimize.ppd.storage=true;
> select count(*) from t where c >= '2015-12-10' and c <= '2015-12-10';
> +--+--+
> | _c0 |
> +--+--+
> | 1|
> +--+--+
> {code}
> Disabling hive.optimize.ppd.storage resolves the problem. This query now
> correctly returns 1:
> {code}
> set hive.optimize.index.filter=true;
> set hive.optimize.ppd.storage=false;
> select count(*) from t where c >= '2015-12-10' and c <= '2015-12-10';
> +--+--+
> | _c0 |
> +--+--+
> | 1|
> +--+--+
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-12678) BETWEEN relational operator sometimes returns incorrect results against PARQUET tables
[
https://issues.apache.org/jira/browse/HIVE-12678?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15187118#comment-15187118
]
Chaoyu Tang commented on HIVE-12678:
Tested in upstream for the queries which return incorrect results:
{code}
hive> set hive.optimize.index.filter=true;
hive> set hive.optimize.ppd.storage=true;
hive> select count(*) from t where c between '2015-12-09' and '2015-12-11';
Total MapReduce CPU Time Spent: 0 msec
OK
3
hive> select count(*) from t where c between '2015-12-10' and '2015-12-10';
OK
1
{code}
After disabling the hive.compute.query.using.stats, these queries also return
the correct result. Somehow the issue has been fixed.
> BETWEEN relational operator sometimes returns incorrect results against
> PARQUET tables
> --
>
> Key: HIVE-12678
> URL: https://issues.apache.org/jira/browse/HIVE-12678
> Project: Hive
> Issue Type: Bug
>Affects Versions: 1.1.0, 1.2.1
>Reporter: Nicholas Brenwald
>Assignee: Chaoyu Tang
>
> When querying a parquet table, the BETWEEN relational operator returns
> incorrect results when hive.optimize.index.filter and
> hive.optimize.ppd.storage are enabled
> Create a parquet table:
> {code}
> create table t(c string) stored as parquet;
> {code}
> Insert some strings representing dates
> {code}
> insert into t select '2015-12-09' from default.dual limit 1;
> insert into t select '2015-12-10' from default.dual limit 1;
> insert into t select '2015-12-11' from default.dual limit 1;
> {code}
> h3. Example 1
> This query correctly returns 3:
> {code}
> set hive.optimize.index.filter=true;
> set hive.optimize.ppd.storage=true;
> select count(*) from t where c >= '2015-12-09' and c <= '2015-12-11';
> +--+--+
> | _c0 |
> +--+--+
> | 3|
> +--+--+
> {code}
> This query incorrectly returns 1:
> {code}
> set hive.optimize.index.filter=true;
> set hive.optimize.ppd.storage=true;
> select count(*) from t where c between '2015-12-09' and '2015-12-11';
> +--+--+
> | _c0 |
> +--+--+
> | 1|
> +--+--+
> {code}
> Disabling hive.optimize.findex.filter resolves the problem. This query now
> correctly returns 3:
> {code}
> set hive.optimize.index.filter=false;
> set hive.optimize.ppd.storage=true;
> select count(*) from t where c between '2015-12-09' and '2015-12-11';
> +--+--+
> | _c0 |
> +--+--+
> | 3|
> +--+--+
> {code}
> Disabling hive.optimize.ppd.storage resolves the problem. This query now
> correctly returns 3:
> {code}
> set hive.optimize.index.filter=true;
> set hive.optimize.ppd.storage=false;
> select count(*) from t where c between '2015-12-09' and '2015-12-11';
> +--+--+
> | _c0 |
> +--+--+
> | 3|
> +--+--+
> {code}
> h3. Example 2
> This query correctly returns 1:
> {code}
> set hive.optimize.index.filter=true;
> set hive.optimize.ppd.storage=true;
> select count(*) from t where c >= '2015-12-10' and c <= '2015-12-10';
> +--+--+
> | _c0 |
> +--+--+
> | 1|
> +--+--+
> {code}
> This query incorrectly returns 0:
> {code}
> set hive.optimize.index.filter=true;
> set hive.optimize.ppd.storage=true;
> select count(*) from t where c between '2015-12-10' and '2015-12-10';
> +--+--+
> | _c0 |
> +--+--+
> | 0|
> +--+--+
> {code}
> Disabling hive.optimize.findex.filter resolves the problem. This query now
> correctly returns 1:
> {code}
> set hive.optimize.index.filter=false;
> set hive.optimize.ppd.storage=true;
> select count(*) from t where c >= '2015-12-10' and c <= '2015-12-10';
> +--+--+
> | _c0 |
> +--+--+
> | 1|
> +--+--+
> {code}
> Disabling hive.optimize.ppd.storage resolves the problem. This query now
> correctly returns 1:
> {code}
> set hive.optimize.index.filter=true;
> set hive.optimize.ppd.storage=false;
> select count(*) from t where c >= '2015-12-10' and c <= '2015-12-10';
> +--+--+
> | _c0 |
> +--+--+
> | 1|
> +--+--+
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Assigned] (HIVE-12678) BETWEEN relational operator sometimes returns incorrect results against PARQUET tables
[
https://issues.apache.org/jira/browse/HIVE-12678?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chaoyu Tang reassigned HIVE-12678:
--
Assignee: Chaoyu Tang
> BETWEEN relational operator sometimes returns incorrect results against
> PARQUET tables
> --
>
> Key: HIVE-12678
> URL: https://issues.apache.org/jira/browse/HIVE-12678
> Project: Hive
> Issue Type: Bug
>Affects Versions: 1.1.0, 1.2.1
>Reporter: Nicholas Brenwald
>Assignee: Chaoyu Tang
>
> When querying a parquet table, the BETWEEN relational operator returns
> incorrect results when hive.optimize.index.filter and
> hive.optimize.ppd.storage are enabled
> Create a parquet table:
> {code}
> create table t(c string) stored as parquet;
> {code}
> Insert some strings representing dates
> {code}
> insert into t select '2015-12-09' from default.dual limit 1;
> insert into t select '2015-12-10' from default.dual limit 1;
> insert into t select '2015-12-11' from default.dual limit 1;
> {code}
> h3. Example 1
> This query correctly returns 3:
> {code}
> set hive.optimize.index.filter=true;
> set hive.optimize.ppd.storage=true;
> select count(*) from t where c >= '2015-12-09' and c <= '2015-12-11';
> +--+--+
> | _c0 |
> +--+--+
> | 3|
> +--+--+
> {code}
> This query incorrectly returns 1:
> {code}
> set hive.optimize.index.filter=true;
> set hive.optimize.ppd.storage=true;
> select count(*) from t where c between '2015-12-09' and '2015-12-11';
> +--+--+
> | _c0 |
> +--+--+
> | 1|
> +--+--+
> {code}
> Disabling hive.optimize.findex.filter resolves the problem. This query now
> correctly returns 3:
> {code}
> set hive.optimize.index.filter=false;
> set hive.optimize.ppd.storage=true;
> select count(*) from t where c between '2015-12-09' and '2015-12-11';
> +--+--+
> | _c0 |
> +--+--+
> | 3|
> +--+--+
> {code}
> Disabling hive.optimize.ppd.storage resolves the problem. This query now
> correctly returns 3:
> {code}
> set hive.optimize.index.filter=true;
> set hive.optimize.ppd.storage=false;
> select count(*) from t where c between '2015-12-09' and '2015-12-11';
> +--+--+
> | _c0 |
> +--+--+
> | 3|
> +--+--+
> {code}
> h3. Example 2
> This query correctly returns 1:
> {code}
> set hive.optimize.index.filter=true;
> set hive.optimize.ppd.storage=true;
> select count(*) from t where c >= '2015-12-10' and c <= '2015-12-10';
> +--+--+
> | _c0 |
> +--+--+
> | 1|
> +--+--+
> {code}
> This query incorrectly returns 0:
> {code}
> set hive.optimize.index.filter=true;
> set hive.optimize.ppd.storage=true;
> select count(*) from t where c between '2015-12-10' and '2015-12-10';
> +--+--+
> | _c0 |
> +--+--+
> | 0|
> +--+--+
> {code}
> Disabling hive.optimize.findex.filter resolves the problem. This query now
> correctly returns 1:
> {code}
> set hive.optimize.index.filter=false;
> set hive.optimize.ppd.storage=true;
> select count(*) from t where c >= '2015-12-10' and c <= '2015-12-10';
> +--+--+
> | _c0 |
> +--+--+
> | 1|
> +--+--+
> {code}
> Disabling hive.optimize.ppd.storage resolves the problem. This query now
> correctly returns 1:
> {code}
> set hive.optimize.index.filter=true;
> set hive.optimize.ppd.storage=false;
> select count(*) from t where c >= '2015-12-10' and c <= '2015-12-10';
> +--+--+
> | _c0 |
> +--+--+
> | 1|
> +--+--+
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-13111) Fix timestamp / interval_day_time wrong results with HIVE-9862
[ https://issues.apache.org/jira/browse/HIVE-13111?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt McCline updated HIVE-13111: Attachment: HIVE-13111.03.patch > Fix timestamp / interval_day_time wrong results with HIVE-9862 > --- > > Key: HIVE-13111 > URL: https://issues.apache.org/jira/browse/HIVE-13111 > Project: Hive > Issue Type: Bug >Reporter: Matt McCline >Assignee: Matt McCline >Priority: Critical > Attachments: HIVE-13111.01.patch, HIVE-13111.02.patch, > HIVE-13111.03.patch > > > Fix timestamp / interval_day_time issues discovered when testing the > Vectorized Text patch. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-13111) Fix timestamp / interval_day_time wrong results with HIVE-9862
[ https://issues.apache.org/jira/browse/HIVE-13111?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt McCline updated HIVE-13111: Status: Patch Available (was: In Progress) > Fix timestamp / interval_day_time wrong results with HIVE-9862 > --- > > Key: HIVE-13111 > URL: https://issues.apache.org/jira/browse/HIVE-13111 > Project: Hive > Issue Type: Bug >Reporter: Matt McCline >Assignee: Matt McCline >Priority: Critical > Attachments: HIVE-13111.01.patch, HIVE-13111.02.patch, > HIVE-13111.03.patch > > > Fix timestamp / interval_day_time issues discovered when testing the > Vectorized Text patch. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-13111) Fix timestamp / interval_day_time wrong results with HIVE-9862
[ https://issues.apache.org/jira/browse/HIVE-13111?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt McCline updated HIVE-13111: Attachment: (was: HIVE-13111.03.patch) > Fix timestamp / interval_day_time wrong results with HIVE-9862 > --- > > Key: HIVE-13111 > URL: https://issues.apache.org/jira/browse/HIVE-13111 > Project: Hive > Issue Type: Bug >Reporter: Matt McCline >Assignee: Matt McCline >Priority: Critical > Attachments: HIVE-13111.01.patch, HIVE-13111.02.patch, > HIVE-13111.03.patch > > > Fix timestamp / interval_day_time issues discovered when testing the > Vectorized Text patch. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-11484) Fix ObjectInspector for Char and VarChar
[ https://issues.apache.org/jira/browse/HIVE-11484?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Deepak Barr updated HIVE-11484: --- Status: Open (was: Patch Available) > Fix ObjectInspector for Char and VarChar > > > Key: HIVE-11484 > URL: https://issues.apache.org/jira/browse/HIVE-11484 > Project: Hive > Issue Type: Bug > Components: Serializers/Deserializers >Reporter: Amareshwari Sriramadasu >Assignee: Deepak Barr > Attachments: HIVE-11484.master.patch > > > The creation of HiveChar and Varchar is not happening through ObjectInspector. > Here is fix we pushed internally : > https://github.com/InMobi/hive/commit/fe95c7850e7130448209141155f28b25d3504216 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
