[jira] [Assigned] (HIVE-12360) Bad seek in uncompressed ORC with predicate pushdown
[
https://issues.apache.org/jira/browse/HIVE-12360?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Teddy Choi reassigned HIVE-12360:
-
Assignee: Teddy Choi
> Bad seek in uncompressed ORC with predicate pushdown
>
>
> Key: HIVE-12360
> URL: https://issues.apache.org/jira/browse/HIVE-12360
> Project: Hive
> Issue Type: Bug
> Components: File Formats, Hive
>Affects Versions: 1.2.1
> Environment: Oracle Linux 6.4, HDP 2.3.2.0-2950
>Reporter: Gabriel C Balan
>Assignee: Teddy Choi
> Attachments: numtab_100k.csv.gz, orc_test.hive
>
>
> Reading from an ORC file bombs in HDP-2.3.2 when pushing down predicate:
> {noformat:title=Error message in CLI}
> Failed with exception java.io.IOException:java.lang.IllegalArgumentException:
> Seek in index to 4613 is outside of the data
> {noformat}
> {noformat:title=Stack trace in log4j file}
> 2015-11-06 09:48:11,873 ERROR [main]: CliDriver
> (SessionState.java:printError(960)) - Failed with exception
> java.io.IOException:java.lang.IllegalArgumentException: Seek in index to 4613
> is outside of the data
> java.io.IOException: java.lang.IllegalArgumentException: Seek in index to
> 4613 is outside of the data
> at
> org.apache.hadoop.hive.ql.exec.FetchOperator.getNextRow(FetchOperator.java:508)
> at
> org.apache.hadoop.hive.ql.exec.FetchOperator.pushRow(FetchOperator.java:415)
> at org.apache.hadoop.hive.ql.exec.FetchTask.fetch(FetchTask.java:140)
> at org.apache.hadoop.hive.ql.Driver.getResults(Driver.java:1672)
> at
> org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:233)
> at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:165)
> at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:376)
> at
> org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:736)
> at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:681)
> at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:621)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:601)
> at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:136)
> Caused by: java.lang.IllegalArgumentException: Seek in index to 4613 is
> outside of the data
> at
> org.apache.hadoop.hive.ql.io.orc.InStream$UncompressedStream.seek(InStream.java:139)
> at
> org.apache.hadoop.hive.ql.io.orc.InStream$UncompressedStream.read(InStream.java:87)
> at java.io.InputStream.read(InputStream.java:102)
> at
> com.google.protobuf.CodedInputStream.refillBuffer(CodedInputStream.java:737)
> at
> com.google.protobuf.CodedInputStream.isAtEnd(CodedInputStream.java:701)
> at
> com.google.protobuf.CodedInputStream.readTag(CodedInputStream.java:99)
> at
> org.apache.hadoop.hive.ql.io.orc.OrcProto$RowIndex.(OrcProto.java:7429)
> at
> org.apache.hadoop.hive.ql.io.orc.OrcProto$RowIndex.(OrcProto.java:7393)
> at
> org.apache.hadoop.hive.ql.io.orc.OrcProto$RowIndex$1.parsePartialFrom(OrcProto.java:7482)
> at
> org.apache.hadoop.hive.ql.io.orc.OrcProto$RowIndex$1.parsePartialFrom(OrcProto.java:7477)
> at
> com.google.protobuf.AbstractParser.parsePartialFrom(AbstractParser.java:200)
> at com.google.protobuf.AbstractParser.parseFrom(AbstractParser.java:217)
> at com.google.protobuf.AbstractParser.parseFrom(AbstractParser.java:223)
> at com.google.protobuf.AbstractParser.parseFrom(AbstractParser.java:49)
> at
> org.apache.hadoop.hive.ql.io.orc.OrcProto$RowIndex.parseFrom(OrcProto.java:7593)
> at
> org.apache.hadoop.hive.ql.io.orc.MetadataReader.readRowIndex(MetadataReader.java:88)
> at
> org.apache.hadoop.hive.ql.io.orc.RecordReaderImpl.readRowIndex(RecordReaderImpl.java:1166)
> at
> org.apache.hadoop.hive.ql.io.orc.RecordReaderImpl.readRowIndex(RecordReaderImpl.java:1151)
> at
> org.apache.hadoop.hive.ql.io.orc.RecordReaderImpl.pickRowGroups(RecordReaderImpl.java:750)
> at
> org.apache.hadoop.hive.ql.io.orc.RecordReaderImpl.readStripe(RecordReaderImpl.java:777)
> at
> org.apache.hadoop.hive.ql.io.orc.RecordReaderImpl.advanceStripe(RecordReaderImpl.java:986)
> at
> org.apache.hadoop.hive.ql.io.orc.RecordReaderImpl.advanceToNextRow(RecordReaderImpl.java:1019)
> at
> org.apache.hadoop.hive.ql.io.orc.RecordReaderImpl.(RecordReaderImpl.java:205)
> at
> org.apache.hadoop.hive.ql.io.orc.ReaderImpl.rowsOptions(ReaderImpl.java:598)
> at
>
[jira] [Updated] (HIVE-17954) Implement pool, user, group and trigger to pool management API's.
[ https://issues.apache.org/jira/browse/HIVE-17954?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Harish Jaiprakash updated HIVE-17954: - Attachment: HIVE-17954.05.patch Thanks [~prasanth_j]. Fixed the sql statement for mysql. The others have the statements outside the create table. But not sure if they have other syntax errors. > Implement pool, user, group and trigger to pool management API's. > - > > Key: HIVE-17954 > URL: https://issues.apache.org/jira/browse/HIVE-17954 > Project: Hive > Issue Type: Sub-task >Reporter: Harish Jaiprakash >Assignee: Harish Jaiprakash > Attachments: HIVE-17954.01.patch, HIVE-17954.02.patch, > HIVE-17954.03.patch, HIVE-17954.04.patch, HIVE-17954.05.patch > > > Implement the following commands: > -- Pool management. > CREATE POOL `resource_plan`.`pool_path` WITH > ALLOC_FRACTION `fraction` > QUERY_PARALLELISM `parallelism` > SCHEDULING_POLICY `policy`; > ALTER POOL `resource_plan`.`pool_path` SET > PATH = `new_path`, > ALLOC_FRACTION = `fraction`, > QUERY_PARALLELISM = `parallelism`, > SCHEDULING_POLICY = `policy`; > DROP POOL `resource_plan`.`pool_path`; > -- Trigger to pool mappings. > ALTER RESOURCE PLAN `resource_plan` > ADD TRIGGER `trigger_name` TO `pool_path`; > ALTER RESOURCE PLAN `resource_plan` > DROP TRIGGER `trigger_name` TO `pool_path`; > -- User/Group to pool mappings. > CREATE USER|GROUP MAPPING `resource_plan`.`group_or_user_name` > TO `pool_path` WITH ORDERING `order_no`; > DROP USER|GROUP MAPPING `resource_plan`.`group_or_user_name`; -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HIVE-16756) Vectorization: LongColModuloLongColumn throws "java.lang.ArithmeticException: / by zero"
[
https://issues.apache.org/jira/browse/HIVE-16756?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16253038#comment-16253038
]
Hive QA commented on HIVE-16756:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12897622/HIVE-16756.01.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 9 failed/errored test(s), 11386 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[dbtxnmgr_showlocks]
(batchId=77)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[unionDistinct_1]
(batchId=146)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[insert_values_orig_table_use_metadata]
(batchId=162)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[llap_acid_fast]
(batchId=157)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[ppd_union_view]
(batchId=154)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[sysdb]
(batchId=156)
org.apache.hadoop.hive.cli.control.TestDanglingQOuts.checkDanglingQOut
(batchId=206)
org.apache.hadoop.hive.ql.parse.TestReplicationScenarios.testConstraints
(batchId=223)
org.apache.hive.beeline.TestBeeLineWithArgs.testQueryProgressParallel
(batchId=226)
{noformat}
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/7818/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/7818/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-7818/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 9 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12897622 - PreCommit-HIVE-Build
> Vectorization: LongColModuloLongColumn throws "java.lang.ArithmeticException:
> / by zero"
>
>
> Key: HIVE-16756
> URL: https://issues.apache.org/jira/browse/HIVE-16756
> Project: Hive
> Issue Type: Bug
> Components: Hive
>Affects Versions: 2.3.0
>Reporter: Matt McCline
>Assignee: Vihang Karajgaonkar
>Priority: Critical
> Attachments: HIVE-16756.01.patch
>
>
> vectorization_div0.q needs to test the long data type testing.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HIVE-17954) Implement pool, user, group and trigger to pool management API's.
[ https://issues.apache.org/jira/browse/HIVE-17954?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16253019#comment-16253019 ] Prasanth Jayachandran commented on HIVE-17954: -- mysql scripts are broken because of ',' in last column of WM_POOL table. hive-schema-3.0.0.mysql.sql and 046-HIVE-17566.mysql.sql should be fixed. Maybe other DBs have same issue but haven't tested. > Implement pool, user, group and trigger to pool management API's. > - > > Key: HIVE-17954 > URL: https://issues.apache.org/jira/browse/HIVE-17954 > Project: Hive > Issue Type: Sub-task >Reporter: Harish Jaiprakash >Assignee: Harish Jaiprakash > Attachments: HIVE-17954.01.patch, HIVE-17954.02.patch, > HIVE-17954.03.patch, HIVE-17954.04.patch > > > Implement the following commands: > -- Pool management. > CREATE POOL `resource_plan`.`pool_path` WITH > ALLOC_FRACTION `fraction` > QUERY_PARALLELISM `parallelism` > SCHEDULING_POLICY `policy`; > ALTER POOL `resource_plan`.`pool_path` SET > PATH = `new_path`, > ALLOC_FRACTION = `fraction`, > QUERY_PARALLELISM = `parallelism`, > SCHEDULING_POLICY = `policy`; > DROP POOL `resource_plan`.`pool_path`; > -- Trigger to pool mappings. > ALTER RESOURCE PLAN `resource_plan` > ADD TRIGGER `trigger_name` TO `pool_path`; > ALTER RESOURCE PLAN `resource_plan` > DROP TRIGGER `trigger_name` TO `pool_path`; > -- User/Group to pool mappings. > CREATE USER|GROUP MAPPING `resource_plan`.`group_or_user_name` > TO `pool_path` WITH ORDERING `order_no`; > DROP USER|GROUP MAPPING `resource_plan`.`group_or_user_name`; -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Resolved] (HIVE-18067) Remove extraneous golden files
[ https://issues.apache.org/jira/browse/HIVE-18067?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashutosh Chauhan resolved HIVE-18067. - Resolution: Fixed Fix Version/s: 3.0.0 Pushed to master. > Remove extraneous golden files > -- > > Key: HIVE-18067 > URL: https://issues.apache.org/jira/browse/HIVE-18067 > Project: Hive > Issue Type: Bug > Components: Tests >Affects Versions: 3.0.0 >Reporter: Ashutosh Chauhan >Assignee: Ashutosh Chauhan > Fix For: 3.0.0 > > Attachments: HIVE-18067.patch > > > TestDanglingQouts makes sure that there are no unneeded files in repo. This > is currently failing. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HIVE-18067) Remove extraneous golden files
[ https://issues.apache.org/jira/browse/HIVE-18067?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashutosh Chauhan updated HIVE-18067: Attachment: HIVE-18067.patch > Remove extraneous golden files > -- > > Key: HIVE-18067 > URL: https://issues.apache.org/jira/browse/HIVE-18067 > Project: Hive > Issue Type: Bug > Components: Tests >Affects Versions: 3.0.0 >Reporter: Ashutosh Chauhan >Assignee: Ashutosh Chauhan > Attachments: HIVE-18067.patch > > > TestDanglingQouts makes sure that there are no unneeded files in repo. This > is currently failing. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Assigned] (HIVE-18067) Remove extraneous golden files
[ https://issues.apache.org/jira/browse/HIVE-18067?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashutosh Chauhan reassigned HIVE-18067: --- > Remove extraneous golden files > -- > > Key: HIVE-18067 > URL: https://issues.apache.org/jira/browse/HIVE-18067 > Project: Hive > Issue Type: Bug > Components: Tests >Affects Versions: 3.0.0 >Reporter: Ashutosh Chauhan >Assignee: Ashutosh Chauhan > > TestDanglingQouts makes sure that there are no unneeded files in repo. This > is currently failing. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HIVE-15018) ALTER rewriting flag in materialized view
[
https://issues.apache.org/jira/browse/HIVE-15018?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16252973#comment-16252973
]
Hive QA commented on HIVE-15018:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12897617/HIVE-15018.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 13 failed/errored test(s), 11384 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestBeeLineDriver.testCliDriver[materialized_view_create_rewrite]
(batchId=243)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[auto_sortmerge_join_2]
(batchId=47)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[dbtxnmgr_showlocks]
(batchId=77)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[unionDistinct_1]
(batchId=146)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[insert_values_orig_table_use_metadata]
(batchId=162)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[ppd_union_view]
(batchId=154)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[sysdb]
(batchId=156)
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver[spark_dynamic_partition_pruning]
(batchId=173)
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver[spark_use_op_stats]
(batchId=173)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[subquery_multi]
(batchId=111)
org.apache.hadoop.hive.cli.control.TestDanglingQOuts.checkDanglingQOut
(batchId=206)
org.apache.hadoop.hive.ql.parse.TestReplicationScenarios.testConstraints
(batchId=223)
org.apache.hadoop.hive.ql.security.authorization.plugin.TestHiveOperationType.checkHiveOperationTypeMatch
(batchId=286)
{noformat}
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/7817/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/7817/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-7817/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 13 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12897617 - PreCommit-HIVE-Build
> ALTER rewriting flag in materialized view
> --
>
> Key: HIVE-15018
> URL: https://issues.apache.org/jira/browse/HIVE-15018
> Project: Hive
> Issue Type: Sub-task
> Components: Materialized views
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
> Attachments: HIVE-15018.patch
>
>
> We should extend the ALTER statement in case we want to change the rewriting
> behavior of the materialized view after we have created it.
> {code:sql}
> ALTER MATERIALIZED VIEW [db_name.]materialized_view_name DISABLE REWRITE;
> {code}
> {code:sql}
> ALTER MATERIALIZED VIEW [db_name.]materialized_view_name ENABLE REWRITE;
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HIVE-17361) Support LOAD DATA for transactional tables
[ https://issues.apache.org/jira/browse/HIVE-17361?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eugene Koifman updated HIVE-17361: -- Attachment: HIVE-17361.11.patch > Support LOAD DATA for transactional tables > -- > > Key: HIVE-17361 > URL: https://issues.apache.org/jira/browse/HIVE-17361 > Project: Hive > Issue Type: New Feature > Components: Transactions >Reporter: Wei Zheng >Assignee: Eugene Koifman >Priority: Critical > Attachments: HIVE-17361.07.patch, HIVE-17361.08.patch, > HIVE-17361.09.patch, HIVE-17361.1.patch, HIVE-17361.10.patch, > HIVE-17361.11.patch, HIVE-17361.2.patch, HIVE-17361.3.patch, > HIVE-17361.4.patch > > > LOAD DATA was not supported since ACID was introduced. Need to fill this gap > between ACID table and regular hive table. > Current Documentation is under [DML > Operations|https://cwiki.apache.org/confluence/display/Hive/GettingStarted#GettingStarted-DMLOperations] > and [Loading files into > tables|https://cwiki.apache.org/confluence/display/Hive/LanguageManual+DML#LanguageManualDML-Loadingfilesintotables]: > \\ > * Load Data performs very limited validations of the data, in particular it > uses the input file name which may not be in 0_0 which can break some > read logic. (Certainly will for Acid). > * It does not check the schema of the file. This may be a non issue for Acid > which requires ORC which is self describing so Schema Evolution may handle > this seamlessly. (Assuming Schema is not too different). > * It does check that _InputFormat_S are compatible. > * Bucketed (and thus sorted) tables don't support Load Data (but only if > hive.strict.checks.bucketing=true (default)). Will keep this restriction for > Acid. > * Load Data supports OVERWRITE clause > * What happens to file permissions/ownership: rename vs copy differences > \\ > The implementation will follow the same idea as in HIVE-14988 and use a > base_N/ dir for OVERWRITE clause. > \\ > How is minor compaction going to handle delta/base with original files? -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HIVE-17934) Merging Statistics are promoted to COMPLETE (most of the time)
[ https://issues.apache.org/jira/browse/HIVE-17934?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashutosh Chauhan updated HIVE-17934: Resolution: Fixed Fix Version/s: 3.0.0 Status: Resolved (was: Patch Available) Pushed to master. Thanks, Zoltan! > Merging Statistics are promoted to COMPLETE (most of the time) > -- > > Key: HIVE-17934 > URL: https://issues.apache.org/jira/browse/HIVE-17934 > Project: Hive > Issue Type: Sub-task > Components: Statistics >Reporter: Zoltan Haindrich >Assignee: Zoltan Haindrich > Fix For: 3.0.0 > > Attachments: HIVE-17934.01.patch, HIVE-17934.02.patch, > HIVE-17934.03.patch, HIVE-17934.04.patch, HIVE-17934.05.patch, > HIVE-17934.06.patch, HIVE-17934.06.patch, HIVE-17934.06wip01.patch > > > in case multiple partition statistics are merged the STATS state is computed > based on the datasize and rowcount; > the merge may hide away non-existent stats in case there are other partition > or operators which do contribute to the datasize and the rowcount. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HIVE-17954) Implement pool, user, group and trigger to pool management API's.
[ https://issues.apache.org/jira/browse/HIVE-17954?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Harish Jaiprakash updated HIVE-17954: - Attachment: HIVE-17954.04.patch Rebased again. > Implement pool, user, group and trigger to pool management API's. > - > > Key: HIVE-17954 > URL: https://issues.apache.org/jira/browse/HIVE-17954 > Project: Hive > Issue Type: Sub-task >Reporter: Harish Jaiprakash >Assignee: Harish Jaiprakash > Attachments: HIVE-17954.01.patch, HIVE-17954.02.patch, > HIVE-17954.03.patch, HIVE-17954.04.patch > > > Implement the following commands: > -- Pool management. > CREATE POOL `resource_plan`.`pool_path` WITH > ALLOC_FRACTION `fraction` > QUERY_PARALLELISM `parallelism` > SCHEDULING_POLICY `policy`; > ALTER POOL `resource_plan`.`pool_path` SET > PATH = `new_path`, > ALLOC_FRACTION = `fraction`, > QUERY_PARALLELISM = `parallelism`, > SCHEDULING_POLICY = `policy`; > DROP POOL `resource_plan`.`pool_path`; > -- Trigger to pool mappings. > ALTER RESOURCE PLAN `resource_plan` > ADD TRIGGER `trigger_name` TO `pool_path`; > ALTER RESOURCE PLAN `resource_plan` > DROP TRIGGER `trigger_name` TO `pool_path`; > -- User/Group to pool mappings. > CREATE USER|GROUP MAPPING `resource_plan`.`group_or_user_name` > TO `pool_path` WITH ORDERING `order_no`; > DROP USER|GROUP MAPPING `resource_plan`.`group_or_user_name`; -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HIVE-17651) TableScanOperator might miss vectorization on flag
[
https://issues.apache.org/jira/browse/HIVE-17651?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16252933#comment-16252933
]
Hive QA commented on HIVE-17651:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12897606/HIVE-17651.03.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 10 failed/errored test(s), 11384 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[auto_sortmerge_join_2]
(batchId=47)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[dbtxnmgr_showlocks]
(batchId=77)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[unionDistinct_1]
(batchId=146)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[insert_values_orig_table_use_metadata]
(batchId=162)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[ppd_union_view]
(batchId=154)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[sysdb]
(batchId=156)
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[explainanalyze_2]
(batchId=102)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[subquery_multi]
(batchId=111)
org.apache.hadoop.hive.cli.control.TestDanglingQOuts.checkDanglingQOut
(batchId=206)
org.apache.hadoop.hive.ql.parse.TestReplicationScenarios.testConstraints
(batchId=223)
{noformat}
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/7816/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/7816/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-7816/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 10 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12897606 - PreCommit-HIVE-Build
> TableScanOperator might miss vectorization on flag
> --
>
> Key: HIVE-17651
> URL: https://issues.apache.org/jira/browse/HIVE-17651
> Project: Hive
> Issue Type: Sub-task
>Reporter: Zoltan Haindrich
>Assignee: Zoltan Haindrich
> Attachments: HIVE-17651.01ex.patch, HIVE-17651.02rc.patch,
> HIVE-17651.02rc.patch, HIVE-17651.03.patch
>
>
> https://github.com/apache/hive/blob/3bfcfdde0c0be2aab1afdf5b1bc71fdcc9e77360/ql/src/java/org/apache/hadoop/hive/ql/exec/TableScanOperator.java#L259-L273
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HIVE-18026) Hive webhcat principal configuration optimization
[ https://issues.apache.org/jira/browse/HIVE-18026?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ZhangBing Lin updated HIVE-18026: - Status: Patch Available (was: Open) > Hive webhcat principal configuration optimization > - > > Key: HIVE-18026 > URL: https://issues.apache.org/jira/browse/HIVE-18026 > Project: Hive > Issue Type: Bug > Components: WebHCat >Affects Versions: 3.0.0 >Reporter: ZhangBing Lin >Assignee: ZhangBing Lin > Attachments: HIVE-18026.1.patch > > > Hive webhcat principal configuration optimization，when you configure: > > templeton.kerberos.principal > HTTP/_HOST@ > > The '_HOST' should be replaced by specific host name. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HIVE-18026) Hive webhcat principal configuration optimization
[ https://issues.apache.org/jira/browse/HIVE-18026?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ZhangBing Lin updated HIVE-18026: - Attachment: (was: HIVE-18026.1.patch) > Hive webhcat principal configuration optimization > - > > Key: HIVE-18026 > URL: https://issues.apache.org/jira/browse/HIVE-18026 > Project: Hive > Issue Type: Bug > Components: WebHCat >Affects Versions: 3.0.0 >Reporter: ZhangBing Lin >Assignee: ZhangBing Lin > Attachments: HIVE-18026.1.patch > > > Hive webhcat principal configuration optimization，when you configure: > > templeton.kerberos.principal > HTTP/_HOST@ > > The '_HOST' should be replaced by specific host name. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HIVE-18026) Hive webhcat principal configuration optimization
[ https://issues.apache.org/jira/browse/HIVE-18026?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ZhangBing Lin updated HIVE-18026: - Attachment: HIVE-18026.1.patch > Hive webhcat principal configuration optimization > - > > Key: HIVE-18026 > URL: https://issues.apache.org/jira/browse/HIVE-18026 > Project: Hive > Issue Type: Bug > Components: WebHCat >Affects Versions: 3.0.0 >Reporter: ZhangBing Lin >Assignee: ZhangBing Lin > Attachments: HIVE-18026.1.patch > > > Hive webhcat principal configuration optimization，when you configure: > > templeton.kerberos.principal > HTTP/_HOST@ > > The '_HOST' should be replaced by specific host name. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HIVE-18026) Hive webhcat principal configuration optimization
[ https://issues.apache.org/jira/browse/HIVE-18026?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ZhangBing Lin updated HIVE-18026: - Status: Open (was: Patch Available) > Hive webhcat principal configuration optimization > - > > Key: HIVE-18026 > URL: https://issues.apache.org/jira/browse/HIVE-18026 > Project: Hive > Issue Type: Bug > Components: WebHCat >Affects Versions: 3.0.0 >Reporter: ZhangBing Lin >Assignee: ZhangBing Lin > Attachments: HIVE-18026.1.patch > > > Hive webhcat principal configuration optimization，when you configure: > > templeton.kerberos.principal > HTTP/_HOST@ > > The '_HOST' should be replaced by specific host name. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HIVE-17934) Merging Statistics are promoted to COMPLETE (most of the time)
[
https://issues.apache.org/jira/browse/HIVE-17934?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16252883#comment-16252883
]
Hive QA commented on HIVE-17934:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12897626/HIVE-17934.06.patch
{color:green}SUCCESS:{color} +1 due to 2 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 14 failed/errored test(s), 11386 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestBlobstoreCliDriver.testCliDriver[insert_overwrite_dynamic_partitions_move_only]
(batchId=246)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[dbtxnmgr_showlocks]
(batchId=77)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[vectorization_parquet_projection]
(batchId=42)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[unionDistinct_1]
(batchId=146)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[constprog_dpp]
(batchId=151)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[insert_values_orig_table_use_metadata]
(batchId=162)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[ppd_union_view]
(batchId=154)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[semijoin_hint]
(batchId=154)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[sysdb]
(batchId=156)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[union_view]
(batchId=109)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[vectorization_parquet_projection]
(batchId=122)
org.apache.hadoop.hive.cli.control.TestDanglingQOuts.checkDanglingQOut
(batchId=206)
org.apache.hadoop.hive.ql.parse.TestReplicationScenarios.testConstraints
(batchId=223)
org.apache.hive.beeline.TestBeeLineWithArgs.testQueryProgressParallel
(batchId=226)
{noformat}
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/7815/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/7815/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-7815/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 14 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12897626 - PreCommit-HIVE-Build
> Merging Statistics are promoted to COMPLETE (most of the time)
> --
>
> Key: HIVE-17934
> URL: https://issues.apache.org/jira/browse/HIVE-17934
> Project: Hive
> Issue Type: Sub-task
> Components: Statistics
>Reporter: Zoltan Haindrich
>Assignee: Zoltan Haindrich
> Attachments: HIVE-17934.01.patch, HIVE-17934.02.patch,
> HIVE-17934.03.patch, HIVE-17934.04.patch, HIVE-17934.05.patch,
> HIVE-17934.06.patch, HIVE-17934.06.patch, HIVE-17934.06wip01.patch
>
>
> in case multiple partition statistics are merged the STATS state is computed
> based on the datasize and rowcount;
> the merge may hide away non-existent stats in case there are other partition
> or operators which do contribute to the datasize and the rowcount.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HIVE-17898) Explain plan output enhancement
[ https://issues.apache.org/jira/browse/HIVE-17898?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vineet Garg updated HIVE-17898: --- Attachment: HIVE-17898.3.patch > Explain plan output enhancement > --- > > Key: HIVE-17898 > URL: https://issues.apache.org/jira/browse/HIVE-17898 > Project: Hive > Issue Type: Improvement > Components: Query Planning >Reporter: Vineet Garg >Assignee: Vineet Garg > Attachments: HIVE-17898.1.patch, HIVE-17898.2.patch, > HIVE-17898.3.patch > > > We would like to enhance the explain plan output to display additional > information e.g.: > TableScan operator should have following additional info > * Actual table name (currently only alias name is displayed) > * Database name > * Column names being scanned -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HIVE-17898) Explain plan output enhancement
[ https://issues.apache.org/jira/browse/HIVE-17898?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vineet Garg updated HIVE-17898: --- Status: Patch Available (was: Open) > Explain plan output enhancement > --- > > Key: HIVE-17898 > URL: https://issues.apache.org/jira/browse/HIVE-17898 > Project: Hive > Issue Type: Improvement > Components: Query Planning >Reporter: Vineet Garg >Assignee: Vineet Garg > Attachments: HIVE-17898.1.patch, HIVE-17898.2.patch, > HIVE-17898.3.patch > > > We would like to enhance the explain plan output to display additional > information e.g.: > TableScan operator should have following additional info > * Actual table name (currently only alias name is displayed) > * Database name > * Column names being scanned -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HIVE-17964) HoS: some spark configs doesn't require re-creating a session
[
https://issues.apache.org/jira/browse/HIVE-17964?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16252858#comment-16252858
]
Xuefu Zhang commented on HIVE-17964:
That's good then.
> HoS: some spark configs doesn't require re-creating a session
> -
>
> Key: HIVE-17964
> URL: https://issues.apache.org/jira/browse/HIVE-17964
> Project: Hive
> Issue Type: Improvement
> Components: Spark
>Reporter: Rui Li
>Assignee: Rui Li
>Priority: Minor
> Attachments: HIVE-17964.1.patch, HIVE-17964.2.patch
>
>
> I guess the {{hive.spark.}} configs were initially intended for the RSC.
> Therefore when they're changed, we'll re-create the session for them to take
> effect. There're some configs not related to RSC that also start with
> {{hive.spark.}}. We'd better rename them so that we don't unnecessarily
> re-create sessions, which is usually time consuming.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HIVE-17964) HoS: some spark configs doesn't require re-creating a session
[
https://issues.apache.org/jira/browse/HIVE-17964?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16252855#comment-16252855
]
Rui Li commented on HIVE-17964:
---
[~xuefuz], changing configs that start with {{spark.}} will still result in a
new session. This patch only changes how we handle {{hive.spark.}} configs.
> HoS: some spark configs doesn't require re-creating a session
> -
>
> Key: HIVE-17964
> URL: https://issues.apache.org/jira/browse/HIVE-17964
> Project: Hive
> Issue Type: Improvement
> Components: Spark
>Reporter: Rui Li
>Assignee: Rui Li
>Priority: Minor
> Attachments: HIVE-17964.1.patch, HIVE-17964.2.patch
>
>
> I guess the {{hive.spark.}} configs were initially intended for the RSC.
> Therefore when they're changed, we'll re-create the session for them to take
> effect. There're some configs not related to RSC that also start with
> {{hive.spark.}}. We'd better rename them so that we don't unnecessarily
> re-create sessions, which is usually time consuming.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HIVE-14495) Add SHOW MATERIALIZED VIEWS statement
[
https://issues.apache.org/jira/browse/HIVE-14495?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16252849#comment-16252849
]
Hive QA commented on HIVE-14495:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12897600/HIVE-14495.01.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 12 failed/errored test(s), 11385 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[dbtxnmgr_showlocks]
(batchId=77)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[unionDistinct_1]
(batchId=146)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[insert_values_orig_table_use_metadata]
(batchId=162)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[ppd_union_view]
(batchId=154)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[sysdb]
(batchId=156)
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[explainanalyze_2]
(batchId=102)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[subquery_multi]
(batchId=111)
org.apache.hadoop.hive.cli.control.TestDanglingQOuts.checkDanglingQOut
(batchId=206)
org.apache.hadoop.hive.ql.parse.TestReplicationScenarios.testConstraints
(batchId=223)
org.apache.hadoop.hive.ql.parse.authorization.plugin.sqlstd.TestOperation2Privilege.checkHiveOperationTypeMatch
(batchId=271)
org.apache.hive.hcatalog.pig.TestTextFileHCatStorer.testWriteDecimalX
(batchId=187)
org.apache.hive.hcatalog.pig.TestTextFileHCatStorer.testWriteTimestamp
(batchId=187)
{noformat}
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/7814/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/7814/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-7814/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 12 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12897600 - PreCommit-HIVE-Build
> Add SHOW MATERIALIZED VIEWS statement
> -
>
> Key: HIVE-14495
> URL: https://issues.apache.org/jira/browse/HIVE-14495
> Project: Hive
> Issue Type: Sub-task
> Components: Materialized views
>Affects Versions: 2.2.0
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
> Attachments: HIVE-14495.01.patch, HIVE-14495.patch
>
>
> In the spirit of {{SHOW TABLES}}, we should support the following statement:
> {code:sql}
> SHOW MATERIALIZED VIEWS [IN database_name] ['identifier_with_wildcards'];
> {code}
> In contrast to {{SHOW TABLES}}, this command would only list the materialized
> views.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HIVE-9020) When dropping external tables, Hive should not verify whether user has access to the data.
[ https://issues.apache.org/jira/browse/HIVE-9020?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Carl Steinbach updated HIVE-9020: - Status: Open (was: Patch Available) Changes have been requested so I'm setting the status to open. > When dropping external tables, Hive should not verify whether user has access > to the data. > --- > > Key: HIVE-9020 > URL: https://issues.apache.org/jira/browse/HIVE-9020 > Project: Hive > Issue Type: Bug >Affects Versions: 0.13.1 >Reporter: Anant Nag > Attachments: dropExternal.patch > > > When dropping tables, hive verifies whether the user has access to the data > on hdfs. It fails, if user doesn't have access. It makes sense for internal > tables since the data has to be deleted when dropping internal tables but for > external tables, Hive should not check for data access. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HIVE-14495) Add SHOW MATERIALIZED VIEWS statement
[
https://issues.apache.org/jira/browse/HIVE-14495?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16252813#comment-16252813
]
Ashutosh Chauhan commented on HIVE-14495:
-
+1
> Add SHOW MATERIALIZED VIEWS statement
> -
>
> Key: HIVE-14495
> URL: https://issues.apache.org/jira/browse/HIVE-14495
> Project: Hive
> Issue Type: Sub-task
> Components: Materialized views
>Affects Versions: 2.2.0
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
> Attachments: HIVE-14495.01.patch, HIVE-14495.patch
>
>
> In the spirit of {{SHOW TABLES}}, we should support the following statement:
> {code:sql}
> SHOW MATERIALIZED VIEWS [IN database_name] ['identifier_with_wildcards'];
> {code}
> In contrast to {{SHOW TABLES}}, this command would only list the materialized
> views.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HIVE-17528) Add more q-tests for Hive-on-Spark with Parquet vectorized reader
[ https://issues.apache.org/jira/browse/HIVE-17528?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ferdinand Xu updated HIVE-17528: Attachment: HIVE-17528.4.patch Failed queries are not related. [~vihangk1] could you have a review on it? > Add more q-tests for Hive-on-Spark with Parquet vectorized reader > - > > Key: HIVE-17528 > URL: https://issues.apache.org/jira/browse/HIVE-17528 > Project: Hive > Issue Type: Sub-task >Reporter: Vihang Karajgaonkar >Assignee: Ferdinand Xu > Attachments: HIVE-17528.1.patch, HIVE-17528.2.patch, > HIVE-17528.3.patch, HIVE-17528.4.patch, HIVE-17528.patch > > > Most of the vectorization related q-tests operate on ORC tables using Tez. It > would be good to add more coverage on a different combination of engine and > file-format. We can model existing q-tests using parquet tables and run it > using TestSparkCliDriver -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HIVE-18057) remove PostExecute / PreExecute hook support
[ https://issues.apache.org/jira/browse/HIVE-18057?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashutosh Chauhan updated HIVE-18057: Resolution: Fixed Fix Version/s: 3.0.0 Status: Resolved (was: Patch Available) Pushed to master. Thanks, Zoltan! > remove PostExecute / PreExecute hook support > > > Key: HIVE-18057 > URL: https://issues.apache.org/jira/browse/HIVE-18057 > Project: Hive > Issue Type: Sub-task >Reporter: Zoltan Haindrich >Assignee: Zoltan Haindrich > Fix For: 3.0.0 > > Attachments: HIVE-18057.01.patch > > > * deprecated since 2010 > * they are needlessly complicate the dispatch logic > * the current dispatch logic just silently accepts pre/post hooks if they > don't implement neither interface -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HIVE-18054) Make Lineage work with concurrent queries on a Session
[ https://issues.apache.org/jira/browse/HIVE-18054?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Sherman updated HIVE-18054: -- Attachment: HIVE-18054.2.patch > Make Lineage work with concurrent queries on a Session > --- > > Key: HIVE-18054 > URL: https://issues.apache.org/jira/browse/HIVE-18054 > Project: Hive > Issue Type: Bug >Reporter: Andrew Sherman >Assignee: Andrew Sherman > Attachments: HIVE-18054.1.patch, HIVE-18054.2.patch > > > A Hive Session can contain multiple concurrent sql Operations. > Lineage is currently tracked in SessionState and is cleared when a query > completes. This results in Lineage for other running queries being lost. > To fix this, move LineageState from SessionState to QueryState. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HIVE-18046) Metastore: default IS_REWRITE_ENABLED=false instead of NULL
[
https://issues.apache.org/jira/browse/HIVE-18046?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16252783#comment-16252783
]
Hive QA commented on HIVE-18046:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12897595/HIVE-18046.01.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 10 failed/errored test(s), 10989 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[dbtxnmgr_showlocks]
(batchId=77)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[unionDistinct_1]
(batchId=146)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[insert_values_orig_table_use_metadata]
(batchId=162)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[ppd_union_view]
(batchId=154)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[sysdb]
(batchId=156)
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[explainanalyze_2]
(batchId=102)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.org.apache.hadoop.hive.cli.TestNegativeCliDriver
(batchId=91)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[subquery_multi]
(batchId=111)
org.apache.hadoop.hive.cli.control.TestDanglingQOuts.checkDanglingQOut
(batchId=206)
org.apache.hadoop.hive.ql.parse.TestReplicationScenarios.testConstraints
(batchId=223)
{noformat}
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/7813/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/7813/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-7813/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 10 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12897595 - PreCommit-HIVE-Build
> Metastore: default IS_REWRITE_ENABLED=false instead of NULL
> ---
>
> Key: HIVE-18046
> URL: https://issues.apache.org/jira/browse/HIVE-18046
> Project: Hive
> Issue Type: Bug
> Components: Materialized views, Metastore
>Affects Versions: 3.0.0
>Reporter: Gopal V
>Assignee: Jesus Camacho Rodriguez
>Priority: Minor
> Attachments: HIVE-18046.01.patch, HIVE-18046.patch
>
>
> The materialized view impl breaks old metastore sql write access, by
> complaining that the new table creation does not set this column up.
> {code}
> `IS_REWRITE_ENABLED` bit(1) NOT NULL,
> {code}
> {{NOT NULL DEFAULT 0}} would allow old metastore direct sql compatibility
> (not thrift).
> {code}
> 2017-11-09T07:11:58,331 ERROR [HiveServer2-Background-Pool: Thread-2354]
> metastore.RetryingHMSHandler: Retrying HMSHandler after 2000 ms (attempt 1 of
> 10) with error: javax.jdo.JDODataStoreException: Insert of object
> "org.apache.hadoop.hive.metastore.model.MTable@249dbf1" using statement
> "INSERT INTO `TBLS`
> (`TBL_ID`,`CREATE_TIME`,`DB_ID`,`LAST_ACCESS_TIME`,`OWNER`,`RETENTION`,`SD_ID`,`TBL_NAME`,`TBL_TYPE`,`VIEW_EXPANDED_TEXT`,`VIEW_ORIGINAL_TEXT`)
> VALUES (?,?,?,?,?,?,?,?,?,?,?)" failed : Field 'IS_REWRITE_ENABLED' doesn't
> have a default value
> at
> org.datanucleus.api.jdo.NucleusJDOHelper.getJDOExceptionForNucleusException(NucleusJDOHelper.java:543)
> at
> org.datanucleus.api.jdo.JDOPersistenceManager.jdoMakePersistent(JDOPersistenceManager.java:720)
> at
> org.datanucleus.api.jdo.JDOPersistenceManager.makePersistent(JDOPersistenceManager.java:740)
> at
> org.apache.hadoop.hive.metastore.ObjectStore.createTable(ObjectStore.java:1038)
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HIVE-17714) move custom SerDe schema considerations into metastore from QL
[ https://issues.apache.org/jira/browse/HIVE-17714?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16252765#comment-16252765 ] Vihang Karajgaonkar commented on HIVE-17714: bq. metastore would duplicate the schema from deserializer into the metastore columns (2.1 in my previous comment). So, in most cases (unless either the user or the serde messed with it), the schema returned would actually be the real schema. Sounds like before HIVE-11985 all the APIs would have returned consistent schema. Why did we change that in HIVE-11985? (traced it to [this comment | https://issues.apache.org/jira/browse/HIVE-11985?focusedCommentId=14949665=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14949665] by [~xuefuz] on that JIRA bq. "If we spend time on this, I'd rather solve the problem in the generic way, regardless the serde type and db type. The obvious inconsistency I see here is that we store for avro the schema if it's less than 2000 while storing a constant string for anything over that. If we determine that it's not necessary to store it for avro, don't store it at all. Or if we can solve the length problem for all serdes, then that's probably the the right way to go." Based on what I understand so far (please forgive me if I am repeating the obvious) the inconsistency in the returned schema is only in the cases of tables where the schema should be derived from deserializer because in HIVE-11985 we decided not to store the such schemas in metastore. And the reason why we don't store these schemas in metastore in the first places is due to the 4000 character limit. The patch for HIVE-12274 changed the column type to CLOB. Shouldn't the original problem which is causing all this not exist anymore? > move custom SerDe schema considerations into metastore from QL > -- > > Key: HIVE-17714 > URL: https://issues.apache.org/jira/browse/HIVE-17714 > Project: Hive > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Alan Gates > > Columns in metastore for tables that use external schema don't have the type > information (since HIVE-11985) and may be entirely inconsistent (since > forever, due to issues like HIVE-17713; or for SerDes that allow an URL for > the schema, due to a change in the underlying file). > Currently, if you trace the usage of ConfVars.SERDESUSINGMETASTOREFORSCHEMA, > and to MetaStoreUtils.getFieldsFromDeserializer, you'd see that the code in > QL handles this in Hive. So, for the most part metastore just returns > whatever is stored for columns in the database. > One exception appears to be get_fields_with_environment_context, which is > interesting... so getTable will return incorrect columns (potentially), but > get_fields/get_schema will return correct ones from SerDe as far as I can > tell. > As part of separating the metastore, we should make sure all the APIs return > the correct schema for the columns; it's not a good idea to have everyone > reimplement getFieldsFromDeserializer. > Note: this should also remove a flag introduced in HIVE-17731 -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HIVE-15016) Run tests with Hadoop 3.0.0-beta1
[ https://issues.apache.org/jira/browse/HIVE-15016?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16252740#comment-16252740 ] Ashutosh Chauhan commented on HIVE-15016: - yes.. thats correct. > Run tests with Hadoop 3.0.0-beta1 > - > > Key: HIVE-15016 > URL: https://issues.apache.org/jira/browse/HIVE-15016 > Project: Hive > Issue Type: Sub-task > Components: Hive >Affects Versions: 3.0.0 >Reporter: Sergio Peña >Assignee: Aihua Xu > Fix For: 3.0.0 > > Attachments: HIVE-15016.10.patch, HIVE-15016.2.patch, > HIVE-15016.3.patch, HIVE-15016.4.patch, HIVE-15016.5.patch, > HIVE-15016.6.patch, HIVE-15016.7.patch, HIVE-15016.8.patch, > HIVE-15016.9.patch, HIVE-15016.patch, Hadoop3Upstream.patch > > > Hadoop 3.0.0-alpha1 was released back on Sep/16 to allow other components run > tests against this new version before GA. > We should start running tests with Hive to validate compatibility against > Hadoop 3.0. > NOTE: The patch used to test must not be committed to Hive until Hadoop 3.0 > GA is released. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HIVE-17672) Upgrade Calcite version to 1.14
[ https://issues.apache.org/jira/browse/HIVE-17672?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jesus Camacho Rodriguez updated HIVE-17672: --- Fix Version/s: 3.0.0 > Upgrade Calcite version to 1.14 > --- > > Key: HIVE-17672 > URL: https://issues.apache.org/jira/browse/HIVE-17672 > Project: Hive > Issue Type: Task >Affects Versions: 3.0.0 >Reporter: Jesus Camacho Rodriguez >Assignee: Jesus Camacho Rodriguez > Fix For: 3.0.0 > > Attachments: HIVE-17672.01.patch, HIVE-17672.02.patch, > HIVE-17672.03.patch, HIVE-17672.04.patch, HIVE-17672.05.patch > > > Calcite 1.14.0 has been recently released. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HIVE-17373) Upgrade some dependency versions
[
https://issues.apache.org/jira/browse/HIVE-17373?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16252712#comment-16252712
]
Hive QA commented on HIVE-17373:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12897598/HIVE-17373.3.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 14 failed/errored test(s), 11384 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestAccumuloCliDriver.testCliDriver[accumulo_joins]
(batchId=235)
org.apache.hadoop.hive.cli.TestAccumuloCliDriver.testCliDriver[accumulo_predicate_pushdown]
(batchId=235)
org.apache.hadoop.hive.cli.TestAccumuloCliDriver.testCliDriver[accumulo_queries]
(batchId=235)
org.apache.hadoop.hive.cli.TestAccumuloCliDriver.testCliDriver[accumulo_single_sourced_multi_insert]
(batchId=235)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[auto_sortmerge_join_2]
(batchId=47)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[dbtxnmgr_showlocks]
(batchId=77)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[unionDistinct_1]
(batchId=146)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[insert_values_orig_table_use_metadata]
(batchId=162)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[ppd_union_view]
(batchId=154)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[sysdb]
(batchId=156)
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[explainanalyze_2]
(batchId=102)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[subquery_multi]
(batchId=111)
org.apache.hadoop.hive.cli.control.TestDanglingQOuts.checkDanglingQOut
(batchId=206)
org.apache.hadoop.hive.ql.parse.TestReplicationScenarios.testConstraints
(batchId=223)
{noformat}
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/7812/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/7812/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-7812/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 14 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12897598 - PreCommit-HIVE-Build
> Upgrade some dependency versions
>
>
> Key: HIVE-17373
> URL: https://issues.apache.org/jira/browse/HIVE-17373
> Project: Hive
> Issue Type: Improvement
> Components: Hive
>Affects Versions: 3.0.0
>Reporter: Aihua Xu
>Assignee: Aihua Xu
> Attachments: HIVE-17373.1.patch, HIVE-17373.2.patch,
> HIVE-17373.3.patch
>
>
> Upgrade some libraries including log4j to 2.8.2, accumulo to 1.8.1 and
> commons-httpclient to 3.1.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HIVE-15016) Run tests with Hadoop 3.0.0-beta1
[ https://issues.apache.org/jira/browse/HIVE-15016?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16252671#comment-16252671 ] Sergey Shelukhin commented on HIVE-15016: - Hmm... the changes to some files (e.g. ShuffleHandler.java) make this runtime-incompatible with Hadoop 2. The normal approach for such cases used to be to create a shim. Are we saying Hive 3 will only support Hadoop 3? > Run tests with Hadoop 3.0.0-beta1 > - > > Key: HIVE-15016 > URL: https://issues.apache.org/jira/browse/HIVE-15016 > Project: Hive > Issue Type: Sub-task > Components: Hive >Affects Versions: 3.0.0 >Reporter: Sergio Peña >Assignee: Aihua Xu > Fix For: 3.0.0 > > Attachments: HIVE-15016.10.patch, HIVE-15016.2.patch, > HIVE-15016.3.patch, HIVE-15016.4.patch, HIVE-15016.5.patch, > HIVE-15016.6.patch, HIVE-15016.7.patch, HIVE-15016.8.patch, > HIVE-15016.9.patch, HIVE-15016.patch, Hadoop3Upstream.patch > > > Hadoop 3.0.0-alpha1 was released back on Sep/16 to allow other components run > tests against this new version before GA. > We should start running tests with Hive to validate compatibility against > Hadoop 3.0. > NOTE: The patch used to test must not be committed to Hive until Hadoop 3.0 > GA is released. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Comment Edited] (HIVE-17714) move custom SerDe schema considerations into metastore from QL
[ https://issues.apache.org/jira/browse/HIVE-17714?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16252660#comment-16252660 ] Sergey Shelukhin edited comment on HIVE-17714 at 11/14/17 11:14 PM: The reason people never hit it before is that before HIVE-11985, which people are only starting to use, metastore would duplicate the schema from deserializer into the metastore columns (2.1 in my previous comment). So, in most cases (unless either the user or the serde messed with it), the schema returned would actually be the real schema. I actually filed this JIRA based on a case where someone was using a Hive table from Presto, and in addition to the problems introduced by HIVE-11985 for that scenario (presto would not get the correct type; but in this case it worked anyway because it was a text-based serde so it was anyway always reading string), also managed to modify the columns manually so they were out of sync with the SerDe was (Author: sershe): The reason people never hit it before is that before HIVE-11985, which people are only starting to use, metastore would duplicate the schema from deserializer into the metastore columns (2.1 in my previous comment). So, in most cases (unless either the user or the serde messed with it), the schema returned would actually be the real schema. I actually filed this JIRA based on a case where someone was using a Hive table from Presto, and in addition to the problems introduced by HIVE-11985 for that scenario, also managed to modify the columns manually so they were out of sync with the SerDe > move custom SerDe schema considerations into metastore from QL > -- > > Key: HIVE-17714 > URL: https://issues.apache.org/jira/browse/HIVE-17714 > Project: Hive > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Alan Gates > > Columns in metastore for tables that use external schema don't have the type > information (since HIVE-11985) and may be entirely inconsistent (since > forever, due to issues like HIVE-17713; or for SerDes that allow an URL for > the schema, due to a change in the underlying file). > Currently, if you trace the usage of ConfVars.SERDESUSINGMETASTOREFORSCHEMA, > and to MetaStoreUtils.getFieldsFromDeserializer, you'd see that the code in > QL handles this in Hive. So, for the most part metastore just returns > whatever is stored for columns in the database. > One exception appears to be get_fields_with_environment_context, which is > interesting... so getTable will return incorrect columns (potentially), but > get_fields/get_schema will return correct ones from SerDe as far as I can > tell. > As part of separating the metastore, we should make sure all the APIs return > the correct schema for the columns; it's not a good idea to have everyone > reimplement getFieldsFromDeserializer. > Note: this should also remove a flag introduced in HIVE-17731 -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HIVE-17714) move custom SerDe schema considerations into metastore from QL
[
https://issues.apache.org/jira/browse/HIVE-17714?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16252656#comment-16252656
]
Vihang Karajgaonkar commented on HIVE-17714:
bq. However, that means that most metastore APIs return bogus fields for such
tables (only get_schema/get_fields return correct fields - by calling the
deserializer inside metastore).
Can you please give an example? If the other APIs should really be using
Deserializer till now, I am surprised that nobody is hit with this issue till
now. If there is a way to reproduce this perhaps it will make it easier to
understand.
bq. screwing everyone who wants to read Hive tables without intricate
understanding of SerDe/Hive internals. I know for sure that it will break
Presto, but I suspect it will actually break everyone trying to use metastore
at this time And I'm not even sure how non-Java users can support this.
Again, if this was such a fundamental problem I don't know why anyone has not
seen this till now since only two APIs currently get the schema from serde
while rest just query from the database.
bq. forcing the table creation and updates to externally recreate the schema
for the benefit of the readers. This is not as bad as messing with readers,
cause those tables are mostly created by Hive, but still bad (if external users
do create the tables) and also doesn't solve the external schema case.
Just to clarify are you saying that anybody who is reading this table
(Hive/Impala/Presto etc) will have to recreate the schema using Deserializer if
the table schema is not stored in metastore? Isn't that happening right now
anyways? I looked at the describe table implementation in Hive and it gets the
schema from deserializer. The {{getTable}} API by default does not retrieve the
storageDescriptor and columnDescriptor currently.
> move custom SerDe schema considerations into metastore from QL
> --
>
> Key: HIVE-17714
> URL: https://issues.apache.org/jira/browse/HIVE-17714
> Project: Hive
> Issue Type: Bug
>Reporter: Sergey Shelukhin
>Assignee: Alan Gates
>
> Columns in metastore for tables that use external schema don't have the type
> information (since HIVE-11985) and may be entirely inconsistent (since
> forever, due to issues like HIVE-17713; or for SerDes that allow an URL for
> the schema, due to a change in the underlying file).
> Currently, if you trace the usage of ConfVars.SERDESUSINGMETASTOREFORSCHEMA,
> and to MetaStoreUtils.getFieldsFromDeserializer, you'd see that the code in
> QL handles this in Hive. So, for the most part metastore just returns
> whatever is stored for columns in the database.
> One exception appears to be get_fields_with_environment_context, which is
> interesting... so getTable will return incorrect columns (potentially), but
> get_fields/get_schema will return correct ones from SerDe as far as I can
> tell.
> As part of separating the metastore, we should make sure all the APIs return
> the correct schema for the columns; it's not a good idea to have everyone
> reimplement getFieldsFromDeserializer.
> Note: this should also remove a flag introduced in HIVE-17731
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Comment Edited] (HIVE-17714) move custom SerDe schema considerations into metastore from QL
[ https://issues.apache.org/jira/browse/HIVE-17714?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16252660#comment-16252660 ] Sergey Shelukhin edited comment on HIVE-17714 at 11/14/17 11:14 PM: The reason people never hit it before is that before HIVE-11985, which people are only starting to use, metastore would duplicate the schema from deserializer into the metastore columns (2.1 in my previous comment). So, in most cases (unless either the user or the serde messed with it), the schema returned would actually be the real schema. I actually filed this JIRA based on a case where someone was using a Hive table from Presto, and in addition to the problems introduced by HIVE-11985 for that scenario, also managed to modify the columns manually so they were out of sync with the SerDe was (Author: sershe): The reason people never hit it before is that before HIVE-11985, which people are only starting to use, metastore would duplicate the schema from deserializer into the metastore columns (2.1 in my previous comment). So, in most cases (unless either the user or the serde messed with it), the schema returned would actually be the real schema. > move custom SerDe schema considerations into metastore from QL > -- > > Key: HIVE-17714 > URL: https://issues.apache.org/jira/browse/HIVE-17714 > Project: Hive > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Alan Gates > > Columns in metastore for tables that use external schema don't have the type > information (since HIVE-11985) and may be entirely inconsistent (since > forever, due to issues like HIVE-17713; or for SerDes that allow an URL for > the schema, due to a change in the underlying file). > Currently, if you trace the usage of ConfVars.SERDESUSINGMETASTOREFORSCHEMA, > and to MetaStoreUtils.getFieldsFromDeserializer, you'd see that the code in > QL handles this in Hive. So, for the most part metastore just returns > whatever is stored for columns in the database. > One exception appears to be get_fields_with_environment_context, which is > interesting... so getTable will return incorrect columns (potentially), but > get_fields/get_schema will return correct ones from SerDe as far as I can > tell. > As part of separating the metastore, we should make sure all the APIs return > the correct schema for the columns; it's not a good idea to have everyone > reimplement getFieldsFromDeserializer. > Note: this should also remove a flag introduced in HIVE-17731 -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HIVE-17714) move custom SerDe schema considerations into metastore from QL
[ https://issues.apache.org/jira/browse/HIVE-17714?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16252660#comment-16252660 ] Sergey Shelukhin commented on HIVE-17714: - The reason people never hit it before is that before HIVE-11985, which people are only starting to use, metastore would duplicate the schema from deserializer into the metastore columns (2.1 in my previous comment). So, in most cases (unless either the user or the serde messed with it), the schema returned would actually be the real schema. > move custom SerDe schema considerations into metastore from QL > -- > > Key: HIVE-17714 > URL: https://issues.apache.org/jira/browse/HIVE-17714 > Project: Hive > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Alan Gates > > Columns in metastore for tables that use external schema don't have the type > information (since HIVE-11985) and may be entirely inconsistent (since > forever, due to issues like HIVE-17713; or for SerDes that allow an URL for > the schema, due to a change in the underlying file). > Currently, if you trace the usage of ConfVars.SERDESUSINGMETASTOREFORSCHEMA, > and to MetaStoreUtils.getFieldsFromDeserializer, you'd see that the code in > QL handles this in Hive. So, for the most part metastore just returns > whatever is stored for columns in the database. > One exception appears to be get_fields_with_environment_context, which is > interesting... so getTable will return incorrect columns (potentially), but > get_fields/get_schema will return correct ones from SerDe as far as I can > tell. > As part of separating the metastore, we should make sure all the APIs return > the correct schema for the columns; it's not a good idea to have everyone > reimplement getFieldsFromDeserializer. > Note: this should also remove a flag introduced in HIVE-17731 -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HIVE-17964) HoS: some spark configs doesn't require re-creating a session
[
https://issues.apache.org/jira/browse/HIVE-17964?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16252653#comment-16252653
]
Xuefu Zhang commented on HIVE-17964:
[~lirui] What's your thought on configurations such as {{spark.driver.cores}}?
> HoS: some spark configs doesn't require re-creating a session
> -
>
> Key: HIVE-17964
> URL: https://issues.apache.org/jira/browse/HIVE-17964
> Project: Hive
> Issue Type: Improvement
> Components: Spark
>Reporter: Rui Li
>Assignee: Rui Li
>Priority: Minor
> Attachments: HIVE-17964.1.patch, HIVE-17964.2.patch
>
>
> I guess the {{hive.spark.}} configs were initially intended for the RSC.
> Therefore when they're changed, we'll re-create the session for them to take
> effect. There're some configs not related to RSC that also start with
> {{hive.spark.}}. We'd better rename them so that we don't unnecessarily
> re-create sessions, which is usually time consuming.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HIVE-17954) Implement pool, user, group and trigger to pool management API's.
[
https://issues.apache.org/jira/browse/HIVE-17954?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16252632#comment-16252632
]
Hive QA commented on HIVE-17954:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12897584/HIVE-17954.03.patch
{color:red}ERROR:{color} -1 due to build exiting with an error
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/7810/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/7810/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-7810/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Tests exited with: NonZeroExitCodeException
Command 'bash /data/hiveptest/working/scratch/source-prep.sh' failed with exit
status 1 and output '+ date '+%Y-%m-%d %T.%3N'
2017-11-14 22:50:17.848
+ [[ -n /usr/lib/jvm/java-8-openjdk-amd64 ]]
+ export JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
+ JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
+ export
PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+
PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+ export 'ANT_OPTS=-Xmx1g -XX:MaxPermSize=256m '
+ ANT_OPTS='-Xmx1g -XX:MaxPermSize=256m '
+ export 'MAVEN_OPTS=-Xmx1g '
+ MAVEN_OPTS='-Xmx1g '
+ cd /data/hiveptest/working/
+ tee /data/hiveptest/logs/PreCommit-HIVE-Build-7810/source-prep.txt
+ [[ false == \t\r\u\e ]]
+ mkdir -p maven ivy
+ [[ git = \s\v\n ]]
+ [[ git = \g\i\t ]]
+ [[ -z master ]]
+ [[ -d apache-github-source-source ]]
+ [[ ! -d apache-github-source-source/.git ]]
+ [[ ! -d apache-github-source-source ]]
+ date '+%Y-%m-%d %T.%3N'
2017-11-14 22:50:17.852
+ cd apache-github-source-source
+ git fetch origin
>From https://github.com/apache/hive
857a9fd..2fa7a63 branch-2.3 -> origin/branch-2.3
+ git reset --hard HEAD
HEAD is now at d295849 HIVE-18002 : add group support for pool mappings (Sergey
Shelukhin, reviewed by Prasanth Jayachandran)
+ git clean -f -d
+ git checkout master
Already on 'master'
Your branch is up-to-date with 'origin/master'.
+ git reset --hard origin/master
HEAD is now at d295849 HIVE-18002 : add group support for pool mappings (Sergey
Shelukhin, reviewed by Prasanth Jayachandran)
+ git merge --ff-only origin/master
Already up-to-date.
+ date '+%Y-%m-%d %T.%3N'
2017-11-14 22:50:24.551
+ patchCommandPath=/data/hiveptest/working/scratch/smart-apply-patch.sh
+ patchFilePath=/data/hiveptest/working/scratch/build.patch
+ [[ -f /data/hiveptest/working/scratch/build.patch ]]
+ chmod +x /data/hiveptest/working/scratch/smart-apply-patch.sh
+ /data/hiveptest/working/scratch/smart-apply-patch.sh
/data/hiveptest/working/scratch/build.patch
error: patch failed:
ql/src/java/org/apache/hadoop/hive/ql/exec/tez/UserPoolMapping.java:68
error: ql/src/java/org/apache/hadoop/hive/ql/exec/tez/UserPoolMapping.java:
patch does not apply
error: patch failed:
ql/src/test/org/apache/hadoop/hive/ql/exec/tez/TestWorkloadManager.java:130
error: ql/src/test/org/apache/hadoop/hive/ql/exec/tez/TestWorkloadManager.java:
patch does not apply
The patch does not appear to apply with p0, p1, or p2
+ exit 1
'
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12897584 - PreCommit-HIVE-Build
> Implement pool, user, group and trigger to pool management API's.
> -
>
> Key: HIVE-17954
> URL: https://issues.apache.org/jira/browse/HIVE-17954
> Project: Hive
> Issue Type: Sub-task
>Reporter: Harish Jaiprakash
>Assignee: Harish Jaiprakash
> Attachments: HIVE-17954.01.patch, HIVE-17954.02.patch,
> HIVE-17954.03.patch
>
>
> Implement the following commands:
> -- Pool management.
> CREATE POOL `resource_plan`.`pool_path` WITH
> ALLOC_FRACTION `fraction`
> QUERY_PARALLELISM `parallelism`
> SCHEDULING_POLICY `policy`;
> ALTER POOL `resource_plan`.`pool_path` SET
> PATH = `new_path`,
> ALLOC_FRACTION = `fraction`,
> QUERY_PARALLELISM = `parallelism`,
> SCHEDULING_POLICY = `policy`;
> DROP POOL `resource_plan`.`pool_path`;
> -- Trigger to pool mappings.
> ALTER RESOURCE PLAN `resource_plan`
> ADD TRIGGER `trigger_name` TO `pool_path`;
> ALTER RESOURCE PLAN `resource_plan`
> DROP TRIGGER `trigger_name` TO `pool_path`;
> -- User/Group to pool mappings.
> CREATE USER|GROUP MAPPING `resource_plan`.`group_or_user_name`
> TO `pool_path` WITH ORDERING `order_no`;
> DROP USER|GROUP MAPPING `resource_plan`.`group_or_user_name`;
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HIVE-15680) Incorrect results when hive.optimize.index.filter=true and same ORC table is referenced twice in query
[
https://issues.apache.org/jira/browse/HIVE-15680?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16252634#comment-16252634
]
Hive QA commented on HIVE-15680:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12850112/HIVE-15680.6.patch
{color:red}ERROR:{color} -1 due to build exiting with an error
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/7811/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/7811/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-7811/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Tests exited with: NonZeroExitCodeException
Command 'bash /data/hiveptest/working/scratch/source-prep.sh' failed with exit
status 1 and output '+ date '+%Y-%m-%d %T.%3N'
2017-11-14 22:50:57.198
+ [[ -n /usr/lib/jvm/java-8-openjdk-amd64 ]]
+ export JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
+ JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
+ export
PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+
PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+ export 'ANT_OPTS=-Xmx1g -XX:MaxPermSize=256m '
+ ANT_OPTS='-Xmx1g -XX:MaxPermSize=256m '
+ export 'MAVEN_OPTS=-Xmx1g '
+ MAVEN_OPTS='-Xmx1g '
+ cd /data/hiveptest/working/
+ tee /data/hiveptest/logs/PreCommit-HIVE-Build-7811/source-prep.txt
+ [[ false == \t\r\u\e ]]
+ mkdir -p maven ivy
+ [[ git = \s\v\n ]]
+ [[ git = \g\i\t ]]
+ [[ -z master ]]
+ [[ -d apache-github-source-source ]]
+ [[ ! -d apache-github-source-source/.git ]]
+ [[ ! -d apache-github-source-source ]]
+ date '+%Y-%m-%d %T.%3N'
2017-11-14 22:50:57.201
+ cd apache-github-source-source
+ git fetch origin
+ git reset --hard HEAD
HEAD is now at d295849 HIVE-18002 : add group support for pool mappings (Sergey
Shelukhin, reviewed by Prasanth Jayachandran)
+ git clean -f -d
+ git checkout master
Already on 'master'
Your branch is up-to-date with 'origin/master'.
+ git reset --hard origin/master
HEAD is now at d295849 HIVE-18002 : add group support for pool mappings (Sergey
Shelukhin, reviewed by Prasanth Jayachandran)
+ git merge --ff-only origin/master
Already up-to-date.
+ date '+%Y-%m-%d %T.%3N'
2017-11-14 22:50:57.727
+ patchCommandPath=/data/hiveptest/working/scratch/smart-apply-patch.sh
+ patchFilePath=/data/hiveptest/working/scratch/build.patch
+ [[ -f /data/hiveptest/working/scratch/build.patch ]]
+ chmod +x /data/hiveptest/working/scratch/smart-apply-patch.sh
+ /data/hiveptest/working/scratch/smart-apply-patch.sh
/data/hiveptest/working/scratch/build.patch
error: patch failed:
ql/src/java/org/apache/hadoop/hive/ql/io/HiveInputFormat.java:23
error: ql/src/java/org/apache/hadoop/hive/ql/io/HiveInputFormat.java: patch
does not apply
error: patch failed:
serde/src/java/org/apache/hadoop/hive/serde2/ColumnProjectionUtils.java:117
error: serde/src/java/org/apache/hadoop/hive/serde2/ColumnProjectionUtils.java:
patch does not apply
The patch does not appear to apply with p0, p1, or p2
+ exit 1
'
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12850112 - PreCommit-HIVE-Build
> Incorrect results when hive.optimize.index.filter=true and same ORC table is
> referenced twice in query
> --
>
> Key: HIVE-15680
> URL: https://issues.apache.org/jira/browse/HIVE-15680
> Project: Hive
> Issue Type: Bug
>Affects Versions: 1.1.0, 2.2.0
>Reporter: Anthony Hsu
>Assignee: Anthony Hsu
> Attachments: HIVE-15680.1.patch, HIVE-15680.2.patch,
> HIVE-15680.3.patch, HIVE-15680.4.patch, HIVE-15680.5.patch, HIVE-15680.6.patch
>
>
> To repro:
> {noformat}
> set hive.optimize.index.filter=true;
> create table test_table(number int) stored as ORC;
> -- Two insertions will create two files, with one stripe each
> insert into table test_table VALUES (1);
> insert into table test_table VALUES (2);
> -- This should and does return 2 records
> select * from test_table;
> -- These should and do each return 1 record
> select * from test_table where number = 1;
> select * from test_table where number = 2;
> -- This should return 2 records but only returns 1 record
> select * from test_table where number = 1
> union all
> select * from test_table where number = 2;
> {noformat}
> What's happening is only the last predicate is being pushed down.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HIVE-18057) remove PostExecute / PreExecute hook support
[
https://issues.apache.org/jira/browse/HIVE-18057?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16252629#comment-16252629
]
Hive QA commented on HIVE-18057:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12897573/HIVE-18057.01.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 10 failed/errored test(s), 11384 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[auto_sortmerge_join_2]
(batchId=47)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[dbtxnmgr_showlocks]
(batchId=77)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[unionDistinct_1]
(batchId=146)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[insert_values_orig_table_use_metadata]
(batchId=162)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[ppd_union_view]
(batchId=154)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[sysdb]
(batchId=156)
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[explainanalyze_2]
(batchId=102)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[subquery_multi]
(batchId=111)
org.apache.hadoop.hive.cli.control.TestDanglingQOuts.checkDanglingQOut
(batchId=206)
org.apache.hadoop.hive.ql.parse.TestReplicationScenarios.testConstraints
(batchId=223)
{noformat}
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/7809/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/7809/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-7809/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 10 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12897573 - PreCommit-HIVE-Build
> remove PostExecute / PreExecute hook support
>
>
> Key: HIVE-18057
> URL: https://issues.apache.org/jira/browse/HIVE-18057
> Project: Hive
> Issue Type: Sub-task
>Reporter: Zoltan Haindrich
>Assignee: Zoltan Haindrich
> Attachments: HIVE-18057.01.patch
>
>
> * deprecated since 2010
> * they are needlessly complicate the dispatch logic
> * the current dispatch logic just silently accepts pre/post hooks if they
> don't implement neither interface
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HIVE-10378) Hive Update statement set keyword work with lower case only and doesn't give any error if wrong column name specified in the set clause.
[
https://issues.apache.org/jira/browse/HIVE-10378?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sahil Takiar updated HIVE-10378:
Fix Version/s: (was: 2.3.2)
> Hive Update statement set keyword work with lower case only and doesn't give
> any error if wrong column name specified in the set clause.
>
>
> Key: HIVE-10378
> URL: https://issues.apache.org/jira/browse/HIVE-10378
> Project: Hive
> Issue Type: Bug
> Components: Transactions
>Affects Versions: 1.0.0, 1.1.0
> Environment: Hadoop: 2.6.0
> Hive : 1.0.0/1.1.0
> OS:Linux
>Reporter: Vineet Kandpal
>Assignee: Oleksiy Sayankin
> Attachments: HIVE-10378.1.patch
>
>
> Brief: Hive Update statement set keyword work with lower case only and
> doesn't give any error if wrong column name specified in the set clause.
> Steps to reproduce:
> following are the steps performed for the same:
> 1. Create Table with transactional properties.
> create table customer(id int ,name string, email string) clustered by (id)
> into 2 buckets stored as orc TBLPROPERTIES('transactional'='true')
> 2. Insert data into transactional table:
> insert into table customer values
> (1,'user1','us...@user1.com'),(2,'user2','us...@user1.com'),(3,'user3','us...@gmail.com')
> 3. Search result:
> 0: jdbc:hive2://localhost:1> select * from customer;
> +--++--+--+
> | customer.id | customer.name | customer.email |
> +--++--+--+
> | 2| user2 | us...@user1.com |
> | 3| user3 | us...@gmail.com |
> | 1| user1 | us...@user1.com |
> +--++--+--+
> 3 rows selected (0.299 seconds)
> 4. Update table column name with some clause In below column name is used in
> the UPPER case (NAME) and it is not updating the column value :
> 0: jdbc:hive2://localhost:1> update customer set NAME =
> 'notworking' where id = 1;
> INFO : Table default.customer stats: [numFiles=10, numRows=3,
> totalSize=6937, rawDataSize=0]
> No rows affected (20.343 seconds)
> 0: jdbc:hive2://localhost:1> select * from customer;
> +--++--+--+
> | customer.id | customer.name | customer.email |
> +--++--+--+
> | 2| user2 | us...@user1.com |
> | 3| user3 | us...@gmail.com |
> | 1| user1 | us...@user1.com |
> +--++--+--+
> 3 rows selected (0.321 seconds)
> 5. Update table column name with some clause In below column name is used in
> the LOWER case (name) and it is updating the column value
> 0: jdbc:hive2://localhost:1> update customer set name = 'working'
> where id = 1;
> INFO : Table default.customer stats: [numFiles=11, numRows=3,
> totalSize=7699, rawDataSize=0]
> No rows affected (19.74 seconds)
> 0: jdbc:hive2://localhost:1> select * from customer;
> +--++--+--+
> | customer.id | customer.name | customer.email |
> +--++--+--+
> | 2| user2 | us...@user1.com |
> | 3| user3 | us...@gmail.com |
> | 1| working| us...@user1.com |
> +--++--+--+
> 3 rows selected (0.333 seconds)
> 0: jdbc:hive2://localhost:1>
> 6. We have also seen that if we put the column name incorrect in set keyword
> of the update statement it accept the query and execute job. There should
> validation on the column name used in the set clause.
> 0: jdbc:hive2://localhost:1> update customer set name_44 =
> 'working' where id = 1;
>
>
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HIVE-17966) org.apache.hadoop.hive.ql.io.parquet.serde.ParquetHiveArrayInspector - Review
[ https://issues.apache.org/jira/browse/HIVE-17966?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sahil Takiar updated HIVE-17966: Fix Version/s: (was: 2.3.2) > org.apache.hadoop.hive.ql.io.parquet.serde.ParquetHiveArrayInspector - Review > - > > Key: HIVE-17966 > URL: https://issues.apache.org/jira/browse/HIVE-17966 > Project: Hive > Issue Type: Bug > Components: HiveServer2, Serializers/Deserializers >Affects Versions: 3.0.0 >Reporter: BELUGA BEHR >Assignee: BELUGA BEHR >Priority: Trivial > Fix For: 3.0.0 > > Attachments: HIVE-17966.1.patch, HIVE-17966.2.patch > > > * Simplify > * Make faster - perform bulk operations instead of iterating > * Remove compilation warnings -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HIVE-17948) Hive 2.3.2 Release Planning
[ https://issues.apache.org/jira/browse/HIVE-17948?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sahil Takiar updated HIVE-17948: Resolution: Fixed Status: Resolved (was: Patch Available) > Hive 2.3.2 Release Planning > --- > > Key: HIVE-17948 > URL: https://issues.apache.org/jira/browse/HIVE-17948 > Project: Hive > Issue Type: Bug >Affects Versions: 2.3.2 >Reporter: Sahil Takiar >Assignee: Sahil Takiar > Fix For: 2.3.2 > > Attachments: HIVE-17948-branch-2.3.patch, > HIVE-17948.2-branch-2.3.patch, HIVE-17948.3-branch-2.3.patch, > HIVE-17948.4-branch-2.3.patch > > > Release planning for Hive 2.3.2 -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HIVE-15862) beeline always have a warning "Hive does not support autoCommit=false"
[ https://issues.apache.org/jira/browse/HIVE-15862?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashutosh Chauhan updated HIVE-15862: Resolution: Fixed Fix Version/s: 2.2.0 Target Version/s: 2.1.1, 2.1.0 (was: 2.1.0, 2.1.1) Status: Resolved (was: Patch Available) > beeline always have a warning "Hive does not support autoCommit=false" > -- > > Key: HIVE-15862 > URL: https://issues.apache.org/jira/browse/HIVE-15862 > Project: Hive > Issue Type: Bug > Components: Beeline >Affects Versions: 2.1.0 > Environment: hive2.1.0 >Reporter: yangfang >Assignee: yangfang > Fix For: 2.2.0 > > Attachments: HIVE-15862.1.patch > > > When we use beeline, there is always a warning::"Request to set autoCommit to > false; Hive does not support autoCommit=false", > For example, we use beeline excute a sql: > beeline -u "jdbc:hive2://localhost:1/default" -n mr -e "show tables" > --hivevar taskNO=10111, we got this warning: > . > Connecting to jdbc:hive2://localhost:1/default > Connected to: Apache Hive (version 2.1.0-zdh2.7.3) > Driver: Hive JDBC (version 2.1.0-zdh2.7.3) > 17/02/10 15:10:10 [main]: WARN jdbc.HiveConnection: Request to set autoCommit > to false; Hive does not support autoCommit=false. > When I looked at the hive source code, I found the BeeLineOpts set the > autoCommit default value to false, It always triggers an alarm when beeline > connect to the hiveserver2 in DatabaseConnection.connect: > getConnection().setAutoCommit(beeLine.getOpts().getAutoCommit()); -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HIVE-18063) Make CommandProcessorResponse an exception instead of a return class
[
https://issues.apache.org/jira/browse/HIVE-18063?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16252227#comment-16252227
]
Hive QA commented on HIVE-18063:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12897572/HIVE-18063.01.patch
{color:green}SUCCESS:{color} +1 due to 4 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 43 failed/errored test(s), 11383 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[auto_sortmerge_join_2]
(batchId=47)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[char_udf1] (batchId=88)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[dbtxnmgr_showlocks]
(batchId=77)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[unionDistinct_1]
(batchId=146)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[insert_values_orig_table_use_metadata]
(batchId=162)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[ppd_union_view]
(batchId=154)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[sysdb]
(batchId=156)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[authorization_fail_2]
(batchId=91)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[authorization_fail_3]
(batchId=91)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[authorization_fail_4]
(batchId=91)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[authorization_fail_5]
(batchId=91)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[authorization_fail_6]
(batchId=93)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[authorization_fail_7]
(batchId=91)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[authorization_fail_create_db]
(batchId=92)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[authorization_fail_drop_db]
(batchId=92)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[authorization_part]
(batchId=92)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[authorization_view_1]
(batchId=91)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[authorization_view_2]
(batchId=92)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[authorization_view_3]
(batchId=91)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[authorization_view_4]
(batchId=91)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[authorization_view_5]
(batchId=92)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[authorization_view_6]
(batchId=92)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[authorization_view_7]
(batchId=92)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[authorization_view_disable_cbo_1]
(batchId=92)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[authorization_view_disable_cbo_2]
(batchId=91)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[authorization_view_disable_cbo_3]
(batchId=91)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[authorization_view_disable_cbo_4]
(batchId=91)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[authorization_view_disable_cbo_5]
(batchId=92)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[authorization_view_disable_cbo_6]
(batchId=91)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[authorization_view_disable_cbo_7]
(batchId=92)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[exim_22_export_authfail]
(batchId=92)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[exim_23_import_exist_authfail]
(batchId=92)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[exim_24_import_part_authfail]
(batchId=92)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[exim_25_import_nonexist_authfail]
(batchId=91)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[load_exist_part_authfail]
(batchId=91)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[load_nonpart_authfail]
(batchId=92)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[load_part_authfail]
(batchId=92)
org.apache.hadoop.hive.cli.TestNegativeMinimrCliDriver.testCliDriver[ct_noperm_loc]
(batchId=94)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[subquery_multi]
(batchId=111)
org.apache.hadoop.hive.cli.control.TestDanglingQOuts.checkDanglingQOut
(batchId=206)
org.apache.hadoop.hive.ql.lockmgr.TestDbTxnManager2.testLockBlockedBy
(batchId=287)
org.apache.hadoop.hive.ql.parse.TestReplicationScenarios.testConstraints
(batchId=223)
org.apache.hadoop.hive.ql.security.TestClientSideAuthorizationProvider.testSimplePrivileges
(batchId=225)
{noformat}
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/7808/testReport
Console output:
[jira] [Assigned] (HIVE-18065) Exception when pushing postaggregates into Druid
[
https://issues.apache.org/jira/browse/HIVE-18065?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jesus Camacho Rodriguez reassigned HIVE-18065:
--
> Exception when pushing postaggregates into Druid
>
>
> Key: HIVE-18065
> URL: https://issues.apache.org/jira/browse/HIVE-18065
> Project: Hive
> Issue Type: Bug
> Components: Druid integration
>Affects Versions: 3.0.0
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
>
> After Calcite is upgraded to 1.14 and the rule to push post-aggregations to
> Druid is enabled, the following query will fail:
> {code}
> EXPLAIN
> SELECT language, robot, sum(added) - sum(delta) AS a
> FROM druid_table_1
> WHERE extract (week from `__time`) IN (10,11)
> AND robot='Bird Call'
> GROUP BY language, robot;
> {code}
> The error we get is the following:
> {code}
> Cannot add expression of different type to set:
> set type is RecordType(VARCHAR(2147483647) CHARACTER SET "UTF-16LE" COLLATE
> "ISO-8859-1$en_US$primary" language, VARCHAR(2147483647) CHARACTER SET
> "UTF-16LE" COLLATE "ISO-8859-1$en_US$primary" robot, DOUBLE a) NOT NULL
> expression type is RecordType(VARCHAR(2147483647) CHARACTER SET "UTF-16LE"
> COLLATE "ISO-8859-1$en_US$primary" language, DOUBLE postagg#0) NOT NULL
> set is
> rel#1507:HiveProject.HIVE.[](input=HepRelVertex#1514,language=$0,robot=CAST(_UTF-16LE'Bird
> Call'):VARCHAR(2147483647) CHARACTER SET "UTF-16LE" COLLATE
> "ISO-8859-1$en_US$primary",a=-($1, $2))
> expression is DruidQuery#1516
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HIVE-18057) remove PostExecute / PreExecute hook support
[ https://issues.apache.org/jira/browse/HIVE-18057?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16252221#comment-16252221 ] Ashutosh Chauhan commented on HIVE-18057: - +1 pending tests > remove PostExecute / PreExecute hook support > > > Key: HIVE-18057 > URL: https://issues.apache.org/jira/browse/HIVE-18057 > Project: Hive > Issue Type: Sub-task >Reporter: Zoltan Haindrich >Assignee: Zoltan Haindrich > Attachments: HIVE-18057.01.patch > > > * deprecated since 2010 > * they are needlessly complicate the dispatch logic > * the current dispatch logic just silently accepts pre/post hooks if they > don't implement neither interface -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HIVE-18064) Hive on Tez parallel order by
[ https://issues.apache.org/jira/browse/HIVE-18064?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhiyuan Yang updated HIVE-18064: Status: Patch Available (was: Open) > Hive on Tez parallel order by > - > > Key: HIVE-18064 > URL: https://issues.apache.org/jira/browse/HIVE-18064 > Project: Hive > Issue Type: New Feature >Reporter: Zhiyuan Yang >Assignee: Zhiyuan Yang > Attachments: HIVE-18064.1.patch > > > We've built parallel sorting in TEZ-3837. It does sampling as output is > generated and figure out a range partitioner for shuffle edge. Each reducer > output a sorted span. This is mainly for external consumption since output > files need to be read in certain order. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HIVE-18064) Hive on Tez parallel order by
[ https://issues.apache.org/jira/browse/HIVE-18064?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhiyuan Yang updated HIVE-18064: Status: Open (was: Patch Available) > Hive on Tez parallel order by > - > > Key: HIVE-18064 > URL: https://issues.apache.org/jira/browse/HIVE-18064 > Project: Hive > Issue Type: New Feature >Reporter: Zhiyuan Yang >Assignee: Zhiyuan Yang > Attachments: HIVE-18064.1.patch > > > We've built parallel sorting in TEZ-3837. It does sampling as output is > generated and figure out a range partitioner for shuffle edge. Each reducer > output a sorted span. This is mainly for external consumption since output > files need to be read in certain order. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HIVE-18002) add group support for pool mappings
[ https://issues.apache.org/jira/browse/HIVE-18002?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Shelukhin updated HIVE-18002: Resolution: Fixed Fix Version/s: 3.0.0 Status: Resolved (was: Patch Available) Committed; thanks for the review! > add group support for pool mappings > --- > > Key: HIVE-18002 > URL: https://issues.apache.org/jira/browse/HIVE-18002 > Project: Hive > Issue Type: Sub-task >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin > Fix For: 3.0.0 > > Attachments: HIVE-18002.01.patch, HIVE-18002.02.patch, > HIVE-18002.02.patch, HIVE-18002.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HIVE-18064) Hive on Tez parallel order by
[ https://issues.apache.org/jira/browse/HIVE-18064?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhiyuan Yang updated HIVE-18064: Attachment: HIVE-18064.1.patch > Hive on Tez parallel order by > - > > Key: HIVE-18064 > URL: https://issues.apache.org/jira/browse/HIVE-18064 > Project: Hive > Issue Type: New Feature >Reporter: Zhiyuan Yang >Assignee: Zhiyuan Yang > Attachments: HIVE-18064.1.patch > > > We've built parallel sorting in TEZ-3837. It does sampling as output is > generated and figure out a range partitioner for shuffle edge. Each reducer > output a sorted span. This is mainly for external consumption since output > files need to be read in certain order. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Assigned] (HIVE-18064) Hive on Tez parallel order by
[ https://issues.apache.org/jira/browse/HIVE-18064?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhiyuan Yang reassigned HIVE-18064: --- > Hive on Tez parallel order by > - > > Key: HIVE-18064 > URL: https://issues.apache.org/jira/browse/HIVE-18064 > Project: Hive > Issue Type: Bug >Reporter: Zhiyuan Yang >Assignee: Zhiyuan Yang > > We've built parallel sorting in TEZ-3837. It does sampling as output is > generated and figure out a range partitioner for shuffle edge. Each reducer > output a sorted span. This is mainly for external consumption since output > files need to be read in certain order. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HIVE-18064) Hive on Tez parallel order by
[ https://issues.apache.org/jira/browse/HIVE-18064?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhiyuan Yang updated HIVE-18064: Issue Type: New Feature (was: Bug) > Hive on Tez parallel order by > - > > Key: HIVE-18064 > URL: https://issues.apache.org/jira/browse/HIVE-18064 > Project: Hive > Issue Type: New Feature >Reporter: Zhiyuan Yang >Assignee: Zhiyuan Yang > > We've built parallel sorting in TEZ-3837. It does sampling as output is > generated and figure out a range partitioner for shuffle edge. Each reducer > output a sorted span. This is mainly for external consumption since output > files need to be read in certain order. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HIVE-18002) add group support for pool mappings
[ https://issues.apache.org/jira/browse/HIVE-18002?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16252196#comment-16252196 ] Prasanth Jayachandran commented on HIVE-18002: -- +1 > add group support for pool mappings > --- > > Key: HIVE-18002 > URL: https://issues.apache.org/jira/browse/HIVE-18002 > Project: Hive > Issue Type: Sub-task >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin > Attachments: HIVE-18002.01.patch, HIVE-18002.02.patch, > HIVE-18002.02.patch, HIVE-18002.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HIVE-18012) fix ct_noperm_loc test
[
https://issues.apache.org/jira/browse/HIVE-18012?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ashutosh Chauhan updated HIVE-18012:
Resolution: Fixed
Fix Version/s: 3.0.0
Status: Resolved (was: Patch Available)
Pushed to master. Thanks, Zoltan!
> fix ct_noperm_loc test
> --
>
> Key: HIVE-18012
> URL: https://issues.apache.org/jira/browse/HIVE-18012
> Project: Hive
> Issue Type: Bug
>Reporter: Zoltan Haindrich
>Assignee: Zoltan Haindrich
> Fix For: 3.0.0
>
> Attachments: HIVE-18012.001.patch, HIVE-18012.02.patch
>
>
> the goal of the test is to check that hive doesn't let user1 to create a
> table with a location under an unowned path.
> I've bisected this test to be broken by
> 5250ef450430fcdeed0a2cb7a770f48647987cd3 (HIVE-12408).
> the original exception was (which have been by that sole masked line):
> {code}
> FAILED: HiveAccessControlException Permission denied: Principal [name=user1,
> type=USER] does not have following privileges for operation CREATETABLE
> [[OBJECT OWNERSHIP] on Object [type=DFS_URI,
> name=hdfs://localhost:35753/tmp/ct_noperm_loc_foo0]]
> {code}
> the current semanticexception shouldnt be accepted ; because it's unrelated
> to the tests goal.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HIVE-17904) handle internal Tez AM restart in registry and WM
[ https://issues.apache.org/jira/browse/HIVE-17904?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Shelukhin updated HIVE-17904: Attachment: HIVE-17904.06.patch Rebased. There's some possibility of conflicts, so I'm going to wait for HiveQA again. > handle internal Tez AM restart in registry and WM > - > > Key: HIVE-17904 > URL: https://issues.apache.org/jira/browse/HIVE-17904 > Project: Hive > Issue Type: Sub-task >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin > Attachments: HIVE-17904.01.patch, HIVE-17904.02.patch, > HIVE-17904.03.patch, HIVE-17904.04.patch, HIVE-17904.05.patch, > HIVE-17904.06.patch, HIVE-17904.patch, HIVE-17904.patch > > > After the plan update patch is committed. The current code doesn't account > very well for it; registry may have races, and an event needs to be added to > WM when some AM resets, at least to make sure we discard the update errors > that pertain to the old AM. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HIVE-17906) use kill query mechanics to kill queries in WM
[ https://issues.apache.org/jira/browse/HIVE-17906?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Shelukhin updated HIVE-17906: Resolution: Fixed Fix Version/s: 3.0.0 Status: Resolved (was: Patch Available) Committed to master. Thanks for the reviews! > use kill query mechanics to kill queries in WM > -- > > Key: HIVE-17906 > URL: https://issues.apache.org/jira/browse/HIVE-17906 > Project: Hive > Issue Type: Sub-task >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin > Fix For: 3.0.0 > > Attachments: HIVE-17906.01.patch, HIVE-17906.02.patch, > HIVE-17906.03.patch, HIVE-17906.03.patch, HIVE-17906.04.patch, > HIVE-17906.05.patch, HIVE-17906.patch > > > Right now it just closes the session (see HIVE-17841). The sessions would > need to be reused after the kill, or closed after the kill if the total QP > has decreased -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HIVE-18050) LlapServiceDriver shoud split HIVE_AUX_JARS_PATH by ':' instead of ','
[
https://issues.apache.org/jira/browse/HIVE-18050?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16252152#comment-16252152
]
Sergey Shelukhin commented on HIVE-18050:
-
+1
> LlapServiceDriver shoud split HIVE_AUX_JARS_PATH by ':' instead of ','
> --
>
> Key: HIVE-18050
> URL: https://issues.apache.org/jira/browse/HIVE-18050
> Project: Hive
> Issue Type: Bug
> Components: CLI, Clients
>Affects Versions: 2.3.0
>Reporter: Aegeaner
>Assignee: Aegeaner
> Labels: pull-request-available
>
> LlapServiceDriver shoud split HIVE_AUX_JARS_PATH by ':' instead of ',' ,
> since in hive script the environment variable has been replaced:
> {code:java}
> elif [ "${HIVE_AUX_JARS_PATH}" != "" ]; then
> HIVE_AUX_JARS_PATH=`echo $HIVE_AUX_JARS_PATH | sed 's/,/:/g'`
> if $cygwin; then
> HIVE_AUX_JARS_PATH=`cygpath -p -w "$HIVE_AUX_JARS_PATH"`
> HIVE_AUX_JARS_PATH=`echo $HIVE_AUX_JARS_PATH | sed 's/;/,/g'`
> fi
> AUX_CLASSPATH=${AUX_CLASSPATH}:${HIVE_AUX_JARS_PATH}
> AUX_PARAM="file://$(echo ${HIVE_AUX_JARS_PATH} | sed 's/:/,file:\/\//g')"
> fi
> {code}
> But in the LLAP Service Driver, it's processed as :
> {code:java}
> private void addAuxJarsToSet(HashSet auxJarSet, String auxJars) {
> if (auxJars != null && !auxJars.isEmpty()) {
> // TODO: transitive dependencies warning?
> String[] jarPaths = auxJars.split(",");
> for (String jarPath : jarPaths) {
> if (!jarPath.isEmpty()) {
> auxJarSet.add(jarPath);
> }
> }
> }
> }
> };
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HIVE-17934) Merging Statistics are promoted to COMPLETE (most of the time)
[ https://issues.apache.org/jira/browse/HIVE-17934?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zoltan Haindrich updated HIVE-17934: Attachment: HIVE-17934.06.patch > Merging Statistics are promoted to COMPLETE (most of the time) > -- > > Key: HIVE-17934 > URL: https://issues.apache.org/jira/browse/HIVE-17934 > Project: Hive > Issue Type: Sub-task > Components: Statistics >Reporter: Zoltan Haindrich >Assignee: Zoltan Haindrich > Attachments: HIVE-17934.01.patch, HIVE-17934.02.patch, > HIVE-17934.03.patch, HIVE-17934.04.patch, HIVE-17934.05.patch, > HIVE-17934.06.patch, HIVE-17934.06.patch, HIVE-17934.06wip01.patch > > > in case multiple partition statistics are merged the STATS state is computed > based on the datasize and rowcount; > the merge may hide away non-existent stats in case there are other partition > or operators which do contribute to the datasize and the rowcount. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HIVE-17906) use kill query mechanics to kill queries in WM
[ https://issues.apache.org/jira/browse/HIVE-17906?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16252144#comment-16252144 ] Sergey Shelukhin commented on HIVE-17906: - The new failures are unrelated - the timed-out test OOMed (GC overhead limit), and the HCat one had a connection error. > use kill query mechanics to kill queries in WM > -- > > Key: HIVE-17906 > URL: https://issues.apache.org/jira/browse/HIVE-17906 > Project: Hive > Issue Type: Sub-task >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin > Attachments: HIVE-17906.01.patch, HIVE-17906.02.patch, > HIVE-17906.03.patch, HIVE-17906.03.patch, HIVE-17906.04.patch, > HIVE-17906.05.patch, HIVE-17906.patch > > > Right now it just closes the session (see HIVE-17841). The sessions would > need to be reused after the kill, or closed after the kill if the total QP > has decreased -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HIVE-17361) Support LOAD DATA for transactional tables
[
https://issues.apache.org/jira/browse/HIVE-17361?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16252128#comment-16252128
]
Hive QA commented on HIVE-17361:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12897571/HIVE-17361.10.patch
{color:green}SUCCESS:{color} +1 due to 7 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 21 failed/errored test(s), 11388 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[acid_table_stats]
(batchId=52)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[autoColumnStats_4]
(batchId=12)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[dbtxnmgr_showlocks]
(batchId=77)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[mm_default] (batchId=81)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[udf_case_column_pruning]
(batchId=76)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[unionDistinct_1]
(batchId=146)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[insert_values_orig_table_use_metadata]
(batchId=162)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[llap_acid_fast]
(batchId=157)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[ppd_union_view]
(batchId=154)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[sysdb]
(batchId=156)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[load_data_into_acid]
(batchId=91)
org.apache.hadoop.hive.cli.TestNegativeMinimrCliDriver.testCliDriver[ct_noperm_loc]
(batchId=94)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[subquery_multi]
(batchId=111)
org.apache.hadoop.hive.cli.control.TestDanglingQOuts.checkDanglingQOut
(batchId=206)
org.apache.hadoop.hive.ql.TestTxnLoadData.loadData (batchId=254)
org.apache.hadoop.hive.ql.TestTxnLoadData.loadDataNonAcid2AcidConversion
(batchId=254)
org.apache.hadoop.hive.ql.TestTxnLoadData.loadDataPartitioned (batchId=254)
org.apache.hadoop.hive.ql.io.orc.TestInputOutputFormat.testACIDReaderFooterSerializeWithDeltas
(batchId=267)
org.apache.hadoop.hive.ql.io.orc.TestInputOutputFormat.testACIDReaderNoFooterSerializeWithDeltas
(batchId=267)
org.apache.hadoop.hive.ql.io.orc.TestInputOutputFormat.testEtlCombinedStrategy
(batchId=267)
org.apache.hadoop.hive.ql.parse.TestReplicationScenarios.testConstraints
(batchId=223)
{noformat}
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/7807/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/7807/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-7807/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 21 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12897571 - PreCommit-HIVE-Build
> Support LOAD DATA for transactional tables
> --
>
> Key: HIVE-17361
> URL: https://issues.apache.org/jira/browse/HIVE-17361
> Project: Hive
> Issue Type: New Feature
> Components: Transactions
>Reporter: Wei Zheng
>Assignee: Eugene Koifman
>Priority: Critical
> Attachments: HIVE-17361.07.patch, HIVE-17361.08.patch,
> HIVE-17361.09.patch, HIVE-17361.1.patch, HIVE-17361.10.patch,
> HIVE-17361.2.patch, HIVE-17361.3.patch, HIVE-17361.4.patch
>
>
> LOAD DATA was not supported since ACID was introduced. Need to fill this gap
> between ACID table and regular hive table.
> Current Documentation is under [DML
> Operations|https://cwiki.apache.org/confluence/display/Hive/GettingStarted#GettingStarted-DMLOperations]
> and [Loading files into
> tables|https://cwiki.apache.org/confluence/display/Hive/LanguageManual+DML#LanguageManualDML-Loadingfilesintotables]:
> \\
> * Load Data performs very limited validations of the data, in particular it
> uses the input file name which may not be in 0_0 which can break some
> read logic. (Certainly will for Acid).
> * It does not check the schema of the file. This may be a non issue for Acid
> which requires ORC which is self describing so Schema Evolution may handle
> this seamlessly. (Assuming Schema is not too different).
> * It does check that _InputFormat_S are compatible.
> * Bucketed (and thus sorted) tables don't support Load Data (but only if
> hive.strict.checks.bucketing=true (default)). Will keep this restriction for
> Acid.
> * Load Data supports OVERWRITE clause
> * What happens to file permissions/ownership: rename vs copy differences
> \\
> The implementation will follow the same idea as in HIVE-14988 and use a
>
[jira] [Updated] (HIVE-16756) Vectorization: LongColModuloLongColumn throws "java.lang.ArithmeticException: / by zero"
[ https://issues.apache.org/jira/browse/HIVE-16756?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vihang Karajgaonkar updated HIVE-16756: --- Attachment: HIVE-16756.01.patch > Vectorization: LongColModuloLongColumn throws "java.lang.ArithmeticException: > / by zero" > > > Key: HIVE-16756 > URL: https://issues.apache.org/jira/browse/HIVE-16756 > Project: Hive > Issue Type: Bug > Components: Hive >Affects Versions: 2.3.0 >Reporter: Matt McCline >Assignee: Vihang Karajgaonkar >Priority: Critical > Attachments: HIVE-16756.01.patch > > > vectorization_div0.q needs to test the long data type testing. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HIVE-16756) Vectorization: LongColModuloLongColumn throws "java.lang.ArithmeticException: / by zero"
[ https://issues.apache.org/jira/browse/HIVE-16756?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vihang Karajgaonkar updated HIVE-16756: --- Status: Patch Available (was: Open) > Vectorization: LongColModuloLongColumn throws "java.lang.ArithmeticException: > / by zero" > > > Key: HIVE-16756 > URL: https://issues.apache.org/jira/browse/HIVE-16756 > Project: Hive > Issue Type: Bug > Components: Hive >Affects Versions: 2.3.0 >Reporter: Matt McCline >Assignee: Vihang Karajgaonkar >Priority: Critical > Attachments: HIVE-16756.01.patch > > > vectorization_div0.q needs to test the long data type testing. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Comment Edited] (HIVE-17714) move custom SerDe schema considerations into metastore from QL
[ https://issues.apache.org/jira/browse/HIVE-17714?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16252091#comment-16252091 ] Sergey Shelukhin edited comment on HIVE-17714 at 11/14/17 8:10 PM: --- Let me summarize on the high level. Metastore not creating the schema for such SerDes is the current state after HIVE-11985 However, that means that most metastore APIs return bogus fields for such tables (only get_schema/get_fields return correct fields - by calling the deserializer inside metastore). So that means that everyone who wants to use metastore for such tables needs to know about these shennanigans (in particular the internal Hive SerDe list from HiveConf, and the fromDeserializer stuff). And also, metastore compile-time depends on SerDe interface and runtime-depends on SerDe jars. We can resolve this either by either: # removing the SerDe dependency, and either ## screwing everyone who wants to read Hive tables without intricate understanding of SerDe/Hive internals. I know for sure that it will break Presto, but I suspect it will actually break everyone trying to use metastore at this time :) And I'm not even sure how non-Java users can support this. ## forcing the table creation and updates to externally recreate the schema for the benefit of the readers. This is not as bad as messing with readers, cause those tables are mostly created by Hive, but still bad (if external users do create the tables) and also doesn't solve the external schema case. # keeping the SerDe dependency ## recreating the schema. The old metastore approach before HIVE-11985 that nobody seems to like. ## changing get_table/etc. APIs to return the correct schema from SerDe (with in-memory caching for most cases, based on internal config?). The last one IMO is the right solution. At compile time, the main dependency that metastore would need is "Deserializer" interface, not individual SerDes, so it's a reasonable addition to storage-api (or a new module). At runtime, I think it's reasonable to expect the user to deploy jars with metastore if they want to use the table, since they'd likely need the same jars anyway to read from the table using the SerDe (although it does present some inconvenience to non-Java readers). Also, if jars are not available we can output an error; and we can optionally add a compat flag for users that are aware of Hive internals and can override the jar requirement. was (Author: sershe): Let me summarize on the high level. Metastore not creating the schema for such SerDes is the current state after HIVE-11985 However, that means that most metastore APIs return bogus fields for such tables (only get_schema/get_fields return correct fields - by calling the deserializer inside metastore). So that means that everyone who wants to use metastore for such tables needs to know about these shennanigans (in particular the internal Hive SerDe list from HiveConf, and the fromDeserializer stuff). And also, metastore compile-time depends on SerDe interface and runtime-depends on SerDe jars. We can resolve this either by either: # removing the SerDe dependency, and either #.# screwing everyone who wants to read Hive tables without intricate understanding of SerDe/Hive internals. I know for sure that it will break Presto, but I suspect it will actually break everyone trying to use metastore at this time :) And I'm not even sure how non-Java users can support this. #.# forcing the table creation and updates to externally recreate the schema for the benefit of the readers. This is not as bad as messing with readers, cause those tables are mostly created by Hive, but still bad (if external users do create the tables) and also doesn't solve the external schema case. # keeping the SerDe dependency #.# recreating the schema. The old metastore approach before HIVE-11985 that nobody seems to like. ## changing get_table/etc. APIs to return the correct schema from SerDe (with in-memory caching for most cases, based on internal config?). This IMO is the right solution. At compile time, the main dependency that metastore would need is "Deserializer" interface, not individual SerDes, so it's a reasonable addition to storage-api (or a new module). At runtime, I think it's reasonable to expect the user to deploy jars with metastore if they want to use the table, since they'd likely need the same jars anyway to read from the table using the SerDe (although it does present some inconvenience to non-Java readers). Also, if jars are not available we can output an error; and we can optionally add a compat flag for users that are aware of Hive internals and can override the jar requirement. > move custom SerDe schema considerations into metastore from QL > -- > > Key: HIVE-17714 > URL:
[jira] [Comment Edited] (HIVE-17714) move custom SerDe schema considerations into metastore from QL
[ https://issues.apache.org/jira/browse/HIVE-17714?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16252091#comment-16252091 ] Sergey Shelukhin edited comment on HIVE-17714 at 11/14/17 8:09 PM: --- Let me summarize on the high level. Metastore not creating the schema for such SerDes is the current state after HIVE-11985 However, that means that most metastore APIs return bogus fields for such tables (only get_schema/get_fields return correct fields - by calling the deserializer inside metastore). So that means that everyone who wants to use metastore for such tables needs to know about these shennanigans (in particular the internal Hive SerDe list from HiveConf, and the fromDeserializer stuff). And also, metastore compile-time depends on SerDe interface and runtime-depends on SerDe jars. We can resolve this either by either: # removing the SerDe dependency, and either #.# screwing everyone who wants to read Hive tables without intricate understanding of SerDe/Hive internals. I know for sure that it will break Presto, but I suspect it will actually break everyone trying to use metastore at this time :) And I'm not even sure how non-Java users can support this. #.# forcing the table creation and updates to externally recreate the schema for the benefit of the readers. This is not as bad as messing with readers, cause those tables are mostly created by Hive, but still bad (if external users do create the tables) and also doesn't solve the external schema case. # keeping the SerDe dependency #.# recreating the schema. The old metastore approach before HIVE-11985 that nobody seems to like. ## changing get_table/etc. APIs to return the correct schema from SerDe (with in-memory caching for most cases, based on internal config?). This IMO is the right solution. At compile time, the main dependency that metastore would need is "Deserializer" interface, not individual SerDes, so it's a reasonable addition to storage-api (or a new module). At runtime, I think it's reasonable to expect the user to deploy jars with metastore if they want to use the table, since they'd likely need the same jars anyway to read from the table using the SerDe (although it does present some inconvenience to non-Java readers). Also, if jars are not available we can output an error; and we can optionally add a compat flag for users that are aware of Hive internals and can override the jar requirement. was (Author: sershe): Let me summarize on the high level. Metastore not creating the schema for such SerDes is the current state after HIVE-11985 However, that means that most metastore APIs return bogus fields for such tables (only get_schema/get_fields return correct fields - by calling the deserializer inside metastore). So that means that everyone who wants to use metastore for such tables needs to know about these shennanigans (in particular the internal Hive SerDe list from HiveConf, and the fromDeserializer stuff). And also, metastore compile-time depends on SerDe interface and runtime-depends on SerDe jars. We can resolve this either by either: # removing the SerDe dependency, and either #.# screwing everyone who wants to read Hive tables without intricate understanding of SerDe/Hive internals. I know for sure that it will break Presto, but I suspect it will actually break everyone trying to use metastore at this time :) And I'm not even sure how non-Java users can support this. #.# forcing the table creation and updates to externally recreate the schema for the benefit of the readers. This is not as bad as messing with readers, cause those tables are mostly created by Hive, but still bad (if external users do create the tables) and also doesn't solve the external schema case. # keeping the SerDe dependency #.# recreating the schema. The old metastore approach before HIVE-11985 that nobody seems to like. #.# changing get_table/etc. APIs to return the correct schema from SerDe (with in-memory caching for most cases, based on internal config?). This IMO is the right solution. At compile time, the main dependency that metastore would need is "Deserializer" interface, not individual SerDes, so it's a reasonable addition to storage-api (or a new module). At runtime, I think it's reasonable to expect the user to deploy jars with metastore if they want to use the table, since they'd likely need the same jars anyway to read from the table using the SerDe (although it does present some inconvenience to non-Java readers). Also, if jars are not available we can output an error; and we can optionally add a compat flag for users that are aware of Hive internals and can override the jar requirement. > move custom SerDe schema considerations into metastore from QL > -- > > Key: HIVE-17714 > URL:
[jira] [Commented] (HIVE-17714) move custom SerDe schema considerations into metastore from QL
[ https://issues.apache.org/jira/browse/HIVE-17714?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16252091#comment-16252091 ] Sergey Shelukhin commented on HIVE-17714: - Let me summarize on the high level. Metastore not creating the schema for such SerDes is the current state after HIVE-11985 However, that means that most metastore APIs return bogus fields for such tables (only get_schema/get_fields return correct fields - by calling the deserializer inside metastore). So that means that everyone who wants to use metastore for such tables needs to know about these shennanigans (in particular the internal Hive SerDe list from HiveConf, and the fromDeserializer stuff). And also, metastore compile-time depends on SerDe interface and runtime-depends on SerDe jars. We can resolve this either by either: # removing the SerDe dependency, and either #.# screwing everyone who wants to read Hive tables without intricate understanding of SerDe/Hive internals. I know for sure that it will break Presto, but I suspect it will actually break everyone trying to use metastore at this time :) And I'm not even sure how non-Java users can support this. #.# forcing the table creation and updates to externally recreate the schema for the benefit of the readers. This is not as bad as messing with readers, cause those tables are mostly created by Hive, but still bad (if external users do create the tables) and also doesn't solve the external schema case. # keeping the SerDe dependency #.# recreating the schema. The old metastore approach before HIVE-11985 that nobody seems to like. #.# changing get_table/etc. APIs to return the correct schema from SerDe (with in-memory caching for most cases, based on internal config?). This IMO is the right solution. At compile time, the main dependency that metastore would need is "Deserializer" interface, not individual SerDes, so it's a reasonable addition to storage-api (or a new module). At runtime, I think it's reasonable to expect the user to deploy jars with metastore if they want to use the table, since they'd likely need the same jars anyway to read from the table using the SerDe (although it does present some inconvenience to non-Java readers). Also, if jars are not available we can output an error; and we can optionally add a compat flag for users that are aware of Hive internals and can override the jar requirement. > move custom SerDe schema considerations into metastore from QL > -- > > Key: HIVE-17714 > URL: https://issues.apache.org/jira/browse/HIVE-17714 > Project: Hive > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Alan Gates > > Columns in metastore for tables that use external schema don't have the type > information (since HIVE-11985) and may be entirely inconsistent (since > forever, due to issues like HIVE-17713; or for SerDes that allow an URL for > the schema, due to a change in the underlying file). > Currently, if you trace the usage of ConfVars.SERDESUSINGMETASTOREFORSCHEMA, > and to MetaStoreUtils.getFieldsFromDeserializer, you'd see that the code in > QL handles this in Hive. So, for the most part metastore just returns > whatever is stored for columns in the database. > One exception appears to be get_fields_with_environment_context, which is > interesting... so getTable will return incorrect columns (potentially), but > get_fields/get_schema will return correct ones from SerDe as far as I can > tell. > As part of separating the metastore, we should make sure all the APIs return > the correct schema for the columns; it's not a good idea to have everyone > reimplement getFieldsFromDeserializer. > Note: this should also remove a flag introduced in HIVE-17731 -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HIVE-15018) ALTER rewriting flag in materialized view
[
https://issues.apache.org/jira/browse/HIVE-15018?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jesus Camacho Rodriguez updated HIVE-15018:
---
Attachment: HIVE-15018.patch
> ALTER rewriting flag in materialized view
> --
>
> Key: HIVE-15018
> URL: https://issues.apache.org/jira/browse/HIVE-15018
> Project: Hive
> Issue Type: Sub-task
> Components: Materialized views
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
> Attachments: HIVE-15018.patch
>
>
> We should extend the ALTER statement in case we want to change the rewriting
> behavior of the materialized view after we have created it.
> {code:sql}
> ALTER MATERIALIZED VIEW [db_name.]materialized_view_name DISABLE REWRITE;
> {code}
> {code:sql}
> ALTER MATERIALIZED VIEW [db_name.]materialized_view_name ENABLE REWRITE;
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HIVE-15018) ALTER rewriting flag in materialized view
[
https://issues.apache.org/jira/browse/HIVE-15018?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jesus Camacho Rodriguez updated HIVE-15018:
---
Status: Patch Available (was: In Progress)
> ALTER rewriting flag in materialized view
> --
>
> Key: HIVE-15018
> URL: https://issues.apache.org/jira/browse/HIVE-15018
> Project: Hive
> Issue Type: Sub-task
> Components: Materialized views
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
>
> We should extend the ALTER statement in case we want to change the rewriting
> behavior of the materialized view after we have created it.
> {code:sql}
> ALTER MATERIALIZED VIEW [db_name.]materialized_view_name DISABLE REWRITE;
> {code}
> {code:sql}
> ALTER MATERIALIZED VIEW [db_name.]materialized_view_name ENABLE REWRITE;
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Work started] (HIVE-15018) ALTER rewriting flag in materialized view
[
https://issues.apache.org/jira/browse/HIVE-15018?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Work on HIVE-15018 started by Jesus Camacho Rodriguez.
--
> ALTER rewriting flag in materialized view
> --
>
> Key: HIVE-15018
> URL: https://issues.apache.org/jira/browse/HIVE-15018
> Project: Hive
> Issue Type: Sub-task
> Components: Materialized views
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
>
> We should extend the ALTER statement in case we want to change the rewriting
> behavior of the materialized view after we have created it.
> {code:sql}
> ALTER MATERIALIZED VIEW [db_name.]materialized_view_name DISABLE REWRITE;
> {code}
> {code:sql}
> ALTER MATERIALIZED VIEW [db_name.]materialized_view_name ENABLE REWRITE;
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HIVE-17651) TableScanOperator might miss vectorization on flag
[ https://issues.apache.org/jira/browse/HIVE-17651?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zoltan Haindrich updated HIVE-17651: Description: https://github.com/apache/hive/blob/3bfcfdde0c0be2aab1afdf5b1bc71fdcc9e77360/ql/src/java/org/apache/hadoop/hive/ql/exec/TableScanOperator.java#L259-L273 was: It seems to me that there might be some overhead when the operator is gathering the stats; also check: it seems that in case vectorization is on ; the collected rowcount would be invalid > TableScanOperator might miss vectorization on flag > -- > > Key: HIVE-17651 > URL: https://issues.apache.org/jira/browse/HIVE-17651 > Project: Hive > Issue Type: Sub-task >Reporter: Zoltan Haindrich >Assignee: Zoltan Haindrich > Attachments: HIVE-17651.01ex.patch, HIVE-17651.02rc.patch, > HIVE-17651.02rc.patch, HIVE-17651.03.patch > > > https://github.com/apache/hive/blob/3bfcfdde0c0be2aab1afdf5b1bc71fdcc9e77360/ql/src/java/org/apache/hadoop/hive/ql/exec/TableScanOperator.java#L259-L273 -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HIVE-17651) TableScanOperator might miss vectorization on flag
[ https://issues.apache.org/jira/browse/HIVE-17651?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zoltan Haindrich updated HIVE-17651: Summary: TableScanOperator might miss vectorization on flag (was: Profile TableScanOperator in case stats gathering is on) > TableScanOperator might miss vectorization on flag > -- > > Key: HIVE-17651 > URL: https://issues.apache.org/jira/browse/HIVE-17651 > Project: Hive > Issue Type: Sub-task >Reporter: Zoltan Haindrich >Assignee: Zoltan Haindrich > Attachments: HIVE-17651.01ex.patch, HIVE-17651.02rc.patch, > HIVE-17651.02rc.patch, HIVE-17651.03.patch > > > It seems to me that there might be some overhead when the operator is > gathering the stats; > also check: it seems that in case vectorization is on ; the collected > rowcount would be invalid -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HIVE-17528) Add more q-tests for Hive-on-Spark with Parquet vectorized reader
[
https://issues.apache.org/jira/browse/HIVE-17528?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16252005#comment-16252005
]
Hive QA commented on HIVE-17528:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12897570/HIVE-17528.3.patch
{color:green}SUCCESS:{color} +1 due to 30 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 12 failed/errored test(s), 11439 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[auto_sortmerge_join_2]
(batchId=48)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[dbtxnmgr_showlocks]
(batchId=78)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[unionDistinct_1]
(batchId=149)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[insert_values_orig_table_use_metadata]
(batchId=165)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[ppd_union_view]
(batchId=157)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[sysdb]
(batchId=159)
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[explainanalyze_2]
(batchId=103)
org.apache.hadoop.hive.cli.TestNegativeMinimrCliDriver.testCliDriver[ct_noperm_loc]
(batchId=95)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[subquery_multi]
(batchId=113)
org.apache.hadoop.hive.cli.control.TestDanglingQOuts.checkDanglingQOut
(batchId=209)
org.apache.hadoop.hive.ql.exec.tez.TestWorkloadManager.testApplyPlanQpChanges
(batchId=284)
org.apache.hadoop.hive.ql.parse.TestReplicationScenarios.testConstraints
(batchId=226)
{noformat}
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/7806/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/7806/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-7806/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 12 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12897570 - PreCommit-HIVE-Build
> Add more q-tests for Hive-on-Spark with Parquet vectorized reader
> -
>
> Key: HIVE-17528
> URL: https://issues.apache.org/jira/browse/HIVE-17528
> Project: Hive
> Issue Type: Sub-task
>Reporter: Vihang Karajgaonkar
>Assignee: Ferdinand Xu
> Attachments: HIVE-17528.1.patch, HIVE-17528.2.patch,
> HIVE-17528.3.patch, HIVE-17528.patch
>
>
> Most of the vectorization related q-tests operate on ORC tables using Tez. It
> would be good to add more coverage on a different combination of engine and
> file-format. We can model existing q-tests using parquet tables and run it
> using TestSparkCliDriver
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HIVE-17651) Profile TableScanOperator in case stats gathering is on
[ https://issues.apache.org/jira/browse/HIVE-17651?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zoltan Haindrich updated HIVE-17651: Attachment: HIVE-17651.03.patch > Profile TableScanOperator in case stats gathering is on > --- > > Key: HIVE-17651 > URL: https://issues.apache.org/jira/browse/HIVE-17651 > Project: Hive > Issue Type: Sub-task >Reporter: Zoltan Haindrich >Assignee: Zoltan Haindrich > Attachments: HIVE-17651.01ex.patch, HIVE-17651.02rc.patch, > HIVE-17651.02rc.patch, HIVE-17651.03.patch > > > It seems to me that there might be some overhead when the operator is > gathering the stats; > also check: it seems that in case vectorization is on ; the collected > rowcount would be invalid -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HIVE-17651) Profile TableScanOperator in case stats gathering is on
[ https://issues.apache.org/jira/browse/HIVE-17651?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16251912#comment-16251912 ] Jesus Camacho Rodriguez commented on HIVE-17651: [~kgyrtkirk], it seems to me that the vectorization flag should be set before the early bail out (so instead of removing the early bail out, just set the flag before it). However, I am not sure now how this can be tested / why no tests failed before because of this. Are any of the ptest failures above related to your patch? > Profile TableScanOperator in case stats gathering is on > --- > > Key: HIVE-17651 > URL: https://issues.apache.org/jira/browse/HIVE-17651 > Project: Hive > Issue Type: Sub-task >Reporter: Zoltan Haindrich >Assignee: Zoltan Haindrich > Attachments: HIVE-17651.01ex.patch, HIVE-17651.02rc.patch, > HIVE-17651.02rc.patch > > > It seems to me that there might be some overhead when the operator is > gathering the stats; > also check: it seems that in case vectorization is on ; the collected > rowcount would be invalid -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HIVE-18041) Add SORT_QUERY_RESULTS to subquery_multi
[
https://issues.apache.org/jira/browse/HIVE-18041?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16251864#comment-16251864
]
Hive QA commented on HIVE-18041:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12897323/HIVE-18041.1.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 9 failed/errored test(s), 11383 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[dbtxnmgr_showlocks]
(batchId=77)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[unionDistinct_1]
(batchId=146)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[insert_values_orig_table_use_metadata]
(batchId=162)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[ppd_union_view]
(batchId=154)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[sysdb]
(batchId=156)
org.apache.hadoop.hive.cli.TestNegativeMinimrCliDriver.testCliDriver[ct_noperm_loc]
(batchId=94)
org.apache.hadoop.hive.cli.control.TestDanglingQOuts.checkDanglingQOut
(batchId=206)
org.apache.hadoop.hive.ql.exec.tez.TestWorkloadManager.testApplyPlanQpChanges
(batchId=281)
org.apache.hadoop.hive.ql.parse.TestReplicationScenarios.testConstraints
(batchId=223)
{noformat}
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/7805/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/7805/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-7805/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 9 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12897323 - PreCommit-HIVE-Build
> Add SORT_QUERY_RESULTS to subquery_multi
>
>
> Key: HIVE-18041
> URL: https://issues.apache.org/jira/browse/HIVE-18041
> Project: Hive
> Issue Type: Test
>Reporter: Rui Li
>Assignee: Rui Li
>Priority: Trivial
> Attachments: HIVE-18041.1.patch
>
>
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HIVE-14495) Add SHOW MATERIALIZED VIEWS statement
[
https://issues.apache.org/jira/browse/HIVE-14495?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jesus Camacho Rodriguez updated HIVE-14495:
---
Attachment: HIVE-14495.01.patch
> Add SHOW MATERIALIZED VIEWS statement
> -
>
> Key: HIVE-14495
> URL: https://issues.apache.org/jira/browse/HIVE-14495
> Project: Hive
> Issue Type: Sub-task
> Components: Materialized views
>Affects Versions: 2.2.0
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
> Attachments: HIVE-14495.01.patch, HIVE-14495.patch
>
>
> In the spirit of {{SHOW TABLES}}, we should support the following statement:
> {code:sql}
> SHOW MATERIALIZED VIEWS [IN database_name] ['identifier_with_wildcards'];
> {code}
> In contrast to {{SHOW TABLES}}, this command would only list the materialized
> views.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HIVE-15680) Incorrect results when hive.optimize.index.filter=true and same ORC table is referenced twice in query
[
https://issues.apache.org/jira/browse/HIVE-15680?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16251834#comment-16251834
]
Prasanth Jayachandran commented on HIVE-15680:
--
[~niklaus.xiao] made some valid points in RB. Set will change order of needed
columns/types. Also the case where aliases require different needed columns
does not look correct. Added one more comment to RB.
> Incorrect results when hive.optimize.index.filter=true and same ORC table is
> referenced twice in query
> --
>
> Key: HIVE-15680
> URL: https://issues.apache.org/jira/browse/HIVE-15680
> Project: Hive
> Issue Type: Bug
>Affects Versions: 1.1.0, 2.2.0
>Reporter: Anthony Hsu
>Assignee: Anthony Hsu
> Attachments: HIVE-15680.1.patch, HIVE-15680.2.patch,
> HIVE-15680.3.patch, HIVE-15680.4.patch, HIVE-15680.5.patch, HIVE-15680.6.patch
>
>
> To repro:
> {noformat}
> set hive.optimize.index.filter=true;
> create table test_table(number int) stored as ORC;
> -- Two insertions will create two files, with one stripe each
> insert into table test_table VALUES (1);
> insert into table test_table VALUES (2);
> -- This should and does return 2 records
> select * from test_table;
> -- These should and do each return 1 record
> select * from test_table where number = 1;
> select * from test_table where number = 2;
> -- This should return 2 records but only returns 1 record
> select * from test_table where number = 1
> union all
> select * from test_table where number = 2;
> {noformat}
> What's happening is only the last predicate is being pushed down.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HIVE-17373) Upgrade some dependency versions
[ https://issues.apache.org/jira/browse/HIVE-17373?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aihua Xu updated HIVE-17373: Status: Patch Available (was: Reopened) > Upgrade some dependency versions > > > Key: HIVE-17373 > URL: https://issues.apache.org/jira/browse/HIVE-17373 > Project: Hive > Issue Type: Improvement > Components: Hive >Affects Versions: 3.0.0 >Reporter: Aihua Xu >Assignee: Aihua Xu > Fix For: 3.0.0 > > Attachments: HIVE-17373.1.patch, HIVE-17373.2.patch, > HIVE-17373.3.patch > > > Upgrade some libraries including log4j to 2.8.2, accumulo to 1.8.1 and > commons-httpclient to 3.1. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HIVE-17373) Upgrade some dependency versions
[ https://issues.apache.org/jira/browse/HIVE-17373?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aihua Xu updated HIVE-17373: Fix Version/s: (was: 3.0.0) > Upgrade some dependency versions > > > Key: HIVE-17373 > URL: https://issues.apache.org/jira/browse/HIVE-17373 > Project: Hive > Issue Type: Improvement > Components: Hive >Affects Versions: 3.0.0 >Reporter: Aihua Xu >Assignee: Aihua Xu > Attachments: HIVE-17373.1.patch, HIVE-17373.2.patch, > HIVE-17373.3.patch > > > Upgrade some libraries including log4j to 2.8.2, accumulo to 1.8.1 and > commons-httpclient to 3.1. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HIVE-17373) Upgrade some dependency versions
[ https://issues.apache.org/jira/browse/HIVE-17373?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aihua Xu updated HIVE-17373: Attachment: HIVE-17373.3.patch > Upgrade some dependency versions > > > Key: HIVE-17373 > URL: https://issues.apache.org/jira/browse/HIVE-17373 > Project: Hive > Issue Type: Improvement > Components: Hive >Affects Versions: 3.0.0 >Reporter: Aihua Xu >Assignee: Aihua Xu > Attachments: HIVE-17373.1.patch, HIVE-17373.2.patch, > HIVE-17373.3.patch > > > Upgrade some libraries including log4j to 2.8.2, accumulo to 1.8.1 and > commons-httpclient to 3.1. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HIVE-17583) Fix test failure TestAccumuloCliDriver caused from the accumulo version upgrade
[ https://issues.apache.org/jira/browse/HIVE-17583?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aihua Xu updated HIVE-17583: Attachment: HIVE-17373.3.patch > Fix test failure TestAccumuloCliDriver caused from the accumulo version > upgrade > --- > > Key: HIVE-17583 > URL: https://issues.apache.org/jira/browse/HIVE-17583 > Project: Hive > Issue Type: Test > Components: Test >Affects Versions: 3.0.0 >Reporter: Aihua Xu >Assignee: Aihua Xu > Attachments: HIVE-17583.1.patch > > > HIVE-17373 increases the accumulo version and it's causing the test failure > for TestAccumuloCliDriver -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HIVE-17583) Fix test failure TestAccumuloCliDriver caused from the accumulo version upgrade
[ https://issues.apache.org/jira/browse/HIVE-17583?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aihua Xu updated HIVE-17583: Attachment: (was: HIVE-17373.3.patch) > Fix test failure TestAccumuloCliDriver caused from the accumulo version > upgrade > --- > > Key: HIVE-17583 > URL: https://issues.apache.org/jira/browse/HIVE-17583 > Project: Hive > Issue Type: Test > Components: Test >Affects Versions: 3.0.0 >Reporter: Aihua Xu >Assignee: Aihua Xu > Attachments: HIVE-17583.1.patch > > > HIVE-17373 increases the accumulo version and it's causing the test failure > for TestAccumuloCliDriver -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HIVE-18046) Metastore: default IS_REWRITE_ENABLED=false instead of NULL
[
https://issues.apache.org/jira/browse/HIVE-18046?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jesus Camacho Rodriguez updated HIVE-18046:
---
Attachment: HIVE-18046.01.patch
Small fix in Derby upgrade scripts (DDL syntax).
> Metastore: default IS_REWRITE_ENABLED=false instead of NULL
> ---
>
> Key: HIVE-18046
> URL: https://issues.apache.org/jira/browse/HIVE-18046
> Project: Hive
> Issue Type: Bug
> Components: Materialized views, Metastore
>Affects Versions: 3.0.0
>Reporter: Gopal V
>Assignee: Jesus Camacho Rodriguez
>Priority: Minor
> Attachments: HIVE-18046.01.patch, HIVE-18046.patch
>
>
> The materialized view impl breaks old metastore sql write access, by
> complaining that the new table creation does not set this column up.
> {code}
> `IS_REWRITE_ENABLED` bit(1) NOT NULL,
> {code}
> {{NOT NULL DEFAULT 0}} would allow old metastore direct sql compatibility
> (not thrift).
> {code}
> 2017-11-09T07:11:58,331 ERROR [HiveServer2-Background-Pool: Thread-2354]
> metastore.RetryingHMSHandler: Retrying HMSHandler after 2000 ms (attempt 1 of
> 10) with error: javax.jdo.JDODataStoreException: Insert of object
> "org.apache.hadoop.hive.metastore.model.MTable@249dbf1" using statement
> "INSERT INTO `TBLS`
> (`TBL_ID`,`CREATE_TIME`,`DB_ID`,`LAST_ACCESS_TIME`,`OWNER`,`RETENTION`,`SD_ID`,`TBL_NAME`,`TBL_TYPE`,`VIEW_EXPANDED_TEXT`,`VIEW_ORIGINAL_TEXT`)
> VALUES (?,?,?,?,?,?,?,?,?,?,?)" failed : Field 'IS_REWRITE_ENABLED' doesn't
> have a default value
> at
> org.datanucleus.api.jdo.NucleusJDOHelper.getJDOExceptionForNucleusException(NucleusJDOHelper.java:543)
> at
> org.datanucleus.api.jdo.JDOPersistenceManager.jdoMakePersistent(JDOPersistenceManager.java:720)
> at
> org.datanucleus.api.jdo.JDOPersistenceManager.makePersistent(JDOPersistenceManager.java:740)
> at
> org.apache.hadoop.hive.metastore.ObjectStore.createTable(ObjectStore.java:1038)
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HIVE-17714) move custom SerDe schema considerations into metastore from QL
[ https://issues.apache.org/jira/browse/HIVE-17714?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16251811#comment-16251811 ] Owen O'Malley commented on HIVE-17714: -- [~alangates] Thanks for watching out for adding dependencies to storage-api. Adding JSON and Avro as recursive dependencies for storage-api would be really painful. Minimizing the size of storage-api also means that fewer changes cross the hive to storage-api artifacts. > move custom SerDe schema considerations into metastore from QL > -- > > Key: HIVE-17714 > URL: https://issues.apache.org/jira/browse/HIVE-17714 > Project: Hive > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Alan Gates > > Columns in metastore for tables that use external schema don't have the type > information (since HIVE-11985) and may be entirely inconsistent (since > forever, due to issues like HIVE-17713; or for SerDes that allow an URL for > the schema, due to a change in the underlying file). > Currently, if you trace the usage of ConfVars.SERDESUSINGMETASTOREFORSCHEMA, > and to MetaStoreUtils.getFieldsFromDeserializer, you'd see that the code in > QL handles this in Hive. So, for the most part metastore just returns > whatever is stored for columns in the database. > One exception appears to be get_fields_with_environment_context, which is > interesting... so getTable will return incorrect columns (potentially), but > get_fields/get_schema will return correct ones from SerDe as far as I can > tell. > As part of separating the metastore, we should make sure all the APIs return > the correct schema for the columns; it's not a good idea to have everyone > reimplement getFieldsFromDeserializer. > Note: this should also remove a flag introduced in HIVE-17731 -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HIVE-17714) move custom SerDe schema considerations into metastore from QL
[ https://issues.apache.org/jira/browse/HIVE-17714?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16251806#comment-16251806 ] Owen O'Malley commented on HIVE-17714: -- I'm also -1 to the metastore using the Serdes to recreate the table schema. The Avro serde is particularly bad in this regard because it can use an external file to store the schema. Thus, the schema of the table can change without notifying the metastore. That is pretty broken. Does anyone know what the original goal of that capability was? I think the long term goal should be to make "load data" should determine if the type is self-describing and invoke an interface to determine the types of the loaded data. For managed tables, the metastore needs to know the types of the tables. The goal should be to remove the functions that allow users to update the data directly without going through Hive. The metastore needs to know the types and have relevant statistics. That is the only way the optimizer has a chance of figuring out the proper plan. > move custom SerDe schema considerations into metastore from QL > -- > > Key: HIVE-17714 > URL: https://issues.apache.org/jira/browse/HIVE-17714 > Project: Hive > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Alan Gates > > Columns in metastore for tables that use external schema don't have the type > information (since HIVE-11985) and may be entirely inconsistent (since > forever, due to issues like HIVE-17713; or for SerDes that allow an URL for > the schema, due to a change in the underlying file). > Currently, if you trace the usage of ConfVars.SERDESUSINGMETASTOREFORSCHEMA, > and to MetaStoreUtils.getFieldsFromDeserializer, you'd see that the code in > QL handles this in Hive. So, for the most part metastore just returns > whatever is stored for columns in the database. > One exception appears to be get_fields_with_environment_context, which is > interesting... so getTable will return incorrect columns (potentially), but > get_fields/get_schema will return correct ones from SerDe as far as I can > tell. > As part of separating the metastore, we should make sure all the APIs return > the correct schema for the columns; it's not a good idea to have everyone > reimplement getFieldsFromDeserializer. > Note: this should also remove a flag introduced in HIVE-17731 -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HIVE-15883) HBase mapped table in Hive insert fail for decimal
[
https://issues.apache.org/jira/browse/HIVE-15883?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16251776#comment-16251776
]
Ashutosh Chauhan commented on HIVE-15883:
-
We have tests for hbase-handler. Both junit as well qfile. You may use either.
> HBase mapped table in Hive insert fail for decimal
> --
>
> Key: HIVE-15883
> URL: https://issues.apache.org/jira/browse/HIVE-15883
> Project: Hive
> Issue Type: Bug
> Components: Hive
>Affects Versions: 1.1.0
>Reporter: Naveen Gangam
>Assignee: Naveen Gangam
> Attachments: HIVE-15883.patch
>
>
> CREATE TABLE hbase_table (
> id int,
> balance decimal(15,2))
> ROW FORMAT DELIMITED
> COLLECTION ITEMS TERMINATED BY '~'
> STORED BY 'org.apache.hadoop.hive.hbase.HBaseStorageHandler'
> WITH SERDEPROPERTIES (
> "hbase.columns.mapping"=":key,cf:balance#b");
> insert into hbase_table values (1,1);
>
> Diagnostic Messages for this Task:
> Error: java.lang.RuntimeException:
> org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while
> processing row {"tmp_values_col1":"1","tmp_values_col2":"1"}
> at
> org.apache.hadoop.hive.ql.exec.mr.ExecMapper.map(ExecMapper.java:179)
> at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:54)
> at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:453)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:343)
> at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:164)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:415)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1783)
> at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)
> Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime
> Error while processing row {"tmp_values_col1":"1","tmp_values_col2":"1"}
> at
> org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:507)
> at
> org.apache.hadoop.hive.ql.exec.mr.ExecMapper.map(ExecMapper.java:170)
> ... 8 more
> Caused by: org.apache.hadoop.hive.ql.metadata.HiveException:
> org.apache.hadoop.hive.serde2.SerDeException: java.lang.RuntimeException:
> Hive internal error.
> at
> org.apache.hadoop.hive.ql.exec.FileSinkOperator.processOp(FileSinkOperator.java:733)
> at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:815)
> at
> org.apache.hadoop.hive.ql.exec.SelectOperator.processOp(SelectOperator.java:84)
> at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:815)
> at
> org.apache.hadoop.hive.ql.exec.TableScanOperator.processOp(TableScanOperator.java:97)
> at
> org.apache.hadoop.hive.ql.exec.MapOperator$MapOpCtx.forward(MapOperator.java:157)
> at
> org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:497)
> ... 9 more
> Caused by: org.apache.hadoop.hive.serde2.SerDeException:
> java.lang.RuntimeException: Hive internal error.
> at
> org.apache.hadoop.hive.hbase.HBaseSerDe.serialize(HBaseSerDe.java:286)
> at
> org.apache.hadoop.hive.ql.exec.FileSinkOperator.processOp(FileSinkOperator.java:668)
> ... 15 more
> Caused by: java.lang.RuntimeException: Hive internal error.
> at
> org.apache.hadoop.hive.serde2.lazy.LazyUtils.writePrimitive(LazyUtils.java:328)
> at
> org.apache.hadoop.hive.hbase.HBaseRowSerializer.serialize(HBaseRowSerializer.java:220)
> at
> org.apache.hadoop.hive.hbase.HBaseRowSerializer.serializeField(HBaseRowSerializer.java:194)
> at
> org.apache.hadoop.hive.hbase.HBaseRowSerializer.serialize(HBaseRowSerializer.java:118)
> at
> org.apache.hadoop.hive.hbase.HBaseSerDe.serialize(HBaseSerDe.java:282)
> ... 16 more
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HIVE-17934) Merging Statistics are promoted to COMPLETE (most of the time)
[ https://issues.apache.org/jira/browse/HIVE-17934?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16251771#comment-16251771 ] Ashutosh Chauhan commented on HIVE-17934: - +1 pending tests. > Merging Statistics are promoted to COMPLETE (most of the time) > -- > > Key: HIVE-17934 > URL: https://issues.apache.org/jira/browse/HIVE-17934 > Project: Hive > Issue Type: Sub-task > Components: Statistics >Reporter: Zoltan Haindrich >Assignee: Zoltan Haindrich > Attachments: HIVE-17934.01.patch, HIVE-17934.02.patch, > HIVE-17934.03.patch, HIVE-17934.04.patch, HIVE-17934.05.patch, > HIVE-17934.06.patch, HIVE-17934.06wip01.patch > > > in case multiple partition statistics are merged the STATS state is computed > based on the datasize and rowcount; > the merge may hide away non-existent stats in case there are other partition > or operators which do contribute to the datasize and the rowcount. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HIVE-17193) HoS: don't combine map works that are targets of different DPPs
[
https://issues.apache.org/jira/browse/HIVE-17193?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16251738#comment-16251738
]
Hive QA commented on HIVE-17193:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12897542/HIVE-17193.2.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 11 failed/errored test(s), 11383 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[auto_sortmerge_join_2]
(batchId=47)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[dbtxnmgr_showlocks]
(batchId=77)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[unionDistinct_1]
(batchId=146)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[insert_values_orig_table_use_metadata]
(batchId=162)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[ppd_union_view]
(batchId=154)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[sysdb]
(batchId=156)
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[explainanalyze_2]
(batchId=102)
org.apache.hadoop.hive.cli.TestNegativeMinimrCliDriver.testCliDriver[ct_noperm_loc]
(batchId=94)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[subquery_multi]
(batchId=111)
org.apache.hadoop.hive.cli.control.TestDanglingQOuts.checkDanglingQOut
(batchId=206)
org.apache.hadoop.hive.ql.parse.TestReplicationScenarios.testConstraints
(batchId=223)
{noformat}
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/7804/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/7804/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-7804/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 11 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12897542 - PreCommit-HIVE-Build
> HoS: don't combine map works that are targets of different DPPs
> ---
>
> Key: HIVE-17193
> URL: https://issues.apache.org/jira/browse/HIVE-17193
> Project: Hive
> Issue Type: Bug
> Components: Spark
>Reporter: Rui Li
>Assignee: Rui Li
> Attachments: HIVE-17193.1.patch, HIVE-17193.2.patch
>
>
> Suppose {{srcpart}} is partitioned by {{ds}}. The following query can trigger
> the issue:
> {code}
> explain
> select * from
> (select srcpart.ds,srcpart.key from srcpart join src on srcpart.ds=src.key)
> a
> join
> (select srcpart.ds,srcpart.key from srcpart join src on
> srcpart.ds=src.value) b
> on a.key=b.key;
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HIVE-17954) Implement pool, user, group and trigger to pool management API's.
[ https://issues.apache.org/jira/browse/HIVE-17954?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Harish Jaiprakash updated HIVE-17954: - Attachment: HIVE-17954.03.patch Thanks. Splitting the file fixed the problem, updated the patch with all the fixes. > Implement pool, user, group and trigger to pool management API's. > - > > Key: HIVE-17954 > URL: https://issues.apache.org/jira/browse/HIVE-17954 > Project: Hive > Issue Type: Sub-task >Reporter: Harish Jaiprakash >Assignee: Harish Jaiprakash > Attachments: HIVE-17954.01.patch, HIVE-17954.02.patch, > HIVE-17954.03.patch > > > Implement the following commands: > -- Pool management. > CREATE POOL `resource_plan`.`pool_path` WITH > ALLOC_FRACTION `fraction` > QUERY_PARALLELISM `parallelism` > SCHEDULING_POLICY `policy`; > ALTER POOL `resource_plan`.`pool_path` SET > PATH = `new_path`, > ALLOC_FRACTION = `fraction`, > QUERY_PARALLELISM = `parallelism`, > SCHEDULING_POLICY = `policy`; > DROP POOL `resource_plan`.`pool_path`; > -- Trigger to pool mappings. > ALTER RESOURCE PLAN `resource_plan` > ADD TRIGGER `trigger_name` TO `pool_path`; > ALTER RESOURCE PLAN `resource_plan` > DROP TRIGGER `trigger_name` TO `pool_path`; > -- User/Group to pool mappings. > CREATE USER|GROUP MAPPING `resource_plan`.`group_or_user_name` > TO `pool_path` WITH ORDERING `order_no`; > DROP USER|GROUP MAPPING `resource_plan`.`group_or_user_name`; -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HIVE-17651) Profile TableScanOperator in case stats gathering is on
[ https://issues.apache.org/jira/browse/HIVE-17651?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16251665#comment-16251665 ] Zoltan Haindrich commented on HIVE-17651: - [~jcamachorodriguez] I guess there is no issue here...but just in case; in the next link ; the stats gathering is *always* off; and because of that the boolean which would capture the vectorization state is never updated... https://github.com/apache/hive/blame/master/ql/src/java/org/apache/hadoop/hive/ql/exec/TableScanOperator.java#L259-L273 do you think this might cause any trouble; or just close this ticket? > Profile TableScanOperator in case stats gathering is on > --- > > Key: HIVE-17651 > URL: https://issues.apache.org/jira/browse/HIVE-17651 > Project: Hive > Issue Type: Sub-task >Reporter: Zoltan Haindrich >Assignee: Zoltan Haindrich > Attachments: HIVE-17651.01ex.patch, HIVE-17651.02rc.patch, > HIVE-17651.02rc.patch > > > It seems to me that there might be some overhead when the operator is > gathering the stats; > also check: it seems that in case vectorization is on ; the collected > rowcount would be invalid -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HIVE-18057) remove PostExecute / PreExecute hook support
[ https://issues.apache.org/jira/browse/HIVE-18057?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zoltan Haindrich updated HIVE-18057: Status: Patch Available (was: Open) > remove PostExecute / PreExecute hook support > > > Key: HIVE-18057 > URL: https://issues.apache.org/jira/browse/HIVE-18057 > Project: Hive > Issue Type: Sub-task >Reporter: Zoltan Haindrich >Assignee: Zoltan Haindrich > Attachments: HIVE-18057.01.patch > > > * deprecated since 2010 > * they are needlessly complicate the dispatch logic > * the current dispatch logic just silently accepts pre/post hooks if they > don't implement neither interface -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HIVE-18063) Make CommandProcessorResponse an exception instead of a return class
[
https://issues.apache.org/jira/browse/HIVE-18063?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zoltan Haindrich updated HIVE-18063:
Status: Patch Available (was: Open)
> Make CommandProcessorResponse an exception instead of a return class
>
>
> Key: HIVE-18063
> URL: https://issues.apache.org/jira/browse/HIVE-18063
> Project: Hive
> Issue Type: Sub-task
> Components: Logical Optimizer
>Reporter: Zoltan Haindrich
>Assignee: Zoltan Haindrich
> Attachments: HIVE-18063.01.patch
>
>
> the usage pattern of the {{CommandProcessorResponse}} class suggests that its
> current role is closer to Exceptions than to return values.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Assigned] (HIVE-18057) remove PostExecute / PreExecute hook support
[ https://issues.apache.org/jira/browse/HIVE-18057?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zoltan Haindrich reassigned HIVE-18057: --- Assignee: Zoltan Haindrich > remove PostExecute / PreExecute hook support > > > Key: HIVE-18057 > URL: https://issues.apache.org/jira/browse/HIVE-18057 > Project: Hive > Issue Type: Sub-task >Reporter: Zoltan Haindrich >Assignee: Zoltan Haindrich > Attachments: HIVE-18057.01.patch > > > * deprecated since 2010 > * they are needlessly complicate the dispatch logic > * the current dispatch logic just silently accepts pre/post hooks if they > don't implement neither interface -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HIVE-17932) Remove option to control partition level basic stats fetching
[
https://issues.apache.org/jira/browse/HIVE-17932?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16251632#comment-16251632
]
Zoltan Haindrich commented on HIVE-17932:
-
Thank you [~leftylev], I've updated the docs! :)
> Remove option to control partition level basic stats fetching
> -
>
> Key: HIVE-17932
> URL: https://issues.apache.org/jira/browse/HIVE-17932
> Project: Hive
> Issue Type: Improvement
> Components: Statistics
>Reporter: Zoltan Haindrich
>Assignee: Zoltan Haindrich
> Labels: TODOC3.0
> Fix For: 3.0.0
>
> Attachments: HIVE-17932.01.patch
>
>
> disabling the fetching of partition
> stats({{hive.stats.fetch.partition.stats}}) may cause problematic cases to
> arise for partitioned tables...the user might just want to disable the cbo
> instead tweaking the fetching of partition stats.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HIVE-17965) Remove HIVELIMITTABLESCANPARTITION support
[ https://issues.apache.org/jira/browse/HIVE-17965?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16251624#comment-16251624 ] Zoltan Haindrich commented on HIVE-17965: - Thank you [~leftylev]! I've updated the docs :) > Remove HIVELIMITTABLESCANPARTITION support > -- > > Key: HIVE-17965 > URL: https://issues.apache.org/jira/browse/HIVE-17965 > Project: Hive > Issue Type: Improvement >Reporter: Zoltan Haindrich >Assignee: Zoltan Haindrich >Priority: Trivial > Labels: TODOC3.0 > Fix For: 3.0.0 > > Attachments: HIVE-17965.01.patch > > > HIVE-13884 marked it as deprecated -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HIVE-15883) HBase mapped table in Hive insert fail for decimal
[
https://issues.apache.org/jira/browse/HIVE-15883?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16251617#comment-16251617
]
Naveen Gangam commented on HIVE-15883:
--
Thanks [~ashutoshc]. I will rebase the patch. Do you know if it is possible to
add a testcase for this as it would need HBase for it as well? Thanks
> HBase mapped table in Hive insert fail for decimal
> --
>
> Key: HIVE-15883
> URL: https://issues.apache.org/jira/browse/HIVE-15883
> Project: Hive
> Issue Type: Bug
> Components: Hive
>Affects Versions: 1.1.0
>Reporter: Naveen Gangam
>Assignee: Naveen Gangam
> Attachments: HIVE-15883.patch
>
>
> CREATE TABLE hbase_table (
> id int,
> balance decimal(15,2))
> ROW FORMAT DELIMITED
> COLLECTION ITEMS TERMINATED BY '~'
> STORED BY 'org.apache.hadoop.hive.hbase.HBaseStorageHandler'
> WITH SERDEPROPERTIES (
> "hbase.columns.mapping"=":key,cf:balance#b");
> insert into hbase_table values (1,1);
>
> Diagnostic Messages for this Task:
> Error: java.lang.RuntimeException:
> org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while
> processing row {"tmp_values_col1":"1","tmp_values_col2":"1"}
> at
> org.apache.hadoop.hive.ql.exec.mr.ExecMapper.map(ExecMapper.java:179)
> at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:54)
> at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:453)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:343)
> at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:164)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:415)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1783)
> at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)
> Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime
> Error while processing row {"tmp_values_col1":"1","tmp_values_col2":"1"}
> at
> org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:507)
> at
> org.apache.hadoop.hive.ql.exec.mr.ExecMapper.map(ExecMapper.java:170)
> ... 8 more
> Caused by: org.apache.hadoop.hive.ql.metadata.HiveException:
> org.apache.hadoop.hive.serde2.SerDeException: java.lang.RuntimeException:
> Hive internal error.
> at
> org.apache.hadoop.hive.ql.exec.FileSinkOperator.processOp(FileSinkOperator.java:733)
> at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:815)
> at
> org.apache.hadoop.hive.ql.exec.SelectOperator.processOp(SelectOperator.java:84)
> at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:815)
> at
> org.apache.hadoop.hive.ql.exec.TableScanOperator.processOp(TableScanOperator.java:97)
> at
> org.apache.hadoop.hive.ql.exec.MapOperator$MapOpCtx.forward(MapOperator.java:157)
> at
> org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:497)
> ... 9 more
> Caused by: org.apache.hadoop.hive.serde2.SerDeException:
> java.lang.RuntimeException: Hive internal error.
> at
> org.apache.hadoop.hive.hbase.HBaseSerDe.serialize(HBaseSerDe.java:286)
> at
> org.apache.hadoop.hive.ql.exec.FileSinkOperator.processOp(FileSinkOperator.java:668)
> ... 15 more
> Caused by: java.lang.RuntimeException: Hive internal error.
> at
> org.apache.hadoop.hive.serde2.lazy.LazyUtils.writePrimitive(LazyUtils.java:328)
> at
> org.apache.hadoop.hive.hbase.HBaseRowSerializer.serialize(HBaseRowSerializer.java:220)
> at
> org.apache.hadoop.hive.hbase.HBaseRowSerializer.serializeField(HBaseRowSerializer.java:194)
> at
> org.apache.hadoop.hive.hbase.HBaseRowSerializer.serialize(HBaseRowSerializer.java:118)
> at
> org.apache.hadoop.hive.hbase.HBaseSerDe.serialize(HBaseSerDe.java:282)
> ... 16 more
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HIVE-18057) remove PostExecute / PreExecute hook support
[ https://issues.apache.org/jira/browse/HIVE-18057?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zoltan Haindrich updated HIVE-18057: Attachment: HIVE-18057.01.patch > remove PostExecute / PreExecute hook support > > > Key: HIVE-18057 > URL: https://issues.apache.org/jira/browse/HIVE-18057 > Project: Hive > Issue Type: Sub-task >Reporter: Zoltan Haindrich > Attachments: HIVE-18057.01.patch > > > * deprecated since 2010 > * they are needlessly complicate the dispatch logic > * the current dispatch logic just silently accepts pre/post hooks if they > don't implement neither interface -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HIVE-18063) Make CommandProcessorResponse an exception instead of a return class
[
https://issues.apache.org/jira/browse/HIVE-18063?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zoltan Haindrich updated HIVE-18063:
Attachment: HIVE-18063.01.patch
> Make CommandProcessorResponse an exception instead of a return class
>
>
> Key: HIVE-18063
> URL: https://issues.apache.org/jira/browse/HIVE-18063
> Project: Hive
> Issue Type: Sub-task
> Components: Logical Optimizer
>Reporter: Zoltan Haindrich
>Assignee: Zoltan Haindrich
> Attachments: HIVE-18063.01.patch
>
>
> the usage pattern of the {{CommandProcessorResponse}} class suggests that its
> current role is closer to Exceptions than to return values.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HIVE-18055) cache like pattern object using map object in like function
[
https://issues.apache.org/jira/browse/HIVE-18055?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16251600#comment-16251600
]
Hive QA commented on HIVE-18055:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12897531/HIVE-18055.2-branch-1.2.patch
{color:red}ERROR:{color} -1 due to build exiting with an error
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/7803/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/7803/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-7803/
Messages:
{noformat}
This message was trimmed, see log for full details
[loading
ZipFileIndexFileObject[/usr/lib/jvm/java-7-openjdk-amd64/lib/ct.sym(META-INF/sym/rt.jar/java/util/Collections.class)]]
[loading
ZipFileIndexFileObject[/usr/lib/jvm/java-7-openjdk-amd64/lib/ct.sym(META-INF/sym/rt.jar/java/util/Comparator.class)]]
[loading
ZipFileIndexFileObject[/usr/lib/jvm/java-7-openjdk-amd64/lib/ct.sym(META-INF/sym/rt.jar/java/util/Iterator.class)]]
[loading
ZipFileIndexFileObject[/usr/lib/jvm/java-7-openjdk-amd64/lib/ct.sym(META-INF/sym/rt.jar/java/util/List.class)]]
[loading
ZipFileIndexFileObject[/usr/lib/jvm/java-7-openjdk-amd64/lib/ct.sym(META-INF/sym/rt.jar/java/util/Map.class)]]
[loading
ZipFileIndexFileObject[/usr/lib/jvm/java-7-openjdk-amd64/lib/ct.sym(META-INF/sym/rt.jar/java/util/StringTokenizer.class)]]
[loading
ZipFileIndexFileObject[/data/hiveptest/working/maven/org/apache/hadoop/hadoop-common/2.6.0/hadoop-common-2.6.0.jar(org/apache/hadoop/conf/Configuration.class)]]
[loading
ZipFileIndexFileObject[/data/hiveptest/working/maven/org/apache/hadoop/hadoop-common/2.6.0/hadoop-common-2.6.0.jar(org/apache/hadoop/fs/Path.class)]]
[loading
ZipFileIndexFileObject[/data/hiveptest/working/maven/org/apache/hadoop/hadoop-common/2.6.0/hadoop-common-2.6.0.jar(org/apache/hadoop/util/StringUtils.class)]]
[loading
ZipFileIndexFileObject[/data/hiveptest/working/maven/org/apache/hadoop/hadoop-common/2.6.0/hadoop-common-2.6.0.jar(org/apache/hadoop/util/VersionInfo.class)]]
[loading
ZipFileIndexFileObject[/usr/lib/jvm/java-7-openjdk-amd64/lib/ct.sym(META-INF/sym/rt.jar/java/lang/Iterable.class)]]
[loading
ZipFileIndexFileObject[/data/hiveptest/working/maven/org/apache/hadoop/hadoop-common/2.6.0/hadoop-common-2.6.0.jar(org/apache/hadoop/io/Writable.class)]]
[loading
ZipFileIndexFileObject[/usr/lib/jvm/java-7-openjdk-amd64/lib/ct.sym(META-INF/sym/rt.jar/java/lang/String.class)]]
[loading
ZipFileIndexFileObject[/data/hiveptest/working/maven/org/eclipse/jetty/aggregate/jetty-all-server/7.6.0.v20120127/jetty-all-server-7.6.0.v20120127.jar(org/eclipse/jetty/http/HttpStatus.class)]]
[loading
ZipFileIndexFileObject[/usr/lib/jvm/java-7-openjdk-amd64/lib/ct.sym(META-INF/sym/rt.jar/java/util/HashMap.class)]]
[loading
ZipFileIndexFileObject[/data/hiveptest/working/maven/com/sun/jersey/jersey-core/1.14/jersey-core-1.14.jar(javax/ws/rs/core/MediaType.class)]]
[loading
ZipFileIndexFileObject[/data/hiveptest/working/maven/com/sun/jersey/jersey-core/1.14/jersey-core-1.14.jar(javax/ws/rs/core/Response.class)]]
[loading
ZipFileIndexFileObject[/data/hiveptest/working/apache-github-branch-1.2-source/ql/target/hive-exec-1.2.3-SNAPSHOT.jar(org/codehaus/jackson/map/ObjectMapper.class)]]
[loading
ZipFileIndexFileObject[/usr/lib/jvm/java-7-openjdk-amd64/lib/ct.sym(META-INF/sym/rt.jar/java/lang/Exception.class)]]
[loading
ZipFileIndexFileObject[/usr/lib/jvm/java-7-openjdk-amd64/lib/ct.sym(META-INF/sym/rt.jar/java/lang/Throwable.class)]]
[loading
ZipFileIndexFileObject[/usr/lib/jvm/java-7-openjdk-amd64/lib/ct.sym(META-INF/sym/rt.jar/java/io/Serializable.class)]]
[loading
ZipFileIndexFileObject[/data/hiveptest/working/maven/com/sun/jersey/jersey-server/1.14/jersey-server-1.14.jar(com/sun/jersey/api/core/PackagesResourceConfig.class)]]
[loading
ZipFileIndexFileObject[/data/hiveptest/working/maven/com/sun/jersey/jersey-servlet/1.14/jersey-servlet-1.14.jar(com/sun/jersey/spi/container/servlet/ServletContainer.class)]]
[loading
ZipFileIndexFileObject[/data/hiveptest/working/apache-github-branch-1.2-source/common/target/hive-common-1.2.3-SNAPSHOT.jar(org/apache/hadoop/hive/common/classification/InterfaceStability.class)]]
[loading
ZipFileIndexFileObject[/data/hiveptest/working/maven/org/apache/hadoop/hadoop-hdfs/2.6.0/hadoop-hdfs-2.6.0.jar(org/apache/hadoop/hdfs/web/AuthFilter.class)]]
[loading
ZipFileIndexFileObject[/data/hiveptest/working/maven/org/apache/hadoop/hadoop-common/2.6.0/hadoop-common-2.6.0.jar(org/apache/hadoop/security/UserGroupInformation.class)]]
[loading
ZipFileIndexFileObject[/data/hiveptest/working/maven/org/apache/hadoop/hadoop-auth/2.6.0/hadoop-auth-2.6.0.jar(org/apache/hadoop/security/authentication/client/PseudoAuthenticator.class)]]
[loading
