[jira] [Updated] (HIVE-22652) TopNKey push through Group by with Grouping sets
[ https://issues.apache.org/jira/browse/HIVE-22652?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Krisztian Kasa updated HIVE-22652: -- Attachment: HIVE-22652.1.patch > TopNKey push through Group by with Grouping sets > > > Key: HIVE-22652 > URL: https://issues.apache.org/jira/browse/HIVE-22652 > Project: Hive > Issue Type: Improvement >Reporter: Krisztian Kasa >Assignee: Krisztian Kasa >Priority: Major > Attachments: HIVE-22652.1.patch > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22652) TopNKey push through Group by with Grouping sets
[ https://issues.apache.org/jira/browse/HIVE-22652?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Krisztian Kasa updated HIVE-22652: -- Status: Patch Available (was: Open) > TopNKey push through Group by with Grouping sets > > > Key: HIVE-22652 > URL: https://issues.apache.org/jira/browse/HIVE-22652 > Project: Hive > Issue Type: Improvement >Reporter: Krisztian Kasa >Assignee: Krisztian Kasa >Priority: Major > Attachments: HIVE-22652.1.patch > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-22088) Dynamic partition insert problem on table with "=" in location path spec
[
https://issues.apache.org/jira/browse/HIVE-22088?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17000620#comment-17000620
]
Naveen Gangam commented on HIVE-22088:
--
[~Bone An] Supporting "=" in the location for the table conflicts with the
semantics of a partition value. This might throw off some coding parsing paths.
I dont know what the support for such paths is across native cloud storage
systems but I feeling like restricting such paths is the probably the best
going forward, instead of supporting this as a feature which adds to the
confusion.
I feel the fix here should be to throw a semanticexception on create table with
such a path.
Thoughts?
> Dynamic partition insert problem on table with "=" in location path spec
>
>
> Key: HIVE-22088
> URL: https://issues.apache.org/jira/browse/HIVE-22088
> Project: Hive
> Issue Type: Bug
>Affects Versions: 4.0.0, 2.3.4
> Environment: Hive 2.6.0.10-2 Executing on Tez.
> OS: Ubuntu 16.04.4 LTS
> Config settings used:
> SET hive.exec.dynamic.partition=true;
> SET hive.exec.dynamic.partition.mode=nonstrict;
>Reporter: Puneet Khatod
>Assignee: Hui An
>Priority: Major
> Attachments: HIVE-22088.patch.1, HIVE-22088.patch.2
>
>
> If external table location spec has a '=' sign (coincidentally partition
> specifier) in it, then dynamic partition loading fails.
> *Use cases:*
> Quite often the same data is used in different contexts by creating different
> external tables on top of the data. Many times the tables have different
> partition depths depending on how data is organized.
> Like in below example, there are individual customer specific tables and
> queries/jobs to insert data partitioned by type. And there is another table
> to give the consolidated data view of all the customers, thus have two level
> partition customer and type.
> The job to insert customer specific data into customer specific table fails
> if we use dynamic partitioning. Static partition insert on same table works
> fine though.
> *Replication:*
> To replicate following simple setup could be done. Below execution is on
> 'Tez'.
> *Source table**-*
> CREATE EXTERNAL TABLE temp_dummy_table
> (id STRING, type STRING)
> ROW FORMAT DELIMITED FIELDS TERMINATED BY '|' LINES TERMINATED BY '\n'
> STORED AS TEXTFILE
> LOCATION '/home/source/';
>
> *Destination Table-*
> CREATE EXTERNAL TABLE temp_dummy_dest_table
> (id STRING)
> PARTITIONED BY (type string)
> ROW FORMAT DELIMITED FIELDS TERMINATED BY '|' LINES TERMINATED BY '\n'
> STORED AS TEXTFILE
> LOCATION '/home/destination/{color:#ff}customer=abc{color}/';
>
> *Insert into destination-*
> insert overwrite table temp_dummy_dest_table partition (type)
> select i.id as id, i.type as type
> from temp_dummy_table i
> where i.type in ('type1','type2');
>
> *Log and Error Msgs on CLI*-
> Loading data to table temp_dummy_dest_table partition (type=null)
> Failed with exception Partition spec \{type=type1, customer=abc} contains
> non-partition columns
> FAILED: Execution Error, return code 1 from
> org.apache.hadoop.hive.ql.exec.MoveTask
>
> *Possible resolution:*
> The dynamic partitioning should consider only those partition specs which are
> under the defined table root/base path. If the path itself has partition
> style format (customer=abc in above example) then that should not be
> considered as partition as it is outside the scope of the table.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22608) Reduce the number of public methods in Driver
[
https://issues.apache.org/jira/browse/HIVE-22608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17000573#comment-17000573
]
Hive QA commented on HIVE-22608:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12989220/HIVE-22608.01.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 17786 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.metastore.TestMetastoreHousekeepingNonLeader.testHouseKeepingThreadExistence
(batchId=247)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/19988/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/19988/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-19988/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 1 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12989220 - PreCommit-HIVE-Build
> Reduce the number of public methods in Driver
> -
>
> Key: HIVE-22608
> URL: https://issues.apache.org/jira/browse/HIVE-22608
> Project: Hive
> Issue Type: Sub-task
> Components: Hive
>Reporter: Miklos Gergely
>Assignee: Miklos Gergely
>Priority: Major
> Fix For: 4.0.0
>
> Attachments: HIVE-22608.01.patch
>
>
> There are 33 public methods in Driver, some of them either don't belong
> there, or should not be public.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22608) Reduce the number of public methods in Driver
[
https://issues.apache.org/jira/browse/HIVE-22608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17000520#comment-17000520
]
Hive QA commented on HIVE-22608:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
57s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
13s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
43s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
56s{color} | {color:blue} ql in master has 1531 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
0s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
28s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
7s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
7s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
39s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
3s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
1s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
14s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 23m 43s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-19988/dev-support/hive-personality.sh
|
| git revision | master / 2e930ab |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.1 |
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-19988/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Reduce the number of public methods in Driver
> -
>
> Key: HIVE-22608
> URL: https://issues.apache.org/jira/browse/HIVE-22608
> Project: Hive
> Issue Type: Sub-task
> Components: Hive
>Reporter: Miklos Gergely
>Assignee: Miklos Gergely
>Priority: Major
> Fix For: 4.0.0
>
> Attachments: HIVE-22608.01.patch
>
>
> There are 33 public methods in Driver, some of them either don't belong
> there, or should not be public.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-22608) Reduce the number of public methods in Driver
[ https://issues.apache.org/jira/browse/HIVE-22608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Miklos Gergely updated HIVE-22608: -- Attachment: (was: HIVE-22608.01.patch) > Reduce the number of public methods in Driver > - > > Key: HIVE-22608 > URL: https://issues.apache.org/jira/browse/HIVE-22608 > Project: Hive > Issue Type: Sub-task > Components: Hive >Reporter: Miklos Gergely >Assignee: Miklos Gergely >Priority: Major > Fix For: 4.0.0 > > Attachments: HIVE-22608.01.patch > > > There are 33 public methods in Driver, some of them either don't belong > there, or should not be public. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22608) Reduce the number of public methods in Driver
[ https://issues.apache.org/jira/browse/HIVE-22608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Miklos Gergely updated HIVE-22608: -- Attachment: HIVE-22608.01.patch > Reduce the number of public methods in Driver > - > > Key: HIVE-22608 > URL: https://issues.apache.org/jira/browse/HIVE-22608 > Project: Hive > Issue Type: Sub-task > Components: Hive >Reporter: Miklos Gergely >Assignee: Miklos Gergely >Priority: Major > Fix For: 4.0.0 > > Attachments: HIVE-22608.01.patch > > > There are 33 public methods in Driver, some of them either don't belong > there, or should not be public. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-22631) Avoid deep copying partition list in listPartitionsByExpr
[ https://issues.apache.org/jira/browse/HIVE-22631?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17000486#comment-17000486 ] Jesus Camacho Rodriguez commented on HIVE-22631: [~vgarg], since these partition objects are not immutable, I assume the call was trying to avoid that other methods upstream can modify the returned partitions? However, may this actually happen (those returned objects being reused for multiple calls)? In our tests, there is certainly no such case. > Avoid deep copying partition list in listPartitionsByExpr > - > > Key: HIVE-22631 > URL: https://issues.apache.org/jira/browse/HIVE-22631 > Project: Hive > Issue Type: Improvement > Components: Hive >Reporter: Vineet Garg >Assignee: Vineet Garg >Priority: Major > Attachments: HIVE-22631.1.patch, HIVE-22631.2.patch > > > This is an expensive call, I am not sure why deepCopy is required. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-20150) TopNKey pushdown
[ https://issues.apache.org/jira/browse/HIVE-20150?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jesus Camacho Rodriguez updated HIVE-20150: --- Fix Version/s: 4.0.0 Resolution: Fixed Status: Resolved (was: Patch Available) Pushed to master. Thanks [~kkasa]! > TopNKey pushdown > > > Key: HIVE-20150 > URL: https://issues.apache.org/jira/browse/HIVE-20150 > Project: Hive > Issue Type: Improvement > Components: Physical Optimizer >Affects Versions: 4.0.0 >Reporter: Teddy Choi >Assignee: Krisztian Kasa >Priority: Major > Labels: pull-request-available > Fix For: 4.0.0 > > Attachments: HIVE-20150.1.patch, HIVE-20150.10.patch, > HIVE-20150.11.patch, HIVE-20150.11.patch, HIVE-20150.14.patch, > HIVE-20150.15.patch, HIVE-20150.16.patch, HIVE-20150.17.patch, > HIVE-20150.17.patch, HIVE-20150.18.patch, HIVE-20150.18.patch, > HIVE-20150.19.patch, HIVE-20150.2.patch, HIVE-20150.20.patch, > HIVE-20150.21.patch, HIVE-20150.22.patch, HIVE-20150.23.patch, > HIVE-20150.24.patch, HIVE-20150.25.patch, HIVE-20150.25.patch, > HIVE-20150.26.patch, HIVE-20150.27.patch, HIVE-20150.28.patch, > HIVE-20150.29.patch, HIVE-20150.29.patch, HIVE-20150.29.patch, > HIVE-20150.29.patch, HIVE-20150.29.patch, HIVE-20150.29.patch, > HIVE-20150.4.patch, HIVE-20150.5.patch, HIVE-20150.6.patch, > HIVE-20150.7.patch, HIVE-20150.8.patch, HIVE-20150.9.patch > > > TopNKey operator is implemented in HIVE-17896, but it needs more work in > pushdown implementation. So this issue covers TopNKey pushdown implementation > with proper tests. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-22649) Fix TestHiveCli: scratchdir should be writable
[
https://issues.apache.org/jira/browse/HIVE-22649?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17000446#comment-17000446
]
Ramesh Kumar Thangarajan commented on HIVE-22649:
-
Considering this is only the tests, the changes look reasonable to me. We can
get this in and then probably work on understanding the root cause.
+1
> Fix TestHiveCli: scratchdir should be writable
> --
>
> Key: HIVE-22649
> URL: https://issues.apache.org/jira/browse/HIVE-22649
> Project: Hive
> Issue Type: Sub-task
>Reporter: Denys Kuzmenko
>Assignee: Denys Kuzmenko
>Priority: Major
> Attachments: HIVE-22649.1.patch, HIVE-22649.2.patch
>
>
> Error applying authorization policy on hive configuration: The dir: /tmp/hive
> on HDFS should be writable. Current permissions are: rwxr-xr-x
> SessionState.java
> {code}
> private Path createRootHDFSDir(HiveConf conf) throws IOException {
> Path rootHDFSDirPath = new Path(HiveConf.getVar(conf,
> HiveConf.ConfVars.SCRATCHDIR));
> *Utilities.ensurePathIsWritable(rootHDFSDirPath, conf);*
> return rootHDFSDirPath;
> }
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22608) Reduce the number of public methods in Driver
[
https://issues.apache.org/jira/browse/HIVE-22608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17000301#comment-17000301
]
Hive QA commented on HIVE-22608:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12989202/HIVE-22608.01.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 24 failed/errored test(s), 17780 tests
executed
*Failed tests:*
{noformat}
org.apache.hive.beeline.cli.TestHiveCli.testCmd (batchId=206)
org.apache.hive.beeline.cli.TestHiveCli.testCommentStripping (batchId=206)
org.apache.hive.beeline.cli.TestHiveCli.testDatabaseOptions (batchId=206)
org.apache.hive.beeline.cli.TestHiveCli.testErrOutput (batchId=206)
org.apache.hive.beeline.cli.TestHiveCli.testHelp (batchId=206)
org.apache.hive.beeline.cli.TestHiveCli.testInValidCmd (batchId=206)
org.apache.hive.beeline.cli.TestHiveCli.testInvalidDatabaseOptions (batchId=206)
org.apache.hive.beeline.cli.TestHiveCli.testInvalidOptions (batchId=206)
org.apache.hive.beeline.cli.TestHiveCli.testInvalidOptions2 (batchId=206)
org.apache.hive.beeline.cli.TestHiveCli.testNoErrorDB (batchId=206)
org.apache.hive.beeline.cli.TestHiveCli.testSetHeaderValue (batchId=206)
org.apache.hive.beeline.cli.TestHiveCli.testSetPromptValue (batchId=206)
org.apache.hive.beeline.cli.TestHiveCli.testSourceCmd (batchId=206)
org.apache.hive.beeline.cli.TestHiveCli.testSourceCmd2 (batchId=206)
org.apache.hive.beeline.cli.TestHiveCli.testSourceCmd3 (batchId=206)
org.apache.hive.beeline.cli.TestHiveCli.testSourceCmd4 (batchId=206)
org.apache.hive.beeline.cli.TestHiveCli.testSqlFromCmd (batchId=206)
org.apache.hive.beeline.cli.TestHiveCli.testSqlFromCmdWithDBName (batchId=206)
org.apache.hive.beeline.cli.TestHiveCli.testUseCurrentDB1 (batchId=206)
org.apache.hive.beeline.cli.TestHiveCli.testUseCurrentDB2 (batchId=206)
org.apache.hive.beeline.cli.TestHiveCli.testUseCurrentDB3 (batchId=206)
org.apache.hive.beeline.cli.TestHiveCli.testUseInvalidDB (batchId=206)
org.apache.hive.beeline.cli.TestHiveCli.testVariables (batchId=206)
org.apache.hive.beeline.cli.TestHiveCli.testVariablesForSource (batchId=206)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/19987/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/19987/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-19987/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 24 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12989202 - PreCommit-HIVE-Build
> Reduce the number of public methods in Driver
> -
>
> Key: HIVE-22608
> URL: https://issues.apache.org/jira/browse/HIVE-22608
> Project: Hive
> Issue Type: Sub-task
> Components: Hive
>Reporter: Miklos Gergely
>Assignee: Miklos Gergely
>Priority: Major
> Fix For: 4.0.0
>
> Attachments: HIVE-22608.01.patch
>
>
> There are 33 public methods in Driver, some of them either don't belong
> there, or should not be public.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22608) Reduce the number of public methods in Driver
[
https://issues.apache.org/jira/browse/HIVE-22608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17000243#comment-17000243
]
Hive QA commented on HIVE-22608:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
31s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
6s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
40s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m
3s{color} | {color:blue} ql in master has 1531 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
58s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
32s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
4s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
4s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
40s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
55s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
57s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
13s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 23m 59s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-19987/dev-support/hive-personality.sh
|
| git revision | master / 0dfecd5 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.1 |
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-19987/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Reduce the number of public methods in Driver
> -
>
> Key: HIVE-22608
> URL: https://issues.apache.org/jira/browse/HIVE-22608
> Project: Hive
> Issue Type: Sub-task
> Components: Hive
>Reporter: Miklos Gergely
>Assignee: Miklos Gergely
>Priority: Major
> Fix For: 4.0.0
>
> Attachments: HIVE-22608.01.patch
>
>
> There are 33 public methods in Driver, some of them either don't belong
> there, or should not be public.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22661) Compaction fails on non bucketed table with data loaded inpath
[
https://issues.apache.org/jira/browse/HIVE-22661?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17000207#comment-17000207
]
Hive QA commented on HIVE-22661:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12989199/HIVE-22661.0.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 20 failed/errored test(s), 17781 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[hybridgrace_hashjoin_2]
(batchId=112)
org.apache.hadoop.hive.ql.TestTxnCommands3.testDeleteEventPruningOff
(batchId=352)
org.apache.hadoop.hive.ql.TestTxnCommands3.testDeleteEventPruningOn
(batchId=352)
org.apache.hadoop.hive.ql.TestTxnExIm.testImport (batchId=337)
org.apache.hadoop.hive.ql.TestTxnExIm.testImportNoTarget (batchId=337)
org.apache.hadoop.hive.ql.TestTxnExIm.testImportVectorized (batchId=337)
org.apache.hadoop.hive.ql.TestTxnLoadData.loadDataUpdate (batchId=313)
org.apache.hadoop.hive.ql.TestTxnLoadData.loadDataUpdateVectorized (batchId=313)
org.apache.hadoop.hive.ql.TestTxnNoBuckets.testNoBuckets (batchId=337)
org.apache.hadoop.hive.ql.TestTxnNoBuckets.testNonAcidToAcidVectorzied
(batchId=337)
org.apache.hadoop.hive.ql.TestTxnNoBuckets.testToAcidConversion02 (batchId=337)
org.apache.hadoop.hive.ql.TestTxnNoBuckets.testToAcidConversionMultiBucket
(batchId=337)
org.apache.hadoop.hive.ql.TestTxnNoBucketsVectorized.testNoBuckets (batchId=337)
org.apache.hadoop.hive.ql.TestTxnNoBucketsVectorized.testNonAcidToAcidVectorzied
(batchId=337)
org.apache.hadoop.hive.ql.TestTxnNoBucketsVectorized.testToAcidConversion02
(batchId=337)
org.apache.hadoop.hive.ql.TestTxnNoBucketsVectorized.testToAcidConversionMultiBucket
(batchId=337)
org.apache.hive.hcatalog.streaming.TestStreaming.testNoBuckets (batchId=221)
org.apache.hive.jdbc.TestJdbcDriver2.testGetQueryLogForReplCommands
(batchId=287)
org.apache.hive.streaming.TestStreaming.testAutoRollTransactionBatch
(batchId=363)
org.apache.hive.streaming.TestStreaming.testNoBuckets (batchId=363)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/19986/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/19986/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-19986/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 20 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12989199 - PreCommit-HIVE-Build
> Compaction fails on non bucketed table with data loaded inpath
> --
>
> Key: HIVE-22661
> URL: https://issues.apache.org/jira/browse/HIVE-22661
> Project: Hive
> Issue Type: Bug
>Reporter: Ádám Szita
>Assignee: Ádám Szita
>Priority: Major
> Attachments: HIVE-22661.0.patch
>
>
> Compaction cannot handle situations where:
> * data was ingested with {{LOAD DATA INPATH}}
> * this ingest method is run multiple times, and
> ** with different number of files getting created in the delta directories
> Therefore, for file/dir structures such as:
> {code:java}
> /warehouse/tablespace/managed/hive/comp3/delta_001_001_
> /warehouse/tablespace/managed/hive/comp3/delta_001_001_/00_0
> /warehouse/tablespace/managed/hive/comp3/delta_001_001_/01_0
> /warehouse/tablespace/managed/hive/comp3/delta_002_002_
> /warehouse/tablespace/managed/hive/comp3/delta_002_002_/00_0
> /warehouse/tablespace/managed/hive/comp3/delta_002_002_/01_0
> /warehouse/tablespace/managed/hive/comp3/delta_002_002_/02_0
> {code}
> Although the table is not bucketed, bucket is calculated from the (raw)
> files' names. Compaction in the above case will fail on delta1-1 not having
> data for 'bucket' 2.
> Steps to repro using small dataset:
> {code:java}
> set tez.grouping.min-size=8;
> set tez.grouping.max-size=8;
> set mapreduce.input.fileinputformat.split.minsize=8;
> set mapreduce.input.fileinputformat.split.minsize=8;
> create external table comp0 (a string);
> insert into comp0 values ("qwertyuiopasdfghjklzxcvbnm");
> insert into comp0 values ("qwertyuiopasdfghjklzxcvbnm");
> create external table comp1 stored as orc as select * from comp0;
> insert into comp0 values ("qwertyuiopasdfghjklzxcvbnm");
> create external table comp2 stored as orc as select * from comp0;
> create table comp3 (a string);
> load data inpath '/warehouse/tablespace/external/hive/comp1'
[jira] [Commented] (HIVE-22653) Remove commons-lang leftovers
[ https://issues.apache.org/jira/browse/HIVE-22653?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17000183#comment-17000183 ] Ashutosh Chauhan commented on HIVE-22653: - +1 > Remove commons-lang leftovers > - > > Key: HIVE-22653 > URL: https://issues.apache.org/jira/browse/HIVE-22653 > Project: Hive > Issue Type: Bug >Reporter: David Lavati >Assignee: David Lavati >Priority: Major > Labels: pull-request-available > Fix For: 4.0.0 > > Attachments: HIVE-22653.01.patch, HIVE-22653.01.patch > > Time Spent: 10m > Remaining Estimate: 0h > > HIVE-7145 removed commons-lang - in favor of commons-lang3 - as a direct > dependency, however a high number of files still refer to commons-lang, which > is transitively brought in either way. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-22661) Compaction fails on non bucketed table with data loaded inpath
[
https://issues.apache.org/jira/browse/HIVE-22661?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17000140#comment-17000140
]
Hive QA commented on HIVE-22661:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
37s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
52s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
46s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
56s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
57s{color} | {color:blue} ql in master has 1531 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
42s{color} | {color:blue} itests/hive-unit in master has 2 extant Findbugs
warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
24s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
27s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
6s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
45s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
45s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
18s{color} | {color:red} itests/hive-unit: The patch generated 3 new + 148
unchanged - 0 fixed = 151 total (was 148) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
45s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
20s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
13s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 29m 20s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-19986/dev-support/hive-personality.sh
|
| git revision | master / 0dfecd5 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.1 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-19986/yetus/diff-checkstyle-itests_hive-unit.txt
|
| modules | C: ql itests/hive-unit U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-19986/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Compaction fails on non bucketed table with data loaded inpath
> --
>
> Key: HIVE-22661
> URL: https://issues.apache.org/jira/browse/HIVE-22661
> Project: Hive
> Issue Type: Bug
>Reporter: Ádám Szita
>Assignee: Ádám Szita
>Priority: Major
> Attachments: HIVE-22661.0.patch
>
>
> Compaction cannot handle situations where:
> * data was ingested with {{LOAD DATA INPATH}}
> * this ingest method is run multiple times, and
> ** with different number of files getting created in the delta directories
> Therefore, for file/dir structures such as:
> {code:java}
> /warehouse/tablespace/managed/hive/comp3/delta_001_001_
> /warehouse/tablespace/managed/hive/comp3/delta_001_001_/00_0
> /warehouse/tablespace/managed/hive/comp3/delta_001_001_/01_0
> /warehouse/tablespace/managed/hive/comp3/delta_002_002_
>
[jira] [Updated] (HIVE-22608) Reduce the number of public methods in Driver
[ https://issues.apache.org/jira/browse/HIVE-22608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Miklos Gergely updated HIVE-22608: -- Attachment: (was: HIVE-22608.01.patch) > Reduce the number of public methods in Driver > - > > Key: HIVE-22608 > URL: https://issues.apache.org/jira/browse/HIVE-22608 > Project: Hive > Issue Type: Sub-task > Components: Hive >Reporter: Miklos Gergely >Assignee: Miklos Gergely >Priority: Major > Fix For: 4.0.0 > > Attachments: HIVE-22608.01.patch > > > There are 33 public methods in Driver, some of them either don't belong > there, or should not be public. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22608) Reduce the number of public methods in Driver
[ https://issues.apache.org/jira/browse/HIVE-22608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Miklos Gergely updated HIVE-22608: -- Attachment: HIVE-22608.01.patch > Reduce the number of public methods in Driver > - > > Key: HIVE-22608 > URL: https://issues.apache.org/jira/browse/HIVE-22608 > Project: Hive > Issue Type: Sub-task > Components: Hive >Reporter: Miklos Gergely >Assignee: Miklos Gergely >Priority: Major > Fix For: 4.0.0 > > Attachments: HIVE-22608.01.patch > > > There are 33 public methods in Driver, some of them either don't belong > there, or should not be public. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22661) Compaction fails on non bucketed table with data loaded inpath
[
https://issues.apache.org/jira/browse/HIVE-22661?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ádám Szita updated HIVE-22661:
--
Status: Patch Available (was: Open)
> Compaction fails on non bucketed table with data loaded inpath
> --
>
> Key: HIVE-22661
> URL: https://issues.apache.org/jira/browse/HIVE-22661
> Project: Hive
> Issue Type: Bug
>Reporter: Ádám Szita
>Assignee: Ádám Szita
>Priority: Major
> Attachments: HIVE-22661.0.patch
>
>
> Compaction cannot handle situations where:
> * data was ingested with {{LOAD DATA INPATH}}
> * this ingest method is run multiple times, and
> ** with different number of files getting created in the delta directories
> Therefore, for file/dir structures such as:
> {code:java}
> /warehouse/tablespace/managed/hive/comp3/delta_001_001_
> /warehouse/tablespace/managed/hive/comp3/delta_001_001_/00_0
> /warehouse/tablespace/managed/hive/comp3/delta_001_001_/01_0
> /warehouse/tablespace/managed/hive/comp3/delta_002_002_
> /warehouse/tablespace/managed/hive/comp3/delta_002_002_/00_0
> /warehouse/tablespace/managed/hive/comp3/delta_002_002_/01_0
> /warehouse/tablespace/managed/hive/comp3/delta_002_002_/02_0
> {code}
> Although the table is not bucketed, bucket is calculated from the (raw)
> files' names. Compaction in the above case will fail on delta1-1 not having
> data for 'bucket' 2.
> Steps to repro using small dataset:
> {code:java}
> set tez.grouping.min-size=8;
> set tez.grouping.max-size=8;
> set mapreduce.input.fileinputformat.split.minsize=8;
> set mapreduce.input.fileinputformat.split.minsize=8;
> create external table comp0 (a string);
> insert into comp0 values ("qwertyuiopasdfghjklzxcvbnm");
> insert into comp0 values ("qwertyuiopasdfghjklzxcvbnm");
> create external table comp1 stored as orc as select * from comp0;
> insert into comp0 values ("qwertyuiopasdfghjklzxcvbnm");
> create external table comp2 stored as orc as select * from comp0;
> create table comp3 (a string);
> load data inpath '/warehouse/tablespace/external/hive/comp1' into table comp3;
> load data inpath '/warehouse/tablespace/external/hive/comp2' into table
> comp3;{code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-22661) Compaction fails on non bucketed table with data loaded inpath
[
https://issues.apache.org/jira/browse/HIVE-22661?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ádám Szita updated HIVE-22661:
--
Attachment: HIVE-22661.0.patch
> Compaction fails on non bucketed table with data loaded inpath
> --
>
> Key: HIVE-22661
> URL: https://issues.apache.org/jira/browse/HIVE-22661
> Project: Hive
> Issue Type: Bug
>Reporter: Ádám Szita
>Assignee: Ádám Szita
>Priority: Major
> Attachments: HIVE-22661.0.patch
>
>
> Compaction cannot handle situations where:
> * data was ingested with {{LOAD DATA INPATH}}
> * this ingest method is run multiple times, and
> ** with different number of files getting created in the delta directories
> Therefore, for file/dir structures such as:
> {code:java}
> /warehouse/tablespace/managed/hive/comp3/delta_001_001_
> /warehouse/tablespace/managed/hive/comp3/delta_001_001_/00_0
> /warehouse/tablespace/managed/hive/comp3/delta_001_001_/01_0
> /warehouse/tablespace/managed/hive/comp3/delta_002_002_
> /warehouse/tablespace/managed/hive/comp3/delta_002_002_/00_0
> /warehouse/tablespace/managed/hive/comp3/delta_002_002_/01_0
> /warehouse/tablespace/managed/hive/comp3/delta_002_002_/02_0
> {code}
> Although the table is not bucketed, bucket is calculated from the (raw)
> files' names. Compaction in the above case will fail on delta1-1 not having
> data for 'bucket' 2.
> Steps to repro using small dataset:
> {code:java}
> set tez.grouping.min-size=8;
> set tez.grouping.max-size=8;
> set mapreduce.input.fileinputformat.split.minsize=8;
> set mapreduce.input.fileinputformat.split.minsize=8;
> create external table comp0 (a string);
> insert into comp0 values ("qwertyuiopasdfghjklzxcvbnm");
> insert into comp0 values ("qwertyuiopasdfghjklzxcvbnm");
> create external table comp1 stored as orc as select * from comp0;
> insert into comp0 values ("qwertyuiopasdfghjklzxcvbnm");
> create external table comp2 stored as orc as select * from comp0;
> create table comp3 (a string);
> load data inpath '/warehouse/tablespace/external/hive/comp1' into table comp3;
> load data inpath '/warehouse/tablespace/external/hive/comp2' into table
> comp3;{code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22489) Reduce Sink operator should order nulls by parameter
[
https://issues.apache.org/jira/browse/HIVE-22489?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=1774#comment-1774
]
Hive QA commented on HIVE-22489:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12989184/HIVE-22489.6.patch
{color:green}SUCCESS:{color} +1 due to 7 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 36 failed/errored test(s), 17781 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[hashjoin] (batchId=3)
org.apache.hadoop.hive.cli.TestEncryptedHDFSCliDriver.testCliDriver[encryption_join_with_different_encryption_keys]
(batchId=194)
org.apache.hadoop.hive.ql.exec.tez.TestVectorMapJoinFastHashTable.checkFast2estimations
(batchId=347)
org.apache.hadoop.hive.ql.exec.vector.TestVectorSerDeRow.testVectorBinarySortableDeserializeRow
(batchId=345)
org.apache.hadoop.hive.ql.exec.vector.TestVectorSerDeRow.testVectorBinarySortableSerializeRow
(batchId=345)
org.apache.hadoop.hive.ql.exec.vector.mapjoin.fast.TestVectorMapJoinFastLongHashMap.testExpand
(batchId=345)
org.apache.hadoop.hive.ql.exec.vector.mapjoin.fast.TestVectorMapJoinFastLongHashMap.testFullMap
(batchId=345)
org.apache.hadoop.hive.ql.exec.vector.mapjoin.fast.TestVectorMapJoinFastLongHashMap.testGetNonExistent
(batchId=345)
org.apache.hadoop.hive.ql.exec.vector.mapjoin.fast.TestVectorMapJoinFastLongHashMap.testLargeAndExpand
(batchId=345)
org.apache.hadoop.hive.ql.exec.vector.mapjoin.fast.TestVectorMapJoinFastLongHashMap.testMultipleKeysMultipleValue
(batchId=345)
org.apache.hadoop.hive.ql.exec.vector.mapjoin.fast.TestVectorMapJoinFastLongHashMap.testMultipleKeysSingleValue
(batchId=345)
org.apache.hadoop.hive.ql.exec.vector.mapjoin.fast.TestVectorMapJoinFastLongHashMap.testOneKey
(batchId=345)
org.apache.hadoop.hive.ql.exec.vector.mapjoin.fast.TestVectorMapJoinFastLongHashMapNonMatched.testExpand
(batchId=345)
org.apache.hadoop.hive.ql.exec.vector.mapjoin.fast.TestVectorMapJoinFastLongHashMapNonMatched.testLargeAndExpand
(batchId=345)
org.apache.hadoop.hive.ql.exec.vector.mapjoin.fast.TestVectorMapJoinFastLongHashMapNonMatched.testMultipleKeysMultipleValue
(batchId=345)
org.apache.hadoop.hive.ql.exec.vector.mapjoin.fast.TestVectorMapJoinFastLongHashMapNonMatched.testMultipleKeysSingleValue
(batchId=345)
org.apache.hadoop.hive.ql.exec.vector.mapjoin.fast.TestVectorMapJoinFastLongHashMapNonMatched.testOneKey
(batchId=345)
org.apache.hadoop.hive.ql.exec.vector.mapjoin.fast.TestVectorMapJoinFastLongHashMultiSet.testExpand
(batchId=346)
org.apache.hadoop.hive.ql.exec.vector.mapjoin.fast.TestVectorMapJoinFastLongHashMultiSet.testFullMap
(batchId=346)
org.apache.hadoop.hive.ql.exec.vector.mapjoin.fast.TestVectorMapJoinFastLongHashMultiSet.testGetNonExistent
(batchId=346)
org.apache.hadoop.hive.ql.exec.vector.mapjoin.fast.TestVectorMapJoinFastLongHashMultiSet.testLargeAndExpand
(batchId=346)
org.apache.hadoop.hive.ql.exec.vector.mapjoin.fast.TestVectorMapJoinFastLongHashMultiSet.testMultipleKeysMultipleValue
(batchId=346)
org.apache.hadoop.hive.ql.exec.vector.mapjoin.fast.TestVectorMapJoinFastLongHashMultiSet.testMultipleKeysSingleValue
(batchId=346)
org.apache.hadoop.hive.ql.exec.vector.mapjoin.fast.TestVectorMapJoinFastLongHashMultiSet.testOneKey
(batchId=346)
org.apache.hadoop.hive.ql.exec.vector.mapjoin.fast.TestVectorMapJoinFastRowHashMap.testBigIntRows
(batchId=345)
org.apache.hadoop.hive.ql.exec.vector.mapjoin.fast.TestVectorMapJoinFastRowHashMap.testBigIntRowsClipped
(batchId=345)
org.apache.hadoop.hive.ql.exec.vector.mapjoin.fast.TestVectorMapJoinFastRowHashMap.testBigIntRowsClippedExact
(batchId=345)
org.apache.hadoop.hive.ql.exec.vector.mapjoin.fast.TestVectorMapJoinFastRowHashMap.testBigIntRowsExact
(batchId=345)
org.apache.hadoop.hive.ql.exec.vector.mapjoin.fast.TestVectorMapJoinFastRowHashMap.testIntRows
(batchId=345)
org.apache.hadoop.hive.ql.exec.vector.mapjoin.fast.TestVectorMapJoinFastRowHashMap.testIntRowsClipped
(batchId=345)
org.apache.hadoop.hive.ql.exec.vector.mapjoin.fast.TestVectorMapJoinFastRowHashMap.testIntRowsClippedExact
(batchId=345)
org.apache.hadoop.hive.ql.exec.vector.mapjoin.fast.TestVectorMapJoinFastRowHashMap.testIntRowsExact
(batchId=345)
org.apache.hadoop.hive.ql.exec.vector.mapjoin.fast.TestVectorMapJoinFastRowHashMap.testStringRows
(batchId=345)
org.apache.hadoop.hive.ql.exec.vector.mapjoin.fast.TestVectorMapJoinFastRowHashMap.testStringRowsClipped
(batchId=345)
org.apache.hadoop.hive.ql.exec.vector.mapjoin.fast.TestVectorMapJoinFastRowHashMap.testStringRowsClippedExact
(batchId=345)
org.apache.hadoop.hive.ql.exec.vector.mapjoin.fast.TestVectorMapJoinFastRowHashMap.testStringRowsExact
(batchId=345)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/19985/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/19985/console
[jira] [Commented] (HIVE-22489) Reduce Sink operator should order nulls by parameter
[
https://issues.apache.org/jira/browse/HIVE-22489?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=1745#comment-1745
]
Hive QA commented on HIVE-22489:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
28s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
58s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
57s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 2m
11s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
38s{color} | {color:blue} serde in master has 198 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
58s{color} | {color:blue} ql in master has 1531 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
31s{color} | {color:blue} accumulo-handler in master has 21 extant Findbugs
warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
23s{color} | {color:blue} contrib in master has 11 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
30s{color} | {color:blue} hbase-handler in master has 15 extant Findbugs
warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
22s{color} | {color:blue} kudu-handler in master has 1 extant Findbugs
warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m
11s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
26s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
20s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
51s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 2m
51s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
22s{color} | {color:red} serde: The patch generated 3 new + 564 unchanged - 2
fixed = 567 total (was 566) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
49s{color} | {color:red} ql: The patch generated 3 new + 695 unchanged - 0
fixed = 698 total (was 695) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
1s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
45s{color} | {color:green} serde generated 0 new + 197 unchanged - 1 fixed =
197 total (was 198) {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 3m
57s{color} | {color:red} ql generated 1 new + 1531 unchanged - 0 fixed = 1532
total (was 1531) {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
34s{color} | {color:green} accumulo-handler in the patch passed. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
29s{color} | {color:green} contrib in the patch passed. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
34s{color} | {color:green} hbase-handler in the patch passed. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
28s{color} | {color:green} kudu-handler in the patch passed. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m
10s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
13s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 48m 8s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| FindBugs | module:ql |
| | The field
org.apache.hadoop.hive.ql.exec.CommonMergeJoinOperator.nullOrdering is
transient but isn't set by deserialization In
[jira] [Updated] (HIVE-22548) Optimise Utilities.removeTempOrDuplicateFiles when moving files to final location
[
https://issues.apache.org/jira/browse/HIVE-22548?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
mahesh kumar behera updated HIVE-22548:
---
Resolution: Fixed
Status: Resolved (was: Patch Available)
Committed to master. Thanks [~rajesh.balamohan] for review.
> Optimise Utilities.removeTempOrDuplicateFiles when moving files to final
> location
> -
>
> Key: HIVE-22548
> URL: https://issues.apache.org/jira/browse/HIVE-22548
> Project: Hive
> Issue Type: Improvement
> Components: Hive
>Affects Versions: 3.1.2
>Reporter: Rajesh Balamohan
>Assignee: mahesh kumar behera
>Priority: Major
> Attachments: HIVE-22548.01.patch, HIVE-22548.02.patch
>
>
> {{Utilities.removeTempOrDuplicateFiles}}
> is very slow with cloud storage, as it executes {{listStatus}} twice and also
> runs in single threaded mode.
> https://github.com/apache/hive/blob/master/ql/src/java/org/apache/hadoop/hive/ql/exec/Utilities.java#L1629
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-22489) Reduce Sink operator should order nulls by parameter
[
https://issues.apache.org/jira/browse/HIVE-22489?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Krisztian Kasa updated HIVE-22489:
--
Status: Open (was: Patch Available)
> Reduce Sink operator should order nulls by parameter
> -
>
> Key: HIVE-22489
> URL: https://issues.apache.org/jira/browse/HIVE-22489
> Project: Hive
> Issue Type: Bug
> Components: Query Planning
>Reporter: Krisztian Kasa
>Assignee: Krisztian Kasa
>Priority: Major
> Attachments: HIVE-22489.1.patch, HIVE-22489.2.patch,
> HIVE-22489.3.patch, HIVE-22489.3.patch, HIVE-22489.4.patch,
> HIVE-22489.5.patch, HIVE-22489.6.patch
>
>
> When the property hive.default.nulls.last is set to true and no null order is
> explicitly specified in the ORDER BY clause of the query null ordering should
> be NULLS LAST.
> But some of the Reduce Sink operators still orders null first.
> {code}
> SET hive.default.nulls.last=true;
> EXPLAIN EXTENDED
> SELECT src1.key, src2.value FROM src src1 JOIN src src2 ON (src1.key =
> src2.key) ORDER BY src1.key LIMIT 5;
> {code}
> {code}
> PREHOOK: query: EXPLAIN EXTENDED
> SELECT src1.key, src2.value FROM src src1 JOIN src src2 ON (src1.key =
> src2.key) ORDER BY src1.key
> PREHOOK: type: QUERY
> PREHOOK: Input: default@src
> A masked pattern was here
> POSTHOOK: query: EXPLAIN EXTENDED
> SELECT src1.key, src2.value FROM src src1 JOIN src src2 ON (src1.key =
> src2.key) ORDER BY src1.key
> POSTHOOK: type: QUERY
> POSTHOOK: Input: default@src
> A masked pattern was here
> OPTIMIZED SQL: SELECT `t0`.`key`, `t2`.`value`
> FROM (SELECT `key`
> FROM `default`.`src`
> WHERE `key` IS NOT NULL) AS `t0`
> INNER JOIN (SELECT `key`, `value`
> FROM `default`.`src`
> WHERE `key` IS NOT NULL) AS `t2` ON `t0`.`key` = `t2`.`key`
> ORDER BY `t0`.`key`
> STAGE DEPENDENCIES:
> Stage-1 is a root stage
> Stage-0 depends on stages: Stage-1
> STAGE PLANS:
> Stage: Stage-1
> Tez
> A masked pattern was here
> Edges:
> Reducer 2 <- Map 1 (SIMPLE_EDGE), Map 4 (SIMPLE_EDGE)
> Reducer 3 <- Reducer 2 (SIMPLE_EDGE)
> A masked pattern was here
> Vertices:
> Map 1
> Map Operator Tree:
> TableScan
> alias: src1
> filterExpr: key is not null (type: boolean)
> Statistics: Num rows: 500 Data size: 43500 Basic stats:
> COMPLETE Column stats: COMPLETE
> GatherStats: false
> Filter Operator
> isSamplingPred: false
> predicate: key is not null (type: boolean)
> Statistics: Num rows: 500 Data size: 43500 Basic stats:
> COMPLETE Column stats: COMPLETE
> Select Operator
> expressions: key (type: string)
> outputColumnNames: _col0
> Statistics: Num rows: 500 Data size: 43500 Basic stats:
> COMPLETE Column stats: COMPLETE
> Reduce Output Operator
> key expressions: _col0 (type: string)
> null sort order: a
> sort order: +
> Map-reduce partition columns: _col0 (type: string)
> Statistics: Num rows: 500 Data size: 43500 Basic
> stats: COMPLETE Column stats: COMPLETE
> tag: 0
> auto parallelism: true
> Execution mode: vectorized, llap
> LLAP IO: no inputs
> Path -> Alias:
> A masked pattern was here
> Path -> Partition:
> A masked pattern was here
> Partition
> base file name: src
> input format: org.apache.hadoop.mapred.TextInputFormat
> output format:
> org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
> properties:
> COLUMN_STATS_ACCURATE
> {"BASIC_STATS":"true","COLUMN_STATS":{"key":"true","value":"true"}}
> bucket_count -1
> bucketing_version 2
> column.name.delimiter ,
> columns key,value
> columns.comments 'default','default'
> columns.types string:string
> A masked pattern was here
> name default.src
> numFiles 1
> numRows 500
> rawDataSize 5312
> serialization.ddl struct src { string key, string value}
> serialization.format 1
> serialization.lib
> org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
> totalSize 5812
> A
[jira] [Updated] (HIVE-22489) Reduce Sink operator should order nulls by parameter
[
https://issues.apache.org/jira/browse/HIVE-22489?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Krisztian Kasa updated HIVE-22489:
--
Status: Patch Available (was: Open)
> Reduce Sink operator should order nulls by parameter
> -
>
> Key: HIVE-22489
> URL: https://issues.apache.org/jira/browse/HIVE-22489
> Project: Hive
> Issue Type: Bug
> Components: Query Planning
>Reporter: Krisztian Kasa
>Assignee: Krisztian Kasa
>Priority: Major
> Attachments: HIVE-22489.1.patch, HIVE-22489.2.patch,
> HIVE-22489.3.patch, HIVE-22489.3.patch, HIVE-22489.4.patch,
> HIVE-22489.5.patch, HIVE-22489.6.patch
>
>
> When the property hive.default.nulls.last is set to true and no null order is
> explicitly specified in the ORDER BY clause of the query null ordering should
> be NULLS LAST.
> But some of the Reduce Sink operators still orders null first.
> {code}
> SET hive.default.nulls.last=true;
> EXPLAIN EXTENDED
> SELECT src1.key, src2.value FROM src src1 JOIN src src2 ON (src1.key =
> src2.key) ORDER BY src1.key LIMIT 5;
> {code}
> {code}
> PREHOOK: query: EXPLAIN EXTENDED
> SELECT src1.key, src2.value FROM src src1 JOIN src src2 ON (src1.key =
> src2.key) ORDER BY src1.key
> PREHOOK: type: QUERY
> PREHOOK: Input: default@src
> A masked pattern was here
> POSTHOOK: query: EXPLAIN EXTENDED
> SELECT src1.key, src2.value FROM src src1 JOIN src src2 ON (src1.key =
> src2.key) ORDER BY src1.key
> POSTHOOK: type: QUERY
> POSTHOOK: Input: default@src
> A masked pattern was here
> OPTIMIZED SQL: SELECT `t0`.`key`, `t2`.`value`
> FROM (SELECT `key`
> FROM `default`.`src`
> WHERE `key` IS NOT NULL) AS `t0`
> INNER JOIN (SELECT `key`, `value`
> FROM `default`.`src`
> WHERE `key` IS NOT NULL) AS `t2` ON `t0`.`key` = `t2`.`key`
> ORDER BY `t0`.`key`
> STAGE DEPENDENCIES:
> Stage-1 is a root stage
> Stage-0 depends on stages: Stage-1
> STAGE PLANS:
> Stage: Stage-1
> Tez
> A masked pattern was here
> Edges:
> Reducer 2 <- Map 1 (SIMPLE_EDGE), Map 4 (SIMPLE_EDGE)
> Reducer 3 <- Reducer 2 (SIMPLE_EDGE)
> A masked pattern was here
> Vertices:
> Map 1
> Map Operator Tree:
> TableScan
> alias: src1
> filterExpr: key is not null (type: boolean)
> Statistics: Num rows: 500 Data size: 43500 Basic stats:
> COMPLETE Column stats: COMPLETE
> GatherStats: false
> Filter Operator
> isSamplingPred: false
> predicate: key is not null (type: boolean)
> Statistics: Num rows: 500 Data size: 43500 Basic stats:
> COMPLETE Column stats: COMPLETE
> Select Operator
> expressions: key (type: string)
> outputColumnNames: _col0
> Statistics: Num rows: 500 Data size: 43500 Basic stats:
> COMPLETE Column stats: COMPLETE
> Reduce Output Operator
> key expressions: _col0 (type: string)
> null sort order: a
> sort order: +
> Map-reduce partition columns: _col0 (type: string)
> Statistics: Num rows: 500 Data size: 43500 Basic
> stats: COMPLETE Column stats: COMPLETE
> tag: 0
> auto parallelism: true
> Execution mode: vectorized, llap
> LLAP IO: no inputs
> Path -> Alias:
> A masked pattern was here
> Path -> Partition:
> A masked pattern was here
> Partition
> base file name: src
> input format: org.apache.hadoop.mapred.TextInputFormat
> output format:
> org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
> properties:
> COLUMN_STATS_ACCURATE
> {"BASIC_STATS":"true","COLUMN_STATS":{"key":"true","value":"true"}}
> bucket_count -1
> bucketing_version 2
> column.name.delimiter ,
> columns key,value
> columns.comments 'default','default'
> columns.types string:string
> A masked pattern was here
> name default.src
> numFiles 1
> numRows 500
> rawDataSize 5312
> serialization.ddl struct src { string key, string value}
> serialization.format 1
> serialization.lib
> org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
> totalSize 5812
> A
[jira] [Updated] (HIVE-22489) Reduce Sink operator should order nulls by parameter
[
https://issues.apache.org/jira/browse/HIVE-22489?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Krisztian Kasa updated HIVE-22489:
--
Attachment: HIVE-22489.6.patch
> Reduce Sink operator should order nulls by parameter
> -
>
> Key: HIVE-22489
> URL: https://issues.apache.org/jira/browse/HIVE-22489
> Project: Hive
> Issue Type: Bug
> Components: Query Planning
>Reporter: Krisztian Kasa
>Assignee: Krisztian Kasa
>Priority: Major
> Attachments: HIVE-22489.1.patch, HIVE-22489.2.patch,
> HIVE-22489.3.patch, HIVE-22489.3.patch, HIVE-22489.4.patch,
> HIVE-22489.5.patch, HIVE-22489.6.patch
>
>
> When the property hive.default.nulls.last is set to true and no null order is
> explicitly specified in the ORDER BY clause of the query null ordering should
> be NULLS LAST.
> But some of the Reduce Sink operators still orders null first.
> {code}
> SET hive.default.nulls.last=true;
> EXPLAIN EXTENDED
> SELECT src1.key, src2.value FROM src src1 JOIN src src2 ON (src1.key =
> src2.key) ORDER BY src1.key LIMIT 5;
> {code}
> {code}
> PREHOOK: query: EXPLAIN EXTENDED
> SELECT src1.key, src2.value FROM src src1 JOIN src src2 ON (src1.key =
> src2.key) ORDER BY src1.key
> PREHOOK: type: QUERY
> PREHOOK: Input: default@src
> A masked pattern was here
> POSTHOOK: query: EXPLAIN EXTENDED
> SELECT src1.key, src2.value FROM src src1 JOIN src src2 ON (src1.key =
> src2.key) ORDER BY src1.key
> POSTHOOK: type: QUERY
> POSTHOOK: Input: default@src
> A masked pattern was here
> OPTIMIZED SQL: SELECT `t0`.`key`, `t2`.`value`
> FROM (SELECT `key`
> FROM `default`.`src`
> WHERE `key` IS NOT NULL) AS `t0`
> INNER JOIN (SELECT `key`, `value`
> FROM `default`.`src`
> WHERE `key` IS NOT NULL) AS `t2` ON `t0`.`key` = `t2`.`key`
> ORDER BY `t0`.`key`
> STAGE DEPENDENCIES:
> Stage-1 is a root stage
> Stage-0 depends on stages: Stage-1
> STAGE PLANS:
> Stage: Stage-1
> Tez
> A masked pattern was here
> Edges:
> Reducer 2 <- Map 1 (SIMPLE_EDGE), Map 4 (SIMPLE_EDGE)
> Reducer 3 <- Reducer 2 (SIMPLE_EDGE)
> A masked pattern was here
> Vertices:
> Map 1
> Map Operator Tree:
> TableScan
> alias: src1
> filterExpr: key is not null (type: boolean)
> Statistics: Num rows: 500 Data size: 43500 Basic stats:
> COMPLETE Column stats: COMPLETE
> GatherStats: false
> Filter Operator
> isSamplingPred: false
> predicate: key is not null (type: boolean)
> Statistics: Num rows: 500 Data size: 43500 Basic stats:
> COMPLETE Column stats: COMPLETE
> Select Operator
> expressions: key (type: string)
> outputColumnNames: _col0
> Statistics: Num rows: 500 Data size: 43500 Basic stats:
> COMPLETE Column stats: COMPLETE
> Reduce Output Operator
> key expressions: _col0 (type: string)
> null sort order: a
> sort order: +
> Map-reduce partition columns: _col0 (type: string)
> Statistics: Num rows: 500 Data size: 43500 Basic
> stats: COMPLETE Column stats: COMPLETE
> tag: 0
> auto parallelism: true
> Execution mode: vectorized, llap
> LLAP IO: no inputs
> Path -> Alias:
> A masked pattern was here
> Path -> Partition:
> A masked pattern was here
> Partition
> base file name: src
> input format: org.apache.hadoop.mapred.TextInputFormat
> output format:
> org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
> properties:
> COLUMN_STATS_ACCURATE
> {"BASIC_STATS":"true","COLUMN_STATS":{"key":"true","value":"true"}}
> bucket_count -1
> bucketing_version 2
> column.name.delimiter ,
> columns key,value
> columns.comments 'default','default'
> columns.types string:string
> A masked pattern was here
> name default.src
> numFiles 1
> numRows 500
> rawDataSize 5312
> serialization.ddl struct src { string key, string value}
> serialization.format 1
> serialization.lib
> org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
> totalSize 5812
> A masked
[jira] [Commented] (HIVE-22546) Postgres schema not using quoted identifiers for certain tables
[
https://issues.apache.org/jira/browse/HIVE-22546?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16999873#comment-16999873
]
Zoltan Chovan commented on HIVE-22546:
--
[~ashutosh.bapat] thanks for the heads up! I guess schema changes like this are
harder to test than I thought. I'll assign HIVE-22663 to myself.
> Postgres schema not using quoted identifiers for certain tables
> ---
>
> Key: HIVE-22546
> URL: https://issues.apache.org/jira/browse/HIVE-22546
> Project: Hive
> Issue Type: Bug
> Components: Database/Schema
>Affects Versions: 4.0.0
>Reporter: Zoltan Chovan
>Assignee: Zoltan Chovan
>Priority: Major
> Fix For: 4.0.0
>
> Attachments: HIVE-22546.2.patch, HIVE-22546.3.patch,
> HIVE-22546.3.patch, HIVE-22546.4.patch, HIVE-22546.5.patch,
> HIVE-22546.6.patch, HIVE-22546.patch
>
>
> In the latest postgresql schema
> (standalone-metastore/metastore-server/src/main/sql/postgres/hive-schema-4.0.0.postgres.sql)
> the following tables have lowercase table and column names:
> {code:java}
> aux_table
> compaction_queue
> completed_compactions
> completed_txn_components
> hive_locks
> materialization_rebuild_locks
> min_history_level
> next_compaction_queue_id
> next_lock_id
> next_txn_id
> next_write_id
> repl_txn_map
> runtime_stats
> txn_components
> txn_to_write_id
> txns
> write_set{code}
> As these tables are referenced from the Hive sys database, the queries to
> these tables will fail with a "Table not found" error.
> The problem is that the table and column names are not enclosed in quotes, so
> postgres will turn these identifiers into lowercase.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Assigned] (HIVE-22663) Quote all table and column names or do not quote any
[
https://issues.apache.org/jira/browse/HIVE-22663?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zoltan Chovan reassigned HIVE-22663:
Assignee: Zoltan Chovan
> Quote all table and column names or do not quote any
>
>
> Key: HIVE-22663
> URL: https://issues.apache.org/jira/browse/HIVE-22663
> Project: Hive
> Issue Type: Bug
> Components: HiveServer2, Standalone Metastore
>Affects Versions: 4.0.0
>Reporter: Ashutosh Bapat
>Assignee: Zoltan Chovan
>Priority: Major
>
> The change in HIVE-22546 is causing following stack trace when I run Hive
> with PostgreSQL as backend db for the metastore.
> 0: jdbc:hive2://localhost:1> create database dumpdb with
> ('repl.source.for'='1,2,3');0: jdbc:hive2://localhost:1> create database
> dumpdb with ('repl.source.for'='1,2,3');Error: Error while compiling
> statement: FAILED: ParseException line 1:28 missing KW_DBPROPERTIES at '('
> near '' (state=42000,code=4)0: jdbc:hive2://localhost:1> create
> database dumpdb with dbproperties ('repl.source.for'='1,2,3');ERROR : FAILED:
> Hive Internal Error: org.apache.hadoop.hive.ql.lockmgr.LockException(Error
> communicating with the
> metastore)org.apache.hadoop.hive.ql.lockmgr.LockException: Error
> communicating with the metastore at
> org.apache.hadoop.hive.ql.lockmgr.DbTxnManager.commitTxn(DbTxnManager.java:541)
> at
> org.apache.hadoop.hive.ql.Driver.releaseLocksAndCommitOrRollback(Driver.java:687)
> at
> org.apache.hadoop.hive.ql.Driver.releaseLocksAndCommitOrRollback(Driver.java:653)
> at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:969)
> ... stack trace clipped
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)Caused by:
> MetaException(message:Unable to update transaction database
> org.postgresql.util.PSQLException: ERROR: relation
> "materialization_rebuild_locks" does not exist Position: 13 at
> org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2440)
> at
> org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:2183)
> at
> org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:308)
> at org.postgresql.jdbc.PgStatement.executeInternal(PgStatement.java:441) at
> org.postgresql.jdbc.PgStatement.execute(PgStatement.java:365) at
> This happens because the table names in all the queries in TxnHandler.java
> (including the one at 1312, which causes this stack trace) are not quoting
> the table names. All the tablenames and column names should be quoted there.
> Just the change in HIVE-22546 won't suffice.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22515) Support cast to decimal64 in Vectorization
[
https://issues.apache.org/jira/browse/HIVE-22515?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16999870#comment-16999870
]
Hive QA commented on HIVE-22515:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12989154/HIVE-22515.8.patch
{color:green}SUCCESS:{color} +1 due to 2 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 11 failed/errored test(s), 17781 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.ql.parse.TestTableLevelReplicationScenarios.testBasicBootstrapWithIncludeAndExcludeList
(batchId=259)
org.apache.hadoop.hive.ql.parse.TestTableLevelReplicationScenarios.testBasicBootstrapWithIncludeList
(batchId=259)
org.apache.hadoop.hive.ql.parse.TestTableLevelReplicationScenarios.testBasicReplaceReplPolicy
(batchId=259)
org.apache.hadoop.hive.ql.parse.TestTableLevelReplicationScenarios.testBootstrapExternalTablesWithIncludeAndExcludeList
(batchId=259)
org.apache.hadoop.hive.ql.parse.TestTableLevelReplicationScenarios.testCaseInSensitiveNatureOfReplPolicy
(batchId=259)
org.apache.hadoop.hive.ql.parse.TestTableLevelReplicationScenarios.testIncorrectTablePolicyInReplDump
(batchId=259)
org.apache.hadoop.hive.ql.parse.TestTableLevelReplicationScenarios.testRenameTableScenariosExternalTable
(batchId=259)
org.apache.hadoop.hive.ql.parse.TestTableLevelReplicationScenarios.testRenameTableScenariosUpgrade
(batchId=259)
org.apache.hadoop.hive.ql.parse.TestTableLevelReplicationScenarios.testRenameTableScenariosWithReplacePolicyDMLOperattion
(batchId=259)
org.apache.hadoop.hive.ql.parse.TestTableLevelReplicationScenarios.testReplacePolicyOnBootstrapAcidTablesIncrementalPhase
(batchId=259)
org.apache.hive.service.server.TestHS2HttpServerPam.org.apache.hive.service.server.TestHS2HttpServerPam
(batchId=241)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/19984/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/19984/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-19984/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 11 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12989154 - PreCommit-HIVE-Build
> Support cast to decimal64 in Vectorization
> --
>
> Key: HIVE-22515
> URL: https://issues.apache.org/jira/browse/HIVE-22515
> Project: Hive
> Issue Type: Bug
>Reporter: Ramesh Kumar Thangarajan
>Assignee: Ramesh Kumar Thangarajan
>Priority: Major
> Attachments: HIVE-22515.5.patch, HIVE-22515.8.patch
>
>
> Support cast to decimal64 in Vectorization
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22515) Support cast to decimal64 in Vectorization
[
https://issues.apache.org/jira/browse/HIVE-22515?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16999843#comment-16999843
]
Hive QA commented on HIVE-22515:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
56s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
50s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
8s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
57s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
43s{color} | {color:blue} ql in master has 1531 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
4s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
24s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
31s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
12s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
12s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
43s{color} | {color:red} ql: The patch generated 34 new + 740 unchanged - 40
fixed = 774 total (was 780) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 3m
59s{color} | {color:red} ql generated 1 new + 1530 unchanged - 1 fixed = 1531
total (was 1531) {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
3s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
13s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 25m 27s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| FindBugs | module:ql |
| | Redundant nullcheck of filterExpr, which is known to be non-null in
org.apache.hadoop.hive.ql.exec.vector.VectorizationContext.createDecimal64VectorExpression(Class,
List, VectorExpressionDescriptor$Mode, boolean, int, TypeInfo,
DataTypePhysicalVariation, boolean, GenericUDF) Redundant null check at
VectorizationContext.java:is known to be non-null in
org.apache.hadoop.hive.ql.exec.vector.VectorizationContext.createDecimal64VectorExpression(Class,
List, VectorExpressionDescriptor$Mode, boolean, int, TypeInfo,
DataTypePhysicalVariation, boolean, GenericUDF) Redundant null check at
VectorizationContext.java:[line 1939] |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-19984/dev-support/hive-personality.sh
|
| git revision | master / d5aec85 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.1 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-19984/yetus/diff-checkstyle-ql.txt
|
| findbugs |
http://104.198.109.242/logs//PreCommit-HIVE-Build-19984/yetus/new-findbugs-ql.html
|
| modules | C: vector-code-gen ql itests U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-19984/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Support cast to decimal64 in Vectorization
> --
>
> Key: HIVE-22515
> URL: https://issues.apache.org/jira/browse/HIVE-22515
> Project: Hive
> Issue Type: Bug
>Reporter: Ramesh Kumar Thangarajan
>Assignee: Ramesh Kumar Thangarajan
>Priority: Major
> Attachments:
