[jira] [Commented] (HIVE-18658) WM: allow not specifying scheduling policy when creating a pool
[
https://issues.apache.org/jira/browse/HIVE-18658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16359301#comment-16359301
]
Hive QA commented on HIVE-18658:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
1s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
28s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
2s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
41s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
55s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
23s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
7s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
7s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
40s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
55s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
12s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 14m 41s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /data/hiveptest/working/yetus/dev-support/hive-personality.sh |
| git revision | master / 89e75c7 |
| Default Java | 1.8.0_111 |
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-9133/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> WM: allow not specifying scheduling policy when creating a pool
> ---
>
> Key: HIVE-18658
> URL: https://issues.apache.org/jira/browse/HIVE-18658
> Project: Hive
> Issue Type: Bug
>Reporter: Sergey Shelukhin
>Assignee: Sergey Shelukhin
>Priority: Major
> Attachments: HIVE-18658.patch
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-18659) add acid version marker to acid files
[
https://issues.apache.org/jira/browse/HIVE-18659?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16359282#comment-16359282
]
Hive QA commented on HIVE-18659:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12909849/HIVE-18659.01.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 27 failed/errored test(s), 13152 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestAccumuloCliDriver.testCliDriver[accumulo_queries]
(batchId=241)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[acid_table_stats]
(batchId=54)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[autoColumnStats_4]
(batchId=12)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[ppd_join5] (batchId=36)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[row__id] (batchId=79)
org.apache.hadoop.hive.cli.TestEncryptedHDFSCliDriver.testCliDriver[encryption_move_tbl]
(batchId=175)
org.apache.hadoop.hive.cli.TestMiniDruidCliDriver.testCliDriver[druidmini_mv]
(batchId=249)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[acid_bucket_pruning]
(batchId=149)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[llap_smb]
(batchId=152)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[bucket_map_join_tez1]
(batchId=172)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[insert_values_orig_table_use_metadata]
(batchId=167)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[llap_acid]
(batchId=171)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[llap_acid_fast]
(batchId=162)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[resourceplan]
(batchId=164)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[sysdb]
(batchId=161)
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver[spark_opt_shuffle_serde]
(batchId=180)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[ppd_join5]
(batchId=122)
org.apache.hadoop.hive.cli.TestSparkPerfCliDriver.testCliDriver[query1]
(batchId=251)
org.apache.hadoop.hive.cli.control.TestDanglingQOuts.checkDanglingQOut
(batchId=222)
org.apache.hadoop.hive.metastore.client.TestTablesList.testListTableNamesByFilterNullDatabase[Embedded]
(batchId=206)
org.apache.hadoop.hive.ql.io.TestDruidRecordWriter.testWrite (batchId=257)
org.apache.hadoop.hive.ql.io.orc.TestInputOutputFormat.testCombinationInputFormatWithAcid
(batchId=273)
org.apache.hive.beeline.cli.TestHiveCli.testNoErrorDB (batchId=188)
org.apache.hive.jdbc.TestSSL.testConnectionMismatch (batchId=235)
org.apache.hive.jdbc.TestSSL.testConnectionWrongCertCN (batchId=235)
org.apache.hive.jdbc.TestSSL.testMetastoreConnectionWrongCertCN (batchId=235)
org.apache.hive.service.cli.TestCLIServiceConnectionLimits.testConnectionLimitPerUserIpAddress
(batchId=218)
{noformat}
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/9132/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/9132/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-9132/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 27 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12909849 - PreCommit-HIVE-Build
> add acid version marker to acid files
> -
>
> Key: HIVE-18659
> URL: https://issues.apache.org/jira/browse/HIVE-18659
> Project: Hive
> Issue Type: Bug
> Components: Transactions
>Reporter: Eugene Koifman
>Assignee: Eugene Koifman
>Priority: Major
> Attachments: HIVE-18659.01.patch
>
>
> add acid version marker to acid files so that we know which version of acid
> wrote the file
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-18218) SMB Join : Handle buckets with no splits.
[ https://issues.apache.org/jira/browse/HIVE-18218?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16359273#comment-16359273 ] Deepak Jaiswal commented on HIVE-18218: --- Fixed the couple of test failures related to this patch. > SMB Join : Handle buckets with no splits. > - > > Key: HIVE-18218 > URL: https://issues.apache.org/jira/browse/HIVE-18218 > Project: Hive > Issue Type: Bug >Reporter: Deepak Jaiswal >Assignee: Deepak Jaiswal >Priority: Major > Attachments: HIVE-18218.1.patch, HIVE-18218.2.patch, > HIVE-18218.3.patch, HIVE-18218.4.patch, HIVE-18218.5.patch > > > While working on HIVE-18208, it was found that with SMB, the results are > incorrect. This most likely is a product issue. > auto_sortmerge_join_16 fails with wrong results due to this. > cc [~hagleitn] > The current logic in CustomPartitionVertex assumes that there is a split for > each bucket whereas in Tez, we can have no splits for empty buckets. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-18218) SMB Join : Handle buckets with no splits.
[ https://issues.apache.org/jira/browse/HIVE-18218?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Deepak Jaiswal updated HIVE-18218: -- Attachment: HIVE-18218.5.patch > SMB Join : Handle buckets with no splits. > - > > Key: HIVE-18218 > URL: https://issues.apache.org/jira/browse/HIVE-18218 > Project: Hive > Issue Type: Bug >Reporter: Deepak Jaiswal >Assignee: Deepak Jaiswal >Priority: Major > Attachments: HIVE-18218.1.patch, HIVE-18218.2.patch, > HIVE-18218.3.patch, HIVE-18218.4.patch, HIVE-18218.5.patch > > > While working on HIVE-18208, it was found that with SMB, the results are > incorrect. This most likely is a product issue. > auto_sortmerge_join_16 fails with wrong results due to this. > cc [~hagleitn] > The current logic in CustomPartitionVertex assumes that there is a split for > each bucket whereas in Tez, we can have no splits for empty buckets. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-18659) add acid version marker to acid files
[
https://issues.apache.org/jira/browse/HIVE-18659?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16359272#comment-16359272
]
Hive QA commented on HIVE-18659:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
48s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
6s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
47s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
56s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
26s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
9s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
9s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
46s{color} | {color:red} ql: The patch generated 7 new + 374 unchanged - 3
fixed = 381 total (was 377) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
56s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
12s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 15m 23s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /data/hiveptest/working/yetus/dev-support/hive-personality.sh |
| git revision | master / 89e75c7 |
| Default Java | 1.8.0_111 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-9132/yetus/diff-checkstyle-ql.txt
|
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-9132/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> add acid version marker to acid files
> -
>
> Key: HIVE-18659
> URL: https://issues.apache.org/jira/browse/HIVE-18659
> Project: Hive
> Issue Type: Bug
> Components: Transactions
>Reporter: Eugene Koifman
>Assignee: Eugene Koifman
>Priority: Major
> Attachments: HIVE-18659.01.patch
>
>
> add acid version marker to acid files so that we know which version of acid
> wrote the file
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-18492) Wrong argument in the WorkloadManager.resetAndQueryKill()
[
https://issues.apache.org/jira/browse/HIVE-18492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16359266#comment-16359266
]
Hive QA commented on HIVE-18492:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12909848/HIVE-18492.04.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 23 failed/errored test(s), 13151 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestAccumuloCliDriver.testCliDriver[accumulo_queries]
(batchId=241)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[mapjoin_hook]
(batchId=13)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[ppd_join5] (batchId=36)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[row__id] (batchId=79)
org.apache.hadoop.hive.cli.TestEncryptedHDFSCliDriver.testCliDriver[encryption_move_tbl]
(batchId=175)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[llap_smb]
(batchId=152)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[bucket_map_join_tez1]
(batchId=172)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[insert_values_orig_table_use_metadata]
(batchId=167)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[llap_acid]
(batchId=171)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[llap_acid_fast]
(batchId=162)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[resourceplan]
(batchId=164)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[sysdb]
(batchId=161)
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver[spark_opt_shuffle_serde]
(batchId=180)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[ppd_join5]
(batchId=122)
org.apache.hadoop.hive.cli.TestSparkPerfCliDriver.testCliDriver[query1]
(batchId=251)
org.apache.hadoop.hive.cli.control.TestDanglingQOuts.checkDanglingQOut
(batchId=222)
org.apache.hadoop.hive.ql.TestAutoPurgeTables.testAutoPurgeUnset (batchId=231)
org.apache.hadoop.hive.ql.io.TestDruidRecordWriter.testWrite (batchId=257)
org.apache.hive.beeline.TestBeeLineWithArgs.testEscapeCRLFInTSV2Output
(batchId=232)
org.apache.hive.beeline.cli.TestHiveCli.testNoErrorDB (batchId=188)
org.apache.hive.jdbc.TestSSL.testConnectionMismatch (batchId=235)
org.apache.hive.jdbc.TestSSL.testConnectionWrongCertCN (batchId=235)
org.apache.hive.jdbc.TestSSL.testMetastoreConnectionWrongCertCN (batchId=235)
{noformat}
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/9131/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/9131/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-9131/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 23 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12909848 - PreCommit-HIVE-Build
> Wrong argument in the WorkloadManager.resetAndQueryKill()
> -
>
> Key: HIVE-18492
> URL: https://issues.apache.org/jira/browse/HIVE-18492
> Project: Hive
> Issue Type: Bug
>Reporter: Oleg Danilov
>Assignee: Oleg Danilov
>Priority: Trivial
> Attachments: HIVE-18492.03.patch, HIVE-18492.04.patch,
> HIVE-18492.2.patch, HIVE-18492.patch
>
>
> Caused by HIVE-18088, [~prasanth_j], could you please check this?
> {code:java}
> private void resetAndQueueKill(Map
> toKillQuery,
> KillQueryContext killQueryContext, Map toReuse) {
> WmTezSession toKill = killQueryContext.session;
> ...
> if (poolState != null) {
> poolState.getSessions().remove(toKill);
> poolState.getInitializingSessions().remove(toKill);
> ...
> {code}
> getInitializingSessions() returns List of SessionInitContext, so toKill
> definitely can't be in this list and therefore no needs to remove it.
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-18676) Usability: Better warnings for Unicode single quotes
[
https://issues.apache.org/jira/browse/HIVE-18676?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16359249#comment-16359249
]
Madhudeep Petwal commented on HIVE-18676:
-
Hi [~gopalv] can you describe this in detail?
Can you provide table used here and other details like hive version you used
for bug reproduction ?
> Usability: Better warnings for Unicode single quotes
>
>
> Key: HIVE-18676
> URL: https://issues.apache.org/jira/browse/HIVE-18676
> Project: Hive
> Issue Type: Improvement
> Components: Parser
>Reporter: Gopal V
>Priority: Major
> Labels: usability
>
> The warning for
> {code}
> select d_date as ‘date’ from date_dim;
> {code}
> is hard to locate than it should be.
> {code}
> Error: Error while compiling statement: FAILED: ParseException line 1:18
> cannot recognize input near 'as' 'date' 'from' in selection target
> (state=42000,code=4)
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Assigned] (HIVE-18676) Usability: Better warnings for Unicode single quotes
[
https://issues.apache.org/jira/browse/HIVE-18676?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Madhudeep Petwal reassigned HIVE-18676:
---
Assignee: Madhudeep Petwal
> Usability: Better warnings for Unicode single quotes
>
>
> Key: HIVE-18676
> URL: https://issues.apache.org/jira/browse/HIVE-18676
> Project: Hive
> Issue Type: Improvement
> Components: Parser
>Reporter: Gopal V
>Assignee: Madhudeep Petwal
>Priority: Major
> Labels: usability
>
> The warning for
> {code}
> select d_date as ‘date’ from date_dim;
> {code}
> is hard to locate than it should be.
> {code}
> Error: Error while compiling statement: FAILED: ParseException line 1:18
> cannot recognize input near 'as' 'date' 'from' in selection target
> (state=42000,code=4)
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-18492) Wrong argument in the WorkloadManager.resetAndQueryKill()
[
https://issues.apache.org/jira/browse/HIVE-18492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16359242#comment-16359242
]
Hive QA commented on HIVE-18492:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
1s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
39s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
3s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
41s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
55s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
23s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
6s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
6s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
42s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
57s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
14s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 14m 57s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /data/hiveptest/working/yetus/dev-support/hive-personality.sh |
| git revision | master / 89e75c7 |
| Default Java | 1.8.0_111 |
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-9131/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Wrong argument in the WorkloadManager.resetAndQueryKill()
> -
>
> Key: HIVE-18492
> URL: https://issues.apache.org/jira/browse/HIVE-18492
> Project: Hive
> Issue Type: Bug
>Reporter: Oleg Danilov
>Assignee: Oleg Danilov
>Priority: Trivial
> Attachments: HIVE-18492.03.patch, HIVE-18492.04.patch,
> HIVE-18492.2.patch, HIVE-18492.patch
>
>
> Caused by HIVE-18088, [~prasanth_j], could you please check this?
> {code:java}
> private void resetAndQueueKill(Map
> toKillQuery,
> KillQueryContext killQueryContext, Map toReuse) {
> WmTezSession toKill = killQueryContext.session;
> ...
> if (poolState != null) {
> poolState.getSessions().remove(toKill);
> poolState.getInitializingSessions().remove(toKill);
> ...
> {code}
> getInitializingSessions() returns List of SessionInitContext, so toKill
> definitely can't be in this list and therefore no needs to remove it.
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-18387) Minimize time that REBUILD locks the materialized view
[
https://issues.apache.org/jira/browse/HIVE-18387?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16359228#comment-16359228
]
Hive QA commented on HIVE-18387:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
35s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m
7s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 7m
45s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 4m
22s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 8m
11s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
21s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
46s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 7m
53s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 7m
53s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
27s{color} | {color:red} standalone-metastore: The patch generated 3 new + 1266
unchanged - 1 fixed = 1269 total (was 1267) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
54s{color} | {color:red} ql: The patch generated 30 new + 1265 unchanged - 2
fixed = 1295 total (was 1267) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 2m
23s{color} | {color:red} root: The patch generated 34 new + 3181 unchanged - 4
fixed = 3215 total (was 3185) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
22s{color} | {color:red} itests/hive-unit: The patch generated 1 new + 630
unchanged - 1 fixed = 631 total (was 631) {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
0s{color} | {color:red} The patch has 39 line(s) that end in whitespace. Use
git apply --whitespace=fix <>. Refer
https://git-scm.com/docs/git-apply {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 8m
5s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
13s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 58m 26s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /data/hiveptest/working/yetus/dev-support/hive-personality.sh |
| git revision | master / 89e75c7 |
| Default Java | 1.8.0_111 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-9130/yetus/diff-checkstyle-standalone-metastore.txt
|
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-9130/yetus/diff-checkstyle-ql.txt
|
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-9130/yetus/diff-checkstyle-root.txt
|
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-9130/yetus/diff-checkstyle-itests_hive-unit.txt
|
| whitespace |
http://104.198.109.242/logs//PreCommit-HIVE-Build-9130/yetus/whitespace-eol.txt
|
| modules | C: standalone-metastore ql . itests/hcatalog-unit itests/hive-unit
U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-9130/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Minimize time that REBUILD locks the materialized view
> --
>
> Key: HIVE-18387
> URL: https://issues.apache.org/jira/browse/HIVE-18387
> Project: Hive
> Issue Type:
[jira] [Updated] (HIVE-18678) fix exim for MM tables and reinstante the test
[ https://issues.apache.org/jira/browse/HIVE-18678?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Shelukhin updated HIVE-18678: Status: Patch Available (was: Open) > fix exim for MM tables and reinstante the test > -- > > Key: HIVE-18678 > URL: https://issues.apache.org/jira/browse/HIVE-18678 > Project: Hive > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin >Priority: Major > Attachments: HIVE-18678.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-18678) fix exim for MM tables and reinstante the test
[ https://issues.apache.org/jira/browse/HIVE-18678?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16359216#comment-16359216 ] Sergey Shelukhin commented on HIVE-18678: - [~steveyeom2017] [~ekoifman] can you take a look? The code changes are pretty small > fix exim for MM tables and reinstante the test > -- > > Key: HIVE-18678 > URL: https://issues.apache.org/jira/browse/HIVE-18678 > Project: Hive > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin >Priority: Major > Attachments: HIVE-18678.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-18678) fix exim for MM tables and reinstante the test
[ https://issues.apache.org/jira/browse/HIVE-18678?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Shelukhin updated HIVE-18678: Attachment: HIVE-18678.patch > fix exim for MM tables and reinstante the test > -- > > Key: HIVE-18678 > URL: https://issues.apache.org/jira/browse/HIVE-18678 > Project: Hive > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin >Priority: Major > Attachments: HIVE-18678.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-18678) fix exim for MM tables and reinstante the test
[ https://issues.apache.org/jira/browse/HIVE-18678?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Shelukhin updated HIVE-18678: Attachment: (was: HIVE-18678.patch) > fix exim for MM tables and reinstante the test > -- > > Key: HIVE-18678 > URL: https://issues.apache.org/jira/browse/HIVE-18678 > Project: Hive > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin >Priority: Major > Attachments: HIVE-18678.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-18218) SMB Join : Handle buckets with no splits.
[
https://issues.apache.org/jira/browse/HIVE-18218?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16359214#comment-16359214
]
Hive QA commented on HIVE-18218:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12910001/HIVE-18218.4.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 24 failed/errored test(s), 13151 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestAccumuloCliDriver.testCliDriver[accumulo_queries]
(batchId=241)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[ppd_join5] (batchId=36)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[row__id] (batchId=79)
org.apache.hadoop.hive.cli.TestEncryptedHDFSCliDriver.testCliDriver[encryption_move_tbl]
(batchId=175)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[bucket_map_join_tez1]
(batchId=172)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[insert_values_orig_table_use_metadata]
(batchId=167)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[llap_acid]
(batchId=171)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[llap_acid_fast]
(batchId=162)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[resourceplan]
(batchId=164)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[sysdb]
(batchId=161)
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver[auto_sortmerge_join_16]
(batchId=180)
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver[spark_opt_shuffle_serde]
(batchId=180)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[bucket_mapjoin_mismatch1]
(batchId=95)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[ppd_join5]
(batchId=122)
org.apache.hadoop.hive.cli.TestSparkPerfCliDriver.testCliDriver[query1]
(batchId=251)
org.apache.hadoop.hive.cli.control.TestDanglingQOuts.checkDanglingQOut
(batchId=222)
org.apache.hadoop.hive.metastore.client.TestTablesCreateDropAlterTruncate.testAlterTableNoSuchTableInThisDatabase[Embedded]
(batchId=206)
org.apache.hadoop.hive.metastore.client.TestTablesCreateDropAlterTruncate.testAlterTableNullStorageDescriptorInNew[Embedded]
(batchId=206)
org.apache.hadoop.hive.ql.exec.tez.TestWorkloadManager.testQueueing
(batchId=288)
org.apache.hadoop.hive.ql.io.TestDruidRecordWriter.testWrite (batchId=257)
org.apache.hive.beeline.cli.TestHiveCli.testNoErrorDB (batchId=188)
org.apache.hive.jdbc.TestSSL.testConnectionMismatch (batchId=235)
org.apache.hive.jdbc.TestSSL.testConnectionWrongCertCN (batchId=235)
org.apache.hive.jdbc.TestSSL.testMetastoreConnectionWrongCertCN (batchId=235)
{noformat}
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/9129/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/9129/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-9129/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 24 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12910001 - PreCommit-HIVE-Build
> SMB Join : Handle buckets with no splits.
> -
>
> Key: HIVE-18218
> URL: https://issues.apache.org/jira/browse/HIVE-18218
> Project: Hive
> Issue Type: Bug
>Reporter: Deepak Jaiswal
>Assignee: Deepak Jaiswal
>Priority: Major
> Attachments: HIVE-18218.1.patch, HIVE-18218.2.patch,
> HIVE-18218.3.patch, HIVE-18218.4.patch
>
>
> While working on HIVE-18208, it was found that with SMB, the results are
> incorrect. This most likely is a product issue.
> auto_sortmerge_join_16 fails with wrong results due to this.
> cc [~hagleitn]
> The current logic in CustomPartitionVertex assumes that there is a split for
> each bucket whereas in Tez, we can have no splits for empty buckets.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-18678) fix exim for MM tables and reinstante the test
[ https://issues.apache.org/jira/browse/HIVE-18678?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16359212#comment-16359212 ] Sergey Shelukhin commented on HIVE-18678: - This was surprisingly easy to fix after some digging. I think the test may need to be expanded for import overwrite... I may do it here or in separate jira, cause I was also going to fix load from export, which may justify a separate jira. > fix exim for MM tables and reinstante the test > -- > > Key: HIVE-18678 > URL: https://issues.apache.org/jira/browse/HIVE-18678 > Project: Hive > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin >Priority: Major > Attachments: HIVE-18678.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-18678) fix exim for MM tables and reinstante the test
[ https://issues.apache.org/jira/browse/HIVE-18678?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Shelukhin updated HIVE-18678: Attachment: HIVE-18678.patch > fix exim for MM tables and reinstante the test > -- > > Key: HIVE-18678 > URL: https://issues.apache.org/jira/browse/HIVE-18678 > Project: Hive > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin >Priority: Major > Attachments: HIVE-18678.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (HIVE-18678) fix exim for MM tables and reinstante the test
[ https://issues.apache.org/jira/browse/HIVE-18678?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Shelukhin reassigned HIVE-18678: --- > fix exim for MM tables and reinstante the test > -- > > Key: HIVE-18678 > URL: https://issues.apache.org/jira/browse/HIVE-18678 > Project: Hive > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin >Priority: Major > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-18668) Really shade guava in ql
[ https://issues.apache.org/jira/browse/HIVE-18668?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16359199#comment-16359199 ] Ashutosh Chauhan commented on HIVE-18668: - +1 pending tests > Really shade guava in ql > > > Key: HIVE-18668 > URL: https://issues.apache.org/jira/browse/HIVE-18668 > Project: Hive > Issue Type: Bug >Reporter: Zoltan Haindrich >Assignee: Zoltan Haindrich >Priority: Major > Attachments: HIVE-18668.01.patch > > > After HIVE-15393 a test started to fail in druid; after some investigation it > turned out that ql doesn't shade it's guava artifact at all...because it > shades 'com.google.guava' instead 'com.google.common' -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-18545) Add UDF to parse complex types from json
[ https://issues.apache.org/jira/browse/HIVE-18545?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16359197#comment-16359197 ] Ashutosh Chauhan commented on HIVE-18545: - We have similar code in org.apache.hive.hcatalog.data.JsonSerDe It would be good if we can refactor it so that it could be reused here. Further, this uses diff json parsing library than JsonSerDe so we will have dependencies on 2 diff libraries. > Add UDF to parse complex types from json > > > Key: HIVE-18545 > URL: https://issues.apache.org/jira/browse/HIVE-18545 > Project: Hive > Issue Type: Improvement >Reporter: Zoltan Haindrich >Assignee: Zoltan Haindrich >Priority: Major > Attachments: HIVE-18545.02.patch, HIVE-18545.03.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-18218) SMB Join : Handle buckets with no splits.
[
https://issues.apache.org/jira/browse/HIVE-18218?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16359188#comment-16359188
]
Hive QA commented on HIVE-18218:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
27s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
1s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
40s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
54s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
19s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
0s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
0s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
43s{color} | {color:red} ql: The patch generated 3 new + 85 unchanged - 0 fixed
= 88 total (was 85) {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
0s{color} | {color:red} The patch has 2 line(s) that end in whitespace. Use git
apply --whitespace=fix <>. Refer https://git-scm.com/docs/git-apply
{color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
0s{color} | {color:red} The patch 48 line(s) with tabs. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
55s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
12s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 14m 29s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /data/hiveptest/working/yetus/dev-support/hive-personality.sh |
| git revision | master / 89e75c7 |
| Default Java | 1.8.0_111 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-9129/yetus/diff-checkstyle-ql.txt
|
| whitespace |
http://104.198.109.242/logs//PreCommit-HIVE-Build-9129/yetus/whitespace-eol.txt
|
| whitespace |
http://104.198.109.242/logs//PreCommit-HIVE-Build-9129/yetus/whitespace-tabs.txt
|
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-9129/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> SMB Join : Handle buckets with no splits.
> -
>
> Key: HIVE-18218
> URL: https://issues.apache.org/jira/browse/HIVE-18218
> Project: Hive
> Issue Type: Bug
>Reporter: Deepak Jaiswal
>Assignee: Deepak Jaiswal
>Priority: Major
> Attachments: HIVE-18218.1.patch, HIVE-18218.2.patch,
> HIVE-18218.3.patch, HIVE-18218.4.patch
>
>
> While working on HIVE-18208, it was found that with SMB, the results are
> incorrect. This most likely is a product issue.
> auto_sortmerge_join_16 fails with wrong results due to this.
> cc [~hagleitn]
> The current logic in CustomPartitionVertex assumes that there is a split for
> each bucket whereas in Tez, we can have no splits for empty buckets.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-17983) Make the standalone metastore generate tarballs etc.
[
https://issues.apache.org/jira/browse/HIVE-17983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16359180#comment-16359180
]
Hive QA commented on HIVE-17983:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12909832/HIVE-17983.5.patch
{color:red}ERROR:{color} -1 due to build exiting with an error
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/9128/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/9128/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-9128/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Tests exited with: NonZeroExitCodeException
Command 'bash /data/hiveptest/working/scratch/source-prep.sh' failed with exit
status 1 and output '+ date '+%Y-%m-%d %T.%3N'
2018-02-10 01:20:55.786
+ [[ -n /usr/lib/jvm/java-8-openjdk-amd64 ]]
+ export JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
+ JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
+ export
PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+
PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+ export 'ANT_OPTS=-Xmx1g -XX:MaxPermSize=256m '
+ ANT_OPTS='-Xmx1g -XX:MaxPermSize=256m '
+ export 'MAVEN_OPTS=-Xmx1g '
+ MAVEN_OPTS='-Xmx1g '
+ cd /data/hiveptest/working/
+ tee /data/hiveptest/logs/PreCommit-HIVE-Build-9128/source-prep.txt
+ [[ false == \t\r\u\e ]]
+ mkdir -p maven ivy
+ [[ git = \s\v\n ]]
+ [[ git = \g\i\t ]]
+ [[ -z master ]]
+ [[ -d apache-github-source-source ]]
+ [[ ! -d apache-github-source-source/.git ]]
+ [[ ! -d apache-github-source-source ]]
+ date '+%Y-%m-%d %T.%3N'
2018-02-10 01:20:55.789
+ cd apache-github-source-source
+ git fetch origin
+ git reset --hard HEAD
HEAD is now at 89e75c7 HIVE-17837: Explicitly check if the HoS Remote Driver
has been lost in the RemoteSparkJobMonitor (Sahil Takiar, reviewed by Rui Li)
+ git clean -f -d
Removing ${project.basedir}/
+ git checkout master
Already on 'master'
Your branch is up-to-date with 'origin/master'.
+ git reset --hard origin/master
HEAD is now at 89e75c7 HIVE-17837: Explicitly check if the HoS Remote Driver
has been lost in the RemoteSparkJobMonitor (Sahil Takiar, reviewed by Rui Li)
+ git merge --ff-only origin/master
Already up-to-date.
+ date '+%Y-%m-%d %T.%3N'
2018-02-10 01:20:58.437
+ rm -rf ../yetus
+ mkdir ../yetus
+ git gc
+ cp -R . ../yetus
+ mkdir /data/hiveptest/logs/PreCommit-HIVE-Build-9128/yetus
+ patchCommandPath=/data/hiveptest/working/scratch/smart-apply-patch.sh
+ patchFilePath=/data/hiveptest/working/scratch/build.patch
+ [[ -f /data/hiveptest/working/scratch/build.patch ]]
+ chmod +x /data/hiveptest/working/scratch/smart-apply-patch.sh
+ /data/hiveptest/working/scratch/smart-apply-patch.sh
/data/hiveptest/working/scratch/build.patch
error: patch failed:
beeline/src/java/org/apache/hive/beeline/HiveSchemaTool.java:159
Falling back to three-way merge...
Applied patch to 'beeline/src/java/org/apache/hive/beeline/HiveSchemaTool.java'
with conflicts.
error: patch failed:
standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/MetaStoreSchemaInfo.java:209
Falling back to three-way merge...
Applied patch to
'standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/MetaStoreSchemaInfo.java'
with conflicts.
error: patch failed:
standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/tools/HiveSchemaHelper.java:56
Falling back to three-way merge...
Applied patch to
'standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/tools/HiveSchemaHelper.java'
with conflicts.
Going to apply patch with: git apply -p0
/data/hiveptest/working/scratch/build.patch:699: trailing whitespace.
You can download the Oracle driver at
/data/hiveptest/working/scratch/build.patch:724: trailing whitespace.
same directory as this NOTICE file.
/data/hiveptest/working/scratch/build.patch:1023: trailing whitespace.
/data/hiveptest/working/scratch/build.patch:3713: trailing whitespace.
elif [ "${METASTORE_AUX_JARS_PATH}" != "" ]; then
/data/hiveptest/working/scratch/build.patch:3731: trailing whitespace.
export HADOOP_HOME_WARN_SUPPRESS=true
error: patch failed:
beeline/src/java/org/apache/hive/beeline/HiveSchemaTool.java:159
Falling back to three-way merge...
Applied patch to 'beeline/src/java/org/apache/hive/beeline/HiveSchemaTool.java'
with conflicts.
error: patch failed:
standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/MetaStoreSchemaInfo.java:209
Falling back to three-way merge...
Applied patch to
'standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/MetaStoreSchemaInfo.java'
with conflicts.
error: patch failed:
standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/tools/HiveSchemaHelper.java:56
Falling
[jira] [Commented] (HIVE-18359) Extend grouping set limits from int to long
[
https://issues.apache.org/jira/browse/HIVE-18359?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16359178#comment-16359178
]
Hive QA commented on HIVE-18359:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12909979/HIVE-18359.12.patch
{color:green}SUCCESS:{color} +1 due to 5 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 25 failed/errored test(s), 13130 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestAccumuloCliDriver.testCliDriver[accumulo_queries]
(batchId=241)
org.apache.hadoop.hive.cli.TestCliDriver.org.apache.hadoop.hive.cli.TestCliDriver
(batchId=36)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[row__id] (batchId=79)
org.apache.hadoop.hive.cli.TestEncryptedHDFSCliDriver.testCliDriver[encryption_move_tbl]
(batchId=175)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[llap_smb]
(batchId=152)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[bucket_map_join_tez1]
(batchId=172)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[insert_values_orig_table_use_metadata]
(batchId=167)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[llap_acid]
(batchId=171)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[llap_acid_fast]
(batchId=162)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[resourceplan]
(batchId=164)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[sysdb]
(batchId=161)
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver[bucketizedhiveinputformat]
(batchId=180)
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver[bucketmapjoin6]
(batchId=180)
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver[dynamic_rdd_cache]
(batchId=180)
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver[spark_opt_shuffle_serde]
(batchId=180)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[ppd_join5]
(batchId=122)
org.apache.hadoop.hive.cli.TestSparkPerfCliDriver.testCliDriver[query1]
(batchId=251)
org.apache.hadoop.hive.cli.control.TestDanglingQOuts.checkDanglingQOut
(batchId=222)
org.apache.hadoop.hive.metastore.client.TestTablesCreateDropAlterTruncate.testAlterTableNullStorageDescriptorInNew[Embedded]
(batchId=206)
org.apache.hadoop.hive.metastore.client.TestTablesList.testListTableNamesByFilterNullDatabase[Embedded]
(batchId=206)
org.apache.hadoop.hive.ql.io.TestDruidRecordWriter.testWrite (batchId=257)
org.apache.hive.beeline.cli.TestHiveCli.testNoErrorDB (batchId=188)
org.apache.hive.jdbc.TestSSL.testConnectionMismatch (batchId=235)
org.apache.hive.jdbc.TestSSL.testConnectionWrongCertCN (batchId=235)
org.apache.hive.jdbc.TestSSL.testMetastoreConnectionWrongCertCN (batchId=235)
{noformat}
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/9127/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/9127/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-9127/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 25 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12909979 - PreCommit-HIVE-Build
> Extend grouping set limits from int to long
> ---
>
> Key: HIVE-18359
> URL: https://issues.apache.org/jira/browse/HIVE-18359
> Project: Hive
> Issue Type: Bug
>Affects Versions: 3.0.0
>Reporter: Prasanth Jayachandran
>Assignee: Prasanth Jayachandran
>Priority: Major
> Attachments: HIVE-18359.1.patch, HIVE-18359.10.patch,
> HIVE-18359.11.patch, HIVE-18359.11.patch, HIVE-18359.12.patch,
> HIVE-18359.2.patch, HIVE-18359.3.patch, HIVE-18359.4.patch,
> HIVE-18359.5.patch, HIVE-18359.6.patch, HIVE-18359.7.patch,
> HIVE-18359.8.patch, HIVE-18359.9.patch
>
>
> Grouping sets is broken for >32 columns because of usage of Int for bitmap
> (also GROUPING__ID virtual column). This assumption breaks grouping
> sets/rollups/cube when number of participating aggregation columns is >32.
> The easier fix would be extend it to Long for now. The correct fix would be
> to use BitSets everywhere but that would require GROUPING__ID column type to
> binary which will make predicates on GROUPING__ID difficult to deal with.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-18550) Keep the hbase table name property as hbase.table.name
[ https://issues.apache.org/jira/browse/HIVE-18550?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16359156#comment-16359156 ] Aihua Xu commented on HIVE-18550: - Patch-2: rebasing to the latest. > Keep the hbase table name property as hbase.table.name > -- > > Key: HIVE-18550 > URL: https://issues.apache.org/jira/browse/HIVE-18550 > Project: Hive > Issue Type: Sub-task > Components: HBase Handler >Affects Versions: 3.0.0 >Reporter: Aihua Xu >Assignee: Aihua Xu >Priority: Major > Attachments: HIVE-18550.1.patch, HIVE-18550.2.patch > > > With hbase 2.0 support, I made some changes to the hbase table name property > change in HIVE-18366 and HIVE-18202. By checking the logic, seems the change > is not necessary since hbase.table.name is internal to hive hbase handler. We > just need to map hbase.table.name to > hbase.mapreduce.hfileoutputformat.table.name for HiveHFileOutputFormat. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-18442) HoS: No FileSystem for scheme: nullscan
[
https://issues.apache.org/jira/browse/HIVE-18442?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16359134#comment-16359134
]
Xuefu Zhang commented on HIVE-18442:
[~lirui] Since it's a hack any way, is it okay just to use
{{spark.yarn.user.classpath.first}} configuration? I know it's not ideal.
> HoS: No FileSystem for scheme: nullscan
> ---
>
> Key: HIVE-18442
> URL: https://issues.apache.org/jira/browse/HIVE-18442
> Project: Hive
> Issue Type: Bug
> Components: Spark
>Reporter: Rui Li
>Assignee: Rui Li
>Priority: Major
> Attachments: HIVE-18442.1.patch, HIVE-18442.2.patch
>
>
> Hit the issue when I run following query in yarn-cluster mode:
> {code}
> select * from (select key from src where false) a left outer join (select key
> from srcpart limit 0) b on a.key=b.key;
> {code}
> Stack trace:
> {noformat}
> Job failed with java.io.IOException: No FileSystem for scheme: nullscan
> at
> org.apache.hadoop.fs.FileSystem.getFileSystemClass(FileSystem.java:2799)
> at
> org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:2810)
> at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:100)
> at
> org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:2849)
> at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:2831)
> at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:389)
> at org.apache.hadoop.fs.Path.getFileSystem(Path.java:356)
> at
> org.apache.hadoop.hive.ql.exec.Utilities.isEmptyPath(Utilities.java:2605)
> at
> org.apache.hadoop.hive.ql.exec.Utilities.isEmptyPath(Utilities.java:2601)
> at
> org.apache.hadoop.hive.ql.exec.Utilities$GetInputPathsCallable.call(Utilities.java:3409)
> at
> org.apache.hadoop.hive.ql.exec.Utilities.getInputPaths(Utilities.java:3347)
> at
> org.apache.hadoop.hive.ql.exec.spark.SparkPlanGenerator.cloneJobConf(SparkPlanGenerator.java:299)
> at
> org.apache.hadoop.hive.ql.exec.spark.SparkPlanGenerator.generate(SparkPlanGenerator.java:222)
> at
> org.apache.hadoop.hive.ql.exec.spark.SparkPlanGenerator.generate(SparkPlanGenerator.java:109)
> at
> org.apache.hadoop.hive.ql.exec.spark.RemoteHiveSparkClient$JobStatusJob.call(RemoteHiveSparkClient.java:354)
> at
> org.apache.hive.spark.client.RemoteDriver$JobWrapper.call(RemoteDriver.java:358)
> at
> org.apache.hive.spark.client.RemoteDriver$JobWrapper.call(RemoteDriver.java:323)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-18034) Improving logging with HoS executors spend lots of time in GC
[ https://issues.apache.org/jira/browse/HIVE-18034?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sahil Takiar updated HIVE-18034: Attachment: HIVE-18034.2.patch > Improving logging with HoS executors spend lots of time in GC > - > > Key: HIVE-18034 > URL: https://issues.apache.org/jira/browse/HIVE-18034 > Project: Hive > Issue Type: Sub-task > Components: Spark >Reporter: Sahil Takiar >Assignee: Sahil Takiar >Priority: Major > Attachments: HIVE-18034.1.patch, HIVE-18034.2.patch > > > There are times when Spark will spend lots of time doing GC. The Spark > History UI shows a bunch of red flags when too much time is spent in GC. It > would be nice if those warnings are propagated to Hive. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-18672) Printed state in RemoteSparkJobMonitor is ambiguous
[
https://issues.apache.org/jira/browse/HIVE-18672?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sahil Takiar updated HIVE-18672:
Attachment: HIVE-18672.1.patch
> Printed state in RemoteSparkJobMonitor is ambiguous
> ---
>
> Key: HIVE-18672
> URL: https://issues.apache.org/jira/browse/HIVE-18672
> Project: Hive
> Issue Type: Sub-task
> Components: Spark
>Reporter: Sahil Takiar
>Priority: Major
> Attachments: HIVE-18672.1.patch
>
>
> There are a few places in {{RemoteSparkJobMonitor}} (e.g. when the Spark job
> is in state QUEUED) where the state of the Spark job is printed, but the info
> is ambiguous (no reference to HoS, or the id of the Spark job).
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-18672) Printed state in RemoteSparkJobMonitor is ambiguous
[
https://issues.apache.org/jira/browse/HIVE-18672?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sahil Takiar updated HIVE-18672:
Status: Patch Available (was: Open)
> Printed state in RemoteSparkJobMonitor is ambiguous
> ---
>
> Key: HIVE-18672
> URL: https://issues.apache.org/jira/browse/HIVE-18672
> Project: Hive
> Issue Type: Sub-task
> Components: Spark
>Reporter: Sahil Takiar
>Priority: Major
> Attachments: HIVE-18672.1.patch
>
>
> There are a few places in {{RemoteSparkJobMonitor}} (e.g. when the Spark job
> is in state QUEUED) where the state of the Spark job is printed, but the info
> is ambiguous (no reference to HoS, or the id of the Spark job).
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Assigned] (HIVE-18672) Printed state in RemoteSparkJobMonitor is ambiguous
[
https://issues.apache.org/jira/browse/HIVE-18672?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sahil Takiar reassigned HIVE-18672:
---
Assignee: Sahil Takiar
> Printed state in RemoteSparkJobMonitor is ambiguous
> ---
>
> Key: HIVE-18672
> URL: https://issues.apache.org/jira/browse/HIVE-18672
> Project: Hive
> Issue Type: Sub-task
> Components: Spark
>Reporter: Sahil Takiar
>Assignee: Sahil Takiar
>Priority: Major
> Attachments: HIVE-18672.1.patch
>
>
> There are a few places in {{RemoteSparkJobMonitor}} (e.g. when the Spark job
> is in state QUEUED) where the state of the Spark job is printed, but the info
> is ambiguous (no reference to HoS, or the id of the Spark job).
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-18663) Logged Spark Job Id contains a UUID instead of the actual id
[
https://issues.apache.org/jira/browse/HIVE-18663?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sahil Takiar updated HIVE-18663:
Status: Patch Available (was: Open)
> Logged Spark Job Id contains a UUID instead of the actual id
>
>
> Key: HIVE-18663
> URL: https://issues.apache.org/jira/browse/HIVE-18663
> Project: Hive
> Issue Type: Sub-task
> Components: Spark
>Reporter: Sahil Takiar
>Assignee: Sahil Takiar
>Priority: Major
> Attachments: HIVE-18663.1.patch
>
>
> We have logs like {{Spark Job[job-id]}} but the {{[job-id]}} is set to a UUID
> that is created by the RSC {{ClientProtocol}}. It should be pretty easy to
> print out the actual job id instead.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-18622) Vectorization: IF Statements, Comparisons, and more do not handle NULLs correctly
[
https://issues.apache.org/jira/browse/HIVE-18622?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16359119#comment-16359119
]
Matt McCline commented on HIVE-18622:
-
Currently #9151 in the queue.
> Vectorization: IF Statements, Comparisons, and more do not handle NULLs
> correctly
> -
>
> Key: HIVE-18622
> URL: https://issues.apache.org/jira/browse/HIVE-18622
> Project: Hive
> Issue Type: Bug
> Components: Hive
>Reporter: Matt McCline
>Assignee: Matt McCline
>Priority: Critical
> Fix For: 3.0.0
>
> Attachments: HIVE-18622.03.patch, HIVE-18622.04.patch,
> HIVE-18622.05.patch, HIVE-18622.06.patch, HIVE-18622.07.patch
>
>
>
> Many vector expression classes are setting noNulls to true which does not
> work if the VRB is a scratch column being reused. The previous use may have
> set noNulls to false and the isNull array will have some rows marked as NULL.
> The result is wrong query results and sometimes NPEs (for BytesColumnVector).
> So, many vector expressions need this:
> {code:java}
> // Carefully handle NULLs...
> /*
>* For better performance on LONG/DOUBLE we don't want the conditional
>* statements inside the for loop.
>*/
> outputColVector.noNulls = false;
> {code}
> And, vector expressions need to make sure the isNull array entry is set when
> outputColVector.noNulls is false.
> And, all place that assign column value need to set noNulls to false when the
> value is NULL.
> Almost all cases where noNulls is set to true are incorrect.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-18218) SMB Join : Handle buckets with no splits.
[ https://issues.apache.org/jira/browse/HIVE-18218?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Deepak Jaiswal updated HIVE-18218: -- Attachment: HIVE-18218.4.patch > SMB Join : Handle buckets with no splits. > - > > Key: HIVE-18218 > URL: https://issues.apache.org/jira/browse/HIVE-18218 > Project: Hive > Issue Type: Bug >Reporter: Deepak Jaiswal >Assignee: Deepak Jaiswal >Priority: Major > Attachments: HIVE-18218.1.patch, HIVE-18218.2.patch, > HIVE-18218.3.patch, HIVE-18218.4.patch > > > While working on HIVE-18208, it was found that with SMB, the results are > incorrect. This most likely is a product issue. > auto_sortmerge_join_16 fails with wrong results due to this. > cc [~hagleitn] > The current logic in CustomPartitionVertex assumes that there is a split for > each bucket whereas in Tez, we can have no splits for empty buckets. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-18238) Driver execution may not have configuration changing sideeffects
[
https://issues.apache.org/jira/browse/HIVE-18238?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16359118#comment-16359118
]
Hive QA commented on HIVE-18238:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12909822/HIVE-18238.10.patch
{color:green}SUCCESS:{color} +1 due to 11 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 26 failed/errored test(s), 13152 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestAccumuloCliDriver.testCliDriver[accumulo_queries]
(batchId=241)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[ppd_join5] (batchId=36)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[row__id] (batchId=79)
org.apache.hadoop.hive.cli.TestEncryptedHDFSCliDriver.testCliDriver[encryption_move_tbl]
(batchId=175)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[llap_smb]
(batchId=152)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[bucket_map_join_tez1]
(batchId=172)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[insert_values_orig_table_use_metadata]
(batchId=167)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[llap_acid]
(batchId=171)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[llap_acid_fast]
(batchId=162)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[resourceplan]
(batchId=164)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[results_cache_1]
(batchId=168)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[sysdb]
(batchId=161)
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver[spark_opt_shuffle_serde]
(batchId=180)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[ppd_join5]
(batchId=122)
org.apache.hadoop.hive.cli.TestSparkPerfCliDriver.testCliDriver[query1]
(batchId=251)
org.apache.hadoop.hive.cli.control.TestDanglingQOuts.checkDanglingQOut
(batchId=222)
org.apache.hadoop.hive.metastore.client.TestTablesGetExists.testGetAllTablesCaseInsensitive[Embedded]
(batchId=206)
org.apache.hadoop.hive.ql.exec.tez.TestWorkloadManager.testApplyPlanQpChanges
(batchId=288)
org.apache.hadoop.hive.ql.io.TestDruidRecordWriter.testWrite (batchId=257)
org.apache.hive.beeline.cli.TestHiveCli.testNoErrorDB (batchId=188)
org.apache.hive.jdbc.TestSSL.testConnectionMismatch (batchId=235)
org.apache.hive.jdbc.TestSSL.testConnectionWrongCertCN (batchId=235)
org.apache.hive.jdbc.TestSSL.testMetastoreConnectionWrongCertCN (batchId=235)
org.apache.hive.jdbc.TestTriggersMoveWorkloadManager.testTriggerMoveAndKill
(batchId=236)
org.apache.hive.jdbc.TestTriggersMoveWorkloadManager.testTriggerMoveBackKill
(batchId=236)
org.apache.hive.jdbc.TestTriggersMoveWorkloadManager.testTriggerMoveConflictKill
(batchId=236)
{noformat}
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/9126/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/9126/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-9126/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 26 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12909822 - PreCommit-HIVE-Build
> Driver execution may not have configuration changing sideeffects
> -
>
> Key: HIVE-18238
> URL: https://issues.apache.org/jira/browse/HIVE-18238
> Project: Hive
> Issue Type: Sub-task
> Components: Logical Optimizer
>Reporter: Zoltan Haindrich
>Assignee: Zoltan Haindrich
>Priority: Major
> Attachments: HIVE-18238.01wip01.patch, HIVE-18238.02.patch,
> HIVE-18238.03.patch, HIVE-18238.04.patch, HIVE-18238.04wip01.patch,
> HIVE-18238.07.patch, HIVE-18238.08.patch, HIVE-18238.09.patch,
> HIVE-18238.10.patch
>
>
> {{Driver}} executes sql statements which use "hiveconf" settings;
> but the {{Driver}} itself may *not* change the configuration...
> I've found an example; which shows how hazardous this is...
> {code}
> set hive.mapred.mode=strict;
> select "${hiveconf:hive.mapred.mode}";
> create table t (a int);
> analyze table t compute statistics;
> select "${hiveconf:hive.mapred.mode}";
> {code}
> currently; the last select returns {{nonstrict}} because of
> [this|https://github.com/apache/hive/blob/7ddd915bf82a68c8ab73b0c4ca409f1a6d43d227/ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java#L1696]
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-18663) Logged Spark Job Id contains a UUID instead of the actual id
[
https://issues.apache.org/jira/browse/HIVE-18663?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sahil Takiar updated HIVE-18663:
Attachment: HIVE-18663.1.patch
> Logged Spark Job Id contains a UUID instead of the actual id
>
>
> Key: HIVE-18663
> URL: https://issues.apache.org/jira/browse/HIVE-18663
> Project: Hive
> Issue Type: Sub-task
> Components: Spark
>Reporter: Sahil Takiar
>Assignee: Sahil Takiar
>Priority: Major
> Attachments: HIVE-18663.1.patch
>
>
> We have logs like {{Spark Job[job-id]}} but the {{[job-id]}} is set to a UUID
> that is created by the RSC {{ClientProtocol}}. It should be pretty easy to
> print out the actual job id instead.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Assigned] (HIVE-18663) Logged Spark Job Id contains a UUID instead of the actual id
[
https://issues.apache.org/jira/browse/HIVE-18663?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sahil Takiar reassigned HIVE-18663:
---
Assignee: Sahil Takiar
> Logged Spark Job Id contains a UUID instead of the actual id
>
>
> Key: HIVE-18663
> URL: https://issues.apache.org/jira/browse/HIVE-18663
> Project: Hive
> Issue Type: Sub-task
> Components: Spark
>Reporter: Sahil Takiar
>Assignee: Sahil Takiar
>Priority: Major
>
> We have logs like {{Spark Job[job-id]}} but the {{[job-id]}} is set to a UUID
> that is created by the RSC {{ClientProtocol}}. It should be pretty easy to
> print out the actual job id instead.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-18586) Upgrade Derby to 10.14.1.0
[ https://issues.apache.org/jira/browse/HIVE-18586?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Janaki Lahorani updated HIVE-18586: --- Attachment: HIVE-18586.5.patch > Upgrade Derby to 10.14.1.0 > -- > > Key: HIVE-18586 > URL: https://issues.apache.org/jira/browse/HIVE-18586 > Project: Hive > Issue Type: Improvement >Reporter: Janaki Lahorani >Assignee: Janaki Lahorani >Priority: Major > Attachments: HIVE-18586.1.patch, HIVE-18586.2.patch, > HIVE-18586.3.patch, HIVE-18586.4.patch, HIVE-18586.5.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-18238) Driver execution may not have configuration changing sideeffects
[
https://issues.apache.org/jira/browse/HIVE-18238?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16359083#comment-16359083
]
Hive QA commented on HIVE-18238:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
45s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m
16s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
59s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 2m
1s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m
8s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
23s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
26s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
58s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 2m
58s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
57s{color} | {color:red} ql: The patch generated 12 new + 1708 unchanged - 9
fixed = 1720 total (was 1717) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
10s{color} | {color:red} cli: The patch generated 1 new + 34 unchanged - 0
fixed = 35 total (was 34) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
12s{color} | {color:red} hcatalog/core: The patch generated 1 new + 32
unchanged - 2 fixed = 33 total (was 34) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m
4s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
13s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 26m 59s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /data/hiveptest/working/yetus/dev-support/hive-personality.sh |
| git revision | master / 717ef18 |
| Default Java | 1.8.0_111 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-9126/yetus/diff-checkstyle-ql.txt
|
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-9126/yetus/diff-checkstyle-cli.txt
|
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-9126/yetus/diff-checkstyle-hcatalog_core.txt
|
| modules | C: ql service cli hcatalog/core hcatalog/hcatalog-pig-adapter
itests/hive-unit U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-9126/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Driver execution may not have configuration changing sideeffects
> -
>
> Key: HIVE-18238
> URL: https://issues.apache.org/jira/browse/HIVE-18238
> Project: Hive
> Issue Type: Sub-task
> Components: Logical Optimizer
>Reporter: Zoltan Haindrich
>Assignee: Zoltan Haindrich
>Priority: Major
> Attachments: HIVE-18238.01wip01.patch, HIVE-18238.02.patch,
> HIVE-18238.03.patch, HIVE-18238.04.patch, HIVE-18238.04wip01.patch,
> HIVE-18238.07.patch, HIVE-18238.08.patch, HIVE-18238.09.patch,
> HIVE-18238.10.patch
>
>
> {{Driver}} executes sql statements which use "hiveconf" settings;
> but the {{Driver}} itself may *not*
[jira] [Updated] (HIVE-17837) Explicitly check if the HoS Remote Driver has been lost in the RemoteSparkJobMonitor
[
https://issues.apache.org/jira/browse/HIVE-17837?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sahil Takiar updated HIVE-17837:
Resolution: Fixed
Fix Version/s: 3.0.0
Status: Resolved (was: Patch Available)
Merged to master. Thanks Rui for the review.
> Explicitly check if the HoS Remote Driver has been lost in the
> RemoteSparkJobMonitor
> -
>
> Key: HIVE-17837
> URL: https://issues.apache.org/jira/browse/HIVE-17837
> Project: Hive
> Issue Type: Sub-task
> Components: Hive
>Reporter: Sahil Takiar
>Assignee: Sahil Takiar
>Priority: Major
> Fix For: 3.0.0
>
> Attachments: HIVE-17837.1.patch, HIVE-17837.2.patch
>
>
> Right now the {{RemoteSparkJobMonitor}} implicitly checks if the connection
> to the Spark remote driver is active. It does this everytime it triggers an
> invocation of the {{Rpc#call}} method (so any call to {{SparkClient#run}}).
> There are scenarios where we have seen the {{RemoteSparkJobMonitor}} when the
> connection to the driver dies, because the implicit call fails to be invoked
> (see HIVE-15860).
> It would be ideal if we made this call explicit, so we fail as soon as we
> know that the connection to the driver has died.
> The fix has the added benefit that it allows us to fail faster in the case
> where the {{RemoteSparkJobMonitor}} is in the QUEUED / SENT state. If its
> stuck in that state, it won't fail until it hits the monitor timeout (by
> default 1 minute), even though we already know the connection has died. The
> error message that is thrown is also a little imprecise, it says there could
> be queue contention, even though we know the real reason is that the
> connection was lost.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-15077) Acid LockManager is unfair
[
https://issues.apache.org/jira/browse/HIVE-15077?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Eugene Koifman updated HIVE-15077:
--
Description:
HIVE-10242 made the acid LM unfair.
In TxnHandler.checkLock(), suppose we are trying to acquire SR5 (the number is
extLockId).
Then
LockInfo[] locks = lockSet.toArray(new LockInfo[lockSet.size()]);
may look like this (all explicitly listed locks are in Waiting state)
{, SR5 SW3 X4}
So the algorithm will find SR5 in the list and start looking backwards (to the
left).
According to IDs, SR5 should wait for X4 to be granted but X4 won't even be
examined and so SR5 may be granted.
Theoretically, this could cause starvation.
The query that generates the list already has
query.append(" and hl_lock_ext_id <= ").append(extLockId);
but it should use "<" rather than "<=" to exclude the locks being checked from
"locks" list which will make the algorithm look at all locks "in front" of a
given lock.
was:
HIVE-10242 made the acid LM unfair.
In TxnHandler.checkLock(), suppose we are trying to acquire SR5 (the number is
extLockId).
Then
LockInfo[] locks = lockSet.toArray(new LockInfo[lockSet.size()]);
may look like this (all explicitly listed locks are in Waiting state)
{, SR5 SW3 X4}
So the algorithm will find SR5 in the list and start looking backwards.
According to IDs, SR5 should wait for X4 to be granted but X4 won't even be
examined and so SR5 may be granted.
Theoretically, this could cause starvation.
The query that generates the list already has
query.append(" and hl_lock_ext_id <= ").append(extLockId);
but it should use "<" rather than "<=" to exclude the locks being checked from
"locks" list which will make the algorithm look at all locks "in front" of a
given lock.
> Acid LockManager is unfair
> --
>
> Key: HIVE-15077
> URL: https://issues.apache.org/jira/browse/HIVE-15077
> Project: Hive
> Issue Type: Bug
> Components: Transactions
>Reporter: Eugene Koifman
>Assignee: Eugene Koifman
>Priority: Blocker
>
> HIVE-10242 made the acid LM unfair.
> In TxnHandler.checkLock(), suppose we are trying to acquire SR5 (the number
> is extLockId).
> Then
> LockInfo[] locks = lockSet.toArray(new LockInfo[lockSet.size()]);
> may look like this (all explicitly listed locks are in Waiting state)
> {, SR5 SW3 X4}
> So the algorithm will find SR5 in the list and start looking backwards (to
> the left).
> According to IDs, SR5 should wait for X4 to be granted but X4 won't even be
> examined and so SR5 may be granted.
> Theoretically, this could cause starvation.
> The query that generates the list already has
> query.append(" and hl_lock_ext_id <= ").append(extLockId);
> but it should use "<" rather than "<=" to exclude the locks being checked
> from "locks" list which will make the algorithm look at all locks "in front"
> of a given lock.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-17835) HS2 Logs print unnecessary stack trace when HoS query is cancelled
[
https://issues.apache.org/jira/browse/HIVE-17835?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sahil Takiar updated HIVE-17835:
Resolution: Fixed
Fix Version/s: 3.0.0
Status: Resolved (was: Patch Available)
Pushed to master. Thanks Chao for the review.
> HS2 Logs print unnecessary stack trace when HoS query is cancelled
> --
>
> Key: HIVE-17835
> URL: https://issues.apache.org/jira/browse/HIVE-17835
> Project: Hive
> Issue Type: Sub-task
> Components: Spark
>Reporter: Sahil Takiar
>Assignee: Sahil Takiar
>Priority: Major
> Fix For: 3.0.0
>
> Attachments: HIVE-17835.1.patch, HIVE-17835.2.patch,
> HIVE-17835.3.patch
>
>
> Example:
> {code}
> 2017-10-05 17:47:11,881 ERROR
> org.apache.hadoop.hive.ql.exec.spark.status.SparkJobMonitor:
> [HiveServer2-Background-Pool: Thread-131]: Failed to monitor Job[ 2] with
> exception 'java.lang.InterruptedException(sleep interrupted)'
> java.lang.InterruptedException: sleep interrupted
> at java.lang.Thread.sleep(Native Method)
> at
> org.apache.hadoop.hive.ql.exec.spark.status.RemoteSparkJobMonitor.startMonitor(RemoteSparkJobMonitor.java:124)
> at
> org.apache.hadoop.hive.ql.exec.spark.status.impl.RemoteSparkJobRef.monitorJob(RemoteSparkJobRef.java:60)
> at
> org.apache.hadoop.hive.ql.exec.spark.SparkTask.execute(SparkTask.java:111)
> at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:214)
> at
> org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:99)
> at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:2052)
> at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1748)
> at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1501)
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1285)
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1280)
> at
> org.apache.hive.service.cli.operation.SQLOperation.runQuery(SQLOperation.java:236)
> at
> org.apache.hive.service.cli.operation.SQLOperation.access$300(SQLOperation.java:89)
> at
> org.apache.hive.service.cli.operation.SQLOperation$3$1.run(SQLOperation.java:301)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1917)
> at
> org.apache.hive.service.cli.operation.SQLOperation$3.run(SQLOperation.java:314)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> 2017-10-05 17:47:11,881 WARN org.apache.hadoop.hive.ql.Driver:
> [HiveServer2-Handler-Pool: Thread-105]: Shutting down task : Stage-2:MAPRED
> 2017-10-05 17:47:11,882 ERROR
> org.apache.hadoop.hive.ql.exec.spark.status.SparkJobMonitor:
> [HiveServer2-Background-Pool: Thread-131]: Failed to monitor Job[ 2] with
> exception 'java.lang.InterruptedException(sleep interrupted)'
> java.lang.InterruptedException: sleep interrupted
> at java.lang.Thread.sleep(Native Method)
> at
> org.apache.hadoop.hive.ql.exec.spark.status.RemoteSparkJobMonitor.startMonitor(RemoteSparkJobMonitor.java:124)
> at
> org.apache.hadoop.hive.ql.exec.spark.status.impl.RemoteSparkJobRef.monitorJob(RemoteSparkJobRef.java:60)
> at
> org.apache.hadoop.hive.ql.exec.spark.SparkTask.execute(SparkTask.java:111)
> at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:214)
> at
> org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:99)
> at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:2052)
> at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1748)
> at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1501)
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1285)
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1280)
> at
> org.apache.hive.service.cli.operation.SQLOperation.runQuery(SQLOperation.java:236)
> at
> org.apache.hive.service.cli.operation.SQLOperation.access$300(SQLOperation.java:89)
> at
> org.apache.hive.service.cli.operation.SQLOperation$3$1.run(SQLOperation.java:301)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1917)
> at
>
[jira] [Updated] (HIVE-18677) SparkClientImpl usage of SessionState.LogHelper doesn't respect isSilent value
[
https://issues.apache.org/jira/browse/HIVE-18677?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sahil Takiar updated HIVE-18677:
Status: Patch Available (was: Open)
> SparkClientImpl usage of SessionState.LogHelper doesn't respect isSilent value
> --
>
> Key: HIVE-18677
> URL: https://issues.apache.org/jira/browse/HIVE-18677
> Project: Hive
> Issue Type: Sub-task
> Components: Spark, Test
>Reporter: Sahil Takiar
>Assignee: Sahil Takiar
>Priority: Major
> Attachments: HIVE-18677.1.patch
>
>
> In HIVE-18389 we added support for printing information to the console stream
> in the {{spark-client}} library (by passing the raw stream into
> {{SparkClientImpl}}. However, usage of the console doesn't respect the value
> is {{isSilent}} in {{SessionState}}. This has been causing a few qtests to
> fail.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-18677) SparkClientImpl usage of SessionState.LogHelper doesn't respect isSilent value
[
https://issues.apache.org/jira/browse/HIVE-18677?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sahil Takiar updated HIVE-18677:
Attachment: HIVE-18677.1.patch
> SparkClientImpl usage of SessionState.LogHelper doesn't respect isSilent value
> --
>
> Key: HIVE-18677
> URL: https://issues.apache.org/jira/browse/HIVE-18677
> Project: Hive
> Issue Type: Sub-task
> Components: Spark, Test
>Reporter: Sahil Takiar
>Assignee: Sahil Takiar
>Priority: Major
> Attachments: HIVE-18677.1.patch
>
>
> In HIVE-18389 we added support for printing information to the console stream
> in the {{spark-client}} library (by passing the raw stream into
> {{SparkClientImpl}}. However, usage of the console doesn't respect the value
> is {{isSilent}} in {{SessionState}}. This has been causing a few qtests to
> fail.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Assigned] (HIVE-18677) SparkClientImpl usage of SessionState.LogHelper doesn't respect isSilent value
[
https://issues.apache.org/jira/browse/HIVE-18677?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sahil Takiar reassigned HIVE-18677:
---
> SparkClientImpl usage of SessionState.LogHelper doesn't respect isSilent value
> --
>
> Key: HIVE-18677
> URL: https://issues.apache.org/jira/browse/HIVE-18677
> Project: Hive
> Issue Type: Sub-task
> Components: Spark, Test
>Reporter: Sahil Takiar
>Assignee: Sahil Takiar
>Priority: Major
>
> In HIVE-18389 we added support for printing information to the console stream
> in the {{spark-client}} library (by passing the raw stream into
> {{SparkClientImpl}}. However, usage of the console doesn't respect the value
> is {{isSilent}} in {{SessionState}}. This has been causing a few qtests to
> fail.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-18448) Drop Support For Indexes From Apache Hive
[
https://issues.apache.org/jira/browse/HIVE-18448?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16359064#comment-16359064
]
Hive QA commented on HIVE-18448:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12909820/HIVE-18448.01wip03.patch
{color:red}ERROR:{color} -1 due to build exiting with an error
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/9125/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/9125/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-9125/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Tests exited with: NonZeroExitCodeException
Command 'bash /data/hiveptest/working/scratch/source-prep.sh' failed with exit
status 1 and output '+ date '+%Y-%m-%d %T.%3N'
2018-02-09 22:38:08.910
+ [[ -n /usr/lib/jvm/java-8-openjdk-amd64 ]]
+ export JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
+ JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
+ export
PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+
PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+ export 'ANT_OPTS=-Xmx1g -XX:MaxPermSize=256m '
+ ANT_OPTS='-Xmx1g -XX:MaxPermSize=256m '
+ export 'MAVEN_OPTS=-Xmx1g '
+ MAVEN_OPTS='-Xmx1g '
+ cd /data/hiveptest/working/
+ tee /data/hiveptest/logs/PreCommit-HIVE-Build-9125/source-prep.txt
+ [[ false == \t\r\u\e ]]
+ mkdir -p maven ivy
+ [[ git = \s\v\n ]]
+ [[ git = \g\i\t ]]
+ [[ -z master ]]
+ [[ -d apache-github-source-source ]]
+ [[ ! -d apache-github-source-source/.git ]]
+ [[ ! -d apache-github-source-source ]]
+ date '+%Y-%m-%d %T.%3N'
2018-02-09 22:38:08.914
+ cd apache-github-source-source
+ git fetch origin
>From https://github.com/apache/hive
58bbfc7..717ef18 master -> origin/master
+ git reset --hard HEAD
HEAD is now at 58bbfc7 HIVE-18580: Create tests to cover exchange partitions
(Marta Kuczora, reviewed by Adam Szita, Peter Vary)
+ git clean -f -d
+ git checkout master
Already on 'master'
Your branch is behind 'origin/master' by 1 commit, and can be fast-forwarded.
(use "git pull" to update your local branch)
+ git reset --hard origin/master
HEAD is now at 717ef18 HIVE-18607 : HBase HFile write does strange things
(Sergey Shelukhin, reviewed by Ashutosh Chauhan)
+ git merge --ff-only origin/master
Already up-to-date.
+ date '+%Y-%m-%d %T.%3N'
2018-02-09 22:38:12.029
+ rm -rf ../yetus
+ mkdir ../yetus
+ git gc
+ cp -R . ../yetus
+ mkdir /data/hiveptest/logs/PreCommit-HIVE-Build-9125/yetus
+ patchCommandPath=/data/hiveptest/working/scratch/smart-apply-patch.sh
+ patchFilePath=/data/hiveptest/working/scratch/build.patch
+ [[ -f /data/hiveptest/working/scratch/build.patch ]]
+ chmod +x /data/hiveptest/working/scratch/smart-apply-patch.sh
+ /data/hiveptest/working/scratch/smart-apply-patch.sh
/data/hiveptest/working/scratch/build.patch
error: patch failed:
ql/src/test/org/apache/hadoop/hive/ql/metadata/TestHive.java:36
Falling back to three-way merge...
Applied patch to 'ql/src/test/org/apache/hadoop/hive/ql/metadata/TestHive.java'
with conflicts.
Going to apply patch with: git apply -p0
error: patch failed:
ql/src/test/org/apache/hadoop/hive/ql/metadata/TestHive.java:36
Falling back to three-way merge...
Applied patch to 'ql/src/test/org/apache/hadoop/hive/ql/metadata/TestHive.java'
with conflicts.
U ql/src/test/org/apache/hadoop/hive/ql/metadata/TestHive.java
+ exit 1
'
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12909820 - PreCommit-HIVE-Build
> Drop Support For Indexes From Apache Hive
> -
>
> Key: HIVE-18448
> URL: https://issues.apache.org/jira/browse/HIVE-18448
> Project: Hive
> Issue Type: Improvement
> Components: Indexing
>Reporter: BELUGA BEHR
>Assignee: Zoltan Haindrich
>Priority: Minor
> Attachments: HIVE-18448.01wip02.patch, HIVE-18448.01wip03.patch
>
>
> If a user needs to look up a small subset of records quickly, they can use
> Apache HBase, if they need fast retrieval of larger sets of data, or fast
> joins, aggregations, they can use Apache Impala. It seems to me that Hive
> indexes do not serve much of a role in the future of Hive.
> Even without moving workloads to other products, columnar file formats with
> their statistics achieve similar goals as Hive indexes.
> Please consider dropping Indexes from the Apache Hive project.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-18607) HBase HFile write does strange things
[ https://issues.apache.org/jira/browse/HIVE-18607?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Shelukhin updated HIVE-18607: Resolution: Fixed Fix Version/s: 3.0.0 Status: Resolved (was: Patch Available) Committed to master. Thanks for the review! > HBase HFile write does strange things > - > > Key: HIVE-18607 > URL: https://issues.apache.org/jira/browse/HIVE-18607 > Project: Hive > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin >Priority: Major > Fix For: 3.0.0 > > Attachments: HIVE-18607.02.patch, HIVE-18607.03.patch, > HIVE-18607.04.patch, HIVE-18607.patch > > > There's some strange code in the output handler that changes output directory > into a file because Hive supposedly wants that. > If you run insert overwrite with a side directory multiple times, the 2nd > insert fails -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-18553) VectorizedParquetReader fails after adding a new column to table
[
https://issues.apache.org/jira/browse/HIVE-18553?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16359059#comment-16359059
]
Hive QA commented on HIVE-18553:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12909934/HIVE-18553.7.patch
{color:green}SUCCESS:{color} +1 due to 5 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 33 failed/errored test(s), 13147 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestAccumuloCliDriver.testCliDriver[accumulo_queries]
(batchId=241)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[parquet_complex_types_vectorization]
(batchId=73)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[parquet_map_type_vectorization]
(batchId=85)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[ppd_join5] (batchId=36)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[row__id] (batchId=79)
org.apache.hadoop.hive.cli.TestEncryptedHDFSCliDriver.testCliDriver[encryption_move_tbl]
(batchId=175)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[parquet_complex_types_vectorization]
(batchId=151)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[parquet_map_type_vectorization]
(batchId=152)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[bucket_map_join_tez1]
(batchId=172)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[insert_values_orig_table_use_metadata]
(batchId=167)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[llap_acid]
(batchId=171)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[llap_acid_fast]
(batchId=162)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[resourceplan]
(batchId=164)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[sysdb]
(batchId=161)
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver[bucketizedhiveinputformat]
(batchId=180)
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver[spark_opt_shuffle_serde]
(batchId=180)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[ppd_join5]
(batchId=122)
org.apache.hadoop.hive.cli.TestSparkPerfCliDriver.testCliDriver[query1]
(batchId=251)
org.apache.hadoop.hive.cli.control.TestDanglingQOuts.checkDanglingQOut
(batchId=222)
org.apache.hadoop.hive.metastore.TestAuthzApiEmbedAuthorizerInRemote.org.apache.hadoop.hive.metastore.TestAuthzApiEmbedAuthorizerInRemote
(batchId=226)
org.apache.hadoop.hive.metastore.TestMarkPartition.testMarkingPartitionSet
(batchId=215)
org.apache.hadoop.hive.ql.io.TestDruidRecordWriter.testWrite (batchId=257)
org.apache.hadoop.hive.ql.io.parquet.TestVectorizedListColumnReader.testListReadEqualOneBatch
(batchId=271)
org.apache.hadoop.hive.ql.io.parquet.TestVectorizedListColumnReader.testListReadLessOneBatch
(batchId=271)
org.apache.hadoop.hive.ql.io.parquet.TestVectorizedListColumnReader.testListReadMoreOneBatch
(batchId=271)
org.apache.hadoop.hive.ql.io.parquet.TestVectorizedMapColumnReader.testMapReadEqualOneBatch
(batchId=272)
org.apache.hadoop.hive.ql.io.parquet.TestVectorizedMapColumnReader.testMapReadLessOneBatch
(batchId=272)
org.apache.hadoop.hive.ql.io.parquet.TestVectorizedMapColumnReader.testMapReadMoreOneBatch
(batchId=272)
org.apache.hive.beeline.cli.TestHiveCli.testNoErrorDB (batchId=188)
org.apache.hive.jdbc.TestSSL.testConnectionMismatch (batchId=235)
org.apache.hive.jdbc.TestSSL.testConnectionWrongCertCN (batchId=235)
org.apache.hive.jdbc.TestSSL.testMetastoreConnectionWrongCertCN (batchId=235)
org.apache.hive.jdbc.TestTriggersMoveWorkloadManager.testTriggerMoveAndKill
(batchId=236)
{noformat}
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/9124/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/9124/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-9124/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 33 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12909934 - PreCommit-HIVE-Build
> VectorizedParquetReader fails after adding a new column to table
>
>
> Key: HIVE-18553
> URL: https://issues.apache.org/jira/browse/HIVE-18553
> Project: Hive
> Issue Type: Sub-task
>Affects Versions: 3.0.0, 2.4.0, 2.3.2
>Reporter: Vihang Karajgaonkar
>Assignee: Ferdinand Xu
>Priority: Major
> Attachments: HIVE-18553.2.patch, HIVE-18553.3.patch,
> HIVE-18553.4.patch,
[jira] [Updated] (HIVE-18676) Usability: Better warnings for Unicode single quotes
[
https://issues.apache.org/jira/browse/HIVE-18676?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Gopal V updated HIVE-18676:
---
Issue Type: Improvement (was: Bug)
> Usability: Better warnings for Unicode single quotes
>
>
> Key: HIVE-18676
> URL: https://issues.apache.org/jira/browse/HIVE-18676
> Project: Hive
> Issue Type: Improvement
> Components: Parser
>Reporter: Gopal V
>Priority: Major
> Labels: usability
>
> The warning for
> {code}
> select d_date as ‘date’ from date_dim;
> {code}
> is hard to locate than it should be.
> {code}
> Error: Error while compiling statement: FAILED: ParseException line 1:18
> cannot recognize input near 'as' 'date' 'from' in selection target
> (state=42000,code=4)
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-18676) Usability: Better warnings for Unicode single quotes
[
https://issues.apache.org/jira/browse/HIVE-18676?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Gopal V updated HIVE-18676:
---
Labels: usability (was: )
> Usability: Better warnings for Unicode single quotes
>
>
> Key: HIVE-18676
> URL: https://issues.apache.org/jira/browse/HIVE-18676
> Project: Hive
> Issue Type: Bug
> Components: Parser
>Reporter: Gopal V
>Priority: Major
> Labels: usability
>
> The warning for
> {code}
> select d_date as ‘date’ from date_dim;
> {code}
> is hard to locate than it should be.
> {code}
> Error: Error while compiling statement: FAILED: ParseException line 1:18
> cannot recognize input near 'as' 'date' 'from' in selection target
> (state=42000,code=4)
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-18674) update Hive to use ORC 1.4.3
[ https://issues.apache.org/jira/browse/HIVE-18674?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16359039#comment-16359039 ] Gopal V commented on HIVE-18674: LGTM - +1 > update Hive to use ORC 1.4.3 > > > Key: HIVE-18674 > URL: https://issues.apache.org/jira/browse/HIVE-18674 > Project: Hive > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin >Priority: Major > Attachments: HIVE-18674.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-15353) Metastore throws NPE if StorageDescriptor.cols is null
[ https://issues.apache.org/jira/browse/HIVE-15353?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16359030#comment-16359030 ] Peter Vary commented on HIVE-15353: --- [~erwaman]: Thanks for confirming that the issue is solved! > Metastore throws NPE if StorageDescriptor.cols is null > -- > > Key: HIVE-15353 > URL: https://issues.apache.org/jira/browse/HIVE-15353 > Project: Hive > Issue Type: Bug >Affects Versions: 1.1.0, 2.2.0 >Reporter: Anthony Hsu >Assignee: Anthony Hsu >Priority: Major > Attachments: HIVE-15353.1.patch, HIVE-15353.2.patch, > HIVE-15353.3.patch, HIVE-15353.4.patch, HIVE-15353.5.patch > > > When using the HiveMetaStoreClient API directly to talk to the metastore, you > get NullPointerExceptions when StorageDescriptor.cols is null in the > Table/Partition object in the following calls: > * create_table > * alter_table > * alter_partition > Calling add_partition with StorageDescriptor.cols set to null causes null to > be stored in the metastore database and subsequent calls to alter_partition > for that partition to fail with an NPE. > Null checks should be added to eliminate the NPEs in the metastore. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-18586) Upgrade Derby to 10.14.1.0
[ https://issues.apache.org/jira/browse/HIVE-18586?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16359028#comment-16359028 ] Aihua Xu commented on HIVE-18586: - [~janulatha] I think it's reasonable to remove SecurityManager from these two tests. Doesn't seem to add a lot of value to me as well. > Upgrade Derby to 10.14.1.0 > -- > > Key: HIVE-18586 > URL: https://issues.apache.org/jira/browse/HIVE-18586 > Project: Hive > Issue Type: Improvement >Reporter: Janaki Lahorani >Assignee: Janaki Lahorani >Priority: Major > Attachments: HIVE-18586.1.patch, HIVE-18586.2.patch, > HIVE-18586.3.patch, HIVE-18586.4.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-18665) LLAP: Ignore cache-affinity if the LLAP IO elevator is disabled
[ https://issues.apache.org/jira/browse/HIVE-18665?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16359014#comment-16359014 ] Gopal V commented on HIVE-18665: Not sure if that ever gets triggered, but it is pretty much a toss-up on whether to force affinity or not. > LLAP: Ignore cache-affinity if the LLAP IO elevator is disabled > --- > > Key: HIVE-18665 > URL: https://issues.apache.org/jira/browse/HIVE-18665 > Project: Hive > Issue Type: Bug > Components: llap >Affects Versions: 3.0.0 >Reporter: Gopal V >Assignee: Gopal V >Priority: Major > Attachments: HIVE-18665.1.patch > > > SplitLocationProvider removes HDFS locality in LLAP, causing more network > traffic when reading file formats which aren't currently cached. > In the absence of the IO elevator, cache affinity should be disabled. > This information is encoded into the MapWork in deriveLlapIoDescString, move > this to an Enum type for comparisons. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-18553) VectorizedParquetReader fails after adding a new column to table
[
https://issues.apache.org/jira/browse/HIVE-18553?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16359013#comment-16359013
]
Hive QA commented on HIVE-18553:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
23s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
6s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
38s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
56s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
20s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
9s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
9s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
42s{color} | {color:red} ql: The patch generated 22 new + 211 unchanged - 87
fixed = 233 total (was 298) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
59s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
14s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 15m 45s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /data/hiveptest/working/yetus/dev-support/hive-personality.sh |
| git revision | master / 58bbfc7 |
| Default Java | 1.8.0_111 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-9124/yetus/diff-checkstyle-ql.txt
|
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-9124/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> VectorizedParquetReader fails after adding a new column to table
>
>
> Key: HIVE-18553
> URL: https://issues.apache.org/jira/browse/HIVE-18553
> Project: Hive
> Issue Type: Sub-task
>Affects Versions: 3.0.0, 2.4.0, 2.3.2
>Reporter: Vihang Karajgaonkar
>Assignee: Ferdinand Xu
>Priority: Major
> Attachments: HIVE-18553.2.patch, HIVE-18553.3.patch,
> HIVE-18553.4.patch, HIVE-18553.5.patch, HIVE-18553.6.patch,
> HIVE-18553.7.patch, HIVE-18553.patch, test_result_based_on_HIVE-18553.xlsx
>
>
> VectorizedParquetReader throws an exception when trying to reading from a
> parquet table on which new columns are added. Steps to reproduce below:
> {code}
> 0: jdbc:hive2://localhost:1/default> desc test_p;
> +---++--+
> | col_name | data_type | comment |
> +---++--+
> | t1| tinyint| |
> | t2| tinyint| |
> | i1| int| |
> | i2| int| |
> +---++--+
> 0: jdbc:hive2://localhost:1/default> set hive.fetch.task.conversion=none;
> 0: jdbc:hive2://localhost:1/default> set
> hive.vectorized.execution.enabled=true;
> 0: jdbc:hive2://localhost:1/default> alter table test_p add columns (ts
> timestamp);
> 0: jdbc:hive2://localhost:1/default> select * from test_p;
> Error: Error while processing statement: FAILED: Execution Error, return code
> 2 from org.apache.hadoop.hive.ql.exec.mr.MapRedTask (state=08S01,code=2)
> {code}
> Following exception is seen in the logs
> {code}
> Caused by:
[jira] [Commented] (HIVE-18665) LLAP: Ignore cache-affinity if the LLAP IO elevator is disabled
[ https://issues.apache.org/jira/browse/HIVE-18665?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16359005#comment-16359005 ] Sergey Shelukhin commented on HIVE-18665: - Why is some inputs false? +1 otherwise pending tests > LLAP: Ignore cache-affinity if the LLAP IO elevator is disabled > --- > > Key: HIVE-18665 > URL: https://issues.apache.org/jira/browse/HIVE-18665 > Project: Hive > Issue Type: Bug > Components: llap >Affects Versions: 3.0.0 >Reporter: Gopal V >Assignee: Gopal V >Priority: Major > Attachments: HIVE-18665.1.patch > > > SplitLocationProvider removes HDFS locality in LLAP, causing more network > traffic when reading file formats which aren't currently cached. > In the absence of the IO elevator, cache affinity should be disabled. > This information is encoded into the MapWork in deriveLlapIoDescString, move > this to an Enum type for comparisons. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (HIVE-18674) update Hive to use ORC 1.4.3
[ https://issues.apache.org/jira/browse/HIVE-18674?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Shelukhin reassigned HIVE-18674: --- Assignee: Sergey Shelukhin > update Hive to use ORC 1.4.3 > > > Key: HIVE-18674 > URL: https://issues.apache.org/jira/browse/HIVE-18674 > Project: Hive > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin >Priority: Major > Attachments: HIVE-18674.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-18674) update Hive to use ORC 1.4.3
[ https://issues.apache.org/jira/browse/HIVE-18674?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Shelukhin updated HIVE-18674: Status: Patch Available (was: Open) [~prasanth_j] can you +1? :) > update Hive to use ORC 1.4.3 > > > Key: HIVE-18674 > URL: https://issues.apache.org/jira/browse/HIVE-18674 > Project: Hive > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin >Priority: Major > Attachments: HIVE-18674.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-18674) update Hive to use ORC 1.4.3
[ https://issues.apache.org/jira/browse/HIVE-18674?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Shelukhin updated HIVE-18674: Attachment: HIVE-18674.patch > update Hive to use ORC 1.4.3 > > > Key: HIVE-18674 > URL: https://issues.apache.org/jira/browse/HIVE-18674 > Project: Hive > Issue Type: Bug >Reporter: Sergey Shelukhin >Priority: Major > Attachments: HIVE-18674.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-18575) ACID properties usage in jobconf is ambiguous for MM tables
[ https://issues.apache.org/jira/browse/HIVE-18575?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Shelukhin updated HIVE-18575: Attachment: HIVE-18575.04.patch > ACID properties usage in jobconf is ambiguous for MM tables > --- > > Key: HIVE-18575 > URL: https://issues.apache.org/jira/browse/HIVE-18575 > Project: Hive > Issue Type: Bug > Components: Transactions >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin >Priority: Major > Attachments: HIVE-18575.01.patch, HIVE-18575.02.patch, > HIVE-18575.03.patch, HIVE-18575.04.patch, HIVE-18575.patch > > > Vectorization checks for ACID table trigger for MM tables where they don't > apply. Other places seem to set the setting for transactional case while most > of the code seems to assume it implies full acid. > Overall, many places in the code use the settings directly or set the ACID > flag without setting the ACID properties. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-18672) Printed state in RemoteSparkJobMonitor is ambiguous
[
https://issues.apache.org/jira/browse/HIVE-18672?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16358984#comment-16358984
]
Sahil Takiar commented on HIVE-18672:
-
Should probably create a new {{LogHelper}} here too so that the logs in the
{{hive.log}} file are printed as coming from their respective classes rather
than {{SessionState}}
> Printed state in RemoteSparkJobMonitor is ambiguous
> ---
>
> Key: HIVE-18672
> URL: https://issues.apache.org/jira/browse/HIVE-18672
> Project: Hive
> Issue Type: Sub-task
> Components: Spark
>Reporter: Sahil Takiar
>Priority: Major
>
> There are a few places in {{RemoteSparkJobMonitor}} (e.g. when the Spark job
> is in state QUEUED) where the state of the Spark job is printed, but the info
> is ambiguous (no reference to HoS, or the id of the Spark job).
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-18604) DropDatabase cascade fails when there is an index in the DB
[
https://issues.apache.org/jira/browse/HIVE-18604?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16358961#comment-16358961
]
Hive QA commented on HIVE-18604:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12909805/HIVE-18604.0.patch
{color:green}SUCCESS:{color} +1 due to 4 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 34 failed/errored test(s), 13147 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestAccumuloCliDriver.testCliDriver[accumulo_queries]
(batchId=241)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[auto_sortmerge_join_2]
(batchId=49)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[ppd_join5] (batchId=36)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[results_cache_1]
(batchId=68)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[row__id] (batchId=79)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[sample6] (batchId=68)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[udf_example_add]
(batchId=68)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[udf_from_unixtime]
(batchId=68)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[windowing_streaming]
(batchId=68)
org.apache.hadoop.hive.cli.TestEncryptedHDFSCliDriver.testCliDriver[encryption_move_tbl]
(batchId=175)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[llap_smb]
(batchId=152)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[bucket_map_join_tez1]
(batchId=172)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[insert_values_orig_table_use_metadata]
(batchId=167)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[llap_acid]
(batchId=171)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[llap_acid_fast]
(batchId=162)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[resourceplan]
(batchId=164)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[sysdb]
(batchId=161)
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver[spark_opt_shuffle_serde]
(batchId=180)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[ppd_join5]
(batchId=122)
org.apache.hadoop.hive.cli.TestSparkPerfCliDriver.testCliDriver[query1]
(batchId=251)
org.apache.hadoop.hive.cli.control.TestDanglingQOuts.checkDanglingQOut
(batchId=222)
org.apache.hadoop.hive.metastore.TestAcidTableSetup.testTransactionalValidation
(batchId=224)
org.apache.hadoop.hive.metastore.TestEmbeddedHiveMetaStore.testGetMetastoreUuid
(batchId=208)
org.apache.hadoop.hive.metastore.TestFilterHooks.org.apache.hadoop.hive.metastore.TestFilterHooks
(batchId=206)
org.apache.hadoop.hive.metastore.TestMetaStoreEventListener.testListener
(batchId=211)
org.apache.hadoop.hive.metastore.client.TestTablesCreateDropAlterTruncate.testAlterTableNullStorageDescriptorInNew[Embedded]
(batchId=206)
org.apache.hadoop.hive.ql.io.TestDruidRecordWriter.testWrite (batchId=257)
org.apache.hive.beeline.cli.TestHiveCli.testNoErrorDB (batchId=188)
org.apache.hive.hcatalog.listener.TestDbNotificationListener.alterIndex
(batchId=243)
org.apache.hive.hcatalog.listener.TestDbNotificationListener.createIndex
(batchId=243)
org.apache.hive.hcatalog.listener.TestDbNotificationListener.dropIndex
(batchId=243)
org.apache.hive.jdbc.TestSSL.testConnectionMismatch (batchId=235)
org.apache.hive.jdbc.TestSSL.testConnectionWrongCertCN (batchId=235)
org.apache.hive.jdbc.TestSSL.testMetastoreConnectionWrongCertCN (batchId=235)
{noformat}
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/9121/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/9121/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-9121/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 34 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12909805 - PreCommit-HIVE-Build
> DropDatabase cascade fails when there is an index in the DB
> ---
>
> Key: HIVE-18604
> URL: https://issues.apache.org/jira/browse/HIVE-18604
> Project: Hive
> Issue Type: Bug
> Components: Metastore
>Reporter: Adam Szita
>Assignee: Adam Szita
>Priority: Major
> Attachments: HIVE-18604.0.patch
>
>
> As seen in [HMS API
> test|https://github.com/apache/hive/blob/master/standalone-metastore/src/test/java/org/apache/hadoop/hive/metastore/client/TestDatabases.java#L452]
> dropping database (even with
[jira] [Updated] (HIVE-18666) Materialized view: "create materialized enable rewrite" should fail if rewriting is not possible
[
https://issues.apache.org/jira/browse/HIVE-18666?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jesus Camacho Rodriguez updated HIVE-18666:
---
Target Version/s: 3.0.0
> Materialized view: "create materialized enable rewrite" should fail if
> rewriting is not possible
>
>
> Key: HIVE-18666
> URL: https://issues.apache.org/jira/browse/HIVE-18666
> Project: Hive
> Issue Type: Bug
> Components: Materialized views
>Reporter: Gopal V
>Assignee: Jesus Camacho Rodriguez
>Priority: Major
>
> {code}
> CREATE MATERIALIZED VIEW TEST_AGG ENABLE REWRITE AS
> select ... from ext_Table;
> {code}
> works, but then
> {code}
> alter materialized view TEST_AGG enable rewrite;
> {code}
> fails with {{SemanticException Automatic rewriting for materialized view
> cannot be enabled if the materialized view uses non-transactional tables}}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-18667) Materialized views: rewrites should be triggered without checks if the time.window=-1
[ https://issues.apache.org/jira/browse/HIVE-18667?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jesus Camacho Rodriguez updated HIVE-18667: --- Target Version/s: 3.0.0 > Materialized views: rewrites should be triggered without checks if the > time.window=-1 > - > > Key: HIVE-18667 > URL: https://issues.apache.org/jira/browse/HIVE-18667 > Project: Hive > Issue Type: Bug > Components: Materialized views >Reporter: Gopal V >Assignee: Jesus Camacho Rodriguez >Priority: Major > > This is useful to check for Calcite failures to rewrite to a view instead of > trying to determine if the rewrite was avoided due to a timeout on the view > freshness. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (HIVE-18667) Materialized views: rewrites should be triggered without checks if the time.window=-1
[ https://issues.apache.org/jira/browse/HIVE-18667?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jesus Camacho Rodriguez reassigned HIVE-18667: -- Assignee: Jesus Camacho Rodriguez > Materialized views: rewrites should be triggered without checks if the > time.window=-1 > - > > Key: HIVE-18667 > URL: https://issues.apache.org/jira/browse/HIVE-18667 > Project: Hive > Issue Type: Bug > Components: Materialized views >Reporter: Gopal V >Assignee: Jesus Camacho Rodriguez >Priority: Major > > This is useful to check for Calcite failures to rewrite to a view instead of > trying to determine if the rewrite was avoided due to a timeout on the view > freshness. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (HIVE-18666) Materialized view: "create materialized enable rewrite" should fail if rewriting is not possible
[
https://issues.apache.org/jira/browse/HIVE-18666?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jesus Camacho Rodriguez reassigned HIVE-18666:
--
Assignee: Jesus Camacho Rodriguez
> Materialized view: "create materialized enable rewrite" should fail if
> rewriting is not possible
>
>
> Key: HIVE-18666
> URL: https://issues.apache.org/jira/browse/HIVE-18666
> Project: Hive
> Issue Type: Bug
> Components: Materialized views
>Reporter: Gopal V
>Assignee: Jesus Camacho Rodriguez
>Priority: Major
>
> {code}
> CREATE MATERIALIZED VIEW TEST_AGG ENABLE REWRITE AS
> select ... from ext_Table;
> {code}
> works, but then
> {code}
> alter materialized view TEST_AGG enable rewrite;
> {code}
> fails with {{SemanticException Automatic rewriting for materialized view
> cannot be enabled if the materialized view uses non-transactional tables}}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-18671) lock not released after Hive on Spark query was cancelled
[
https://issues.apache.org/jira/browse/HIVE-18671?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yongzhi Chen updated HIVE-18671:
Attachment: (was: HIVE-18671.1.patch)
> lock not released after Hive on Spark query was cancelled
> -
>
> Key: HIVE-18671
> URL: https://issues.apache.org/jira/browse/HIVE-18671
> Project: Hive
> Issue Type: Bug
>Affects Versions: 2.3.2
>Reporter: Yongzhi Chen
>Assignee: Yongzhi Chen
>Priority: Major
>
> When cancel the query is running on spark, the SparkJobMonitor can not
> return, therefore the locks hold by the query can not be released. When
> enable debug in log, you will see many log info as following:
> {noformat}
> 2018-02-09 08:27:09,613 INFO
> org.apache.hadoop.hive.ql.exec.spark.status.SparkJobMonitor:
> [HiveServer2-Background-Pool: Thread-80]: state = CANCELLED
> 2018-02-09 08:27:10,613 INFO
> org.apache.hadoop.hive.ql.exec.spark.status.SparkJobMonitor:
> [HiveServer2-Background-Pool: Thread-80]: state = CANCELLED
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-18622) Vectorization: IF Statements, Comparisons, and more do not handle NULLs correctly
[
https://issues.apache.org/jira/browse/HIVE-18622?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Matt McCline updated HIVE-18622:
Status: Patch Available (was: In Progress)
> Vectorization: IF Statements, Comparisons, and more do not handle NULLs
> correctly
> -
>
> Key: HIVE-18622
> URL: https://issues.apache.org/jira/browse/HIVE-18622
> Project: Hive
> Issue Type: Bug
> Components: Hive
>Reporter: Matt McCline
>Assignee: Matt McCline
>Priority: Critical
> Fix For: 3.0.0
>
> Attachments: HIVE-18622.03.patch, HIVE-18622.04.patch,
> HIVE-18622.05.patch, HIVE-18622.06.patch, HIVE-18622.07.patch
>
>
>
> Many vector expression classes are setting noNulls to true which does not
> work if the VRB is a scratch column being reused. The previous use may have
> set noNulls to false and the isNull array will have some rows marked as NULL.
> The result is wrong query results and sometimes NPEs (for BytesColumnVector).
> So, many vector expressions need this:
> {code:java}
> // Carefully handle NULLs...
> /*
>* For better performance on LONG/DOUBLE we don't want the conditional
>* statements inside the for loop.
>*/
> outputColVector.noNulls = false;
> {code}
> And, vector expressions need to make sure the isNull array entry is set when
> outputColVector.noNulls is false.
> And, all place that assign column value need to set noNulls to false when the
> value is NULL.
> Almost all cases where noNulls is set to true are incorrect.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-18622) Vectorization: IF Statements, Comparisons, and more do not handle NULLs correctly
[
https://issues.apache.org/jira/browse/HIVE-18622?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Matt McCline updated HIVE-18622:
Status: In Progress (was: Patch Available)
> Vectorization: IF Statements, Comparisons, and more do not handle NULLs
> correctly
> -
>
> Key: HIVE-18622
> URL: https://issues.apache.org/jira/browse/HIVE-18622
> Project: Hive
> Issue Type: Bug
> Components: Hive
>Reporter: Matt McCline
>Assignee: Matt McCline
>Priority: Critical
> Fix For: 3.0.0
>
> Attachments: HIVE-18622.03.patch, HIVE-18622.04.patch,
> HIVE-18622.05.patch, HIVE-18622.06.patch, HIVE-18622.07.patch
>
>
>
> Many vector expression classes are setting noNulls to true which does not
> work if the VRB is a scratch column being reused. The previous use may have
> set noNulls to false and the isNull array will have some rows marked as NULL.
> The result is wrong query results and sometimes NPEs (for BytesColumnVector).
> So, many vector expressions need this:
> {code:java}
> // Carefully handle NULLs...
> /*
>* For better performance on LONG/DOUBLE we don't want the conditional
>* statements inside the for loop.
>*/
> outputColVector.noNulls = false;
> {code}
> And, vector expressions need to make sure the isNull array entry is set when
> outputColVector.noNulls is false.
> And, all place that assign column value need to set noNulls to false when the
> value is NULL.
> Almost all cases where noNulls is set to true are incorrect.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-18622) Vectorization: IF Statements, Comparisons, and more do not handle NULLs correctly
[
https://issues.apache.org/jira/browse/HIVE-18622?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Matt McCline updated HIVE-18622:
Attachment: HIVE-18622.07.patch
> Vectorization: IF Statements, Comparisons, and more do not handle NULLs
> correctly
> -
>
> Key: HIVE-18622
> URL: https://issues.apache.org/jira/browse/HIVE-18622
> Project: Hive
> Issue Type: Bug
> Components: Hive
>Reporter: Matt McCline
>Assignee: Matt McCline
>Priority: Critical
> Fix For: 3.0.0
>
> Attachments: HIVE-18622.03.patch, HIVE-18622.04.patch,
> HIVE-18622.05.patch, HIVE-18622.06.patch, HIVE-18622.07.patch
>
>
>
> Many vector expression classes are setting noNulls to true which does not
> work if the VRB is a scratch column being reused. The previous use may have
> set noNulls to false and the isNull array will have some rows marked as NULL.
> The result is wrong query results and sometimes NPEs (for BytesColumnVector).
> So, many vector expressions need this:
> {code:java}
> // Carefully handle NULLs...
> /*
>* For better performance on LONG/DOUBLE we don't want the conditional
>* statements inside the for loop.
>*/
> outputColVector.noNulls = false;
> {code}
> And, vector expressions need to make sure the isNull array entry is set when
> outputColVector.noNulls is false.
> And, all place that assign column value need to set noNulls to false when the
> value is NULL.
> Almost all cases where noNulls is set to true are incorrect.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-18671) lock not released after Hive on Spark query was cancelled
[
https://issues.apache.org/jira/browse/HIVE-18671?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yongzhi Chen updated HIVE-18671:
Status: Patch Available (was: Open)
Add the case to handle state "CANCELLED".
> lock not released after Hive on Spark query was cancelled
> -
>
> Key: HIVE-18671
> URL: https://issues.apache.org/jira/browse/HIVE-18671
> Project: Hive
> Issue Type: Bug
>Affects Versions: 2.3.2
>Reporter: Yongzhi Chen
>Assignee: Yongzhi Chen
>Priority: Major
> Attachments: HIVE-18671.1.patch
>
>
> When cancel the query is running on spark, the SparkJobMonitor can not
> return, therefore the locks hold by the query can not be released. When
> enable debug in log, you will see many log info as following:
> {noformat}
> 2018-02-09 08:27:09,613 INFO
> org.apache.hadoop.hive.ql.exec.spark.status.SparkJobMonitor:
> [HiveServer2-Background-Pool: Thread-80]: state = CANCELLED
> 2018-02-09 08:27:10,613 INFO
> org.apache.hadoop.hive.ql.exec.spark.status.SparkJobMonitor:
> [HiveServer2-Background-Pool: Thread-80]: state = CANCELLED
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-18671) lock not released after Hive on Spark query was cancelled
[
https://issues.apache.org/jira/browse/HIVE-18671?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yongzhi Chen updated HIVE-18671:
Attachment: HIVE-18671.1.patch
> lock not released after Hive on Spark query was cancelled
> -
>
> Key: HIVE-18671
> URL: https://issues.apache.org/jira/browse/HIVE-18671
> Project: Hive
> Issue Type: Bug
>Affects Versions: 2.3.2
>Reporter: Yongzhi Chen
>Assignee: Yongzhi Chen
>Priority: Major
> Attachments: HIVE-18671.1.patch
>
>
> When cancel the query is running on spark, the SparkJobMonitor can not
> return, therefore the locks hold by the query can not be released. When
> enable debug in log, you will see many log info as following:
> {noformat}
> 2018-02-09 08:27:09,613 INFO
> org.apache.hadoop.hive.ql.exec.spark.status.SparkJobMonitor:
> [HiveServer2-Background-Pool: Thread-80]: state = CANCELLED
> 2018-02-09 08:27:10,613 INFO
> org.apache.hadoop.hive.ql.exec.spark.status.SparkJobMonitor:
> [HiveServer2-Background-Pool: Thread-80]: state = CANCELLED
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-17284) remove OrcRecordUpdater.deleteEventIndexBuilder
[
https://issues.apache.org/jira/browse/HIVE-17284?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16358915#comment-16358915
]
Eugene Koifman commented on HIVE-17284:
---
this may not be the right thing to do. ORC flattens structs (ROW__ID) and will
maintain min/max for individual columns. To filter events we really need
min/max ROW__ID
> remove OrcRecordUpdater.deleteEventIndexBuilder
> ---
>
> Key: HIVE-17284
> URL: https://issues.apache.org/jira/browse/HIVE-17284
> Project: Hive
> Issue Type: Improvement
> Components: Transactions
>Affects Versions: 3.0.0
>Reporter: Eugene Koifman
>Assignee: Eugene Koifman
>Priority: Minor
>
> There is no point in it. We know how many rows a delete_delta file has from
> ORC and they are all the same type - so no need for AcidStats.
> hive.acid.key.index has no value since delete_delta files are never split
> and are not likely to have more than 1 stripe since they are very small.
> Also can remove KeyIndexBuilder.acidStats - we only have 1 type of event per
> file
>
> if doing this, make sure to fix {{OrcInputFormat.isOriginal(Reader)}} and
> {{OrcInputFormat.isOriginal(Footer)}} etc
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Comment Edited] (HIVE-17284) remove OrcRecordUpdater.deleteEventIndexBuilder
[
https://issues.apache.org/jira/browse/HIVE-17284?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16358915#comment-16358915
]
Eugene Koifman edited comment on HIVE-17284 at 2/9/18 8:31 PM:
---
this may not be the right thing to do. ORC flattens structs ({{ROW__ID}}) and
will maintain min/max for individual columns. To filter events we really need
min/max {{ROW__ID}}
was (Author: ekoifman):
this may not be the right thing to do. ORC flattens structs (ROW__ID) and will
maintain min/max for individual columns. To filter events we really need
min/max ROW__ID
> remove OrcRecordUpdater.deleteEventIndexBuilder
> ---
>
> Key: HIVE-17284
> URL: https://issues.apache.org/jira/browse/HIVE-17284
> Project: Hive
> Issue Type: Improvement
> Components: Transactions
>Affects Versions: 3.0.0
>Reporter: Eugene Koifman
>Assignee: Eugene Koifman
>Priority: Minor
>
> There is no point in it. We know how many rows a delete_delta file has from
> ORC and they are all the same type - so no need for AcidStats.
> hive.acid.key.index has no value since delete_delta files are never split
> and are not likely to have more than 1 stripe since they are very small.
> Also can remove KeyIndexBuilder.acidStats - we only have 1 type of event per
> file
>
> if doing this, make sure to fix {{OrcInputFormat.isOriginal(Reader)}} and
> {{OrcInputFormat.isOriginal(Footer)}} etc
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-18359) Extend grouping set limits from int to long
[ https://issues.apache.org/jira/browse/HIVE-18359?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16358907#comment-16358907 ] Prasanth Jayachandran commented on HIVE-18359: -- Minor golden file updates for 3 test failures. Most other failures cannot be reproduced locally or unrelated to this patch. > Extend grouping set limits from int to long > --- > > Key: HIVE-18359 > URL: https://issues.apache.org/jira/browse/HIVE-18359 > Project: Hive > Issue Type: Bug >Affects Versions: 3.0.0 >Reporter: Prasanth Jayachandran >Assignee: Prasanth Jayachandran >Priority: Major > Attachments: HIVE-18359.1.patch, HIVE-18359.10.patch, > HIVE-18359.11.patch, HIVE-18359.11.patch, HIVE-18359.12.patch, > HIVE-18359.2.patch, HIVE-18359.3.patch, HIVE-18359.4.patch, > HIVE-18359.5.patch, HIVE-18359.6.patch, HIVE-18359.7.patch, > HIVE-18359.8.patch, HIVE-18359.9.patch > > > Grouping sets is broken for >32 columns because of usage of Int for bitmap > (also GROUPING__ID virtual column). This assumption breaks grouping > sets/rollups/cube when number of participating aggregation columns is >32. > The easier fix would be extend it to Long for now. The correct fix would be > to use BitSets everywhere but that would require GROUPING__ID column type to > binary which will make predicates on GROUPING__ID difficult to deal with. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-18359) Extend grouping set limits from int to long
[ https://issues.apache.org/jira/browse/HIVE-18359?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prasanth Jayachandran updated HIVE-18359: - Attachment: HIVE-18359.12.patch > Extend grouping set limits from int to long > --- > > Key: HIVE-18359 > URL: https://issues.apache.org/jira/browse/HIVE-18359 > Project: Hive > Issue Type: Bug >Affects Versions: 3.0.0 >Reporter: Prasanth Jayachandran >Assignee: Prasanth Jayachandran >Priority: Major > Attachments: HIVE-18359.1.patch, HIVE-18359.10.patch, > HIVE-18359.11.patch, HIVE-18359.11.patch, HIVE-18359.12.patch, > HIVE-18359.2.patch, HIVE-18359.3.patch, HIVE-18359.4.patch, > HIVE-18359.5.patch, HIVE-18359.6.patch, HIVE-18359.7.patch, > HIVE-18359.8.patch, HIVE-18359.9.patch > > > Grouping sets is broken for >32 columns because of usage of Int for bitmap > (also GROUPING__ID virtual column). This assumption breaks grouping > sets/rollups/cube when number of participating aggregation columns is >32. > The easier fix would be extend it to Long for now. The correct fix would be > to use BitSets everywhere but that would require GROUPING__ID column type to > binary which will make predicates on GROUPING__ID difficult to deal with. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Issue Comment Deleted] (HIVE-17844) Hive JDBC driver converts floats to doubles, changing values slightly
[ https://issues.apache.org/jira/browse/HIVE-17844?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vihang Karajgaonkar updated HIVE-17844: --- Comment: was deleted (was: [~zi] Can you give a example query which can lead to incorrect results? Thanks for reporting this..) > Hive JDBC driver converts floats to doubles, changing values slightly > - > > Key: HIVE-17844 > URL: https://issues.apache.org/jira/browse/HIVE-17844 > Project: Hive > Issue Type: Bug > Components: JDBC >Reporter: Zoltan Ivanfi >Priority: Major > > When querying data through Hive's JDBC driver, float values are automatically > converted to doubles and will slightly differ from values seen when directly > querying the data using a textual SQL interface. > Please note that by slight difference I don't only mean additional zeroes at > the end, but an actual numeric difference in the displayed value. This is the > result of the different precision used when displaying double values, as > discussed > [here|https://stackoverflow.com/questions/17504833/why-converting-from-float-to-double-changes-the-value], > for example. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-17844) Hive JDBC driver converts floats to doubles, changing values slightly
[ https://issues.apache.org/jira/browse/HIVE-17844?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16358902#comment-16358902 ] Vihang Karajgaonkar commented on HIVE-17844: [~zi] Can you give a example query which can lead to incorrect results? Thanks for reporting this.. > Hive JDBC driver converts floats to doubles, changing values slightly > - > > Key: HIVE-17844 > URL: https://issues.apache.org/jira/browse/HIVE-17844 > Project: Hive > Issue Type: Bug > Components: JDBC >Reporter: Zoltan Ivanfi >Priority: Major > > When querying data through Hive's JDBC driver, float values are automatically > converted to doubles and will slightly differ from values seen when directly > querying the data using a textual SQL interface. > Please note that by slight difference I don't only mean additional zeroes at > the end, but an actual numeric difference in the displayed value. This is the > result of the different precision used when displaying double values, as > discussed > [here|https://stackoverflow.com/questions/17504833/why-converting-from-float-to-double-changes-the-value], > for example. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-17843) UINT32 Parquet columns are handled as signed INT32-s, silently reading incorrect data
[
https://issues.apache.org/jira/browse/HIVE-17843?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16358900#comment-16358900
]
Vihang Karajgaonkar commented on HIVE-17843:
[~zi] Can you give a example query which can lead to incorrect results? Thanks
for reporting this..
> UINT32 Parquet columns are handled as signed INT32-s, silently reading
> incorrect data
> -
>
> Key: HIVE-17843
> URL: https://issues.apache.org/jira/browse/HIVE-17843
> Project: Hive
> Issue Type: Bug
>Reporter: Zoltan Ivanfi
>Priority: Major
>
> An unsigned 32 bit Parquet column, such as
> {noformat}
> optional int32 uint_32_col (UINT_32)
> {noformat}
> is read by Hive as if it were signed, leading to incorrect results.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-18654) Add Hiveserver2 specific HADOOP_OPTS environment variable
[ https://issues.apache.org/jira/browse/HIVE-18654?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vihang Karajgaonkar updated HIVE-18654: --- Attachment: HIVE-18654.01.patch > Add Hiveserver2 specific HADOOP_OPTS environment variable > -- > > Key: HIVE-18654 > URL: https://issues.apache.org/jira/browse/HIVE-18654 > Project: Hive > Issue Type: Improvement > Components: HiveServer2 >Reporter: Vihang Karajgaonkar >Assignee: Vihang Karajgaonkar >Priority: Minor > Attachments: HIVE-18654.01.patch > > > HIVE-2665 added support to include metastore specific HADOOP_OPTS variable. > This is helpful in debugging especially if you want to add some jvm > parameters to metastore's process. A similar setting for Hiveserver2 is > missing and could be very helpful in debugging. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-18654) Add Hiveserver2 specific HADOOP_OPTS environment variable
[ https://issues.apache.org/jira/browse/HIVE-18654?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16358898#comment-16358898 ] Vihang Karajgaonkar commented on HIVE-18654: [~aihuaxu] [~stakiar] can you take a look? > Add Hiveserver2 specific HADOOP_OPTS environment variable > -- > > Key: HIVE-18654 > URL: https://issues.apache.org/jira/browse/HIVE-18654 > Project: Hive > Issue Type: Improvement > Components: HiveServer2 >Reporter: Vihang Karajgaonkar >Assignee: Vihang Karajgaonkar >Priority: Minor > Attachments: HIVE-18654.01.patch > > > HIVE-2665 added support to include metastore specific HADOOP_OPTS variable. > This is helpful in debugging especially if you want to add some jvm > parameters to metastore's process. A similar setting for Hiveserver2 is > missing and could be very helpful in debugging. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-18604) DropDatabase cascade fails when there is an index in the DB
[
https://issues.apache.org/jira/browse/HIVE-18604?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16358895#comment-16358895
]
Hive QA commented on HIVE-18604:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
46s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
39s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
19s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
48s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
39s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
38s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
38s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
20s{color} | {color:green} standalone-metastore: The patch generated 0 new +
396 unchanged - 1 fixed = 396 total (was 397) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
52s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
13s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 12m 29s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /data/hiveptest/working/yetus/dev-support/hive-personality.sh |
| git revision | master / 58bbfc7 |
| Default Java | 1.8.0_111 |
| modules | C: standalone-metastore U: standalone-metastore |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-9121/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> DropDatabase cascade fails when there is an index in the DB
> ---
>
> Key: HIVE-18604
> URL: https://issues.apache.org/jira/browse/HIVE-18604
> Project: Hive
> Issue Type: Bug
> Components: Metastore
>Reporter: Adam Szita
>Assignee: Adam Szita
>Priority: Major
> Attachments: HIVE-18604.0.patch
>
>
> As seen in [HMS API
> test|https://github.com/apache/hive/blob/master/standalone-metastore/src/test/java/org/apache/hadoop/hive/metastore/client/TestDatabases.java#L452]
> dropping database (even with cascade) is failing when an index exists in the
> corresponding database, throwing MetaException:
> {code:java}
> MetaException(message:Exception thrown flushing changes to datastore
> )
> at
> org.apache.hadoop.hive.metastore.RetryingHMSHandler.invokeInternal(RetryingHMSHandler.java:208)
> at
> org.apache.hadoop.hive.metastore.RetryingHMSHandler.invoke(RetryingHMSHandler.java:108)
> at com.sun.proxy.$Proxy35.drop_table_with_environment_context(Unknown Source)
> at
> org.apache.hadoop.hive.metastore.HiveMetaStoreClient.drop_table_with_environment_context(HiveMetaStoreClient.java:2495)
> at
> org.apache.hadoop.hive.metastore.HiveMetaStoreClient.dropTable(HiveMetaStoreClient.java:1092)
> at
> org.apache.hadoop.hive.metastore.HiveMetaStoreClient.dropTable(HiveMetaStoreClient.java:1007)
> at
> org.apache.hadoop.hive.metastore.HiveMetaStoreClient.dropDatabase(HiveMetaStoreClient.java:859)
> at
> org.apache.hadoop.hive.metastore.client.TestDatabases.testDropDatabaseWithIndexCascade(TestDatabases.java:470)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
>
[jira] [Commented] (HIVE-18456) Add some tests for HIVE-18367 to check that the table information contains the query correctly
[ https://issues.apache.org/jira/browse/HIVE-18456?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16358893#comment-16358893 ] Andrew Sherman commented on HIVE-18456: --- Test failures look unrelated (not surprisingly as this is a test-only change). Please can you take a look [~pvary] > Add some tests for HIVE-18367 to check that the table information contains > the query correctly > -- > > Key: HIVE-18456 > URL: https://issues.apache.org/jira/browse/HIVE-18456 > Project: Hive > Issue Type: Bug >Reporter: Andrew Sherman >Assignee: Andrew Sherman >Priority: Major > Attachments: HIVE-18456.1.patch, HIVE-18456.2.patch > > > This cannot be tested with a CliDriver test so add a java test to check the > output of 'describe extended', which is changed by HIVE-18367 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-18625) SessionState Not Checking For Directory Creation Result
[
https://issues.apache.org/jira/browse/HIVE-18625?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16358886#comment-16358886
]
Andrew Sherman commented on HIVE-18625:
---
Made changes, ran tests, and failures look unrelated. Please can you take
another look [~stakiar]
> SessionState Not Checking For Directory Creation Result
> ---
>
> Key: HIVE-18625
> URL: https://issues.apache.org/jira/browse/HIVE-18625
> Project: Hive
> Issue Type: Improvement
> Components: HiveServer2
>Affects Versions: 3.0.0, 2.4.0, 2.3.2
>Reporter: BELUGA BEHR
>Assignee: Andrew Sherman
>Priority: Minor
> Attachments: HIVE-18625.1.patch, HIVE-18625.2.patch
>
>
> https://github.com/apache/hive/blob/master/ql/src/java/org/apache/hadoop/hive/ql/session/SessionState.java#L773
> {code:java}
> private static void createPath(HiveConf conf, Path path, String permission,
> boolean isLocal,
> boolean isCleanUp) throws IOException {
> FsPermission fsPermission = new FsPermission(permission);
> FileSystem fs;
> if (isLocal) {
> fs = FileSystem.getLocal(conf);
> } else {
> fs = path.getFileSystem(conf);
> }
> if (!fs.exists(path)) {
> fs.mkdirs(path, fsPermission);
> String dirType = isLocal ? "local" : "HDFS";
> LOG.info("Created " + dirType + " directory: " + path.toString());
> }
> if (isCleanUp) {
> fs.deleteOnExit(path);
> }
> }
> {code}
> The method {{fs.mkdirs(path, fsPermission)}} returns a boolean value
> indicating if the directory creation was successful or not. Hive ignores
> this return value and therefore could be acting on a directory that doesn't
> exist.
> Please capture the result, check it, and throw an Exception if it failed
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-18673) ErrorMsg.SPARK_JOB_MONITOR_TIMEOUT isn't formatted correctly
[
https://issues.apache.org/jira/browse/HIVE-18673?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sahil Takiar updated HIVE-18673:
Attachment: HIVE-18673.1.patch
> ErrorMsg.SPARK_JOB_MONITOR_TIMEOUT isn't formatted correctly
>
>
> Key: HIVE-18673
> URL: https://issues.apache.org/jira/browse/HIVE-18673
> Project: Hive
> Issue Type: Sub-task
> Components: Spark
>Reporter: Sahil Takiar
>Assignee: Sahil Takiar
>Priority: Major
> Attachments: HIVE-18673.1.patch
>
>
> {{ErrorMsg.SPARK_JOB_MONITOR_TIMEOUT}} doesn't format the amount of time
> waited correctly. Mainly because Java's {{MessageFormat}} class requires
> escaping single quotes with another single quote.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-18673) ErrorMsg.SPARK_JOB_MONITOR_TIMEOUT isn't formatted correctly
[
https://issues.apache.org/jira/browse/HIVE-18673?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sahil Takiar updated HIVE-18673:
Status: Patch Available (was: Open)
> ErrorMsg.SPARK_JOB_MONITOR_TIMEOUT isn't formatted correctly
>
>
> Key: HIVE-18673
> URL: https://issues.apache.org/jira/browse/HIVE-18673
> Project: Hive
> Issue Type: Sub-task
> Components: Spark
>Reporter: Sahil Takiar
>Assignee: Sahil Takiar
>Priority: Major
> Attachments: HIVE-18673.1.patch
>
>
> {{ErrorMsg.SPARK_JOB_MONITOR_TIMEOUT}} doesn't format the amount of time
> waited correctly. Mainly because Java's {{MessageFormat}} class requires
> escaping single quotes with another single quote.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-18541) Secure HS2 web UI with PAM
[
https://issues.apache.org/jira/browse/HIVE-18541?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16358875#comment-16358875
]
Hive QA commented on HIVE-18541:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12909777/HIVE-18541.5.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 21 failed/errored test(s), 13155 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestAccumuloCliDriver.testCliDriver[accumulo_queries]
(batchId=241)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[ppd_join5] (batchId=36)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[row__id] (batchId=79)
org.apache.hadoop.hive.cli.TestEncryptedHDFSCliDriver.testCliDriver[encryption_move_tbl]
(batchId=175)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[llap_smb]
(batchId=152)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[bucket_map_join_tez1]
(batchId=172)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[insert_values_orig_table_use_metadata]
(batchId=167)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[llap_acid]
(batchId=171)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[llap_acid_fast]
(batchId=162)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[resourceplan]
(batchId=164)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[sysdb]
(batchId=161)
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver[spark_opt_shuffle_serde]
(batchId=180)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[ppd_join5]
(batchId=122)
org.apache.hadoop.hive.cli.TestSparkPerfCliDriver.testCliDriver[query1]
(batchId=251)
org.apache.hadoop.hive.cli.control.TestDanglingQOuts.checkDanglingQOut
(batchId=222)
org.apache.hadoop.hive.ql.io.TestDruidRecordWriter.testWrite (batchId=257)
org.apache.hadoop.hive.ql.parse.TestReplicationScenarios.testExchangePartition
(batchId=229)
org.apache.hive.beeline.cli.TestHiveCli.testNoErrorDB (batchId=188)
org.apache.hive.jdbc.TestSSL.testConnectionMismatch (batchId=235)
org.apache.hive.jdbc.TestSSL.testConnectionWrongCertCN (batchId=235)
org.apache.hive.jdbc.TestSSL.testMetastoreConnectionWrongCertCN (batchId=235)
{noformat}
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/9120/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/9120/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-9120/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 21 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12909777 - PreCommit-HIVE-Build
> Secure HS2 web UI with PAM
> --
>
> Key: HIVE-18541
> URL: https://issues.apache.org/jira/browse/HIVE-18541
> Project: Hive
> Issue Type: Sub-task
> Components: HiveServer2
>Reporter: Oleksiy Sayankin
>Assignee: Oleksiy Sayankin
>Priority: Major
> Fix For: 3.0.0
>
> Attachments: HIVE-18541.1.patch, HIVE-18541.2.patch,
> HIVE-18541.5.patch
>
>
> Secure HS2 web UI with PAM. Add two new properties
> * hive.server2.webui.use.pam
> * Default value: false
> * Description: If true, the HiveServer2 WebUI will be secured with PAM
> * hive.server2.webui.pam.authenticator
> * Default value: org.apache.hive.http.security.PamAuthenticator
> * Description: Class for PAM authentication
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-18673) ErrorMsg.SPARK_JOB_MONITOR_TIMEOUT isn't formatted correctly
[
https://issues.apache.org/jira/browse/HIVE-18673?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16358874#comment-16358874
]
Sahil Takiar commented on HIVE-18673:
-
Explanation of {{MessageFormat}} behavior with single quotes:
https://docs.oracle.com/javase/7/docs/api/java/text/MessageFormat.html
> ErrorMsg.SPARK_JOB_MONITOR_TIMEOUT isn't formatted correctly
>
>
> Key: HIVE-18673
> URL: https://issues.apache.org/jira/browse/HIVE-18673
> Project: Hive
> Issue Type: Sub-task
> Components: Spark
>Reporter: Sahil Takiar
>Assignee: Sahil Takiar
>Priority: Major
>
> {{ErrorMsg.SPARK_JOB_MONITOR_TIMEOUT}} doesn't format the amount of time
> waited correctly. Mainly because Java's {{MessageFormat}} class requires
> escaping single quotes with another single quote.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Assigned] (HIVE-18673) ErrorMsg.SPARK_JOB_MONITOR_TIMEOUT isn't formatted correctly
[
https://issues.apache.org/jira/browse/HIVE-18673?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sahil Takiar reassigned HIVE-18673:
---
> ErrorMsg.SPARK_JOB_MONITOR_TIMEOUT isn't formatted correctly
>
>
> Key: HIVE-18673
> URL: https://issues.apache.org/jira/browse/HIVE-18673
> Project: Hive
> Issue Type: Sub-task
> Components: Spark
>Reporter: Sahil Takiar
>Assignee: Sahil Takiar
>Priority: Major
>
> {{ErrorMsg.SPARK_JOB_MONITOR_TIMEOUT}} doesn't format the amount of time
> waited correctly. Mainly because Java's {{MessageFormat}} class requires
> escaping single quotes with another single quote.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-18650) Add compression related tests for blobstores
[ https://issues.apache.org/jira/browse/HIVE-18650?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuzhou Sun updated HIVE-18650: -- Status: Patch Available (was: Open) Update with binary patch > Add compression related tests for blobstores > > > Key: HIVE-18650 > URL: https://issues.apache.org/jira/browse/HIVE-18650 > Project: Hive > Issue Type: Test > Components: Tests >Affects Versions: 2.3.2 >Reporter: Yuzhou Sun >Assignee: Yuzhou Sun >Priority: Major > Attachments: HIVE-18650.patch, over10k.bz2, s_aa.gz, s_ab.gz, > s_ac.gz, s_ad.gz, s_ae.gz, s_af.gz, s_ag.gz, s_ah.gz, s_ai.gz, s_aj.gz > > > This patch introduces the following tests and files into the hive-blobstore > module: > * bzip_read.q -> Test read bzip compressed input data > * bzip_write.q -> Test write gzip compressed output table > * gzip_read.q -> Test read gzip compressed input data > * gzip_write.q -> Test write gzip compressed output table > * gzip_splits.q -> Test read splited gzip compressed input data > * over10k.bz2 -> Bzip compressed file of over10k > * gzip_splits/ -> 10 gzip files of 21-22 KB -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-18650) Add compression related tests for blobstores
[ https://issues.apache.org/jira/browse/HIVE-18650?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuzhou Sun updated HIVE-18650: -- Status: Open (was: Patch Available) Will update it with binary diff patch > Add compression related tests for blobstores > > > Key: HIVE-18650 > URL: https://issues.apache.org/jira/browse/HIVE-18650 > Project: Hive > Issue Type: Test > Components: Tests >Affects Versions: 2.3.2 >Reporter: Yuzhou Sun >Assignee: Yuzhou Sun >Priority: Major > Attachments: HIVE-18650.patch, over10k.bz2, s_aa.gz, s_ab.gz, > s_ac.gz, s_ad.gz, s_ae.gz, s_af.gz, s_ag.gz, s_ah.gz, s_ai.gz, s_aj.gz > > > This patch introduces the following tests and files into the hive-blobstore > module: > * bzip_read.q -> Test read bzip compressed input data > * bzip_write.q -> Test write gzip compressed output table > * gzip_read.q -> Test read gzip compressed input data > * gzip_write.q -> Test write gzip compressed output table > * gzip_splits.q -> Test read splited gzip compressed input data > * over10k.bz2 -> Bzip compressed file of over10k > * gzip_splits/ -> 10 gzip files of 21-22 KB -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-18650) Add compression related tests for blobstores
[ https://issues.apache.org/jira/browse/HIVE-18650?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuzhou Sun updated HIVE-18650: -- Attachment: (was: HIVE-18650.patch) > Add compression related tests for blobstores > > > Key: HIVE-18650 > URL: https://issues.apache.org/jira/browse/HIVE-18650 > Project: Hive > Issue Type: Test > Components: Tests >Affects Versions: 2.3.2 >Reporter: Yuzhou Sun >Assignee: Yuzhou Sun >Priority: Major > Attachments: HIVE-18650.patch, over10k.bz2, s_aa.gz, s_ab.gz, > s_ac.gz, s_ad.gz, s_ae.gz, s_af.gz, s_ag.gz, s_ah.gz, s_ai.gz, s_aj.gz > > > This patch introduces the following tests and files into the hive-blobstore > module: > * bzip_read.q -> Test read bzip compressed input data > * bzip_write.q -> Test write gzip compressed output table > * gzip_read.q -> Test read gzip compressed input data > * gzip_write.q -> Test write gzip compressed output table > * gzip_splits.q -> Test read splited gzip compressed input data > * over10k.bz2 -> Bzip compressed file of over10k > * gzip_splits/ -> 10 gzip files of 21-22 KB -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-18650) Add compression related tests for blobstores
[ https://issues.apache.org/jira/browse/HIVE-18650?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuzhou Sun updated HIVE-18650: -- Attachment: HIVE-18650.patch > Add compression related tests for blobstores > > > Key: HIVE-18650 > URL: https://issues.apache.org/jira/browse/HIVE-18650 > Project: Hive > Issue Type: Test > Components: Tests >Affects Versions: 2.3.2 >Reporter: Yuzhou Sun >Assignee: Yuzhou Sun >Priority: Major > Attachments: HIVE-18650.patch, HIVE-18650.patch, over10k.bz2, > s_aa.gz, s_ab.gz, s_ac.gz, s_ad.gz, s_ae.gz, s_af.gz, s_ag.gz, s_ah.gz, > s_ai.gz, s_aj.gz > > > This patch introduces the following tests and files into the hive-blobstore > module: > * bzip_read.q -> Test read bzip compressed input data > * bzip_write.q -> Test write gzip compressed output table > * gzip_read.q -> Test read gzip compressed input data > * gzip_write.q -> Test write gzip compressed output table > * gzip_splits.q -> Test read splited gzip compressed input data > * over10k.bz2 -> Bzip compressed file of over10k > * gzip_splits/ -> 10 gzip files of 21-22 KB -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-18541) Secure HS2 web UI with PAM
[
https://issues.apache.org/jira/browse/HIVE-18541?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16358810#comment-16358810
]
Hive QA commented on HIVE-18541:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
42s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m
19s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
37s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
27s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
29s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
19s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:red}-1{color} | {color:red} mvninstall {color} | {color:red} 0m
19s{color} | {color:red} service in the patch failed. {color} |
| {color:red}-1{color} | {color:red} compile {color} | {color:red} 0m
20s{color} | {color:red} service in the patch failed. {color} |
| {color:red}-1{color} | {color:red} javac {color} | {color:red} 0m 20s{color}
| {color:red} service in the patch failed. {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
16s{color} | {color:red} common: The patch generated 22 new + 439 unchanged - 0
fixed = 461 total (was 439) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
12s{color} | {color:red} service: The patch generated 9 new + 20 unchanged - 0
fixed = 29 total (was 20) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
1s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
29s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
14s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 12m 37s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc xml compile findbugs
checkstyle |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /data/hiveptest/working/yetus/dev-support/hive-personality.sh |
| git revision | master / 58bbfc7 |
| Default Java | 1.8.0_111 |
| mvninstall |
http://104.198.109.242/logs//PreCommit-HIVE-Build-9120/yetus/patch-mvninstall-service.txt
|
| compile |
http://104.198.109.242/logs//PreCommit-HIVE-Build-9120/yetus/patch-compile-service.txt
|
| javac |
http://104.198.109.242/logs//PreCommit-HIVE-Build-9120/yetus/patch-compile-service.txt
|
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-9120/yetus/diff-checkstyle-common.txt
|
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-9120/yetus/diff-checkstyle-service.txt
|
| modules | C: common service U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-9120/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Secure HS2 web UI with PAM
> --
>
> Key: HIVE-18541
> URL: https://issues.apache.org/jira/browse/HIVE-18541
> Project: Hive
> Issue Type: Sub-task
> Components: HiveServer2
>Reporter: Oleksiy Sayankin
>Assignee: Oleksiy Sayankin
>Priority: Major
> Fix For: 3.0.0
>
> Attachments: HIVE-18541.1.patch, HIVE-18541.2.patch,
> HIVE-18541.5.patch
>
>
> Secure HS2 web UI with PAM. Add two new properties
> * hive.server2.webui.use.pam
> * Default value: false
> * Description: If true, the HiveServer2 WebUI will be secured with PAM
[jira] [Commented] (HIVE-18622) Vectorization: IF Statements, Comparisons, and more do not handle NULLs correctly
[
https://issues.apache.org/jira/browse/HIVE-18622?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16358796#comment-16358796
]
Hive QA commented on HIVE-18622:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12909891/HIVE-18622.06.patch
{color:green}SUCCESS:{color} +1 due to 4 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 38 failed/errored test(s), 13151 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestAccumuloCliDriver.testCliDriver[accumulo_queries]
(batchId=241)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[mapjoin_hook]
(batchId=13)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[ppd_join5] (batchId=36)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[row__id] (batchId=79)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[vector_date_1]
(batchId=21)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[vector_interval_1]
(batchId=16)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[vectorization_nested_udf]
(batchId=66)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[vectorized_case]
(batchId=58)
org.apache.hadoop.hive.cli.TestEncryptedHDFSCliDriver.testCliDriver[encryption_move_tbl]
(batchId=175)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[llap_smb]
(batchId=152)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[bucket_map_join_tez1]
(batchId=172)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[insert_values_orig_table_use_metadata]
(batchId=167)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[llap_acid]
(batchId=171)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[llap_acid_fast]
(batchId=162)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[resourceplan]
(batchId=164)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[sysdb]
(batchId=161)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[vector_interval_1]
(batchId=156)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[vector_ptf_part_simple]
(batchId=160)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[vectorization_nested_udf]
(batchId=167)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[vectorized_case]
(batchId=166)
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver[spark_opt_shuffle_serde]
(batchId=180)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[ppd_join5]
(batchId=122)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[vectorization_nested_udf]
(batchId=136)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[vectorized_case]
(batchId=132)
org.apache.hadoop.hive.cli.TestSparkPerfCliDriver.testCliDriver[query1]
(batchId=251)
org.apache.hadoop.hive.cli.control.TestDanglingQOuts.checkDanglingQOut
(batchId=222)
org.apache.hadoop.hive.metastore.client.TestTablesCreateDropAlterTruncate.testAlterTableNullStorageDescriptorInNew[Embedded]
(batchId=206)
org.apache.hadoop.hive.metastore.client.TestTablesGetExists.testGetAllTablesCaseInsensitive[Embedded]
(batchId=206)
org.apache.hadoop.hive.metastore.client.TestTablesList.testListTableNamesByFilterNullDatabase[Embedded]
(batchId=206)
org.apache.hadoop.hive.ql.exec.vector.expressions.TestVectorConditionalExpressions.testDoubleColumnColumnIfExpr
(batchId=284)
org.apache.hadoop.hive.ql.exec.vector.expressions.TestVectorConditionalExpressions.testIfExprStringColumnStringScalar
(batchId=284)
org.apache.hadoop.hive.ql.exec.vector.expressions.TestVectorConditionalExpressions.testIfExprStringScalarStringColumn
(batchId=284)
org.apache.hadoop.hive.ql.exec.vector.expressions.TestVectorConditionalExpressions.testLongColumnColumnIfExpr
(batchId=284)
org.apache.hadoop.hive.ql.io.TestDruidRecordWriter.testWrite (batchId=257)
org.apache.hive.beeline.cli.TestHiveCli.testNoErrorDB (batchId=188)
org.apache.hive.jdbc.TestSSL.testConnectionMismatch (batchId=235)
org.apache.hive.jdbc.TestSSL.testConnectionWrongCertCN (batchId=235)
org.apache.hive.jdbc.TestSSL.testMetastoreConnectionWrongCertCN (batchId=235)
{noformat}
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/9119/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/9119/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-9119/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 38 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12909891 - PreCommit-HIVE-Build
> Vectorization: IF Statements,
[jira] [Assigned] (HIVE-18671) lock not released after Hive on Spark query was cancelled
[
https://issues.apache.org/jira/browse/HIVE-18671?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yongzhi Chen reassigned HIVE-18671:
---
> lock not released after Hive on Spark query was cancelled
> -
>
> Key: HIVE-18671
> URL: https://issues.apache.org/jira/browse/HIVE-18671
> Project: Hive
> Issue Type: Bug
>Affects Versions: 2.3.2
>Reporter: Yongzhi Chen
>Assignee: Yongzhi Chen
>Priority: Major
>
> When cancel the query is running on spark, the SparkJobMonitor can not
> return, therefore the locks hold by the query can not be released. When
> enable debug in log, you will see many log info as following:
> {noformat}
> 2018-02-09 08:27:09,613 INFO
> org.apache.hadoop.hive.ql.exec.spark.status.SparkJobMonitor:
> [HiveServer2-Background-Pool: Thread-80]: state = CANCELLED
> 2018-02-09 08:27:10,613 INFO
> org.apache.hadoop.hive.ql.exec.spark.status.SparkJobMonitor:
> [HiveServer2-Background-Pool: Thread-80]: state = CANCELLED
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-16125) Split work between reducers.
[ https://issues.apache.org/jira/browse/HIVE-16125?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] slim bouguerra updated HIVE-16125: -- Attachment: HIVE-16125.4.patch > Split work between reducers. > > > Key: HIVE-16125 > URL: https://issues.apache.org/jira/browse/HIVE-16125 > Project: Hive > Issue Type: Bug > Components: Druid integration >Reporter: slim bouguerra >Assignee: slim bouguerra >Priority: Major > Attachments: HIVE-16125.4.patch, HIVE-16125.patch > > > Split work between reducer. > currently we have one reducer per segment granularity even if the interval > will be partitioned over multiple partitions. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-18622) Vectorization: IF Statements, Comparisons, and more do not handle NULLs correctly
[
https://issues.apache.org/jira/browse/HIVE-18622?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16358754#comment-16358754
]
Hive QA commented on HIVE-18622:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
20s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m
53s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
19s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
51s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
15s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
24s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
42s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
24s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
24s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
12s{color} | {color:red} storage-api: The patch generated 1 new + 116 unchanged
- 12 fixed = 117 total (was 128) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
13s{color} | {color:red} vector-code-gen: The patch generated 2 new + 308
unchanged - 0 fixed = 310 total (was 308) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 1m
32s{color} | {color:red} ql: The patch generated 125 new + 4298 unchanged - 35
fixed = 4423 total (was 4333) {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
0s{color} | {color:red} The patch has 4 line(s) that end in whitespace. Use git
apply --whitespace=fix <>. Refer https://git-scm.com/docs/git-apply
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
13s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
13s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 19m 52s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /data/hiveptest/working/yetus/dev-support/hive-personality.sh |
| git revision | master / 58bbfc7 |
| Default Java | 1.8.0_111 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-9119/yetus/diff-checkstyle-storage-api.txt
|
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-9119/yetus/diff-checkstyle-vector-code-gen.txt
|
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-9119/yetus/diff-checkstyle-ql.txt
|
| whitespace |
http://104.198.109.242/logs//PreCommit-HIVE-Build-9119/yetus/whitespace-eol.txt
|
| modules | C: storage-api vector-code-gen ql U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-9119/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Vectorization: IF Statements, Comparisons, and more do not handle NULLs
> correctly
> -
>
> Key: HIVE-18622
> URL: https://issues.apache.org/jira/browse/HIVE-18622
> Project: Hive
> Issue Type: Bug
> Components: Hive
>Reporter: Matt McCline
>Assignee: Matt McCline
>Priority: Critical
> Fix For: 3.0.0
>
> Attachments: HIVE-18622.03.patch, HIVE-18622.04.patch,
> HIVE-18622.05.patch, HIVE-18622.06.patch
>
>
>
> Many vector
[jira] [Commented] (HIVE-18238) Driver execution may not have configuration changing sideeffects
[
https://issues.apache.org/jira/browse/HIVE-18238?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16358720#comment-16358720
]
Hive QA commented on HIVE-18238:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12909822/HIVE-18238.10.patch
{color:green}SUCCESS:{color} +1 due to 11 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 23 failed/errored test(s), 13152 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestAccumuloCliDriver.testCliDriver[accumulo_queries]
(batchId=241)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[alter_table_partition_drop]
(batchId=15)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[mapjoin_hook]
(batchId=13)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[ppd_join5] (batchId=36)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[row__id] (batchId=79)
org.apache.hadoop.hive.cli.TestEncryptedHDFSCliDriver.testCliDriver[encryption_move_tbl]
(batchId=175)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[llap_smb]
(batchId=152)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[bucket_map_join_tez1]
(batchId=172)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[insert_values_orig_table_use_metadata]
(batchId=167)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[llap_acid]
(batchId=171)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[llap_acid_fast]
(batchId=162)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[resourceplan]
(batchId=164)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[results_cache_1]
(batchId=168)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[sysdb]
(batchId=161)
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver[spark_opt_shuffle_serde]
(batchId=180)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[ppd_join5]
(batchId=122)
org.apache.hadoop.hive.cli.TestSparkPerfCliDriver.testCliDriver[query1]
(batchId=251)
org.apache.hadoop.hive.cli.control.TestDanglingQOuts.checkDanglingQOut
(batchId=222)
org.apache.hadoop.hive.ql.io.TestDruidRecordWriter.testWrite (batchId=257)
org.apache.hive.beeline.cli.TestHiveCli.testNoErrorDB (batchId=188)
org.apache.hive.jdbc.TestSSL.testConnectionMismatch (batchId=235)
org.apache.hive.jdbc.TestSSL.testConnectionWrongCertCN (batchId=235)
org.apache.hive.jdbc.TestSSL.testMetastoreConnectionWrongCertCN (batchId=235)
{noformat}
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/9118/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/9118/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-9118/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 23 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12909822 - PreCommit-HIVE-Build
> Driver execution may not have configuration changing sideeffects
> -
>
> Key: HIVE-18238
> URL: https://issues.apache.org/jira/browse/HIVE-18238
> Project: Hive
> Issue Type: Sub-task
> Components: Logical Optimizer
>Reporter: Zoltan Haindrich
>Assignee: Zoltan Haindrich
>Priority: Major
> Attachments: HIVE-18238.01wip01.patch, HIVE-18238.02.patch,
> HIVE-18238.03.patch, HIVE-18238.04.patch, HIVE-18238.04wip01.patch,
> HIVE-18238.07.patch, HIVE-18238.08.patch, HIVE-18238.09.patch,
> HIVE-18238.10.patch
>
>
> {{Driver}} executes sql statements which use "hiveconf" settings;
> but the {{Driver}} itself may *not* change the configuration...
> I've found an example; which shows how hazardous this is...
> {code}
> set hive.mapred.mode=strict;
> select "${hiveconf:hive.mapred.mode}";
> create table t (a int);
> analyze table t compute statistics;
> select "${hiveconf:hive.mapred.mode}";
> {code}
> currently; the last select returns {{nonstrict}} because of
> [this|https://github.com/apache/hive/blob/7ddd915bf82a68c8ab73b0c4ca409f1a6d43d227/ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java#L1696]
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-18421) Vectorized execution handles overflows in a different manner than non-vectorized execution
[
https://issues.apache.org/jira/browse/HIVE-18421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16358701#comment-16358701
]
Aihua Xu commented on HIVE-18421:
-
Originally I thought non-vectorization version was handling the overflow but
since both are returning the incorrect results, I'm fine with current patch. I
will leave to [~mmccline] and [~gopalv] for a second review since you are more
familiar on this part.
> Vectorized execution handles overflows in a different manner than
> non-vectorized execution
> --
>
> Key: HIVE-18421
> URL: https://issues.apache.org/jira/browse/HIVE-18421
> Project: Hive
> Issue Type: Bug
> Components: Vectorization
>Affects Versions: 2.1.1, 2.2.0, 3.0.0, 2.3.2
>Reporter: Vihang Karajgaonkar
>Assignee: Vihang Karajgaonkar
>Priority: Major
> Attachments: HIVE-18421.01.patch, HIVE-18421.02.patch,
> HIVE-18421.03.patch, HIVE-18421.04.patch, HIVE-18421.05.patch,
> HIVE-18421.06.patch, HIVE-18421.07.patch
>
>
> In vectorized execution arithmetic operations which cause integer overflows
> can give wrong results. Issue is reproducible in both Orc and parquet.
> Simple test case to reproduce this issue
> {noformat}
> set hive.vectorized.execution.enabled=true;
> create table parquettable (t1 tinyint, t2 tinyint) stored as parquet;
> insert into parquettable values (-104, 25), (-112, 24), (54, 9);
> select t1, t2, (t1-t2) as diff from parquettable where (t1-t2) < 50 order by
> diff desc;
> +---+-+---+
> | t1 | t2 | diff |
> +---+-+---+
> | -104 | 25 | 127 |
> | -112 | 24 | 120 |
> | 54| 9 | 45|
> +---+-+---+
> {noformat}
> When vectorization is turned off the same query produces only one row.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-18670) Prevent DROP DATABASE If Other Data Exists
[
https://issues.apache.org/jira/browse/HIVE-18670?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
BELUGA BEHR updated HIVE-18670:
---
Description:
A user is not able to drop a database if it has tables under it unless they
include the _CASCADE_ keyword to their DROP DATABASE statement.
[https://cwiki.apache.org/confluence/display/Hive/LanguageManual+DDL#LanguageManualDDL-DropTable]
I would like to propose that, if {{hive.mapred.mode}} is set to 'strict', Hive
also checks for other data before dropping the database.
For example, if the database is stored within HDFS, then Hive should check if
there exists other data, not even necessarily related to Hive, within the
database's HDFS directory, before dropping.
The examples are:
{code:java|title=Example 1}
/hive/my_database
/hive/my_database/my_table
-- Does not succeed because 'my_table' exists
> DROP DATABASE my_database;
-- Succeeds and removes the root directory /hive/my_database
> DROP DATABASE my_database CASCADE;
{code}
{code:java|title=Example 2}
/hive/my_database
/hive/my_database/my_important_file.txt
-- Succeeds because no tables exist, but I just lost my "important" file
> DROP DATABASE my_database;
{code}
This "feature" is just to prevent people from shooting themselves in the foot,
even if they shouldn't be using Hive space for storing unrelated data.
was:
A user is not able to drop a database if it has tables under it unless they
include the _CASCADE_ keyword to their DROP DATABASE statement.
[https://cwiki.apache.org/confluence/display/Hive/LanguageManual+DDL#LanguageManualDDL-DropTable]
I would like to propose that, if {{hive.mapred.mode}} is set to 'strict', Hive
also checks for other data before dropping the database.
For example, if the database is stored within HDFS, then Hive should check if
there exists other data, not even necessarily related to Hive, within the
database's HDFS directory, before dropping.
The examples are:
{code:java|title=Example 1}
/hive/my_database
/hive/my_database/my_table
-- Does not succeed because 'my_table' exists
> DROP DATABASE my_database;
-- Succeeds and removes the root directory /hive/my_database
> DROP DATABASE my_database CASCADE;
{code}
{code:java|title=Example 2}
/hive/my_database
/hive/my_database/my_important_file.txt
-- Succeeds because no tables exist, but I just lost my "important" file
> DROP DATABASE my_database;
{code}
> Prevent DROP DATABASE If Other Data Exists
> --
>
> Key: HIVE-18670
> URL: https://issues.apache.org/jira/browse/HIVE-18670
> Project: Hive
> Issue Type: Improvement
> Components: HiveServer2
>Affects Versions: 3.0.0, 2.3.2
>Reporter: BELUGA BEHR
>Priority: Major
>
> A user is not able to drop a database if it has tables under it unless they
> include the _CASCADE_ keyword to their DROP DATABASE statement.
> [https://cwiki.apache.org/confluence/display/Hive/LanguageManual+DDL#LanguageManualDDL-DropTable]
> I would like to propose that, if {{hive.mapred.mode}} is set to 'strict',
> Hive also checks for other data before dropping the database.
> For example, if the database is stored within HDFS, then Hive should check if
> there exists other data, not even necessarily related to Hive, within the
> database's HDFS directory, before dropping.
> The examples are:
> {code:java|title=Example 1}
> /hive/my_database
> /hive/my_database/my_table
> -- Does not succeed because 'my_table' exists
> > DROP DATABASE my_database;
> -- Succeeds and removes the root directory /hive/my_database
> > DROP DATABASE my_database CASCADE;
> {code}
> {code:java|title=Example 2}
> /hive/my_database
> /hive/my_database/my_important_file.txt
> -- Succeeds because no tables exist, but I just lost my "important" file
> > DROP DATABASE my_database;
> {code}
> This "feature" is just to prevent people from shooting themselves in the
> foot, even if they shouldn't be using Hive space for storing unrelated data.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-18670) Prevent DROP DATABASE If Other Data Exists
[
https://issues.apache.org/jira/browse/HIVE-18670?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
BELUGA BEHR updated HIVE-18670:
---
Summary: Prevent DROP DATABASE If Other Data Exists (was: Prevent DROP
DATABASE If Data Exists in HDFS)
> Prevent DROP DATABASE If Other Data Exists
> --
>
> Key: HIVE-18670
> URL: https://issues.apache.org/jira/browse/HIVE-18670
> Project: Hive
> Issue Type: Improvement
> Components: HiveServer2
>Affects Versions: 3.0.0, 2.3.2
>Reporter: BELUGA BEHR
>Priority: Major
>
> A user is not able to drop a database if it has tables under it unless they
> include the _CASCADE_ keyword to their DROP DATABASE statement.
> [https://cwiki.apache.org/confluence/display/Hive/LanguageManual+DDL#LanguageManualDDL-DropTable]
> I would like to propose that, if {{hive.mapred.mode}} is set to 'strict',
> Hive also checks for other data before dropping the database.
> For example, if the database is stored within HDFS, then Hive should check if
> there exists other data, not even necessarily related to Hive, within the
> database's HDFS directory, before dropping.
> The examples are:
> {code:java|title=Example 1}
> /hive/my_database
> /hive/my_database/my_table
> -- Does not succeed because 'my_table' exists
> > DROP DATABASE my_database;
> -- Succeeds and removes the root directory /hive/my_database
> > DROP DATABASE my_database CASCADE;
> {code}
> {code:java|title=Example 2}
> /hive/my_database
> /hive/my_database/my_important_file.txt
> -- Succeeds because no tables exist, but I just lost my "important" file
> > DROP DATABASE my_database;
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
