[jira] [Commented] (HIVE-19064) Add mode to support delimited identifiers enclosed within double quotation
[
https://issues.apache.org/jira/browse/HIVE-19064?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17096157#comment-17096157
]
Hive QA commented on HIVE-19064:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
59s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
27s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 3m
11s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
55s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 2m
33s{color} | {color:blue} standalone-metastore/metastore-common in master has
35 extant Findbugs warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
35s{color} | {color:blue} common in master has 63 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
53s{color} | {color:blue} parser in master has 3 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 1m
13s{color} | {color:blue} standalone-metastore/metastore-server in master has
187 extant Findbugs warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
43s{color} | {color:blue} ql in master has 1528 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
49s{color} | {color:blue} itests/util in master has 53 extant Findbugs
warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 3m
7s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
29s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
48s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 3m
13s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 3m
13s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
12s{color} | {color:green} standalone-metastore/metastore-common: The patch
generated 0 new + 109 unchanged - 2 fixed = 109 total (was 111) {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
16s{color} | {color:green} common: The patch generated 0 new + 377 unchanged -
1 fixed = 377 total (was 378) {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
9s{color} | {color:green} The patch parser passed checkstyle {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
23s{color} | {color:green} The patch metastore-server passed checkstyle {color}
|
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
45s{color} | {color:red} ql: The patch generated 4 new + 419 unchanged - 10
fixed = 423 total (was 429) {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
13s{color} | {color:green} The patch util passed checkstyle {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
1s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
43s{color} | {color:green} metastore-common in the patch passed. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
40s{color} | {color:green} common in the patch passed. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
12s{color} | {color:green} parser in the patch passed. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
24s{color} | {color:green} metastore-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
47s{color} | {color:green} ql generated 0
[jira] [Commented] (HIVE-23252) Change spark related tests to be optional
[
https://issues.apache.org/jira/browse/HIVE-23252?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17096128#comment-17096128
]
Hive QA commented on HIVE-23252:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/13001582/HIVE-23252.01.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 2 failed/errored test(s), 17175 tests
executed
*Failed tests:*
{noformat}
TestStatsReplicationScenariosACID - did not produce a TEST-*.xml file (likely
timed out) (batchId=184)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[schq_ingest]
(batchId=102)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/22025/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/22025/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-22025/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 2 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 13001582 - PreCommit-HIVE-Build
> Change spark related tests to be optional
> -
>
> Key: HIVE-23252
> URL: https://issues.apache.org/jira/browse/HIVE-23252
> Project: Hive
> Issue Type: Sub-task
> Components: Test
>Reporter: Zoltan Haindrich
>Assignee: Zoltan Haindrich
>Priority: Major
> Attachments: HIVE-23252.01.patch, HIVE-23252.01.patch,
> HIVE-23252.01.patch, HIVE-23252.01.patch, HIVE-23252.01.patch,
> HIVE-23252.01.patch, HIVE-23252.01.patch
>
>
> HIVE-23137 have disabled the execution of some spark related tests; but they
> would be still considered by a plain maven command - and the spark artifacts
> are (unneccessarily) still downloaded
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-23252) Change spark related tests to be optional
[
https://issues.apache.org/jira/browse/HIVE-23252?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17096125#comment-17096125
]
Hive QA commented on HIVE-23252:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 2m
2s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
4s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 7m
31s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 7m
35s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
27s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
21s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 7m
13s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 7m
13s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
3s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 7m
7s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
13s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 49m 1s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc xml compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-22025/dev-support/hive-personality.sh
|
| git revision | master / 9c12a4c |
| Default Java | 1.8.0_111 |
| modules | C: . itests itests/hive-unit U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-22025/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Change spark related tests to be optional
> -
>
> Key: HIVE-23252
> URL: https://issues.apache.org/jira/browse/HIVE-23252
> Project: Hive
> Issue Type: Sub-task
> Components: Test
>Reporter: Zoltan Haindrich
>Assignee: Zoltan Haindrich
>Priority: Major
> Attachments: HIVE-23252.01.patch, HIVE-23252.01.patch,
> HIVE-23252.01.patch, HIVE-23252.01.patch, HIVE-23252.01.patch,
> HIVE-23252.01.patch, HIVE-23252.01.patch
>
>
> HIVE-23137 have disabled the execution of some spark related tests; but they
> would be still considered by a plain maven command - and the spark artifacts
> are (unneccessarily) still downloaded
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-23322) Update some tests to better tolerate a more busy environment
[
https://issues.apache.org/jira/browse/HIVE-23322?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17096088#comment-17096088
]
Hive QA commented on HIVE-23322:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/13001581/HIVE-23322.01.patch
{color:green}SUCCESS:{color} +1 due to 3 test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 17179 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/22024/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/22024/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-22024/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 13001581 - PreCommit-HIVE-Build
> Update some tests to better tolerate a more busy environment
>
>
> Key: HIVE-23322
> URL: https://issues.apache.org/jira/browse/HIVE-23322
> Project: Hive
> Issue Type: Sub-task
>Reporter: Zoltan Haindrich
>Assignee: Zoltan Haindrich
>Priority: Major
> Attachments: HIVE-23322.01.patch
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-23322) Update some tests to better tolerate a more busy environment
[
https://issues.apache.org/jira/browse/HIVE-23322?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17096058#comment-17096058
]
Hive QA commented on HIVE-23322:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 2m
18s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
5s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
34s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
2s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
33s{color} | {color:blue} common in master has 63 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
24s{color} | {color:blue} llap-tez in master has 17 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
38s{color} | {color:blue} ql in master has 1528 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
21s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
28s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
59s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
33s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
33s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
12s{color} | {color:green} The patch common passed checkstyle {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
10s{color} | {color:green} The patch llap-tez passed checkstyle {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
38s{color} | {color:green} ql: The patch generated 0 new + 92 unchanged - 2
fixed = 92 total (was 94) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 5m
2s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
18s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
14s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 31m 21s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-22024/dev-support/hive-personality.sh
|
| git revision | master / 9c12a4c |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.1 |
| modules | C: common llap-tez ql U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-22024/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Update some tests to better tolerate a more busy environment
>
>
> Key: HIVE-23322
> URL: https://issues.apache.org/jira/browse/HIVE-23322
> Project: Hive
> Issue Type: Sub-task
>Reporter: Zoltan Haindrich
>Assignee: Zoltan Haindrich
>Priority: Major
> Attachments: HIVE-23322.01.patch
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-23320) Running masking_mv.q breaks strict_pruning.q
[
https://issues.apache.org/jira/browse/HIVE-23320?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17096030#comment-17096030
]
Hive QA commented on HIVE-23320:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/13001580/HIVE-23320.01.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 2 failed/errored test(s), 17179 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[strict_pruning_2]
(batchId=45)
org.apache.hive.service.server.TestInformationSchemaWithPrivilegeZookeeperSSL.org.apache.hive.service.server.TestInformationSchemaWithPrivilegeZookeeperSSL
(batchId=211)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/22023/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/22023/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-22023/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 2 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 13001580 - PreCommit-HIVE-Build
> Running masking_mv.q breaks strict_pruning.q
>
>
> Key: HIVE-23320
> URL: https://issues.apache.org/jira/browse/HIVE-23320
> Project: Hive
> Issue Type: Sub-task
>Reporter: Zoltan Haindrich
>Assignee: Zoltan Haindrich
>Priority: Major
> Attachments: HIVE-23320.01.patch
>
>
> {code}
> mvn install -pl itests/qtest -Dtest=TestNegativeCliDriver
> -Dqfile=masking_mv.q,strict_pruning.q
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-10161) LLAP: ORC file contains compression buffers larger than bufferSize (OR reader has a bug)
[
https://issues.apache.org/jira/browse/HIVE-10161?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17096017#comment-17096017
]
hbprotoss commented on HIVE-10161:
--
[~harishk15] Have you resolved this issue?
> LLAP: ORC file contains compression buffers larger than bufferSize (OR reader
> has a bug)
>
>
> Key: HIVE-10161
> URL: https://issues.apache.org/jira/browse/HIVE-10161
> Project: Hive
> Issue Type: Sub-task
>Affects Versions: llap
>Reporter: Gopal Vijayaraghavan
>Assignee: Sergey Shelukhin
>Priority: Major
> Fix For: llap
>
>
> The EncodedReaderImpl will die when reading from the cache, when reading data
> written by the regular ORC writer
> {code}
> Caused by: java.io.IOException: java.lang.IllegalArgumentException: Buffer
> size too small. size = 262144 needed = 3919246
> at
> org.apache.hadoop.hive.llap.io.api.impl.LlapInputFormat$LlapRecordReader.rethrowErrorIfAny(LlapInputFormat.java:249)
> at
> org.apache.hadoop.hive.llap.io.api.impl.LlapInputFormat$LlapRecordReader.nextCvb(LlapInputFormat.java:201)
> at
> org.apache.hadoop.hive.llap.io.api.impl.LlapInputFormat$LlapRecordReader.next(LlapInputFormat.java:140)
> at
> org.apache.hadoop.hive.llap.io.api.impl.LlapInputFormat$LlapRecordReader.next(LlapInputFormat.java:96)
> at
> org.apache.hadoop.hive.ql.io.HiveContextAwareRecordReader.doNext(HiveContextAwareRecordReader.java:350)
> ... 22 more
> Caused by: java.lang.IllegalArgumentException: Buffer size too small. size =
> 262144 needed = 3919246
> at
> org.apache.hadoop.hive.ql.io.orc.InStream.addOneCompressionBuffer(InStream.java:780)
> at
> org.apache.hadoop.hive.ql.io.orc.InStream.uncompressStream(InStream.java:628)
> at
> org.apache.hadoop.hive.ql.io.orc.EncodedReaderImpl.readEncodedColumns(EncodedReaderImpl.java:309)
> at
> org.apache.hadoop.hive.llap.io.encoded.OrcEncodedDataReader.callInternal(OrcEncodedDataReader.java:278)
> at
> org.apache.hadoop.hive.llap.io.encoded.OrcEncodedDataReader.callInternal(OrcEncodedDataReader.java:48)
> at
> org.apache.hadoop.hive.common.CallableWithNdc.call(CallableWithNdc.java:37)
> ... 4 more
> ]], Vertex failed as one or more tasks failed. failedTasks:1, Vertex
> vertex_1424502260528_1945_1_00 [Map 1] killed/failed due to:null]
> {code}
> Turning off hive.llap.io.enabled makes the error go away.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-23320) Running masking_mv.q breaks strict_pruning.q
[
https://issues.apache.org/jira/browse/HIVE-23320?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17095992#comment-17095992
]
Hive QA commented on HIVE-23320:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
53s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 1m 37s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-22023/dev-support/hive-personality.sh
|
| git revision | master / 9c12a4c |
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-22023/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Running masking_mv.q breaks strict_pruning.q
>
>
> Key: HIVE-23320
> URL: https://issues.apache.org/jira/browse/HIVE-23320
> Project: Hive
> Issue Type: Sub-task
>Reporter: Zoltan Haindrich
>Assignee: Zoltan Haindrich
>Priority: Major
> Attachments: HIVE-23320.01.patch
>
>
> {code}
> mvn install -pl itests/qtest -Dtest=TestNegativeCliDriver
> -Dqfile=masking_mv.q,strict_pruning.q
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-23327) Investigate why the file size has changed at acid_stats2.q
[ https://issues.apache.org/jira/browse/HIVE-23327?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17095991#comment-17095991 ] Jesus Camacho Rodriguez commented on HIVE-23327: I think this could have happened if the order of the rows while creating the file has changed. > Investigate why the file size has changed at acid_stats2.q > -- > > Key: HIVE-23327 > URL: https://issues.apache.org/jira/browse/HIVE-23327 > Project: Hive > Issue Type: Sub-task >Reporter: Miklos Gergely >Priority: Major > > The file size was modified at acid_stats2.q, find out why. See > [https://reviews.apache.org/r/72421/#comment308835] -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-23326) Investigate minReductionHashAggr modifications
[ https://issues.apache.org/jira/browse/HIVE-23326?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17095988#comment-17095988 ] Jesus Camacho Rodriguez commented on HIVE-23326: This is expected. The optimization is not available for MR and default value was 0.99, that is why you were finding that value in the q.out files before. You can close this as not an issue. > Investigate minReductionHashAggr modifications > -- > > Key: HIVE-23326 > URL: https://issues.apache.org/jira/browse/HIVE-23326 > Project: Hive > Issue Type: Sub-task >Reporter: Miklos Gergely >Priority: Major > > At some q tests the minReductionHashAggr was modified in the output after > moving to TestMiniLlapLocalCliDriver, for example acid_nullscan.q, > correlationoptimizer9.q. Check > [https://reviews.apache.org/r/72421/#comment308835] for details. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-23321) Tolerate in tests that metastore is not removing rows from the skewed_string_list_values table
[
https://issues.apache.org/jira/browse/HIVE-23321?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17095987#comment-17095987
]
Hive QA commented on HIVE-23321:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/13001574/HIVE-23321.01.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 3 failed/errored test(s), 17175 tests
executed
*Failed tests:*
{noformat}
TestStatsReplicationScenariosACID - did not produce a TEST-*.xml file (likely
timed out) (batchId=184)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[infer_bucket_sort_reducers_power_two]
(batchId=7)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[multi_insert_partitioned]
(batchId=98)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/22022/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/22022/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-22022/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 3 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 13001574 - PreCommit-HIVE-Build
> Tolerate in tests that metastore is not removing rows from the
> skewed_string_list_values table
> --
>
> Key: HIVE-23321
> URL: https://issues.apache.org/jira/browse/HIVE-23321
> Project: Hive
> Issue Type: Sub-task
>Reporter: Zoltan Haindrich
>Assignee: Zoltan Haindrich
>Priority: Major
> Attachments: HIVE-23321.01.patch
>
>
> I've found this issue; and filed: HIVE-23289 - however fixing it is not easy
> at all..
> * foreign keys were not used
> * there were 0 effort to remove these rows before...I don't think when the
> feature was added the removal was considered...
> * unfortunately designed table hierarchy makes it hard to work with it...
> * unclear contracts (at least to me) between skewed_values and sd stuff
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-23321) Tolerate in tests that metastore is not removing rows from the skewed_string_list_values table
[
https://issues.apache.org/jira/browse/HIVE-23321?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17095976#comment-17095976
]
Hive QA commented on HIVE-23321:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
55s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 1m 41s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-22022/dev-support/hive-personality.sh
|
| git revision | master / 9c12a4c |
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-22022/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Tolerate in tests that metastore is not removing rows from the
> skewed_string_list_values table
> --
>
> Key: HIVE-23321
> URL: https://issues.apache.org/jira/browse/HIVE-23321
> Project: Hive
> Issue Type: Sub-task
>Reporter: Zoltan Haindrich
>Assignee: Zoltan Haindrich
>Priority: Major
> Attachments: HIVE-23321.01.patch
>
>
> I've found this issue; and filed: HIVE-23289 - however fixing it is not easy
> at all..
> * foreign keys were not used
> * there were 0 effort to remove these rows before...I don't think when the
> feature was added the removal was considered...
> * unfortunately designed table hierarchy makes it hard to work with it...
> * unclear contracts (at least to me) between skewed_values and sd stuff
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-23284) Remove dependency on mariadb-java-client
[
https://issues.apache.org/jira/browse/HIVE-23284?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17095974#comment-17095974
]
Hive QA commented on HIVE-23284:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/13001564/HIVE-23284.03.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 17179 tests
executed
*Failed tests:*
{noformat}
org.apache.hive.jdbc.TestJdbcWithMiniLlapArrow.testComplexQuery (batchId=214)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/22021/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/22021/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-22021/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 1 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 13001564 - PreCommit-HIVE-Build
> Remove dependency on mariadb-java-client
>
>
> Key: HIVE-23284
> URL: https://issues.apache.org/jira/browse/HIVE-23284
> Project: Hive
> Issue Type: Improvement
>Reporter: Karen Coppage
>Assignee: Karen Coppage
>Priority: Major
> Attachments: HIVE-23284.01.patch, HIVE-23284.01.patch,
> HIVE-23284.01.patch, HIVE-23284.02.patch, HIVE-23284.03.patch,
> HIVE-23284.03.patch, HIVE-23284.03.patch
>
>
> It has GNU Lesser General Public License which is [Category
> X|https://www.apache.org/legal/resolved.html#category-x].
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-23215) Make FilterContext and MutableFilterContext interfaces
[ https://issues.apache.org/jira/browse/HIVE-23215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17095971#comment-17095971 ] Panagiotis Garefalakis commented on HIVE-23215: --- Food catch [~jcamachorodriguez] ! Updated > Make FilterContext and MutableFilterContext interfaces > -- > > Key: HIVE-23215 > URL: https://issues.apache.org/jira/browse/HIVE-23215 > Project: Hive > Issue Type: Bug > Components: storage-api >Reporter: Owen O'Malley >Assignee: Owen O'Malley >Priority: Major > Labels: pull-request-available > Attachments: HIVE-23215.01.patch, HIVE-23215.patch > > Time Spent: 1h 20m > Remaining Estimate: 0h > > HIVE-22959 introduced FilterContext to support ORC-577. The duplication of > fields between the FilterContext and VectorizedRowBatch seems likely to cause > user confusion. This patch makes them interfaces that VectorizedRowBatch > implements. > Thus, there is a single copy of the data and no need to copy them back and > forth. LLAP can make its own implementation of the interfaces if it doesn't > want to use VectorizedRowBatch. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-23215) Make FilterContext and MutableFilterContext interfaces
[ https://issues.apache.org/jira/browse/HIVE-23215?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Panagiotis Garefalakis updated HIVE-23215: -- Attachment: (was: HIVE-23215.01.patch) > Make FilterContext and MutableFilterContext interfaces > -- > > Key: HIVE-23215 > URL: https://issues.apache.org/jira/browse/HIVE-23215 > Project: Hive > Issue Type: Bug > Components: storage-api >Reporter: Owen O'Malley >Assignee: Panagiotis Garefalakis >Priority: Major > Labels: pull-request-available > Attachments: HIVE-23215.01.patch, HIVE-23215.patch > > Time Spent: 1h 20m > Remaining Estimate: 0h > > HIVE-22959 introduced FilterContext to support ORC-577. The duplication of > fields between the FilterContext and VectorizedRowBatch seems likely to cause > user confusion. This patch makes them interfaces that VectorizedRowBatch > implements. > Thus, there is a single copy of the data and no need to copy them back and > forth. LLAP can make its own implementation of the interfaces if it doesn't > want to use VectorizedRowBatch. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Assigned] (HIVE-23215) Make FilterContext and MutableFilterContext interfaces
[ https://issues.apache.org/jira/browse/HIVE-23215?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Panagiotis Garefalakis reassigned HIVE-23215: - Assignee: Owen O'Malley (was: Panagiotis Garefalakis) > Make FilterContext and MutableFilterContext interfaces > -- > > Key: HIVE-23215 > URL: https://issues.apache.org/jira/browse/HIVE-23215 > Project: Hive > Issue Type: Bug > Components: storage-api >Reporter: Owen O'Malley >Assignee: Owen O'Malley >Priority: Major > Labels: pull-request-available > Attachments: HIVE-23215.01.patch, HIVE-23215.patch > > Time Spent: 1h 20m > Remaining Estimate: 0h > > HIVE-22959 introduced FilterContext to support ORC-577. The duplication of > fields between the FilterContext and VectorizedRowBatch seems likely to cause > user confusion. This patch makes them interfaces that VectorizedRowBatch > implements. > Thus, there is a single copy of the data and no need to copy them back and > forth. LLAP can make its own implementation of the interfaces if it doesn't > want to use VectorizedRowBatch. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Assigned] (HIVE-23215) Make FilterContext and MutableFilterContext interfaces
[ https://issues.apache.org/jira/browse/HIVE-23215?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Panagiotis Garefalakis reassigned HIVE-23215: - Assignee: Panagiotis Garefalakis (was: Owen O'Malley) > Make FilterContext and MutableFilterContext interfaces > -- > > Key: HIVE-23215 > URL: https://issues.apache.org/jira/browse/HIVE-23215 > Project: Hive > Issue Type: Bug > Components: storage-api >Reporter: Owen O'Malley >Assignee: Panagiotis Garefalakis >Priority: Major > Labels: pull-request-available > Attachments: HIVE-23215.01.patch, HIVE-23215.patch > > Time Spent: 1h 20m > Remaining Estimate: 0h > > HIVE-22959 introduced FilterContext to support ORC-577. The duplication of > fields between the FilterContext and VectorizedRowBatch seems likely to cause > user confusion. This patch makes them interfaces that VectorizedRowBatch > implements. > Thus, there is a single copy of the data and no need to copy them back and > forth. LLAP can make its own implementation of the interfaces if it doesn't > want to use VectorizedRowBatch. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-23215) Make FilterContext and MutableFilterContext interfaces
[ https://issues.apache.org/jira/browse/HIVE-23215?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Panagiotis Garefalakis updated HIVE-23215: -- Attachment: HIVE-23215.01.patch > Make FilterContext and MutableFilterContext interfaces > -- > > Key: HIVE-23215 > URL: https://issues.apache.org/jira/browse/HIVE-23215 > Project: Hive > Issue Type: Bug > Components: storage-api >Reporter: Owen O'Malley >Assignee: Panagiotis Garefalakis >Priority: Major > Labels: pull-request-available > Attachments: HIVE-23215.01.patch, HIVE-23215.patch > > Time Spent: 1h 20m > Remaining Estimate: 0h > > HIVE-22959 introduced FilterContext to support ORC-577. The duplication of > fields between the FilterContext and VectorizedRowBatch seems likely to cause > user confusion. This patch makes them interfaces that VectorizedRowBatch > implements. > Thus, there is a single copy of the data and no need to copy them back and > forth. LLAP can make its own implementation of the interfaces if it doesn't > want to use VectorizedRowBatch. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-23301) Optimising privilege synchroniser: UDF for updating privileges
[ https://issues.apache.org/jira/browse/HIVE-23301?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Simhadri G updated HIVE-23301: -- Attachment: hive-23301.4.patch Status: Patch Available (was: In Progress) > Optimising privilege synchroniser: UDF for updating privileges > -- > > Key: HIVE-23301 > URL: https://issues.apache.org/jira/browse/HIVE-23301 > Project: Hive > Issue Type: Improvement > Components: Metastore, UDF >Affects Versions: 3.1.1 >Reporter: Simhadri G >Assignee: Simhadri G >Priority: Minor > Attachments: UDFSplitMapPrivs.patch, UDFSplitMapPrivs.patch, > hive-23301-2.patch, hive-23301.3.patch, hive-23301.4.patch > > > Privilege synchronizer pulls the list of databases, tables and columns from > the Hive Metastore. For each of these objects it fetches the privilege > information and invokes HMS API to refresh the privilege information in HMS. > The current UDF Maps a bit string to a privilege based on if the privilege > is granted or not. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-23301) Optimising privilege synchroniser: UDF for updating privileges
[ https://issues.apache.org/jira/browse/HIVE-23301?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Simhadri G updated HIVE-23301: -- Status: In Progress (was: Patch Available) > Optimising privilege synchroniser: UDF for updating privileges > -- > > Key: HIVE-23301 > URL: https://issues.apache.org/jira/browse/HIVE-23301 > Project: Hive > Issue Type: Improvement > Components: Metastore, UDF >Affects Versions: 3.1.1 >Reporter: Simhadri G >Assignee: Simhadri G >Priority: Minor > Attachments: UDFSplitMapPrivs.patch, UDFSplitMapPrivs.patch, > hive-23301-2.patch, hive-23301.3.patch > > > Privilege synchronizer pulls the list of databases, tables and columns from > the Hive Metastore. For each of these objects it fetches the privilege > information and invokes HMS API to refresh the privilege information in HMS. > The current UDF Maps a bit string to a privilege based on if the privilege > is granted or not. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Assigned] (HIVE-23337) Move q tests to TestMiniLlapLocal from TestCliDriver where the output is different, batch 2
[ https://issues.apache.org/jira/browse/HIVE-23337?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Miklos Gergely reassigned HIVE-23337: - > Move q tests to TestMiniLlapLocal from TestCliDriver where the output is > different, batch 2 > --- > > Key: HIVE-23337 > URL: https://issues.apache.org/jira/browse/HIVE-23337 > Project: Hive > Issue Type: Sub-task >Reporter: Miklos Gergely >Assignee: Miklos Gergely >Priority: Major > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-23284) Remove dependency on mariadb-java-client
[
https://issues.apache.org/jira/browse/HIVE-23284?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17095967#comment-17095967
]
Hive QA commented on HIVE-23284:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
49s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 9m
8s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
33s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
40s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 1m
10s{color} | {color:blue} standalone-metastore/metastore-server in master has
187 extant Findbugs warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
39s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
10s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
45s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
36s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
36s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
39s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
2s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
22s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
34s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
14s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 22m 53s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc xml compile findbugs
checkstyle |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-22021/dev-support/hive-personality.sh
|
| git revision | master / 9c12a4c |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| modules | C: standalone-metastore standalone-metastore/metastore-server U:
standalone-metastore |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-22021/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Remove dependency on mariadb-java-client
>
>
> Key: HIVE-23284
> URL: https://issues.apache.org/jira/browse/HIVE-23284
> Project: Hive
> Issue Type: Improvement
>Reporter: Karen Coppage
>Assignee: Karen Coppage
>Priority: Major
> Attachments: HIVE-23284.01.patch, HIVE-23284.01.patch,
> HIVE-23284.01.patch, HIVE-23284.02.patch, HIVE-23284.03.patch,
> HIVE-23284.03.patch, HIVE-23284.03.patch
>
>
> It has GNU Lesser General Public License which is [Category
> X|https://www.apache.org/legal/resolved.html#category-x].
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-23031) Add option to enable transparent rewrite of count(distinct) into sketch functions
[
https://issues.apache.org/jira/browse/HIVE-23031?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17095951#comment-17095951
]
Hive QA commented on HIVE-23031:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/13001560/HIVE-23031.04.patch
{color:green}SUCCESS:{color} +1 due to 3 test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 17181 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/22020/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/22020/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-22020/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 13001560 - PreCommit-HIVE-Build
> Add option to enable transparent rewrite of count(distinct) into sketch
> functions
> -
>
> Key: HIVE-23031
> URL: https://issues.apache.org/jira/browse/HIVE-23031
> Project: Hive
> Issue Type: Sub-task
>Reporter: Zoltan Haindrich
>Assignee: Zoltan Haindrich
>Priority: Major
> Attachments: HIVE-23031.01.patch, HIVE-23031.02.patch,
> HIVE-23031.03.patch, HIVE-23031.03.patch, HIVE-23031.03.patch,
> HIVE-23031.04.patch, HIVE-23031.04.patch
>
> Time Spent: 2h 40m
> Remaining Estimate: 0h
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-23031) Add option to enable transparent rewrite of count(distinct) into sketch functions
[
https://issues.apache.org/jira/browse/HIVE-23031?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17095927#comment-17095927
]
Hive QA commented on HIVE-23031:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 2m
1s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
12s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
18s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
55s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
32s{color} | {color:blue} common in master has 63 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
40s{color} | {color:blue} ql in master has 1528 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
10s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
27s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
43s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
18s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
18s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
41s{color} | {color:red} ql: The patch generated 3 new + 103 unchanged - 0
fixed = 106 total (was 103) {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
0s{color} | {color:red} The patch has 2 line(s) that end in whitespace. Use git
apply --whitespace=fix <>. Refer https://git-scm.com/docs/git-apply
{color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 3m
47s{color} | {color:red} ql generated 2 new + 1528 unchanged - 0 fixed = 1530
total (was 1528) {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
6s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
14s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 28m 45s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| FindBugs | module:ql |
| | Format string "%s" needs argument 2 but only 1 are provided in
org.apache.hadoop.hive.ql.exec.DataSketchesFunctions.getSketchFunction(String,
String) At DataSketchesFunctions.java:2 but only 1 are provided in
org.apache.hadoop.hive.ql.exec.DataSketchesFunctions.getSketchFunction(String,
String) At DataSketchesFunctions.java:[line 112] |
| | Dead store to newAggCalls in
org.apache.hadoop.hive.ql.optimizer.calcite.rules.HiveRewriteCountDistinctToDataSketches.onMatch(RelOptRuleCall)
At
HiveRewriteCountDistinctToDataSketches.java:org.apache.hadoop.hive.ql.optimizer.calcite.rules.HiveRewriteCountDistinctToDataSketches.onMatch(RelOptRuleCall)
At HiveRewriteCountDistinctToDataSketches.java:[line 74] |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-22020/dev-support/hive-personality.sh
|
| git revision | master / 9c12a4c |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.1 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-22020/yetus/diff-checkstyle-ql.txt
|
| whitespace |
http://104.198.109.242/logs//PreCommit-HIVE-Build-22020/yetus/whitespace-eol.txt
|
| findbugs |
http://104.198.109.242/logs//PreCommit-HIVE-Build-22020/yetus/new-findbugs-ql.html
|
| modules | C: common ql itests U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-22020/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
[jira] [Commented] (HIVE-23215) Make FilterContext and MutableFilterContext interfaces
[ https://issues.apache.org/jira/browse/HIVE-23215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17095914#comment-17095914 ] Jesus Camacho Rodriguez commented on HIVE-23215: I think this patch should not contain the change in the pom.xml file? > Make FilterContext and MutableFilterContext interfaces > -- > > Key: HIVE-23215 > URL: https://issues.apache.org/jira/browse/HIVE-23215 > Project: Hive > Issue Type: Bug > Components: storage-api >Reporter: Owen O'Malley >Assignee: Owen O'Malley >Priority: Major > Labels: pull-request-available > Attachments: HIVE-23215.01.patch, HIVE-23215.patch > > Time Spent: 1h 20m > Remaining Estimate: 0h > > HIVE-22959 introduced FilterContext to support ORC-577. The duplication of > fields between the FilterContext and VectorizedRowBatch seems likely to cause > user confusion. This patch makes them interfaces that VectorizedRowBatch > implements. > Thus, there is a single copy of the data and no need to copy them back and > forth. LLAP can make its own implementation of the interfaces if it doesn't > want to use VectorizedRowBatch. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-23293) Locks: Implement zero-wait readers
[
https://issues.apache.org/jira/browse/HIVE-23293?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17095907#comment-17095907
]
Hive QA commented on HIVE-23293:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/13001571/HIVE-23293.3.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 17181 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/22019/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/22019/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-22019/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 13001571 - PreCommit-HIVE-Build
> Locks: Implement zero-wait readers
> ---
>
> Key: HIVE-23293
> URL: https://issues.apache.org/jira/browse/HIVE-23293
> Project: Hive
> Issue Type: Sub-task
>Reporter: Denys Kuzmenko
>Assignee: Denys Kuzmenko
>Priority: Major
> Attachments: HIVE-23293.1.patch, HIVE-23293.2.patch,
> HIVE-23293.3.patch
>
>
> With a new lock type (EXCL_WRITE) for INSERT_OVERWRITE, SHARED_READ does not
> have to wait for any lock - it can fails fast for a pending EXCLUSIVE,
> because even if there is an EXCL_WRITE or SHARED_WRITE pending, there's no
> semantic reason to wait for them.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-23293) Locks: Implement zero-wait readers
[
https://issues.apache.org/jira/browse/HIVE-23293?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17095898#comment-17095898
]
Hive QA commented on HIVE-23293:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
58s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
13s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
21s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
27s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 2m
32s{color} | {color:blue} standalone-metastore/metastore-common in master has
35 extant Findbugs warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
34s{color} | {color:blue} common in master has 63 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 1m
14s{color} | {color:blue} standalone-metastore/metastore-server in master has
187 extant Findbugs warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
39s{color} | {color:blue} ql in master has 1528 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m
28s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
27s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
49s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
17s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 2m
17s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
21s{color} | {color:red} standalone-metastore/metastore-server: The patch
generated 2 new + 478 unchanged - 3 fixed = 480 total (was 481) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
40s{color} | {color:red} ql: The patch generated 4 new + 137 unchanged - 0
fixed = 141 total (was 137) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 1m
19s{color} | {color:red} standalone-metastore/metastore-server generated 1 new
+ 186 unchanged - 1 fixed = 187 total (was 187) {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m
27s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
14s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 43m 46s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| FindBugs | module:standalone-metastore/metastore-server |
| | org.apache.hadoop.hive.metastore.txn.TxnHandler.checkLock(Connection,
long, long, boolean) passes a nonconstant String to an execute or addBatch
method on an SQL statement At TxnHandler.java:a nonconstant String to an
execute or addBatch method on an SQL statement At TxnHandler.java:[line 4384] |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-22019/dev-support/hive-personality.sh
|
| git revision | master / 9c12a4c |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-22019/yetus/diff-checkstyle-standalone-metastore_metastore-server.txt
|
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-22019/yetus/diff-checkstyle-ql.txt
|
| findbugs |
[jira] [Work logged] (HIVE-23031) Add option to enable transparent rewrite of count(distinct) into sketch functions
[
https://issues.apache.org/jira/browse/HIVE-23031?focusedWorklogId=428743=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-428743
]
ASF GitHub Bot logged work on HIVE-23031:
-
Author: ASF GitHub Bot
Created on: 29/Apr/20 20:56
Start Date: 29/Apr/20 20:56
Worklog Time Spent: 10m
Work Description: jcamachor commented on a change in pull request #988:
URL: https://github.com/apache/hive/pull/988#discussion_r417606413
##
File path: common/src/java/org/apache/hadoop/hive/conf/HiveConf.java
##
@@ -2465,6 +2465,19 @@ private static void
populateLlapDaemonVarsSet(Set llapDaemonVarsSetLocal
"If the number of references to a CTE clause exceeds this threshold,
Hive will materialize it\n" +
"before executing the main query block. -1 will disable this
feature."),
+HIVE_OPTIMIZE_BI_ENABLED("hive.optimize.bi.enabled", false,
+"Enables query rewrites based on approximate functions(sketches)."),
+
+
HIVE_OPTIMIZE_BI_REWRITE_COUNTDISTINCT_ENABLED("hive.optimize.bi.rewrite.countdistinct.enabled",
+true,
+"Enables to rewrite COUNT(DISTINCT(X)) queries to be rewritten to use
sketch functions."),
+
+HIVE_OPTIMIZE_BI_REWRITE_COUNT_DISTINCT_SKETCH(
+"hive.optimize.bi.rewrite.countdistinct.sketch", "hll",
+new StringSet("hll", "cpc", "theta"),
Review comment:
Can we limit the algorithm choices to a single one for the time being?
The reason I am asking this is that this will not work with materialized
views. Since we are not storing in the SQL view definition the algorithm that
we used to generate the column, if the property value changes, this would lead
to errors.
The multi-algorithm supports needs a little bit more work. One option would
be to store this information in the MV table properties so we know how to
interpret them when HS2 needs to load them (and thus parse them). What do you
think?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 428743)
Time Spent: 2h 40m (was: 2.5h)
> Add option to enable transparent rewrite of count(distinct) into sketch
> functions
> -
>
> Key: HIVE-23031
> URL: https://issues.apache.org/jira/browse/HIVE-23031
> Project: Hive
> Issue Type: Sub-task
>Reporter: Zoltan Haindrich
>Assignee: Zoltan Haindrich
>Priority: Major
> Attachments: HIVE-23031.01.patch, HIVE-23031.02.patch,
> HIVE-23031.03.patch, HIVE-23031.03.patch, HIVE-23031.03.patch,
> HIVE-23031.04.patch, HIVE-23031.04.patch
>
> Time Spent: 2h 40m
> Remaining Estimate: 0h
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22737) Concurrency: FunctionRegistry::getFunctionInfo is static object locked
[
https://issues.apache.org/jira/browse/HIVE-22737?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17095877#comment-17095877
]
Hive QA commented on HIVE-22737:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/13001538/HIVE-22737.5.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 17179 tests
executed
*Failed tests:*
{noformat}
org.apache.hive.jdbc.TestSSL.testMetastoreWithSSL (batchId=213)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/22018/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/22018/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-22018/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 1 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 13001538 - PreCommit-HIVE-Build
> Concurrency: FunctionRegistry::getFunctionInfo is static object locked
> --
>
> Key: HIVE-22737
> URL: https://issues.apache.org/jira/browse/HIVE-22737
> Project: Hive
> Issue Type: Bug
> Components: Logical Optimizer, UDF
>Reporter: Gopal Vijayaraghavan
>Assignee: Ashutosh Chauhan
>Priority: Major
> Attachments: FunctionRegistry-lock.png, HIVE-22737.2.patch,
> HIVE-22737.3.patch, HIVE-22737.4.patch, HIVE-22737.5.patch, HIVE-22737.patch
>
>
> The lock is inside a HS2-wide static object
> https://github.com/apache/hive/blob/master/ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionRegistry.java#L191
> {code}
> // registry for system functions
> private static final Registry system = new Registry(true);
> {code}
> And this is the lock itself
> https://github.com/apache/hive/blob/master/ql/src/java/org/apache/hadoop/hive/ql/exec/Registry.java#L332
> {code}
> public FunctionInfo getFunctionInfo(String functionName) throws
> SemanticException {
> lock.lock();
> {code}
> !FunctionRegistry-lock.png!
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (HIVE-23031) Add option to enable transparent rewrite of count(distinct) into sketch functions
[
https://issues.apache.org/jira/browse/HIVE-23031?focusedWorklogId=428741=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-428741
]
ASF GitHub Bot logged work on HIVE-23031:
-
Author: ASF GitHub Bot
Created on: 29/Apr/20 20:36
Start Date: 29/Apr/20 20:36
Worklog Time Spent: 10m
Work Description: jcamachor commented on a change in pull request #988:
URL: https://github.com/apache/hive/pull/988#discussion_r417595617
##
File path: ql/src/java/org/apache/hadoop/hive/ql/exec/DataSketchesFunctions.java
##
@@ -128,14 +141,26 @@ private void buildCalciteFns() {
OperandTypes.family(),
unionFn);
+
unionSFD.setCalciteFunction(unionFn);
sketchSFD.setCalciteFunction(sketchFn);
+ if (estimateSFD != null) {
+SqlFunction estimateFn = new HiveSqlFunction(estimateSFD.name,
+SqlKind.OTHER_FUNCTION,
+ReturnTypes.explicit(SqlTypeName.DOUBLE),
Review comment:
Do we have a follow-up? Would you mind to create it so we can keep track?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 428741)
Time Spent: 2.5h (was: 2h 20m)
> Add option to enable transparent rewrite of count(distinct) into sketch
> functions
> -
>
> Key: HIVE-23031
> URL: https://issues.apache.org/jira/browse/HIVE-23031
> Project: Hive
> Issue Type: Sub-task
>Reporter: Zoltan Haindrich
>Assignee: Zoltan Haindrich
>Priority: Major
> Attachments: HIVE-23031.01.patch, HIVE-23031.02.patch,
> HIVE-23031.03.patch, HIVE-23031.03.patch, HIVE-23031.03.patch,
> HIVE-23031.04.patch, HIVE-23031.04.patch
>
> Time Spent: 2.5h
> Remaining Estimate: 0h
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-21304) Make bucketing version usage more robust
[ https://issues.apache.org/jira/browse/HIVE-21304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17095863#comment-17095863 ] Jesus Camacho Rodriguez commented on HIVE-21304: +1 (pending tests) > Make bucketing version usage more robust > > > Key: HIVE-21304 > URL: https://issues.apache.org/jira/browse/HIVE-21304 > Project: Hive > Issue Type: Bug >Reporter: Deepak Jaiswal >Assignee: Zoltan Haindrich >Priority: Major > Attachments: HIVE-21304.01.patch, HIVE-21304.02.patch, > HIVE-21304.03.patch, HIVE-21304.04.patch, HIVE-21304.05.patch, > HIVE-21304.06.patch, HIVE-21304.07.patch, HIVE-21304.08.patch, > HIVE-21304.09.patch, HIVE-21304.10.patch, HIVE-21304.11.patch, > HIVE-21304.12.patch, HIVE-21304.13.patch, HIVE-21304.14.patch, > HIVE-21304.15.patch, HIVE-21304.16.patch, HIVE-21304.17.patch, > HIVE-21304.18.patch, HIVE-21304.19.patch, HIVE-21304.20.patch, > HIVE-21304.21.patch, HIVE-21304.22.patch, HIVE-21304.23.patch, > HIVE-21304.24.patch, HIVE-21304.25.patch, HIVE-21304.26.patch, > HIVE-21304.27.patch, HIVE-21304.28.patch, HIVE-21304.29.patch, > HIVE-21304.30.patch, HIVE-21304.31.patch, HIVE-21304.32.patch, > HIVE-21304.33.patch, HIVE-21304.33.patch, HIVE-21304.33.patch > > Time Spent: 50m > Remaining Estimate: 0h > > * Show Bucketing version for ReduceSinkOp in explain extended plan - this > helps identify what hashing algorithm is being used by by ReduceSinkOp. > * move the actually selected version to the "conf" so that it doesn't get lost > * replace trait related logic with a separate optimizer rule > * do version selection based on a group of operator - this is more reliable > * skip bucketingversion selection for tables with 1 buckets > * prefer to use version 2 if possible > * fix operator creations which didn't set a new conf -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-22737) Concurrency: FunctionRegistry::getFunctionInfo is static object locked
[
https://issues.apache.org/jira/browse/HIVE-22737?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17095858#comment-17095858
]
Hive QA commented on HIVE-22737:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 9m
41s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
1s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
39s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
31s{color} | {color:blue} ql in master has 1528 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
52s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
26s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
2s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
2s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
41s{color} | {color:red} ql: The patch generated 19 new + 27 unchanged - 0
fixed = 46 total (was 27) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
48s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
53s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
14s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 24m 25s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-22018/dev-support/hive-personality.sh
|
| git revision | master / dc5ea5a |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.1 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-22018/yetus/diff-checkstyle-ql.txt
|
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-22018/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Concurrency: FunctionRegistry::getFunctionInfo is static object locked
> --
>
> Key: HIVE-22737
> URL: https://issues.apache.org/jira/browse/HIVE-22737
> Project: Hive
> Issue Type: Bug
> Components: Logical Optimizer, UDF
>Reporter: Gopal Vijayaraghavan
>Assignee: Ashutosh Chauhan
>Priority: Major
> Attachments: FunctionRegistry-lock.png, HIVE-22737.2.patch,
> HIVE-22737.3.patch, HIVE-22737.4.patch, HIVE-22737.5.patch, HIVE-22737.patch
>
>
> The lock is inside a HS2-wide static object
> https://github.com/apache/hive/blob/master/ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionRegistry.java#L191
> {code}
> // registry for system functions
> private static final Registry system = new Registry(true);
> {code}
> And this is the lock itself
> https://github.com/apache/hive/blob/master/ql/src/java/org/apache/hadoop/hive/ql/exec/Registry.java#L332
> {code}
> public FunctionInfo getFunctionInfo(String functionName) throws
> SemanticException {
> lock.lock();
> {code}
> !FunctionRegistry-lock.png!
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-23089) Add constraint checks to CBO plan
[
https://issues.apache.org/jira/browse/HIVE-23089?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jesus Camacho Rodriguez updated HIVE-23089:
---
Fix Version/s: 4.0.0
Resolution: Fixed
Status: Resolved (was: Patch Available)
Pushed to master, thanks [~kkasa]!
> Add constraint checks to CBO plan
> -
>
> Key: HIVE-23089
> URL: https://issues.apache.org/jira/browse/HIVE-23089
> Project: Hive
> Issue Type: Improvement
> Components: CBO
>Reporter: Krisztian Kasa
>Assignee: Krisztian Kasa
>Priority: Major
> Fix For: 4.0.0
>
> Attachments: HIVE-23089.1.patch, HIVE-23089.2.patch,
> HIVE-23089.3.patch, HIVE-23089.4.patch, HIVE-23089.5.patch,
> HIVE-23089.6.patch, HIVE-23089.6.patch
>
>
> {code}
> create table acid_uami(i int,
> de decimal(5,2) constraint nn1 not null enforced,
> vc varchar(128) constraint nn2 not null enforced) clustered
> by (i) into 2 buckets stored as orc TBLPROPERTIES ('transactional'='true');
> explain
> update acid_uami set de=null where i=1;
> {code}
> Non-CBO path:
> {code:java}
> Map Operator Tree:
> TableScan
> alias: acid_uami
> filterExpr: ((i = 1) and enforce_constraint(vc is not null))
> (type: boolean)
> Statistics: Num rows: 1 Data size: 216 Basic stats: COMPLETE
> Column stats: NONE
> Filter Operator
> predicate: ((i = 1) and enforce_constraint(vc is not null))
> (type: boolean)
> {code}
> CBO path:
> {code:java}
> Map Reduce
> Map Operator Tree:
> TableScan
> alias: acid_uami
> filterExpr: (i = 1) (type: boolean)
> Statistics: Num rows: 1 Data size: 216 Basic stats: COMPLETE
> Column stats: NONE
> Filter Operator
> predicate: (i = 1) (type: boolean)
> ...
> Reduce Operator Tree:
> ...
> Filter Operator
> predicate: enforce_constraint((null is not null and _col3 is not
> null)) (type: boolean)
> {code}
> In CBO path the enforce_constraint function is added to the plan when CBO
> plan is already generated and optimized.
> {code}
> HiveSortExchange(distribution=[any], collation=[[0]])
> HiveProject(row__id=[$5], i=[CAST(1):INTEGER], _o__c2=[null:NULL], vc=[$2])
> HiveFilter(condition=[=($0, 1)])
> HiveTableScan(table=[[default, acid_uami]], table:alias=[acid_uami])
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-23318) TxnHandler need not delete from MATERIALIZATION_REBUILD_LOCKS on need basis
[
https://issues.apache.org/jira/browse/HIVE-23318?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17095842#comment-17095842
]
Peter Vary commented on HIVE-23318:
---
CC: [~dkuzmenko], what are your thoughts? For the first glance I do not really
like the CommitTxnRequest change, maybe a specific TxnType for materialized
views, or any other idea?
> TxnHandler need not delete from MATERIALIZATION_REBUILD_LOCKS on need basis
> ---
>
> Key: HIVE-23318
> URL: https://issues.apache.org/jira/browse/HIVE-23318
> Project: Hive
> Issue Type: Improvement
> Components: Hive, Materialized views
>Reporter: Rajesh Balamohan
>Priority: Minor
>
> Observed the following queries when materialized view or any of its feature
> was not used.
> TxnHandler need not clear this part of txn commit. It would help in reducing
> the sql parsing time in server side as well.
> {noformat}
> Gdelete from MATERIALIZATION_REBUILD_LOCKS where mrl_txn_id = 120398082
> Gdelete from MATERIALIZATION_REBUILD_LOCKS where mrl_txn_id = 120398084
> Gdelete from MATERIALIZATION_REBUILD_LOCKS where mrl_txn_id = 120398112
> {noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-23089) Add constraint checks to CBO plan
[
https://issues.apache.org/jira/browse/HIVE-23089?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17095839#comment-17095839
]
Hive QA commented on HIVE-23089:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/13001537/HIVE-23089.6.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 17179 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/22017/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/22017/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-22017/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 13001537 - PreCommit-HIVE-Build
> Add constraint checks to CBO plan
> -
>
> Key: HIVE-23089
> URL: https://issues.apache.org/jira/browse/HIVE-23089
> Project: Hive
> Issue Type: Improvement
> Components: CBO
>Reporter: Krisztian Kasa
>Assignee: Krisztian Kasa
>Priority: Major
> Attachments: HIVE-23089.1.patch, HIVE-23089.2.patch,
> HIVE-23089.3.patch, HIVE-23089.4.patch, HIVE-23089.5.patch,
> HIVE-23089.6.patch, HIVE-23089.6.patch
>
>
> {code}
> create table acid_uami(i int,
> de decimal(5,2) constraint nn1 not null enforced,
> vc varchar(128) constraint nn2 not null enforced) clustered
> by (i) into 2 buckets stored as orc TBLPROPERTIES ('transactional'='true');
> explain
> update acid_uami set de=null where i=1;
> {code}
> Non-CBO path:
> {code:java}
> Map Operator Tree:
> TableScan
> alias: acid_uami
> filterExpr: ((i = 1) and enforce_constraint(vc is not null))
> (type: boolean)
> Statistics: Num rows: 1 Data size: 216 Basic stats: COMPLETE
> Column stats: NONE
> Filter Operator
> predicate: ((i = 1) and enforce_constraint(vc is not null))
> (type: boolean)
> {code}
> CBO path:
> {code:java}
> Map Reduce
> Map Operator Tree:
> TableScan
> alias: acid_uami
> filterExpr: (i = 1) (type: boolean)
> Statistics: Num rows: 1 Data size: 216 Basic stats: COMPLETE
> Column stats: NONE
> Filter Operator
> predicate: (i = 1) (type: boolean)
> ...
> Reduce Operator Tree:
> ...
> Filter Operator
> predicate: enforce_constraint((null is not null and _col3 is not
> null)) (type: boolean)
> {code}
> In CBO path the enforce_constraint function is added to the plan when CBO
> plan is already generated and optimized.
> {code}
> HiveSortExchange(distribution=[any], collation=[[0]])
> HiveProject(row__id=[$5], i=[CAST(1):INTEGER], _o__c2=[null:NULL], vc=[$2])
> HiveFilter(condition=[=($0, 1)])
> HiveTableScan(table=[[default, acid_uami]], table:alias=[acid_uami])
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-23133) Numeric operations can have different result across hardware archs
[ https://issues.apache.org/jira/browse/HIVE-23133?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17095838#comment-17095838 ] Chinna Rao Lalam commented on HIVE-23133: - Thanks for the patch. StrictMath helps portability of java programs, so using strictmath is good. I am considering these 2 things about strictmath. # Performance issue, as published jdk performance comparison differences are small. # Backward compatibility , but with strictmath it is fixing the problem completely. Any thoughts on this [~hashutosh] [~gopalv] [~jcamachorodriguez] > Numeric operations can have different result across hardware archs > -- > > Key: HIVE-23133 > URL: https://issues.apache.org/jira/browse/HIVE-23133 > Project: Hive > Issue Type: Sub-task >Reporter: Zhenyu Zheng >Assignee: Yikun Jiang >Priority: Major > Attachments: HIVE-23133.1.patch, HIVE-23133.2.patch, > HIVE-23133.3.patch > > > Currently, we have set up an ARM CI to test out how Hive works on ARM > platform: > [https://builds.apache.org/view/H-L/view/Hive/job/Hive-linux-ARM-trunk/] > Among the failures, we have observed that some numeric operations can have > different result across hardware archs, such as: > [https://builds.apache.org/view/H-L/view/Hive/job/Hive-linux-ARM-trunk/25/testReport/org.apache.hadoop.hive.cli/TestCliDriver/testCliDriver_vector_decimal_udf2_/] > [https://builds.apache.org/view/H-L/view/Hive/job/Hive-linux-ARM-trunk/25/testReport/org.apache.hadoop.hive.cli/TestSparkCliDriver/testCliDriver_subquery_select_/] > [https://builds.apache.org/view/H-L/view/Hive/job/Hive-linux-ARM-trunk/25/testReport/org.apache.hadoop.hive.cli/TestSparkCliDriver/testCliDriver_vectorized_math_funcs_/] > we can see that the calculation results of log, exp, cos, toRadians etc is > slitly different than the .out file results that we are > comparing(they are tested and wrote on X86 machines), this is because of we > use [Math > Library|https://docs.oracle.com/javase/6/docs/api/java/lang/Math.html] for > these kind of calculations. > and according to the > [illustration|https://docs.oracle.com/javase/6/docs/api/java/lang/Math.html]: > _Unlike some of the numeric methods of class StrictMath, all implementations > of the equivalent functions of class Math are not_ > _defined to return the bit-for-bit same results. This relaxation permits > better-performing implementations where strict reproducibility_ > _is not required._ > _By default many of the Math methods simply call the equivalent method in > StrictMath for their implementation._ > _Code generators are encouraged to use platform-specific native libraries or > microprocessor instructions, where available,_ > _to provide higher-performance implementations of Math methods._ > so the result will have difference across hardware archs. > On the other hand, JAVA provided another library > [StrictMath|https://docs.oracle.com/javase/6/docs/api/java/lang/StrictMath.html] > that will not have this kind of problem as according to its' > [reference|https://docs.oracle.com/javase/6/docs/api/java/lang/StrictMath.html]: > _To help ensure portability of Java programs, the definitions of some of the > numeric functions in this package require that they produce_ > _the same results as certain published algorithms._ > So in order to fix the above mentioned problem, we have to consider switch to > use StrictMath instead of Math. > > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-23274) Move q tests to TestMiniLlapLocal from TestCliDriver where the output is different, batch 1
[ https://issues.apache.org/jira/browse/HIVE-23274?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Miklos Gergely updated HIVE-23274: -- Attachment: HIVE-23274.02.patch > Move q tests to TestMiniLlapLocal from TestCliDriver where the output is > different, batch 1 > --- > > Key: HIVE-23274 > URL: https://issues.apache.org/jira/browse/HIVE-23274 > Project: Hive > Issue Type: Sub-task >Reporter: Miklos Gergely >Assignee: Miklos Gergely >Priority: Major > Attachments: HIVE-23274.01.patch, HIVE-23274.02.patch > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-23318) TxnHandler need not delete from MATERIALIZATION_REBUILD_LOCKS on need basis
[
https://issues.apache.org/jira/browse/HIVE-23318?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17095808#comment-17095808
]
Jesus Camacho Rodriguez commented on HIVE-23318:
Yes, to indicate whether the commit is coming from a materialized view rebuild
or not.
> TxnHandler need not delete from MATERIALIZATION_REBUILD_LOCKS on need basis
> ---
>
> Key: HIVE-23318
> URL: https://issues.apache.org/jira/browse/HIVE-23318
> Project: Hive
> Issue Type: Improvement
> Components: Hive, Materialized views
>Reporter: Rajesh Balamohan
>Priority: Minor
>
> Observed the following queries when materialized view or any of its feature
> was not used.
> TxnHandler need not clear this part of txn commit. It would help in reducing
> the sql parsing time in server side as well.
> {noformat}
> Gdelete from MATERIALIZATION_REBUILD_LOCKS where mrl_txn_id = 120398082
> Gdelete from MATERIALIZATION_REBUILD_LOCKS where mrl_txn_id = 120398084
> Gdelete from MATERIALIZATION_REBUILD_LOCKS where mrl_txn_id = 120398112
> {noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-23089) Add constraint checks to CBO plan
[
https://issues.apache.org/jira/browse/HIVE-23089?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17095807#comment-17095807
]
Hive QA commented on HIVE-23089:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 10m
2s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
1s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
44s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
40s{color} | {color:blue} ql in master has 1528 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
54s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
27s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
1s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
1s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
44s{color} | {color:green} ql: The patch generated 0 new + 450 unchanged - 10
fixed = 450 total (was 460) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
52s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
55s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
15s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 25m 4s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-22017/dev-support/hive-personality.sh
|
| git revision | master / 213de03 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.1 |
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-22017/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Add constraint checks to CBO plan
> -
>
> Key: HIVE-23089
> URL: https://issues.apache.org/jira/browse/HIVE-23089
> Project: Hive
> Issue Type: Improvement
> Components: CBO
>Reporter: Krisztian Kasa
>Assignee: Krisztian Kasa
>Priority: Major
> Attachments: HIVE-23089.1.patch, HIVE-23089.2.patch,
> HIVE-23089.3.patch, HIVE-23089.4.patch, HIVE-23089.5.patch,
> HIVE-23089.6.patch, HIVE-23089.6.patch
>
>
> {code}
> create table acid_uami(i int,
> de decimal(5,2) constraint nn1 not null enforced,
> vc varchar(128) constraint nn2 not null enforced) clustered
> by (i) into 2 buckets stored as orc TBLPROPERTIES ('transactional'='true');
> explain
> update acid_uami set de=null where i=1;
> {code}
> Non-CBO path:
> {code:java}
> Map Operator Tree:
> TableScan
> alias: acid_uami
> filterExpr: ((i = 1) and enforce_constraint(vc is not null))
> (type: boolean)
> Statistics: Num rows: 1 Data size: 216 Basic stats: COMPLETE
> Column stats: NONE
> Filter Operator
> predicate: ((i = 1) and enforce_constraint(vc is not null))
> (type: boolean)
> {code}
> CBO path:
> {code:java}
> Map Reduce
> Map Operator Tree:
> TableScan
> alias: acid_uami
> filterExpr: (i = 1) (type: boolean)
> Statistics: Num rows: 1 Data size: 216 Basic stats: COMPLETE
> Column stats: NONE
> Filter Operator
> predicate: (i = 1) (type: boolean)
> ...
>
[jira] [Updated] (HIVE-23306) RESET command does not work if there is a config set by System.getProperty
[
https://issues.apache.org/jira/browse/HIVE-23306?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Peter Vary updated HIVE-23306:
--
Fix Version/s: 4.0.0
Resolution: Fixed
Status: Resolved (was: Patch Available)
Pushed to master.
Thanks for the review [~kuczoram]!
> RESET command does not work if there is a config set by System.getProperty
> --
>
> Key: HIVE-23306
> URL: https://issues.apache.org/jira/browse/HIVE-23306
> Project: Hive
> Issue Type: Bug
> Components: HiveServer2
>Reporter: Peter Vary
>Assignee: Peter Vary
>Priority: Major
> Fix For: 4.0.0
>
> Attachments: HIVE-23306.02.patch, HIVE-23306.03.patch,
> HIVE-23306.patch
>
>
> If a property set is restricted by {{hive.conf.restricted.list}} and the
> setting is set by system property, then {{RESET}} command fails with:
> {code}
> Caused by: java.lang.IllegalArgumentException: Cannot modify
> hive.query.max.length at runtime. It is in the list of parameters that can't
> be modified at runtime or is prefixed by a restricted variable
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-23318) TxnHandler need not delete from MATERIALIZATION_REBUILD_LOCKS on need basis
[
https://issues.apache.org/jira/browse/HIVE-23318?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17095805#comment-17095805
]
Ashutosh Chauhan commented on HIVE-23318:
-
I am not sure I followed. Are you saying we can fix this by introducing new
parameter in CommitTxnRequest struct?
> TxnHandler need not delete from MATERIALIZATION_REBUILD_LOCKS on need basis
> ---
>
> Key: HIVE-23318
> URL: https://issues.apache.org/jira/browse/HIVE-23318
> Project: Hive
> Issue Type: Improvement
> Components: Hive, Materialized views
>Reporter: Rajesh Balamohan
>Priority: Minor
>
> Observed the following queries when materialized view or any of its feature
> was not used.
> TxnHandler need not clear this part of txn commit. It would help in reducing
> the sql parsing time in server side as well.
> {noformat}
> Gdelete from MATERIALIZATION_REBUILD_LOCKS where mrl_txn_id = 120398082
> Gdelete from MATERIALIZATION_REBUILD_LOCKS where mrl_txn_id = 120398084
> Gdelete from MATERIALIZATION_REBUILD_LOCKS where mrl_txn_id = 120398112
> {noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-23261) Check whether encryption is enabled in the cluster before moving files
[ https://issues.apache.org/jira/browse/HIVE-23261?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashutosh Chauhan updated HIVE-23261: Fix Version/s: 4.0.0 Resolution: Fixed Status: Resolved (was: Patch Available) Pushed to master. Thanks, Ramesh! > Check whether encryption is enabled in the cluster before moving files > -- > > Key: HIVE-23261 > URL: https://issues.apache.org/jira/browse/HIVE-23261 > Project: Hive > Issue Type: Improvement >Reporter: Rajesh Balamohan >Assignee: Ramesh Kumar Thangarajan >Priority: Minor > Fix For: 4.0.0 > > Attachments: HIVE-23261.1.patch, HIVE-23261.2.patch, > HIVE-23261.3.patch > > > Similar to HIVE-23212, there is an unwanted check of encryption paths during > file move operation. > [https://github.com/apache/hive/blob/master/ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java#L4546] > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-23261) Check whether encryption is enabled in the cluster before moving files
[
https://issues.apache.org/jira/browse/HIVE-23261?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17095759#comment-17095759
]
Hive QA commented on HIVE-23261:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/13001523/HIVE-23261.3.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 17179 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/22016/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/22016/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-22016/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 13001523 - PreCommit-HIVE-Build
> Check whether encryption is enabled in the cluster before moving files
> --
>
> Key: HIVE-23261
> URL: https://issues.apache.org/jira/browse/HIVE-23261
> Project: Hive
> Issue Type: Improvement
>Reporter: Rajesh Balamohan
>Assignee: Ramesh Kumar Thangarajan
>Priority: Minor
> Attachments: HIVE-23261.1.patch, HIVE-23261.2.patch,
> HIVE-23261.3.patch
>
>
> Similar to HIVE-23212, there is an unwanted check of encryption paths during
> file move operation.
> [https://github.com/apache/hive/blob/master/ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java#L4546]
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-23318) TxnHandler need not delete from MATERIALIZATION_REBUILD_LOCKS on need basis
[
https://issues.apache.org/jira/browse/HIVE-23318?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jesus Camacho Rodriguez updated HIVE-23318:
---
Component/s: Materialized views
> TxnHandler need not delete from MATERIALIZATION_REBUILD_LOCKS on need basis
> ---
>
> Key: HIVE-23318
> URL: https://issues.apache.org/jira/browse/HIVE-23318
> Project: Hive
> Issue Type: Improvement
> Components: Hive, Materialized views
>Reporter: Rajesh Balamohan
>Priority: Minor
>
> Observed the following queries when materialized view or any of its feature
> was not used.
> TxnHandler need not clear this part of txn commit. It would help in reducing
> the sql parsing time in server side as well.
> {noformat}
> Gdelete from MATERIALIZATION_REBUILD_LOCKS where mrl_txn_id = 120398082
> Gdelete from MATERIALIZATION_REBUILD_LOCKS where mrl_txn_id = 120398084
> Gdelete from MATERIALIZATION_REBUILD_LOCKS where mrl_txn_id = 120398112
> {noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-23261) Check whether encryption is enabled in the cluster before moving files
[
https://issues.apache.org/jira/browse/HIVE-23261?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17095742#comment-17095742
]
Hive QA commented on HIVE-23261:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 10m
1s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
2s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
40s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
37s{color} | {color:blue} ql in master has 1528 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
53s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
25s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
3s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
3s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
41s{color} | {color:red} ql: The patch generated 1 new + 193 unchanged - 1
fixed = 194 total (was 194) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
49s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
54s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
13s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 24m 46s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-22016/dev-support/hive-personality.sh
|
| git revision | master / 213de03 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.1 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-22016/yetus/diff-checkstyle-ql.txt
|
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-22016/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Check whether encryption is enabled in the cluster before moving files
> --
>
> Key: HIVE-23261
> URL: https://issues.apache.org/jira/browse/HIVE-23261
> Project: Hive
> Issue Type: Improvement
>Reporter: Rajesh Balamohan
>Assignee: Ramesh Kumar Thangarajan
>Priority: Minor
> Attachments: HIVE-23261.1.patch, HIVE-23261.2.patch,
> HIVE-23261.3.patch
>
>
> Similar to HIVE-23212, there is an unwanted check of encryption paths during
> file move operation.
> [https://github.com/apache/hive/blob/master/ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java#L4546]
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-23230) "get_splits" udf ignores limit constraint while creating splits
[
https://issues.apache.org/jira/browse/HIVE-23230?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Adesh Kumar Rao updated HIVE-23230:
---
Status: In Progress (was: Patch Available)
> "get_splits" udf ignores limit constraint while creating splits
> ---
>
> Key: HIVE-23230
> URL: https://issues.apache.org/jira/browse/HIVE-23230
> Project: Hive
> Issue Type: Bug
> Components: HiveServer2
>Affects Versions: 3.1.0
>Reporter: Adesh Kumar Rao
>Assignee: Adesh Kumar Rao
>Priority: Major
> Attachments: HIVE-23230.1.patch, HIVE-23230.2.patch,
> HIVE-23230.3.patch, HIVE-23230.4.patch, HIVE-23230.5.patch, HIVE-23230.patch
>
>
> Issue: Running the query {noformat}select * from limit n{noformat}
> from spark via hive warehouse connector may return more rows than "n".
> This happens because "get_splits" udf creates splits ignoring the limit
> constraint. These splits when submitted to multiple llap daemons will return
> "n" rows each.
> How to reproduce: Needs spark-shell, hive-warehouse-connector and hive on
> llap with more that 1 llap daemons running.
> run below commands via beeline to create and populate the table
>
> {noformat}
> create table test (id int);
> insert into table test values (1);
> insert into table test values (2);
> insert into table test values (3);
> insert into table test values (4);
> insert into table test values (5);
> insert into table test values (6);
> insert into table test values (7);
> delete from test where id = 7;{noformat}
> now running below query via spark-shell
> {noformat}
> import com.hortonworks.hwc.HiveWarehouseSession
> val hive = HiveWarehouseSession.session(spark).build()
> hive.executeQuery("select * from test limit 1").show()
> {noformat}
> will return more than 1 rows.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-23230) "get_splits" udf ignores limit constraint while creating splits
[
https://issues.apache.org/jira/browse/HIVE-23230?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Adesh Kumar Rao updated HIVE-23230:
---
Attachment: HIVE-23230.5.patch
> "get_splits" udf ignores limit constraint while creating splits
> ---
>
> Key: HIVE-23230
> URL: https://issues.apache.org/jira/browse/HIVE-23230
> Project: Hive
> Issue Type: Bug
> Components: HiveServer2
>Affects Versions: 3.1.0
>Reporter: Adesh Kumar Rao
>Assignee: Adesh Kumar Rao
>Priority: Major
> Attachments: HIVE-23230.1.patch, HIVE-23230.2.patch,
> HIVE-23230.3.patch, HIVE-23230.4.patch, HIVE-23230.5.patch, HIVE-23230.patch
>
>
> Issue: Running the query {noformat}select * from limit n{noformat}
> from spark via hive warehouse connector may return more rows than "n".
> This happens because "get_splits" udf creates splits ignoring the limit
> constraint. These splits when submitted to multiple llap daemons will return
> "n" rows each.
> How to reproduce: Needs spark-shell, hive-warehouse-connector and hive on
> llap with more that 1 llap daemons running.
> run below commands via beeline to create and populate the table
>
> {noformat}
> create table test (id int);
> insert into table test values (1);
> insert into table test values (2);
> insert into table test values (3);
> insert into table test values (4);
> insert into table test values (5);
> insert into table test values (6);
> insert into table test values (7);
> delete from test where id = 7;{noformat}
> now running below query via spark-shell
> {noformat}
> import com.hortonworks.hwc.HiveWarehouseSession
> val hive = HiveWarehouseSession.session(spark).build()
> hive.executeQuery("select * from test limit 1").show()
> {noformat}
> will return more than 1 rows.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-23230) "get_splits" udf ignores limit constraint while creating splits
[
https://issues.apache.org/jira/browse/HIVE-23230?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Adesh Kumar Rao updated HIVE-23230:
---
Status: Patch Available (was: In Progress)
> "get_splits" udf ignores limit constraint while creating splits
> ---
>
> Key: HIVE-23230
> URL: https://issues.apache.org/jira/browse/HIVE-23230
> Project: Hive
> Issue Type: Bug
> Components: HiveServer2
>Affects Versions: 3.1.0
>Reporter: Adesh Kumar Rao
>Assignee: Adesh Kumar Rao
>Priority: Major
> Attachments: HIVE-23230.1.patch, HIVE-23230.2.patch,
> HIVE-23230.3.patch, HIVE-23230.4.patch, HIVE-23230.5.patch, HIVE-23230.patch
>
>
> Issue: Running the query {noformat}select * from limit n{noformat}
> from spark via hive warehouse connector may return more rows than "n".
> This happens because "get_splits" udf creates splits ignoring the limit
> constraint. These splits when submitted to multiple llap daemons will return
> "n" rows each.
> How to reproduce: Needs spark-shell, hive-warehouse-connector and hive on
> llap with more that 1 llap daemons running.
> run below commands via beeline to create and populate the table
>
> {noformat}
> create table test (id int);
> insert into table test values (1);
> insert into table test values (2);
> insert into table test values (3);
> insert into table test values (4);
> insert into table test values (5);
> insert into table test values (6);
> insert into table test values (7);
> delete from test where id = 7;{noformat}
> now running below query via spark-shell
> {noformat}
> import com.hortonworks.hwc.HiveWarehouseSession
> val hive = HiveWarehouseSession.session(spark).build()
> hive.executeQuery("select * from test limit 1").show()
> {noformat}
> will return more than 1 rows.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Assigned] (HIVE-23336) HIVE-23230 follow up: Fix get_llap_udf skipped unit tests
[ https://issues.apache.org/jira/browse/HIVE-23336?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Adesh Kumar Rao reassigned HIVE-23336: -- Assignee: Adesh Kumar Rao > HIVE-23230 follow up: Fix get_llap_udf skipped unit tests > - > > Key: HIVE-23336 > URL: https://issues.apache.org/jira/browse/HIVE-23336 > Project: Hive > Issue Type: Task >Reporter: Adesh Kumar Rao >Assignee: Adesh Kumar Rao >Priority: Major > > Fix the unit tests skipped in HIVE-23230 (multiple llap daemons executing > limit query return more rows than specified by limit clause). -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-23282) Reduce number of DB calls in ObjectStore::getPartitionsByExprInternal
[ https://issues.apache.org/jira/browse/HIVE-23282?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Attila Magyar updated HIVE-23282: - Attachment: (was: HIVE-23282.5.patch) > Reduce number of DB calls in ObjectStore::getPartitionsByExprInternal > - > > Key: HIVE-23282 > URL: https://issues.apache.org/jira/browse/HIVE-23282 > Project: Hive > Issue Type: Improvement > Components: Metastore >Reporter: Rajesh Balamohan >Assignee: Attila Magyar >Priority: Major > Attachments: HIVE-23282.1.patch, HIVE-23282.3.patch, > HIVE-23282.5.patch, image-2020-04-23-14-07-06-077.png > > > ObjectStore::getPartitionsByExprInternal internally uses Table information > for getting partitionKeys, table, catalog name. > > For this, it ends up populating entire table data from DB (including skew > column, parameters, sort, bucket cols etc). This makes it a lot more > expensive call. It would be good to check if MTable itself can be used > instead of Table. > [https://github.com/apache/hive/blob/master/standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/ObjectStore.java#L3327] > [https://github.com/apache/hive/blob/master/standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/ObjectStore.java#L3669] > > !image-2020-04-23-14-07-06-077.png|width=665,height=592! -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-23282) Reduce number of DB calls in ObjectStore::getPartitionsByExprInternal
[ https://issues.apache.org/jira/browse/HIVE-23282?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Attila Magyar updated HIVE-23282: - Status: Open (was: Patch Available) > Reduce number of DB calls in ObjectStore::getPartitionsByExprInternal > - > > Key: HIVE-23282 > URL: https://issues.apache.org/jira/browse/HIVE-23282 > Project: Hive > Issue Type: Improvement > Components: Metastore >Reporter: Rajesh Balamohan >Assignee: Attila Magyar >Priority: Major > Attachments: HIVE-23282.1.patch, HIVE-23282.3.patch, > HIVE-23282.5.patch, image-2020-04-23-14-07-06-077.png > > > ObjectStore::getPartitionsByExprInternal internally uses Table information > for getting partitionKeys, table, catalog name. > > For this, it ends up populating entire table data from DB (including skew > column, parameters, sort, bucket cols etc). This makes it a lot more > expensive call. It would be good to check if MTable itself can be used > instead of Table. > [https://github.com/apache/hive/blob/master/standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/ObjectStore.java#L3327] > [https://github.com/apache/hive/blob/master/standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/ObjectStore.java#L3669] > > !image-2020-04-23-14-07-06-077.png|width=665,height=592! -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-23282) Reduce number of DB calls in ObjectStore::getPartitionsByExprInternal
[ https://issues.apache.org/jira/browse/HIVE-23282?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Attila Magyar updated HIVE-23282: - Status: Patch Available (was: Open) > Reduce number of DB calls in ObjectStore::getPartitionsByExprInternal > - > > Key: HIVE-23282 > URL: https://issues.apache.org/jira/browse/HIVE-23282 > Project: Hive > Issue Type: Improvement > Components: Metastore >Reporter: Rajesh Balamohan >Assignee: Attila Magyar >Priority: Major > Attachments: HIVE-23282.1.patch, HIVE-23282.3.patch, > HIVE-23282.5.patch, image-2020-04-23-14-07-06-077.png > > > ObjectStore::getPartitionsByExprInternal internally uses Table information > for getting partitionKeys, table, catalog name. > > For this, it ends up populating entire table data from DB (including skew > column, parameters, sort, bucket cols etc). This makes it a lot more > expensive call. It would be good to check if MTable itself can be used > instead of Table. > [https://github.com/apache/hive/blob/master/standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/ObjectStore.java#L3327] > [https://github.com/apache/hive/blob/master/standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/ObjectStore.java#L3669] > > !image-2020-04-23-14-07-06-077.png|width=665,height=592! -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-23282) Reduce number of DB calls in ObjectStore::getPartitionsByExprInternal
[ https://issues.apache.org/jira/browse/HIVE-23282?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Attila Magyar updated HIVE-23282: - Attachment: HIVE-23282.5.patch > Reduce number of DB calls in ObjectStore::getPartitionsByExprInternal > - > > Key: HIVE-23282 > URL: https://issues.apache.org/jira/browse/HIVE-23282 > Project: Hive > Issue Type: Improvement > Components: Metastore >Reporter: Rajesh Balamohan >Assignee: Attila Magyar >Priority: Major > Attachments: HIVE-23282.1.patch, HIVE-23282.3.patch, > HIVE-23282.5.patch, image-2020-04-23-14-07-06-077.png > > > ObjectStore::getPartitionsByExprInternal internally uses Table information > for getting partitionKeys, table, catalog name. > > For this, it ends up populating entire table data from DB (including skew > column, parameters, sort, bucket cols etc). This makes it a lot more > expensive call. It would be good to check if MTable itself can be used > instead of Table. > [https://github.com/apache/hive/blob/master/standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/ObjectStore.java#L3327] > [https://github.com/apache/hive/blob/master/standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/ObjectStore.java#L3669] > > !image-2020-04-23-14-07-06-077.png|width=665,height=592! -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Assigned] (HIVE-23215) Make FilterContext and MutableFilterContext interfaces
[ https://issues.apache.org/jira/browse/HIVE-23215?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Panagiotis Garefalakis reassigned HIVE-23215: - Assignee: Panagiotis Garefalakis (was: Owen O'Malley) > Make FilterContext and MutableFilterContext interfaces > -- > > Key: HIVE-23215 > URL: https://issues.apache.org/jira/browse/HIVE-23215 > Project: Hive > Issue Type: Bug > Components: storage-api >Reporter: Owen O'Malley >Assignee: Panagiotis Garefalakis >Priority: Major > Labels: pull-request-available > Attachments: HIVE-23215.01.patch, HIVE-23215.patch > > Time Spent: 1h 20m > Remaining Estimate: 0h > > HIVE-22959 introduced FilterContext to support ORC-577. The duplication of > fields between the FilterContext and VectorizedRowBatch seems likely to cause > user confusion. This patch makes them interfaces that VectorizedRowBatch > implements. > Thus, there is a single copy of the data and no need to copy them back and > forth. LLAP can make its own implementation of the interfaces if it doesn't > want to use VectorizedRowBatch. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Assigned] (HIVE-23215) Make FilterContext and MutableFilterContext interfaces
[ https://issues.apache.org/jira/browse/HIVE-23215?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Panagiotis Garefalakis reassigned HIVE-23215: - Assignee: Owen O'Malley (was: Panagiotis Garefalakis) > Make FilterContext and MutableFilterContext interfaces > -- > > Key: HIVE-23215 > URL: https://issues.apache.org/jira/browse/HIVE-23215 > Project: Hive > Issue Type: Bug > Components: storage-api >Reporter: Owen O'Malley >Assignee: Owen O'Malley >Priority: Major > Labels: pull-request-available > Attachments: HIVE-23215.01.patch, HIVE-23215.patch > > Time Spent: 1h 20m > Remaining Estimate: 0h > > HIVE-22959 introduced FilterContext to support ORC-577. The duplication of > fields between the FilterContext and VectorizedRowBatch seems likely to cause > user confusion. This patch makes them interfaces that VectorizedRowBatch > implements. > Thus, there is a single copy of the data and no need to copy them back and > forth. LLAP can make its own implementation of the interfaces if it doesn't > want to use VectorizedRowBatch. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-23215) Make FilterContext and MutableFilterContext interfaces
[ https://issues.apache.org/jira/browse/HIVE-23215?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Panagiotis Garefalakis updated HIVE-23215: -- Status: Patch Available (was: Open) > Make FilterContext and MutableFilterContext interfaces > -- > > Key: HIVE-23215 > URL: https://issues.apache.org/jira/browse/HIVE-23215 > Project: Hive > Issue Type: Bug > Components: storage-api >Reporter: Owen O'Malley >Assignee: Panagiotis Garefalakis >Priority: Major > Labels: pull-request-available > Attachments: HIVE-23215.01.patch, HIVE-23215.patch > > Time Spent: 1h 20m > Remaining Estimate: 0h > > HIVE-22959 introduced FilterContext to support ORC-577. The duplication of > fields between the FilterContext and VectorizedRowBatch seems likely to cause > user confusion. This patch makes them interfaces that VectorizedRowBatch > implements. > Thus, there is a single copy of the data and no need to copy them back and > forth. LLAP can make its own implementation of the interfaces if it doesn't > want to use VectorizedRowBatch. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-21304) Make bucketing version usage more robust
[
https://issues.apache.org/jira/browse/HIVE-21304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17095711#comment-17095711
]
Hive QA commented on HIVE-21304:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/13001590/HIVE-21304.33.patch
{color:green}SUCCESS:{color} +1 due to 4 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 17180 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[infer_bucket_sort_reducers_power_two]
(batchId=7)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/22015/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/22015/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-22015/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 1 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 13001590 - PreCommit-HIVE-Build
> Make bucketing version usage more robust
>
>
> Key: HIVE-21304
> URL: https://issues.apache.org/jira/browse/HIVE-21304
> Project: Hive
> Issue Type: Bug
>Reporter: Deepak Jaiswal
>Assignee: Zoltan Haindrich
>Priority: Major
> Attachments: HIVE-21304.01.patch, HIVE-21304.02.patch,
> HIVE-21304.03.patch, HIVE-21304.04.patch, HIVE-21304.05.patch,
> HIVE-21304.06.patch, HIVE-21304.07.patch, HIVE-21304.08.patch,
> HIVE-21304.09.patch, HIVE-21304.10.patch, HIVE-21304.11.patch,
> HIVE-21304.12.patch, HIVE-21304.13.patch, HIVE-21304.14.patch,
> HIVE-21304.15.patch, HIVE-21304.16.patch, HIVE-21304.17.patch,
> HIVE-21304.18.patch, HIVE-21304.19.patch, HIVE-21304.20.patch,
> HIVE-21304.21.patch, HIVE-21304.22.patch, HIVE-21304.23.patch,
> HIVE-21304.24.patch, HIVE-21304.25.patch, HIVE-21304.26.patch,
> HIVE-21304.27.patch, HIVE-21304.28.patch, HIVE-21304.29.patch,
> HIVE-21304.30.patch, HIVE-21304.31.patch, HIVE-21304.32.patch,
> HIVE-21304.33.patch, HIVE-21304.33.patch, HIVE-21304.33.patch
>
> Time Spent: 50m
> Remaining Estimate: 0h
>
> * Show Bucketing version for ReduceSinkOp in explain extended plan - this
> helps identify what hashing algorithm is being used by by ReduceSinkOp.
> * move the actually selected version to the "conf" so that it doesn't get lost
> * replace trait related logic with a separate optimizer rule
> * do version selection based on a group of operator - this is more reliable
> * skip bucketingversion selection for tables with 1 buckets
> * prefer to use version 2 if possible
> * fix operator creations which didn't set a new conf
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-23215) Make FilterContext and MutableFilterContext interfaces
[ https://issues.apache.org/jira/browse/HIVE-23215?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Panagiotis Garefalakis updated HIVE-23215: -- Attachment: HIVE-23215.01.patch > Make FilterContext and MutableFilterContext interfaces > -- > > Key: HIVE-23215 > URL: https://issues.apache.org/jira/browse/HIVE-23215 > Project: Hive > Issue Type: Bug > Components: storage-api >Reporter: Owen O'Malley >Assignee: Panagiotis Garefalakis >Priority: Major > Labels: pull-request-available > Attachments: HIVE-23215.01.patch, HIVE-23215.patch > > Time Spent: 1h 20m > Remaining Estimate: 0h > > HIVE-22959 introduced FilterContext to support ORC-577. The duplication of > fields between the FilterContext and VectorizedRowBatch seems likely to cause > user confusion. This patch makes them interfaces that VectorizedRowBatch > implements. > Thus, there is a single copy of the data and no need to copy them back and > forth. LLAP can make its own implementation of the interfaces if it doesn't > want to use VectorizedRowBatch. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Assigned] (HIVE-23215) Make FilterContext and MutableFilterContext interfaces
[ https://issues.apache.org/jira/browse/HIVE-23215?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Panagiotis Garefalakis reassigned HIVE-23215: - Assignee: Owen O'Malley (was: Panagiotis Garefalakis) > Make FilterContext and MutableFilterContext interfaces > -- > > Key: HIVE-23215 > URL: https://issues.apache.org/jira/browse/HIVE-23215 > Project: Hive > Issue Type: Bug > Components: storage-api >Reporter: Owen O'Malley >Assignee: Owen O'Malley >Priority: Major > Labels: pull-request-available > Attachments: HIVE-23215.01.patch, HIVE-23215.patch > > Time Spent: 1h 20m > Remaining Estimate: 0h > > HIVE-22959 introduced FilterContext to support ORC-577. The duplication of > fields between the FilterContext and VectorizedRowBatch seems likely to cause > user confusion. This patch makes them interfaces that VectorizedRowBatch > implements. > Thus, there is a single copy of the data and no need to copy them back and > forth. LLAP can make its own implementation of the interfaces if it doesn't > want to use VectorizedRowBatch. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Assigned] (HIVE-23215) Make FilterContext and MutableFilterContext interfaces
[ https://issues.apache.org/jira/browse/HIVE-23215?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Panagiotis Garefalakis reassigned HIVE-23215: - Assignee: Panagiotis Garefalakis (was: Owen O'Malley) > Make FilterContext and MutableFilterContext interfaces > -- > > Key: HIVE-23215 > URL: https://issues.apache.org/jira/browse/HIVE-23215 > Project: Hive > Issue Type: Bug > Components: storage-api >Reporter: Owen O'Malley >Assignee: Panagiotis Garefalakis >Priority: Major > Labels: pull-request-available > Attachments: HIVE-23215.01.patch, HIVE-23215.patch > > Time Spent: 1h 20m > Remaining Estimate: 0h > > HIVE-22959 introduced FilterContext to support ORC-577. The duplication of > fields between the FilterContext and VectorizedRowBatch seems likely to cause > user confusion. This patch makes them interfaces that VectorizedRowBatch > implements. > Thus, there is a single copy of the data and no need to copy them back and > forth. LLAP can make its own implementation of the interfaces if it doesn't > want to use VectorizedRowBatch. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (HIVE-22769) Incorrect query results and query failure during split generation for compressed text files
[
https://issues.apache.org/jira/browse/HIVE-22769?focusedWorklogId=428659=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-428659
]
ASF GitHub Bot logged work on HIVE-22769:
-
Author: ASF GitHub Bot
Created on: 29/Apr/20 16:46
Start Date: 29/Apr/20 16:46
Worklog Time Spent: 10m
Work Description: mustafaiman commented on a change in pull request #998:
URL: https://github.com/apache/hive/pull/998#discussion_r417460328
##
File path:
llap-server/src/java/org/apache/hadoop/hive/llap/io/encoded/PassThruOffsetReader.java
##
@@ -20,23 +20,80 @@
import java.io.IOException;
import
org.apache.hadoop.hive.llap.io.encoded.SerDeEncodedDataReader.ReaderWithOffsets;
+import org.apache.hadoop.hive.ql.exec.FooterBuffer;
+import org.apache.hadoop.hive.ql.exec.Utilities;
import org.apache.hadoop.io.Writable;
+import org.apache.hadoop.io.WritableComparable;
+import org.apache.hadoop.mapred.JobConf;
import org.apache.hadoop.mapred.RecordReader;
@SuppressWarnings("rawtypes") class PassThruOffsetReader implements
ReaderWithOffsets {
protected final RecordReader sourceReader;
protected final Object key;
protected final Writable value;
+ protected final JobConf jobConf;
+ protected final int skipHeaderCnt;
+ protected final int skipFooterCnt;
+ private transient FooterBuffer footerBuffer;
+ private transient boolean initialized = false;
- PassThruOffsetReader(RecordReader sourceReader) {
+ PassThruOffsetReader(RecordReader sourceReader, JobConf jobConf, int
headerCnt, int footerCnt) {
this.sourceReader = sourceReader;
-key = sourceReader.createKey();
-value = (Writable)sourceReader.createValue();
+this.key = sourceReader.createKey();
+this.value = (Writable)sourceReader.createValue();
+this.jobConf = jobConf;
+this.skipHeaderCnt = headerCnt;
+this.skipFooterCnt = footerCnt;
}
@Override
public boolean next() throws IOException {
-return sourceReader.next(key, value);
+try {
+ boolean opNotEOF = true;
+ /**
+ * Start reading a new file.
+ * If file contains header, skip header lines before reading the records.
+ * If file contains footer, used FooterBuffer to cache and remove footer
+ * records at the end of the file.
+ */
+ if (!initialized) {
+// Skip header lines.
+opNotEOF = Utilities.skipHeader(sourceReader, skipFooterCnt, key,
value);
+
+// Initialize footer buffer.
+if (opNotEOF && skipFooterCnt > 0) {
+ footerBuffer = new FooterBuffer();
+ opNotEOF = footerBuffer.initializeBuffer(jobConf, sourceReader,
skipFooterCnt, (WritableComparable) key, value);
+}
+this.initialized = true;
+ }
+
+ if (opNotEOF && footerBuffer == null) {
+/**
+ * When file doesn't end after skipping header line
+ * and there is NO footer lines, read normally.
+ */
+opNotEOF = sourceReader.next(key, value);
+ }
+
+ if (opNotEOF && footerBuffer != null) {
+/**
+ * When file doesn't end after skipping header line
+ * and there IS footer lines, update footerBuffer
+ */
+opNotEOF = footerBuffer.updateBuffer(jobConf, sourceReader,
(WritableComparable) key, value);
+ }
+
+ if (opNotEOF) {
Review comment:
This if-else block can be simplified now
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 428659)
Time Spent: 1h 10m (was: 1h)
> Incorrect query results and query failure during split generation for
> compressed text files
> ---
>
> Key: HIVE-22769
> URL: https://issues.apache.org/jira/browse/HIVE-22769
> Project: Hive
> Issue Type: Bug
> Components: File Formats
>Affects Versions: 3.0.0, 3.1.0
>Reporter: Chiran Ravani
>Assignee: Panagiotis Garefalakis
>Priority: Critical
> Attachments: HIVE-22769.01.patch, HIVE-22769.02.patch,
> testcase1.csv.bz2, testcase2.csv.bz2
>
> Time Spent: 1h 10m
> Remaining Estimate: 0h
>
> Hive Query produces incorrect results when data is in text format and
> compressed and for certain data the query fails during split generation.
> This behavior is seen when skip.header.line.count and skip.footer.line.count
> are set for table.
> Case 1: Select count/aggregate query produces Incorrect row counts/displays
> all rows (when
[jira] [Commented] (HIVE-21304) Make bucketing version usage more robust
[
https://issues.apache.org/jira/browse/HIVE-21304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17095603#comment-17095603
]
Hive QA commented on HIVE-21304:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
41s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
19s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
34s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
9s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
49s{color} | {color:blue} ql in master has 1528 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
15s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
29s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
3s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
36s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
36s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
55s{color} | {color:red} ql: The patch generated 6 new + 1322 unchanged - 12
fixed = 1328 total (was 1334) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 3m
55s{color} | {color:red} ql generated 1 new + 1528 unchanged - 0 fixed = 1529
total (was 1528) {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
18s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
14s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 30m 26s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| FindBugs | module:ql |
| | Nullcheck of table_desc at line 8232 of value previously dereferenced in
org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.createFileSinkDesc(String,
TableDesc, Partition, Path, int, boolean, boolean, boolean, Path,
SemanticAnalyzer$SortBucketRSCtx, DynamicPartitionCtx, ListBucketingCtx,
RowSchema, boolean, Table, Long, boolean, Integer, QB, boolean) At
SemanticAnalyzer.java:8232 of value previously dereferenced in
org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.createFileSinkDesc(String,
TableDesc, Partition, Path, int, boolean, boolean, boolean, Path,
SemanticAnalyzer$SortBucketRSCtx, DynamicPartitionCtx, ListBucketingCtx,
RowSchema, boolean, Table, Long, boolean, Integer, QB, boolean) At
SemanticAnalyzer.java:[line 8225] |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-22015/dev-support/hive-personality.sh
|
| git revision | master / 213de03 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.1 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-22015/yetus/diff-checkstyle-ql.txt
|
| findbugs |
http://104.198.109.242/logs//PreCommit-HIVE-Build-22015/yetus/new-findbugs-ql.html
|
| modules | C: ql itests itests/hive-blobstore U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-22015/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Make bucketing version usage more robust
>
>
> Key: HIVE-21304
> URL: https://issues.apache.org/jira/browse/HIVE-21304
> Project: Hive
> Issue Type: Bug
>Reporter: Deepak Jaiswal
>
[jira] [Commented] (HIVE-23319) multi_insert_partitioned is flaky
[ https://issues.apache.org/jira/browse/HIVE-23319?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17095578#comment-17095578 ] Vineet Garg commented on HIVE-23319: [~kgyrtkirk] I have added the SORT_QUERY_RESULTS to the test file. > multi_insert_partitioned is flaky > - > > Key: HIVE-23319 > URL: https://issues.apache.org/jira/browse/HIVE-23319 > Project: Hive > Issue Type: Bug >Reporter: Zoltan Haindrich >Priority: Major > Attachments: HIVE-23319.1.patch > > > this test shows some incorrect behaviour in a non-deterministcal manner > https://issues.apache.org/jira/browse/HIVE-23317?focusedCommentId=17095238=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-17095238 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-23319) multi_insert_partitioned is flaky
[ https://issues.apache.org/jira/browse/HIVE-23319?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vineet Garg updated HIVE-23319: --- Attachment: HIVE-23319.1.patch > multi_insert_partitioned is flaky > - > > Key: HIVE-23319 > URL: https://issues.apache.org/jira/browse/HIVE-23319 > Project: Hive > Issue Type: Bug >Reporter: Zoltan Haindrich >Priority: Major > Attachments: HIVE-23319.1.patch > > > this test shows some incorrect behaviour in a non-deterministcal manner > https://issues.apache.org/jira/browse/HIVE-23317?focusedCommentId=17095238=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-17095238 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-23319) multi_insert_partitioned is flaky
[ https://issues.apache.org/jira/browse/HIVE-23319?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vineet Garg updated HIVE-23319: --- Status: Patch Available (was: Open) > multi_insert_partitioned is flaky > - > > Key: HIVE-23319 > URL: https://issues.apache.org/jira/browse/HIVE-23319 > Project: Hive > Issue Type: Bug >Reporter: Zoltan Haindrich >Priority: Major > Attachments: HIVE-23319.1.patch > > > this test shows some incorrect behaviour in a non-deterministcal manner > https://issues.apache.org/jira/browse/HIVE-23317?focusedCommentId=17095238=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-17095238 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-23316) Add tests to cover database managed location related DDL and fix minor issues
[
https://issues.apache.org/jira/browse/HIVE-23316?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17095574#comment-17095574
]
Hive QA commented on HIVE-23316:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/13001517/HIVE-23316.01.patch
{color:green}SUCCESS:{color} +1 due to 4 test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 17182 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/22014/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/22014/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-22014/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 13001517 - PreCommit-HIVE-Build
> Add tests to cover database managed location related DDL and fix minor issues
> -
>
> Key: HIVE-23316
> URL: https://issues.apache.org/jira/browse/HIVE-23316
> Project: Hive
> Issue Type: Bug
>Reporter: Miklos Gergely
>Assignee: Miklos Gergely
>Priority: Major
> Attachments: HIVE-23316.01.patch
>
>
> Database managed location was recently introduced, but no tests were added to
> cover it. also the following issues were fixed:
> * ALTER DATABASE ... SET MANAGEDLOCATION ... commands were not handled in a
> separate path as it should, as in DDL each command type have their own
> Analyzer, Desc, and Operation class
> * in case of setting the LOCATION or the MANAGEDLOCATION the location was
> not getting qualified as in the CREATE DATABASE command
> * in case of setting the LOCATION or the MANAGEDLOCATION it was not checked
> if this modification makes the two the same
> * some minor checkstyle issues were fixed as well
> Also the DDL documentation was not modified.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-23335) Investigate why the results have changed for distinct_windowing_no_cbo.q
[ https://issues.apache.org/jira/browse/HIVE-23335?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Miklos Gergely updated HIVE-23335: -- Description: Find out why the result set is different for distinct_windowing_no_cbo.q after moving to TestMiniLlapLocalCliDriver. Check [https://reviews.apache.org/r/72421/#comment308910|https://reviews.apache.org/r/72421/#comment308835] for details. (was: Find out why the result set is different for columnstats_partlvl_dp.q after moving to TestMiniLlapLocalCliDriver. Check [https://reviews.apache.org/r/72421/#comment308875|https://reviews.apache.org/r/72421/#comment308835] for details.) > Investigate why the results have changed for distinct_windowing_no_cbo.q > > > Key: HIVE-23335 > URL: https://issues.apache.org/jira/browse/HIVE-23335 > Project: Hive > Issue Type: Sub-task >Reporter: Miklos Gergely >Priority: Major > > Find out why the result set is different for distinct_windowing_no_cbo.q > after moving to TestMiniLlapLocalCliDriver. Check > [https://reviews.apache.org/r/72421/#comment308910|https://reviews.apache.org/r/72421/#comment308835] > for details. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-23334) Investigate why the results have changed for columnstats_partlvl_dp.q
[ https://issues.apache.org/jira/browse/HIVE-23334?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Miklos Gergely updated HIVE-23334: -- Description: Find out why the result set is different for columnstats_partlvl_dp.q after moving to TestMiniLlapLocalCliDriver. Check [https://reviews.apache.org/r/72421/#comment308875|https://reviews.apache.org/r/72421/#comment308835] for details. (was: Find out why the result set is different for char_udf1.q after moving to TestMiniLlapLocalCliDriver. Check [https://reviews.apache.org/r/72421/#comment308875|https://reviews.apache.org/r/72421/#comment308835] for details.) > Investigate why the results have changed for columnstats_partlvl_dp.q > - > > Key: HIVE-23334 > URL: https://issues.apache.org/jira/browse/HIVE-23334 > Project: Hive > Issue Type: Sub-task >Reporter: Miklos Gergely >Priority: Major > > Find out why the result set is different for columnstats_partlvl_dp.q after > moving to TestMiniLlapLocalCliDriver. Check > [https://reviews.apache.org/r/72421/#comment308875|https://reviews.apache.org/r/72421/#comment308835] > for details. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-23333) Investigate why the results have changed for char_udf1.q
[ https://issues.apache.org/jira/browse/HIVE-2?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Miklos Gergely updated HIVE-2: -- Description: Find out why the result set is different for char_udf1.q after moving to TestMiniLlapLocalCliDriver. Check [https://reviews.apache.org/r/72421/#comment308875|https://reviews.apache.org/r/72421/#comment308835] for details. (was: Find out why the result set is different for cbo_rp_udaf_percentile_approx_23.q after moving to TestMiniLlapLocalCliDriver. Check [https://reviews.apache.org/r/72421/#comment308875|https://reviews.apache.org/r/72421/#comment308835] for details.) > Investigate why the results have changed for char_udf1.q > > > Key: HIVE-2 > URL: https://issues.apache.org/jira/browse/HIVE-2 > Project: Hive > Issue Type: Sub-task >Reporter: Miklos Gergely >Priority: Major > > Find out why the result set is different for char_udf1.q after moving to > TestMiniLlapLocalCliDriver. Check > [https://reviews.apache.org/r/72421/#comment308875|https://reviews.apache.org/r/72421/#comment308835] > for details. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-23332) Investigate why the results have changed for cbo_rp_udaf_percentile_approx_23.q
[ https://issues.apache.org/jira/browse/HIVE-23332?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Miklos Gergely updated HIVE-23332: -- Description: Find out why the result set is different for cbo_rp_udaf_percentile_approx_23.q after moving to TestMiniLlapLocalCliDriver. Check [https://reviews.apache.org/r/72421/#comment308838|https://reviews.apache.org/r/72421/#comment308835] for details. (was: Find out why the result set is different for authorization_9.q after moving to TestMiniLlapLocalCliDriver. Check [https://reviews.apache.org/r/72421/#comment308838|https://reviews.apache.org/r/72421/#comment308835] for details.) > Investigate why the results have changed for > cbo_rp_udaf_percentile_approx_23.q > --- > > Key: HIVE-23332 > URL: https://issues.apache.org/jira/browse/HIVE-23332 > Project: Hive > Issue Type: Sub-task >Reporter: Miklos Gergely >Priority: Major > > Find out why the result set is different for > cbo_rp_udaf_percentile_approx_23.q after moving to > TestMiniLlapLocalCliDriver. Check > [https://reviews.apache.org/r/72421/#comment308838|https://reviews.apache.org/r/72421/#comment308835] > for details. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-23332) Investigate why the results have changed for cbo_rp_udaf_percentile_approx_23.q
[ https://issues.apache.org/jira/browse/HIVE-23332?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Miklos Gergely updated HIVE-23332: -- Description: Find out why the result set is different for cbo_rp_udaf_percentile_approx_23.q after moving to TestMiniLlapLocalCliDriver. Check [https://reviews.apache.org/r/72421/#comment308875|https://reviews.apache.org/r/72421/#comment308835] for details. (was: Find out why the result set is different for cbo_rp_udaf_percentile_approx_23.q after moving to TestMiniLlapLocalCliDriver. Check [https://reviews.apache.org/r/72421/#comment308838|https://reviews.apache.org/r/72421/#comment308835] for details.) > Investigate why the results have changed for > cbo_rp_udaf_percentile_approx_23.q > --- > > Key: HIVE-23332 > URL: https://issues.apache.org/jira/browse/HIVE-23332 > Project: Hive > Issue Type: Sub-task >Reporter: Miklos Gergely >Priority: Major > > Find out why the result set is different for > cbo_rp_udaf_percentile_approx_23.q after moving to > TestMiniLlapLocalCliDriver. Check > [https://reviews.apache.org/r/72421/#comment308875|https://reviews.apache.org/r/72421/#comment308835] > for details. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-23325) Clean up cleanup tasks for TxnHandler/CompactionTxnHandler
[ https://issues.apache.org/jira/browse/HIVE-23325?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marton Bod updated HIVE-23325: -- Attachment: (was: HIVE-23325.1.patch) > Clean up cleanup tasks for TxnHandler/CompactionTxnHandler > -- > > Key: HIVE-23325 > URL: https://issues.apache.org/jira/browse/HIVE-23325 > Project: Hive > Issue Type: Improvement >Reporter: Marton Bod >Assignee: Marton Bod >Priority: Major > Attachments: HIVE-23325.1.patch > > > Currently there are multiple ways to clean up ACID related stuff: > * AcidHouseKeeperService > * AcidWriteSetService > * AcidCompactionHistoryService > * Initiator > * Cleaner > * etc. > We should consolidate them where possible and improve logging. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-23325) Clean up cleanup tasks for TxnHandler/CompactionTxnHandler
[ https://issues.apache.org/jira/browse/HIVE-23325?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marton Bod updated HIVE-23325: -- Attachment: HIVE-23325.1.patch > Clean up cleanup tasks for TxnHandler/CompactionTxnHandler > -- > > Key: HIVE-23325 > URL: https://issues.apache.org/jira/browse/HIVE-23325 > Project: Hive > Issue Type: Improvement >Reporter: Marton Bod >Assignee: Marton Bod >Priority: Major > Attachments: HIVE-23325.1.patch > > > Currently there are multiple ways to clean up ACID related stuff: > * AcidHouseKeeperService > * AcidWriteSetService > * AcidCompactionHistoryService > * Initiator > * Cleaner > * etc. > We should consolidate them where possible and improve logging. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-23331) Investigate why the results have changed for authorization_9.q
[ https://issues.apache.org/jira/browse/HIVE-23331?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Miklos Gergely updated HIVE-23331: -- Description: Find out why the result set is different for authorization_9.q after moving to TestMiniLlapLocalCliDriver. Check [https://reviews.apache.org/r/72421/#comment308838|https://reviews.apache.org/r/72421/#comment308835] for details. (was: Find out why the result set is different for authorization_9.q after moving to TestMiniLlapLocalCliDriver. Check [https://reviews.apache.org/r/72421/#comment308835] for details.) > Investigate why the results have changed for authorization_9.q > -- > > Key: HIVE-23331 > URL: https://issues.apache.org/jira/browse/HIVE-23331 > Project: Hive > Issue Type: Sub-task >Reporter: Miklos Gergely >Priority: Major > > Find out why the result set is different for authorization_9.q after moving > to TestMiniLlapLocalCliDriver. Check > [https://reviews.apache.org/r/72421/#comment308838|https://reviews.apache.org/r/72421/#comment308835] > for details. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-23331) Investigate why the results have changed for authorization_9.q
[ https://issues.apache.org/jira/browse/HIVE-23331?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Miklos Gergely updated HIVE-23331: -- Description: Find out why the result set is different for authorization_9.q after moving to TestMiniLlapLocalCliDriver. Check [https://reviews.apache.org/r/72421/#comment308835] for details. (was: Find out why the result set is different for decimal_udf.q after moving to TestMiniLlapLocalCliDriver. Check [https://reviews.apache.org/r/72421/#comment308835] for details.) > Investigate why the results have changed for authorization_9.q > -- > > Key: HIVE-23331 > URL: https://issues.apache.org/jira/browse/HIVE-23331 > Project: Hive > Issue Type: Sub-task >Reporter: Miklos Gergely >Priority: Major > > Find out why the result set is different for authorization_9.q after moving > to TestMiniLlapLocalCliDriver. Check > [https://reviews.apache.org/r/72421/#comment308835] for details. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-23330) Investigate why the results have changed for decimal_udf.q
[ https://issues.apache.org/jira/browse/HIVE-23330?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Miklos Gergely updated HIVE-23330: -- Description: Find out why the result set is different for decimal_udf.q after moving to TestMiniLlapLocalCliDriver. Check [https://reviews.apache.org/r/72421/#comment308835] for details. (was: Find out why the result set is different for correlationoptimizer14.q after moving to TestMiniLlapLocalCliDriver. Check [https://reviews.apache.org/r/72421/#comment308835] for details.) > Investigate why the results have changed for decimal_udf.q > -- > > Key: HIVE-23330 > URL: https://issues.apache.org/jira/browse/HIVE-23330 > Project: Hive > Issue Type: Sub-task >Reporter: Miklos Gergely >Priority: Major > > Find out why the result set is different for decimal_udf.q after moving to > TestMiniLlapLocalCliDriver. Check > [https://reviews.apache.org/r/72421/#comment308835] for details. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-23316) Add tests to cover database managed location related DDL and fix minor issues
[
https://issues.apache.org/jira/browse/HIVE-23316?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17095547#comment-17095547
]
Hive QA commented on HIVE-23316:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 9m
46s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
0s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
38s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
33s{color} | {color:blue} ql in master has 1528 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
54s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
25s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
3s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
3s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
38s{color} | {color:green} ql: The patch generated 0 new + 17 unchanged - 10
fixed = 17 total (was 27) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
43s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
52s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
14s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 24m 23s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-22014/dev-support/hive-personality.sh
|
| git revision | master / 213de03 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.1 |
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-22014/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Add tests to cover database managed location related DDL and fix minor issues
> -
>
> Key: HIVE-23316
> URL: https://issues.apache.org/jira/browse/HIVE-23316
> Project: Hive
> Issue Type: Bug
>Reporter: Miklos Gergely
>Assignee: Miklos Gergely
>Priority: Major
> Attachments: HIVE-23316.01.patch
>
>
> Database managed location was recently introduced, but no tests were added to
> cover it. also the following issues were fixed:
> * ALTER DATABASE ... SET MANAGEDLOCATION ... commands were not handled in a
> separate path as it should, as in DDL each command type have their own
> Analyzer, Desc, and Operation class
> * in case of setting the LOCATION or the MANAGEDLOCATION the location was
> not getting qualified as in the CREATE DATABASE command
> * in case of setting the LOCATION or the MANAGEDLOCATION it was not checked
> if this modification makes the two the same
> * some minor checkstyle issues were fixed as well
> Also the DDL documentation was not modified.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-23318) TxnHandler need not delete from MATERIALIZATION_REBUILD_LOCKS on need basis
[
https://issues.apache.org/jira/browse/HIVE-23318?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17095546#comment-17095546
]
Jesus Camacho Rodriguez commented on HIVE-23318:
[~rajesh.balamohan], [~ashutoshc], there are two places where we call this.
- One is periodically to find any dangling locks through the materializations
rebuild lock cleaner task. That one can be disabled by removing the class
{{org.apache.hadoop.hive.metastore.MaterializationsRebuildLockCleanerTask}}
from {{metastore.task.threads.remote}}.
- The other one is at commit transaction time. That one is probably the one you
are seeing here repeatedly, since it is executed for each transaction. The
problem is that {{commitTxn}} does not have any information about whether the
table for which you are committing is a materialized view or not.
The right way to fix this would probably be changing the thrift
{{CommitTxnRequest}} to include a new optional field ({{isMaterialization}}?)
which should be set to true when the request is created for materialized view
rebuild; otherwise you would have to retrieve the table from the RDBMS to check
it, which would defeat the purpose of the fix.
That would help all queries independently on whether the feature is used or
not. Disabling it via toggle seems risky because any HS2 that accidentally uses
the feature can create this locks and leave them dangling.
> TxnHandler need not delete from MATERIALIZATION_REBUILD_LOCKS on need basis
> ---
>
> Key: HIVE-23318
> URL: https://issues.apache.org/jira/browse/HIVE-23318
> Project: Hive
> Issue Type: Improvement
> Components: Hive
>Reporter: Rajesh Balamohan
>Priority: Minor
>
> Observed the following queries when materialized view or any of its feature
> was not used.
> TxnHandler need not clear this part of txn commit. It would help in reducing
> the sql parsing time in server side as well.
> {noformat}
> Gdelete from MATERIALIZATION_REBUILD_LOCKS where mrl_txn_id = 120398082
> Gdelete from MATERIALIZATION_REBUILD_LOCKS where mrl_txn_id = 120398084
> Gdelete from MATERIALIZATION_REBUILD_LOCKS where mrl_txn_id = 120398112
> {noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-23325) Clean up cleanup tasks for TxnHandler/CompactionTxnHandler
[ https://issues.apache.org/jira/browse/HIVE-23325?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marton Bod updated HIVE-23325: -- Attachment: HIVE-23325.1.patch Status: Patch Available (was: Open) > Clean up cleanup tasks for TxnHandler/CompactionTxnHandler > -- > > Key: HIVE-23325 > URL: https://issues.apache.org/jira/browse/HIVE-23325 > Project: Hive > Issue Type: Improvement >Reporter: Marton Bod >Assignee: Marton Bod >Priority: Major > Attachments: HIVE-23325.1.patch > > > Currently there are multiple ways to clean up ACID related stuff: > * AcidHouseKeeperService > * AcidWriteSetService > * AcidCompactionHistoryService > * Initiator > * Cleaner > * etc. > We should consolidate them where possible and improve logging. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-23315) Remove empty line from the end of SHOW EXTENDED TABLES and SHOW MATERIALIZED VIEWS
[
https://issues.apache.org/jira/browse/HIVE-23315?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17095514#comment-17095514
]
Hive QA commented on HIVE-23315:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/13001512/HIVE-23315.01.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 17175 tests
executed
*Failed tests:*
{noformat}
TestStatsReplicationScenariosACID - did not produce a TEST-*.xml file (likely
timed out) (batchId=184)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/22013/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/22013/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-22013/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 1 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 13001512 - PreCommit-HIVE-Build
> Remove empty line from the end of SHOW EXTENDED TABLES and SHOW MATERIALIZED
> VIEWS
> --
>
> Key: HIVE-23315
> URL: https://issues.apache.org/jira/browse/HIVE-23315
> Project: Hive
> Issue Type: Bug
> Components: HiveServer2
>Reporter: Miklos Gergely
>Assignee: Miklos Gergely
>Priority: Major
> Attachments: HIVE-23315.01.patch
>
>
> At the end of each SHOW EXTENDED TABLES; and SHOW MATERIALIZED VIEWS; command
> there is an empty line like this:
> {code:java}
> +++
> | tab_name | table_type |
> +++
> | sample_07 | MANAGED_TABLE |
> | sample_08 | MANAGED_TABLE |
> | web_logs | MANAGED_TABLE |
> | | NULL |
> +++
> {code}
> It should be removed.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-23318) TxnHandler need not delete from MATERIALIZATION_REBUILD_LOCKS on need basis
[
https://issues.apache.org/jira/browse/HIVE-23318?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17095512#comment-17095512
]
Ashutosh Chauhan commented on HIVE-23318:
-
[~jcamachorodriguez] safe to remove these sql statements?
> TxnHandler need not delete from MATERIALIZATION_REBUILD_LOCKS on need basis
> ---
>
> Key: HIVE-23318
> URL: https://issues.apache.org/jira/browse/HIVE-23318
> Project: Hive
> Issue Type: Improvement
> Components: Hive
>Reporter: Rajesh Balamohan
>Priority: Minor
>
> Observed the following queries when materialized view or any of its feature
> was not used.
> TxnHandler need not clear this part of txn commit. It would help in reducing
> the sql parsing time in server side as well.
> {noformat}
> Gdelete from MATERIALIZATION_REBUILD_LOCKS where mrl_txn_id = 120398082
> Gdelete from MATERIALIZATION_REBUILD_LOCKS where mrl_txn_id = 120398084
> Gdelete from MATERIALIZATION_REBUILD_LOCKS where mrl_txn_id = 120398112
> {noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-23319) multi_insert_partitioned is flaky
[ https://issues.apache.org/jira/browse/HIVE-23319?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17095503#comment-17095503 ] Zoltan Haindrich commented on HIVE-23319: - I'm starting to think that it's not because of HIVE-23216; and it was there before - it can't be fixed with a SORT becayse the insert overwrite seem to have not changed the table contents. > multi_insert_partitioned is flaky > - > > Key: HIVE-23319 > URL: https://issues.apache.org/jira/browse/HIVE-23319 > Project: Hive > Issue Type: Bug >Reporter: Zoltan Haindrich >Priority: Major > > this test shows some incorrect behaviour in a non-deterministcal manner > https://issues.apache.org/jira/browse/HIVE-23317?focusedCommentId=17095238=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-17095238 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-23266) Remove QueryWrapper from ObjectStore
[
https://issues.apache.org/jira/browse/HIVE-23266?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Mollitor updated HIVE-23266:
--
Attachment: HIVE-23266.6.patch
> Remove QueryWrapper from ObjectStore
>
>
> Key: HIVE-23266
> URL: https://issues.apache.org/jira/browse/HIVE-23266
> Project: Hive
> Issue Type: Improvement
>Reporter: David Mollitor
>Assignee: David Mollitor
>Priority: Major
> Attachments: HIVE-23266.1.patch, HIVE-23266.2.patch,
> HIVE-23266.2.patch, HIVE-23266.3.patch, HIVE-23266.4.patch,
> HIVE-23266.5.patch, HIVE-23266.6.patch
>
>
> There is currently a utility called {{QueryWrapper}} that makes a normal
> {{Query}} auto-closable. However, {{Query}} is now in fact already
> auto-closing, so there is no need for this class. In trying to remove it, I
> realized that this wrapper was being passed around in pretty convoluted ways
> and also it was sometimes being created in a {{try-with-resources}} block but
> then never actually used in any way.
> Remove the {{QueryWrapper}} from the class and simplify some of the DB
> interactions.
> https://github.com/apache/hive/blob/master/standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/ObjectStore.java#L178
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-23282) Reduce number of DB calls in ObjectStore::getPartitionsByExprInternal
[ https://issues.apache.org/jira/browse/HIVE-23282?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Attila Magyar updated HIVE-23282: - Status: Patch Available (was: Open) > Reduce number of DB calls in ObjectStore::getPartitionsByExprInternal > - > > Key: HIVE-23282 > URL: https://issues.apache.org/jira/browse/HIVE-23282 > Project: Hive > Issue Type: Improvement > Components: Metastore >Reporter: Rajesh Balamohan >Assignee: Attila Magyar >Priority: Major > Attachments: HIVE-23282.1.patch, HIVE-23282.3.patch, > HIVE-23282.5.patch, image-2020-04-23-14-07-06-077.png > > > ObjectStore::getPartitionsByExprInternal internally uses Table information > for getting partitionKeys, table, catalog name. > > For this, it ends up populating entire table data from DB (including skew > column, parameters, sort, bucket cols etc). This makes it a lot more > expensive call. It would be good to check if MTable itself can be used > instead of Table. > [https://github.com/apache/hive/blob/master/standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/ObjectStore.java#L3327] > [https://github.com/apache/hive/blob/master/standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/ObjectStore.java#L3669] > > !image-2020-04-23-14-07-06-077.png|width=665,height=592! -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-23282) Reduce number of DB calls in ObjectStore::getPartitionsByExprInternal
[ https://issues.apache.org/jira/browse/HIVE-23282?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Attila Magyar updated HIVE-23282: - Attachment: (was: HIVE-23282.5.patch) > Reduce number of DB calls in ObjectStore::getPartitionsByExprInternal > - > > Key: HIVE-23282 > URL: https://issues.apache.org/jira/browse/HIVE-23282 > Project: Hive > Issue Type: Improvement > Components: Metastore >Reporter: Rajesh Balamohan >Assignee: Attila Magyar >Priority: Major > Attachments: HIVE-23282.1.patch, HIVE-23282.3.patch, > HIVE-23282.5.patch, image-2020-04-23-14-07-06-077.png > > > ObjectStore::getPartitionsByExprInternal internally uses Table information > for getting partitionKeys, table, catalog name. > > For this, it ends up populating entire table data from DB (including skew > column, parameters, sort, bucket cols etc). This makes it a lot more > expensive call. It would be good to check if MTable itself can be used > instead of Table. > [https://github.com/apache/hive/blob/master/standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/ObjectStore.java#L3327] > [https://github.com/apache/hive/blob/master/standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/ObjectStore.java#L3669] > > !image-2020-04-23-14-07-06-077.png|width=665,height=592! -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-23282) Reduce number of DB calls in ObjectStore::getPartitionsByExprInternal
[ https://issues.apache.org/jira/browse/HIVE-23282?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Attila Magyar updated HIVE-23282: - Attachment: HIVE-23282.5.patch > Reduce number of DB calls in ObjectStore::getPartitionsByExprInternal > - > > Key: HIVE-23282 > URL: https://issues.apache.org/jira/browse/HIVE-23282 > Project: Hive > Issue Type: Improvement > Components: Metastore >Reporter: Rajesh Balamohan >Assignee: Attila Magyar >Priority: Major > Attachments: HIVE-23282.1.patch, HIVE-23282.3.patch, > HIVE-23282.5.patch, image-2020-04-23-14-07-06-077.png > > > ObjectStore::getPartitionsByExprInternal internally uses Table information > for getting partitionKeys, table, catalog name. > > For this, it ends up populating entire table data from DB (including skew > column, parameters, sort, bucket cols etc). This makes it a lot more > expensive call. It would be good to check if MTable itself can be used > instead of Table. > [https://github.com/apache/hive/blob/master/standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/ObjectStore.java#L3327] > [https://github.com/apache/hive/blob/master/standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/ObjectStore.java#L3669] > > !image-2020-04-23-14-07-06-077.png|width=665,height=592! -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-23282) Reduce number of DB calls in ObjectStore::getPartitionsByExprInternal
[ https://issues.apache.org/jira/browse/HIVE-23282?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Attila Magyar updated HIVE-23282: - Attachment: (was: HIVE-23282.5.patch) > Reduce number of DB calls in ObjectStore::getPartitionsByExprInternal > - > > Key: HIVE-23282 > URL: https://issues.apache.org/jira/browse/HIVE-23282 > Project: Hive > Issue Type: Improvement > Components: Metastore >Reporter: Rajesh Balamohan >Assignee: Attila Magyar >Priority: Major > Attachments: HIVE-23282.1.patch, HIVE-23282.3.patch, > HIVE-23282.5.patch, image-2020-04-23-14-07-06-077.png > > > ObjectStore::getPartitionsByExprInternal internally uses Table information > for getting partitionKeys, table, catalog name. > > For this, it ends up populating entire table data from DB (including skew > column, parameters, sort, bucket cols etc). This makes it a lot more > expensive call. It would be good to check if MTable itself can be used > instead of Table. > [https://github.com/apache/hive/blob/master/standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/ObjectStore.java#L3327] > [https://github.com/apache/hive/blob/master/standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/ObjectStore.java#L3669] > > !image-2020-04-23-14-07-06-077.png|width=665,height=592! -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-23282) Reduce number of DB calls in ObjectStore::getPartitionsByExprInternal
[ https://issues.apache.org/jira/browse/HIVE-23282?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Attila Magyar updated HIVE-23282: - Attachment: HIVE-23282.5.patch > Reduce number of DB calls in ObjectStore::getPartitionsByExprInternal > - > > Key: HIVE-23282 > URL: https://issues.apache.org/jira/browse/HIVE-23282 > Project: Hive > Issue Type: Improvement > Components: Metastore >Reporter: Rajesh Balamohan >Assignee: Attila Magyar >Priority: Major > Attachments: HIVE-23282.1.patch, HIVE-23282.3.patch, > HIVE-23282.5.patch, image-2020-04-23-14-07-06-077.png > > > ObjectStore::getPartitionsByExprInternal internally uses Table information > for getting partitionKeys, table, catalog name. > > For this, it ends up populating entire table data from DB (including skew > column, parameters, sort, bucket cols etc). This makes it a lot more > expensive call. It would be good to check if MTable itself can be used > instead of Table. > [https://github.com/apache/hive/blob/master/standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/ObjectStore.java#L3327] > [https://github.com/apache/hive/blob/master/standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/ObjectStore.java#L3669] > > !image-2020-04-23-14-07-06-077.png|width=665,height=592! -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-23282) Reduce number of DB calls in ObjectStore::getPartitionsByExprInternal
[ https://issues.apache.org/jira/browse/HIVE-23282?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Attila Magyar updated HIVE-23282: - Status: Open (was: Patch Available) > Reduce number of DB calls in ObjectStore::getPartitionsByExprInternal > - > > Key: HIVE-23282 > URL: https://issues.apache.org/jira/browse/HIVE-23282 > Project: Hive > Issue Type: Improvement > Components: Metastore >Reporter: Rajesh Balamohan >Assignee: Attila Magyar >Priority: Major > Attachments: HIVE-23282.1.patch, HIVE-23282.3.patch, > HIVE-23282.5.patch, image-2020-04-23-14-07-06-077.png > > > ObjectStore::getPartitionsByExprInternal internally uses Table information > for getting partitionKeys, table, catalog name. > > For this, it ends up populating entire table data from DB (including skew > column, parameters, sort, bucket cols etc). This makes it a lot more > expensive call. It would be good to check if MTable itself can be used > instead of Table. > [https://github.com/apache/hive/blob/master/standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/ObjectStore.java#L3327] > [https://github.com/apache/hive/blob/master/standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/ObjectStore.java#L3669] > > !image-2020-04-23-14-07-06-077.png|width=665,height=592! -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-23319) multi_insert_partitioned is flaky
[ https://issues.apache.org/jira/browse/HIVE-23319?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17095495#comment-17095495 ] Vineet Garg commented on HIVE-23319: [~kgyrtkirk] It could be due to HIVE-23216. Will the fix for this be to introduce SORT_QUERY_RESULTS? > multi_insert_partitioned is flaky > - > > Key: HIVE-23319 > URL: https://issues.apache.org/jira/browse/HIVE-23319 > Project: Hive > Issue Type: Bug >Reporter: Zoltan Haindrich >Priority: Major > > this test shows some incorrect behaviour in a non-deterministcal manner > https://issues.apache.org/jira/browse/HIVE-23317?focusedCommentId=17095238=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-17095238 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-23216) Add new api as replacement of get_partitions_by_expr to return PartitionSpec instead of Partitions
[ https://issues.apache.org/jira/browse/HIVE-23216?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17095492#comment-17095492 ] Vineet Garg commented on HIVE-23216: [~kgyrtkirk] I apologize for causing tests to be flaky. I was under the impression that Miklos patch fixed all partition_wise_fileformat tests but I was wrong as it only fixes one of them. Since I didn't see any failure afterwards I thought it was safe to push this change. Thanks for taking care of this. > Add new api as replacement of get_partitions_by_expr to return PartitionSpec > instead of Partitions > -- > > Key: HIVE-23216 > URL: https://issues.apache.org/jira/browse/HIVE-23216 > Project: Hive > Issue Type: Improvement > Components: Metastore >Affects Versions: 4.0.0 >Reporter: Vineet Garg >Assignee: Vineet Garg >Priority: Major > Fix For: 4.0.0 > > Attachments: HIVE-23216.1.patch, HIVE-23216.2.patch, > HIVE-23216.3.patch, HIVE-23216.4.patch, HIVE-23216.5.patch, > HIVE-23216.6.patch, HIVE-23216.7.patch, HIVE-23216.8.patch, HIVE-23216.9.patch > > Time Spent: 1h 50m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-23305) NullPointerException in LlapTaskSchedulerService addNode due to race condition
[
https://issues.apache.org/jira/browse/HIVE-23305?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Attila Magyar updated HIVE-23305:
-
Status: Patch Available (was: Open)
> NullPointerException in LlapTaskSchedulerService addNode due to race condition
> --
>
> Key: HIVE-23305
> URL: https://issues.apache.org/jira/browse/HIVE-23305
> Project: Hive
> Issue Type: Bug
> Components: Hive
>Reporter: Attila Magyar
>Assignee: Attila Magyar
>Priority: Major
> Fix For: 4.0.0
>
> Attachments: HIVE-23305.1.patch
>
>
> {code:java}
> java.lang.NullPointerException at
> org.apache.hadoop.hive.llap.tezplugins.LlapTaskSchedulerService.addNode(LlapTaskSchedulerService.java:1575)
> at
> org.apache.hadoop.hive.llap.tezplugins.LlapTaskSchedulerService.registerAndAddNode(LlapTaskSchedulerService.java:1566)
> at
> org.apache.hadoop.hive.llap.tezplugins.LlapTaskSchedulerService.access$1800(LlapTaskSchedulerService.java:128)
> at
> org.apache.hadoop.hive.llap.tezplugins.LlapTaskSchedulerService$NodeStateChangeListener.onCreate(LlapTaskSchedulerService.java:831)
> at
> org.apache.hadoop.hive.llap.tezplugins.LlapTaskSchedulerService$NodeStateChangeListener.onCreate(LlapTaskSchedulerService.java:823)
> at
> org.apache.hadoop.hive.registry.impl.ZkRegistryBase$InstanceStateChangeListener.childEvent(ZkRegistryBase.java:612)
> at {code}
> The above exception happens when a node registers too fast, before the active
> activeInstances field was initialized.
>
> The registry is started and the listener is registered before initializing
> activeInstances.
> {code:java}
> registry.start();
> registry.registerStateChangeListener(new NodeStateChangeListener());
> activeInstances = registry.getInstances(); {code}
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-23108) Cleanup HiveBaseResultSet.java
[
https://issues.apache.org/jira/browse/HIVE-23108?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Mollitor updated HIVE-23108:
--
Attachment: HIVE-23108.5.patch
> Cleanup HiveBaseResultSet.java
> --
>
> Key: HIVE-23108
> URL: https://issues.apache.org/jira/browse/HIVE-23108
> Project: Hive
> Issue Type: Improvement
>Reporter: David Mollitor
>Assignee: David Mollitor
>Priority: Minor
> Attachments: HIVE-23108.1.patch, HIVE-23108.2.patch,
> HIVE-23108.3.patch, HIVE-23108.4.patch, HIVE-23108.5.patch
>
>
> * Unify the code (there are several implementations of the same thing)
> * Added better error messages
> * In some cases, the code was throwing RuntimeExceptions which is against
> the JDBC Spec
> * Make findColumn a bit more streamlined
> * Remove non-javadoc comments
> * Add {{@Override}} annotations where appropriate
> * Formatting changes
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-23305) NullPointerException in LlapTaskSchedulerService addNode due to race condition
[
https://issues.apache.org/jira/browse/HIVE-23305?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Attila Magyar updated HIVE-23305:
-
Attachment: HIVE-23305.1.patch
> NullPointerException in LlapTaskSchedulerService addNode due to race condition
> --
>
> Key: HIVE-23305
> URL: https://issues.apache.org/jira/browse/HIVE-23305
> Project: Hive
> Issue Type: Bug
> Components: Hive
>Reporter: Attila Magyar
>Assignee: Attila Magyar
>Priority: Major
> Fix For: 4.0.0
>
> Attachments: HIVE-23305.1.patch
>
>
> {code:java}
> java.lang.NullPointerException at
> org.apache.hadoop.hive.llap.tezplugins.LlapTaskSchedulerService.addNode(LlapTaskSchedulerService.java:1575)
> at
> org.apache.hadoop.hive.llap.tezplugins.LlapTaskSchedulerService.registerAndAddNode(LlapTaskSchedulerService.java:1566)
> at
> org.apache.hadoop.hive.llap.tezplugins.LlapTaskSchedulerService.access$1800(LlapTaskSchedulerService.java:128)
> at
> org.apache.hadoop.hive.llap.tezplugins.LlapTaskSchedulerService$NodeStateChangeListener.onCreate(LlapTaskSchedulerService.java:831)
> at
> org.apache.hadoop.hive.llap.tezplugins.LlapTaskSchedulerService$NodeStateChangeListener.onCreate(LlapTaskSchedulerService.java:823)
> at
> org.apache.hadoop.hive.registry.impl.ZkRegistryBase$InstanceStateChangeListener.childEvent(ZkRegistryBase.java:612)
> at {code}
> The above exception happens when a node registers too fast, before the active
> activeInstances field was initialized.
>
> The registry is started and the listener is registered before initializing
> activeInstances.
> {code:java}
> registry.start();
> registry.registerStateChangeListener(new NodeStateChangeListener());
> activeInstances = registry.getInstances(); {code}
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-23305) NullPointerException in LlapTaskSchedulerService addNode due to race condition
[
https://issues.apache.org/jira/browse/HIVE-23305?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Attila Magyar updated HIVE-23305:
-
Attachment: (was: HIVE-23305.1.patch)
> NullPointerException in LlapTaskSchedulerService addNode due to race condition
> --
>
> Key: HIVE-23305
> URL: https://issues.apache.org/jira/browse/HIVE-23305
> Project: Hive
> Issue Type: Bug
> Components: Hive
>Reporter: Attila Magyar
>Assignee: Attila Magyar
>Priority: Major
> Fix For: 4.0.0
>
>
> {code:java}
> java.lang.NullPointerException at
> org.apache.hadoop.hive.llap.tezplugins.LlapTaskSchedulerService.addNode(LlapTaskSchedulerService.java:1575)
> at
> org.apache.hadoop.hive.llap.tezplugins.LlapTaskSchedulerService.registerAndAddNode(LlapTaskSchedulerService.java:1566)
> at
> org.apache.hadoop.hive.llap.tezplugins.LlapTaskSchedulerService.access$1800(LlapTaskSchedulerService.java:128)
> at
> org.apache.hadoop.hive.llap.tezplugins.LlapTaskSchedulerService$NodeStateChangeListener.onCreate(LlapTaskSchedulerService.java:831)
> at
> org.apache.hadoop.hive.llap.tezplugins.LlapTaskSchedulerService$NodeStateChangeListener.onCreate(LlapTaskSchedulerService.java:823)
> at
> org.apache.hadoop.hive.registry.impl.ZkRegistryBase$InstanceStateChangeListener.childEvent(ZkRegistryBase.java:612)
> at {code}
> The above exception happens when a node registers too fast, before the active
> activeInstances field was initialized.
>
> The registry is started and the listener is registered before initializing
> activeInstances.
> {code:java}
> registry.start();
> registry.registerStateChangeListener(new NodeStateChangeListener());
> activeInstances = registry.getInstances(); {code}
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-23305) NullPointerException in LlapTaskSchedulerService addNode due to race condition
[
https://issues.apache.org/jira/browse/HIVE-23305?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Attila Magyar updated HIVE-23305:
-
Status: Open (was: Patch Available)
> NullPointerException in LlapTaskSchedulerService addNode due to race condition
> --
>
> Key: HIVE-23305
> URL: https://issues.apache.org/jira/browse/HIVE-23305
> Project: Hive
> Issue Type: Bug
> Components: Hive
>Reporter: Attila Magyar
>Assignee: Attila Magyar
>Priority: Major
> Fix For: 4.0.0
>
>
> {code:java}
> java.lang.NullPointerException at
> org.apache.hadoop.hive.llap.tezplugins.LlapTaskSchedulerService.addNode(LlapTaskSchedulerService.java:1575)
> at
> org.apache.hadoop.hive.llap.tezplugins.LlapTaskSchedulerService.registerAndAddNode(LlapTaskSchedulerService.java:1566)
> at
> org.apache.hadoop.hive.llap.tezplugins.LlapTaskSchedulerService.access$1800(LlapTaskSchedulerService.java:128)
> at
> org.apache.hadoop.hive.llap.tezplugins.LlapTaskSchedulerService$NodeStateChangeListener.onCreate(LlapTaskSchedulerService.java:831)
> at
> org.apache.hadoop.hive.llap.tezplugins.LlapTaskSchedulerService$NodeStateChangeListener.onCreate(LlapTaskSchedulerService.java:823)
> at
> org.apache.hadoop.hive.registry.impl.ZkRegistryBase$InstanceStateChangeListener.childEvent(ZkRegistryBase.java:612)
> at {code}
> The above exception happens when a node registers too fast, before the active
> activeInstances field was initialized.
>
> The registry is started and the listener is registered before initializing
> activeInstances.
> {code:java}
> registry.start();
> registry.registerStateChangeListener(new NodeStateChangeListener());
> activeInstances = registry.getInstances(); {code}
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-23315) Remove empty line from the end of SHOW EXTENDED TABLES and SHOW MATERIALIZED VIEWS
[
https://issues.apache.org/jira/browse/HIVE-23315?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17095489#comment-17095489
]
Hive QA commented on HIVE-23315:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 9m
53s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
59s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
39s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
38s{color} | {color:blue} ql in master has 1528 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
51s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
24s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
59s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
59s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
40s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
41s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
54s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
14s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 24m 19s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-22013/dev-support/hive-personality.sh
|
| git revision | master / 213de03 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.1 |
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-22013/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Remove empty line from the end of SHOW EXTENDED TABLES and SHOW MATERIALIZED
> VIEWS
> --
>
> Key: HIVE-23315
> URL: https://issues.apache.org/jira/browse/HIVE-23315
> Project: Hive
> Issue Type: Bug
> Components: HiveServer2
>Reporter: Miklos Gergely
>Assignee: Miklos Gergely
>Priority: Major
> Attachments: HIVE-23315.01.patch
>
>
> At the end of each SHOW EXTENDED TABLES; and SHOW MATERIALIZED VIEWS; command
> there is an empty line like this:
> {code:java}
> +++
> | tab_name | table_type |
> +++
> | sample_07 | MANAGED_TABLE |
> | sample_08 | MANAGED_TABLE |
> | web_logs | MANAGED_TABLE |
> | | NULL |
> +++
> {code}
> It should be removed.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-23117) Review of HiveStatement Class
[
https://issues.apache.org/jira/browse/HIVE-23117?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Mollitor updated HIVE-23117:
--
Attachment: HIVE-23117.3.patch
> Review of HiveStatement Class
> -
>
> Key: HIVE-23117
> URL: https://issues.apache.org/jira/browse/HIVE-23117
> Project: Hive
> Issue Type: Improvement
> Components: JDBC
>Reporter: David Mollitor
>Assignee: David Mollitor
>Priority: Minor
> Attachments: HIVE-23117.1.patch, HIVE-23117.2.patch,
> HIVE-23117.2.patch, HIVE-23117.3.patch
>
>
> * Remove unused instance variable(s)
> * Remove non-JavaDoc comments
> * Make inPlaceUpdateStream Optional (and remove NO-OP class) (inconsistent
> behavior with 'null' values)
> * {{getQueryTimeout()}} returns incorrect value
> * Unify and improve Exception messages
> * Checkstyle fixes
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-23117) Review of HiveStatement Class
[
https://issues.apache.org/jira/browse/HIVE-23117?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Mollitor updated HIVE-23117:
--
Attachment: (was: HIVE-23117.3.patch)
> Review of HiveStatement Class
> -
>
> Key: HIVE-23117
> URL: https://issues.apache.org/jira/browse/HIVE-23117
> Project: Hive
> Issue Type: Improvement
> Components: JDBC
>Reporter: David Mollitor
>Assignee: David Mollitor
>Priority: Minor
> Attachments: HIVE-23117.1.patch, HIVE-23117.2.patch,
> HIVE-23117.2.patch
>
>
> * Remove unused instance variable(s)
> * Remove non-JavaDoc comments
> * Make inPlaceUpdateStream Optional (and remove NO-OP class) (inconsistent
> behavior with 'null' values)
> * {{getQueryTimeout()}} returns incorrect value
> * Unify and improve Exception messages
> * Checkstyle fixes
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-23117) Review of HiveStatement Class
[
https://issues.apache.org/jira/browse/HIVE-23117?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Mollitor updated HIVE-23117:
--
Attachment: HIVE-23117.3.patch
> Review of HiveStatement Class
> -
>
> Key: HIVE-23117
> URL: https://issues.apache.org/jira/browse/HIVE-23117
> Project: Hive
> Issue Type: Improvement
> Components: JDBC
>Reporter: David Mollitor
>Assignee: David Mollitor
>Priority: Minor
> Attachments: HIVE-23117.1.patch, HIVE-23117.2.patch,
> HIVE-23117.2.patch, HIVE-23117.3.patch
>
>
> * Remove unused instance variable(s)
> * Remove non-JavaDoc comments
> * Make inPlaceUpdateStream Optional (and remove NO-OP class) (inconsistent
> behavior with 'null' values)
> * {{getQueryTimeout()}} returns incorrect value
> * Unify and improve Exception messages
> * Checkstyle fixes
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
