[jira] [Updated] (HIVE-20985) If select operator inputs are temporary columns vectorization may reuse some of them as output
[ https://issues.apache.org/jira/browse/HIVE-20985?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Dere updated HIVE-20985: -- Attachment: HIVE-20985.03.patch > If select operator inputs are temporary columns vectorization may reuse some > of them as output > -- > > Key: HIVE-20985 > URL: https://issues.apache.org/jira/browse/HIVE-20985 > Project: Hive > Issue Type: Bug > Components: Vectorization >Reporter: Zoltan Haindrich >Assignee: Zoltan Haindrich >Priority: Major > Attachments: HIVE-20985.01.patch, HIVE-20985.02.patch, > HIVE-20985.03.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-20953) Remove a function from function registry when it can not be added to the metastore when creating it.
[
https://issues.apache.org/jira/browse/HIVE-20953?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16704372#comment-16704372
]
Hive QA commented on HIVE-20953:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12950123/HIVE-20953.03
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 2 failed/errored test(s), 15641 tests
executed
*Failed tests:*
{noformat}
TestMiniDruidCliDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=196)
[druidmini_dynamic_partition.q,druidmini_test1.q,druidkafkamini_basic.q,druidmini_test_insert.q,druidkafkamini_delimited.q]
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver[vector_outer_join4]
(batchId=192)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/15126/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/15126/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-15126/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 2 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12950123 - PreCommit-HIVE-Build
> Remove a function from function registry when it can not be added to the
> metastore when creating it.
>
>
> Key: HIVE-20953
> URL: https://issues.apache.org/jira/browse/HIVE-20953
> Project: Hive
> Issue Type: Bug
> Components: Tests
>Affects Versions: 4.0.0
>Reporter: Ashutosh Bapat
>Assignee: Ashutosh Bapat
>Priority: Major
> Labels: pull-request-available
> Fix For: 4.0.0
>
> Attachments: HIVE-20953.01, HIVE-20953.02, HIVE-20953.02,
> HIVE-20953.03, test_func_load_failure_retry.patch
>
> Original Estimate: 24h
> Remaining Estimate: 24h
>
> The testcase is intended to test REPL LOAD with retry. The test creates a
> partitioned table and a function in the source database and loads those to
> the replica. The first attempt to load a dump is intended to fail while
> loading one of the partitions. Based on the order in which the objects get
> loaded, if the function is queued after the table, it will not be available
> in replica after the load failure. But if it's queued before the table, it
> will be available in replica even after the load failure. The test assumes
> the later case, which may not be true always.
> Hence fix the testcase to order the objects by a fixed ordering. By setting
> hive.in.repl.test.files.sorted to true, the objects are ordered by the
> directory names. This ordering is available with minimal changes for testing,
> hence we use it. With this ordering a function gets loaded before a table. So
> changed the test to not expect the function to be available after the failed
> load, but be available after the retry.
> While writing that testcase, I found that even if a function fails to load,
> it's visible through show functions and also is available to be called just
> as if the failure has not happened. Digging further it was found that when
> creating a function we add it to the registry and also to the metastore. If
> the later fails, we do not clean it up from the registry and thus it remains
> visible after failure. Fixed the same.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-20989) JDBC: The GetOperationStatus + log can block query progress via sleep()
[
https://issues.apache.org/jira/browse/HIVE-20989?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16704366#comment-16704366
]
Gopal V commented on HIVE-20989:
A CountDownLatch(1) inside Operation.onNewState(OperationState, OperationState)
might be a good enough way to use .await() instead of sleep() here.
> JDBC: The GetOperationStatus + log can block query progress via sleep()
> ---
>
> Key: HIVE-20989
> URL: https://issues.apache.org/jira/browse/HIVE-20989
> Project: Hive
> Issue Type: Bug
> Components: HiveServer2
>Reporter: Gopal V
>Priority: Major
>
> There is an exponential sleep operation inside the CLIService which can end
> up adding tens of seconds to a query which has already completed.
> {code}
> "HiveServer2-Handler-Pool: Thread-9373" #9373 prio=5 os_prio=0
> tid=0x7f4d5e72d800 nid=0xb634a waiting on condition [0x7f28d06a5000]
> java.lang.Thread.State: TIMED_WAITING (sleeping)
> at java.lang.Thread.sleep(Native Method)
> at
> org.apache.hive.service.cli.CLIService.progressUpdateLog(CLIService.java:506)
> at
> org.apache.hive.service.cli.CLIService.getOperationStatus(CLIService.java:480)
> at
> org.apache.hive.service.cli.thrift.ThriftCLIService.GetOperationStatus(ThriftCLIService.java:695)
> at
> org.apache.hive.service.rpc.thrift.TCLIService$Processor$GetOperationStatus.getResult(TCLIService.java:1757)
> at
> org.apache.hive.service.rpc.thrift.TCLIService$Processor$GetOperationStatus.getResult(TCLIService.java:1742)
> at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)
> at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39)
> at
> org.apache.hive.service.auth.TSetIpAddressProcessor.process(TSetIpAddressProcessor.java:56)
> at
> org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:286)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> {code}
> The sleep loop is on the server side.
> {code}
> private static final long PROGRESS_MAX_WAIT_NS = 30 * 10l;
> private JobProgressUpdate progressUpdateLog(boolean isProgressLogRequested,
> Operation operation, HiveConf conf) {
> ...
> long startTime = System.nanoTime();
> int timeOutMs = 8;
> try {
> while (sessionState.getProgressMonitor() == null &&
> !operation.isDone()) {
> long remainingMs = (PROGRESS_MAX_WAIT_NS - (System.nanoTime() -
> startTime)) / 100l;
> if (remainingMs <= 0) {
> LOG.debug("timed out and hence returning progress log as NULL");
> return new JobProgressUpdate(ProgressMonitor.NULL);
> }
> Thread.sleep(Math.min(remainingMs, timeOutMs));
> timeOutMs <<= 1;
> }
> {code}
> After about 16 seconds of execution of the query, the timeOutMs is 16384 ms,
> which means the next sleep cycle is for min(30 - 17, 16) = 13.
> If the query finishes on the 17th second, the JDBC server will only respond
> after the 30th second when it will check for operation.isDone() and return.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-20989) JDBC: The GetOperationStatus + log can block query progress via sleep()
[
https://issues.apache.org/jira/browse/HIVE-20989?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Gopal V updated HIVE-20989:
---
Component/s: HiveServer2
> JDBC: The GetOperationStatus + log can block query progress via sleep()
> ---
>
> Key: HIVE-20989
> URL: https://issues.apache.org/jira/browse/HIVE-20989
> Project: Hive
> Issue Type: Bug
> Components: HiveServer2
>Reporter: Gopal V
>Priority: Major
>
> There is an exponential sleep operation inside the CLIService which can end
> up adding tens of seconds to a query which has already completed.
> {code}
> "HiveServer2-Handler-Pool: Thread-9373" #9373 prio=5 os_prio=0
> tid=0x7f4d5e72d800 nid=0xb634a waiting on condition [0x7f28d06a5000]
> java.lang.Thread.State: TIMED_WAITING (sleeping)
> at java.lang.Thread.sleep(Native Method)
> at
> org.apache.hive.service.cli.CLIService.progressUpdateLog(CLIService.java:506)
> at
> org.apache.hive.service.cli.CLIService.getOperationStatus(CLIService.java:480)
> at
> org.apache.hive.service.cli.thrift.ThriftCLIService.GetOperationStatus(ThriftCLIService.java:695)
> at
> org.apache.hive.service.rpc.thrift.TCLIService$Processor$GetOperationStatus.getResult(TCLIService.java:1757)
> at
> org.apache.hive.service.rpc.thrift.TCLIService$Processor$GetOperationStatus.getResult(TCLIService.java:1742)
> at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)
> at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39)
> at
> org.apache.hive.service.auth.TSetIpAddressProcessor.process(TSetIpAddressProcessor.java:56)
> at
> org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:286)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> {code}
> The sleep loop is on the server side.
> {code}
> private static final long PROGRESS_MAX_WAIT_NS = 30 * 10l;
> private JobProgressUpdate progressUpdateLog(boolean isProgressLogRequested,
> Operation operation, HiveConf conf) {
> ...
> long startTime = System.nanoTime();
> int timeOutMs = 8;
> try {
> while (sessionState.getProgressMonitor() == null &&
> !operation.isDone()) {
> long remainingMs = (PROGRESS_MAX_WAIT_NS - (System.nanoTime() -
> startTime)) / 100l;
> if (remainingMs <= 0) {
> LOG.debug("timed out and hence returning progress log as NULL");
> return new JobProgressUpdate(ProgressMonitor.NULL);
> }
> Thread.sleep(Math.min(remainingMs, timeOutMs));
> timeOutMs <<= 1;
> }
> {code}
> After about 16 seconds of execution of the query, the timeOutMs is 16384 ms,
> which means the next sleep cycle is for min(30 - 17, 16) = 13.
> If the query finishes on the 17th second, the JDBC server will only respond
> after the 30th second when it will check for operation.isDone() and return.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-20953) Remove a function from function registry when it can not be added to the metastore when creating it.
[
https://issues.apache.org/jira/browse/HIVE-20953?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16704349#comment-16704349
]
Hive QA commented on HIVE-20953:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
38s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m
32s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
36s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
52s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
44s{color} | {color:blue} ql in master has 2312 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
36s{color} | {color:blue} itests/hive-unit in master has 2 extant Findbugs
warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
15s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
25s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
57s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
37s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
37s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
17s{color} | {color:red} itests/hive-unit: The patch generated 2 new + 125
unchanged - 3 fixed = 127 total (was 128) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
32s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
15s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
13s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 27m 41s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-15126/dev-support/hive-personality.sh
|
| git revision | master / 8411ea5 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-15126/yetus/diff-checkstyle-itests_hive-unit.txt
|
| modules | C: ql itests/hive-unit U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-15126/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Remove a function from function registry when it can not be added to the
> metastore when creating it.
>
>
> Key: HIVE-20953
> URL: https://issues.apache.org/jira/browse/HIVE-20953
> Project: Hive
> Issue Type: Bug
> Components: Tests
>Affects Versions: 4.0.0
>Reporter: Ashutosh Bapat
>Assignee: Ashutosh Bapat
>Priority: Major
> Labels: pull-request-available
> Fix For: 4.0.0
>
> Attachments: HIVE-20953.01, HIVE-20953.02, HIVE-20953.02,
> HIVE-20953.03, test_func_load_failure_retry.patch
>
> Original Estimate: 24h
> Remaining Estimate: 24h
>
> The testcase is intended to test REPL LOAD with retry. The test creates a
> partitioned table and a function in the source database and loads those to
> the replica. The first attempt to load a dump is intended to fail while
> loading one of the partitions. Based on
[jira] [Updated] (HIVE-20988) Wrong results for group by queries with primary key on multiple columns
[ https://issues.apache.org/jira/browse/HIVE-20988?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vineet Garg updated HIVE-20988: --- Status: Patch Available (was: Open) > Wrong results for group by queries with primary key on multiple columns > --- > > Key: HIVE-20988 > URL: https://issues.apache.org/jira/browse/HIVE-20988 > Project: Hive > Issue Type: Bug >Affects Versions: 4.0.0 >Reporter: Vineet Garg >Assignee: Vineet Garg >Priority: Major > Attachments: HIVE-20988.1.patch > > > If a table has multi column primary key group by optimization ends up > removing group by which is semantically incorrect. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-20988) Wrong results for group by queries with primary key on multiple columns
[ https://issues.apache.org/jira/browse/HIVE-20988?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vineet Garg updated HIVE-20988: --- Description: If a table has multi column primary key group by optimization ends up removing group by which is semantically incorrect. > Wrong results for group by queries with primary key on multiple columns > --- > > Key: HIVE-20988 > URL: https://issues.apache.org/jira/browse/HIVE-20988 > Project: Hive > Issue Type: Bug >Affects Versions: 4.0.0 >Reporter: Vineet Garg >Assignee: Vineet Garg >Priority: Major > Attachments: HIVE-20988.1.patch > > > If a table has multi column primary key group by optimization ends up > removing group by which is semantically incorrect. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-20988) Wrong results for group by queries with primary key on multiple columns
[ https://issues.apache.org/jira/browse/HIVE-20988?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vineet Garg updated HIVE-20988: --- Attachment: HIVE-20988.1.patch > Wrong results for group by queries with primary key on multiple columns > --- > > Key: HIVE-20988 > URL: https://issues.apache.org/jira/browse/HIVE-20988 > Project: Hive > Issue Type: Bug >Affects Versions: 4.0.0 >Reporter: Vineet Garg >Assignee: Vineet Garg >Priority: Major > Attachments: HIVE-20988.1.patch > > > If a table has multi column primary key group by optimization ends up > removing group by which is semantically incorrect. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (HIVE-20988) Wrong results for group by queries with primary key on multiple columns
[ https://issues.apache.org/jira/browse/HIVE-20988?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vineet Garg reassigned HIVE-20988: -- > Wrong results for group by queries with primary key on multiple columns > --- > > Key: HIVE-20988 > URL: https://issues.apache.org/jira/browse/HIVE-20988 > Project: Hive > Issue Type: Bug >Affects Versions: 4.0.0 >Reporter: Vineet Garg >Assignee: Vineet Garg >Priority: Major > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-20987) Split Druid Tests to avoid Timeouts
[
https://issues.apache.org/jira/browse/HIVE-20987?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16704323#comment-16704323
]
Hive QA commented on HIVE-20987:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12950118/HIVE-20987.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 7 failed/errored test(s), 15645 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[druidmini_test1]
(batchId=81)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[kafka_storage_handler]
(batchId=32)
org.apache.hadoop.hive.cli.TestMiniDruidCliDriver.org.apache.hadoop.hive.cli.TestMiniDruidCliDriver
(batchId=197)
org.apache.hadoop.hive.cli.TestMiniDruidCliDriver.testCliDriver[druidkafkamini_avro]
(batchId=194)
org.apache.hadoop.hive.cli.TestMiniDruidCliDriver.testCliDriver[druidkafkamini_basic]
(batchId=196)
org.apache.hadoop.hive.cli.TestMiniDruidCliDriver.testCliDriver[druidkafkamini_csv]
(batchId=195)
org.apache.hadoop.hive.cli.TestMiniDruidCliDriver.testCliDriver[druidkafkamini_delimited]
(batchId=196)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/15125/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/15125/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-15125/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 7 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12950118 - PreCommit-HIVE-Build
> Split Druid Tests to avoid Timeouts
> ---
>
> Key: HIVE-20987
> URL: https://issues.apache.org/jira/browse/HIVE-20987
> Project: Hive
> Issue Type: Test
>Reporter: slim bouguerra
>Assignee: slim bouguerra
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-20987.patch
>
>
> Currently Druid Tests fail with Timeout issue.
> I am plaining on splitting the test into 2 batches at least to avoid timeouts.
> I will tweak the test code to pick random Druid nodes ports like that
> minimize the collision issue that we saw before.
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Assigned] (HIVE-12876) OpenCSV Serde treats everything as strings.
[
https://issues.apache.org/jira/browse/HIVE-12876?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
qingzhongli reassigned HIVE-12876:
--
Assignee: (was: qingzhongli)
> OpenCSV Serde treats everything as strings.
> ---
>
> Key: HIVE-12876
> URL: https://issues.apache.org/jira/browse/HIVE-12876
> Project: Hive
> Issue Type: Bug
> Components: Hive, Serializers/Deserializers
>Affects Versions: 1.2.1
>Reporter: Carter Shanklin
>Priority: Major
>
> This one caught me by surprise after some wrong results. I'm filing this as
> Brock suggested in HIVE-. Back to Ctrl-A for me it seems.
> To repro:
> {code}
> drop table int_table;
> create table int_table(
> x int
> )
> ROW FORMAT SERDE 'org.apache.hadoop.hive.serde2.OpenCSVSerde';
> show create table int_table;
> {code}
> And note that x is a string.
> Applicable to Hive 1.2.1.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Assigned] (HIVE-12876) OpenCSV Serde treats everything as strings.
[
https://issues.apache.org/jira/browse/HIVE-12876?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
qingzhongli reassigned HIVE-12876:
--
Assignee: qingzhongli
> OpenCSV Serde treats everything as strings.
> ---
>
> Key: HIVE-12876
> URL: https://issues.apache.org/jira/browse/HIVE-12876
> Project: Hive
> Issue Type: Bug
> Components: Hive, Serializers/Deserializers
>Affects Versions: 1.2.1
>Reporter: Carter Shanklin
>Assignee: qingzhongli
>Priority: Major
>
> This one caught me by surprise after some wrong results. I'm filing this as
> Brock suggested in HIVE-. Back to Ctrl-A for me it seems.
> To repro:
> {code}
> drop table int_table;
> create table int_table(
> x int
> )
> ROW FORMAT SERDE 'org.apache.hadoop.hive.serde2.OpenCSVSerde';
> show create table int_table;
> {code}
> And note that x is a string.
> Applicable to Hive 1.2.1.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-20985) If select operator inputs are temporary columns vectorization may reuse some of them as output
[ https://issues.apache.org/jira/browse/HIVE-20985?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16704305#comment-16704305 ] Teddy Choi commented on HIVE-20985: --- LGTM +1. I'll make another follow-up issue to get reuse back. > If select operator inputs are temporary columns vectorization may reuse some > of them as output > -- > > Key: HIVE-20985 > URL: https://issues.apache.org/jira/browse/HIVE-20985 > Project: Hive > Issue Type: Bug > Components: Vectorization >Reporter: Zoltan Haindrich >Assignee: Zoltan Haindrich >Priority: Major > Attachments: HIVE-20985.01.patch, HIVE-20985.02.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-20897) TestJdbcDriver2#testSelectExecAsync2 fails with result set not present error
[ https://issues.apache.org/jira/browse/HIVE-20897?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] mahesh kumar behera updated HIVE-20897: --- Attachment: (was: HIVE-20897.08.patch) > TestJdbcDriver2#testSelectExecAsync2 fails with result set not present error > > > Key: HIVE-20897 > URL: https://issues.apache.org/jira/browse/HIVE-20897 > Project: Hive > Issue Type: Bug > Components: Hive >Reporter: mahesh kumar behera >Assignee: Sankar Hariappan >Priority: Major > Labels: pull-request-available > Attachments: HIVE-20897.01.patch, HIVE-20897.02.patch, > HIVE-20897.03.patch, HIVE-20897.04.patch, HIVE-20897.05.patch, > HIVE-20897.06.patch, HIVE-20897.07.patch > > > if async prepare is enabled, control will be returned to the client before > driver could set of the query has a result set or not. But in current code, > while generating the response for the query, it is not checked if the result > set field is set or not. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (HIVE-20897) TestJdbcDriver2#testSelectExecAsync2 fails with result set not present error
[ https://issues.apache.org/jira/browse/HIVE-20897?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sankar Hariappan reassigned HIVE-20897: --- Assignee: mahesh kumar behera (was: Sankar Hariappan) > TestJdbcDriver2#testSelectExecAsync2 fails with result set not present error > > > Key: HIVE-20897 > URL: https://issues.apache.org/jira/browse/HIVE-20897 > Project: Hive > Issue Type: Bug > Components: Hive >Reporter: mahesh kumar behera >Assignee: mahesh kumar behera >Priority: Major > Labels: pull-request-available > Attachments: HIVE-20897.01.patch, HIVE-20897.02.patch, > HIVE-20897.03.patch, HIVE-20897.04.patch, HIVE-20897.05.patch, > HIVE-20897.06.patch, HIVE-20897.07.patch, HIVE-20897.08.patch > > > if async prepare is enabled, control will be returned to the client before > driver could set of the query has a result set or not. But in current code, > while generating the response for the query, it is not checked if the result > set field is set or not. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-20897) TestJdbcDriver2#testSelectExecAsync2 fails with result set not present error
[ https://issues.apache.org/jira/browse/HIVE-20897?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sankar Hariappan updated HIVE-20897: Attachment: HIVE-20897.08.patch > TestJdbcDriver2#testSelectExecAsync2 fails with result set not present error > > > Key: HIVE-20897 > URL: https://issues.apache.org/jira/browse/HIVE-20897 > Project: Hive > Issue Type: Bug > Components: Hive >Reporter: mahesh kumar behera >Assignee: Sankar Hariappan >Priority: Major > Labels: pull-request-available > Attachments: HIVE-20897.01.patch, HIVE-20897.02.patch, > HIVE-20897.03.patch, HIVE-20897.04.patch, HIVE-20897.05.patch, > HIVE-20897.06.patch, HIVE-20897.07.patch, HIVE-20897.08.patch > > > if async prepare is enabled, control will be returned to the client before > driver could set of the query has a result set or not. But in current code, > while generating the response for the query, it is not checked if the result > set field is set or not. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-20897) TestJdbcDriver2#testSelectExecAsync2 fails with result set not present error
[ https://issues.apache.org/jira/browse/HIVE-20897?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sankar Hariappan updated HIVE-20897: Status: Patch Available (was: Open) Re-attaching same 08.patch as test failure in previous ptest run was flaky. > TestJdbcDriver2#testSelectExecAsync2 fails with result set not present error > > > Key: HIVE-20897 > URL: https://issues.apache.org/jira/browse/HIVE-20897 > Project: Hive > Issue Type: Bug > Components: Hive >Reporter: mahesh kumar behera >Assignee: Sankar Hariappan >Priority: Major > Labels: pull-request-available > Attachments: HIVE-20897.01.patch, HIVE-20897.02.patch, > HIVE-20897.03.patch, HIVE-20897.04.patch, HIVE-20897.05.patch, > HIVE-20897.06.patch, HIVE-20897.07.patch, HIVE-20897.08.patch > > > if async prepare is enabled, control will be returned to the client before > driver could set of the query has a result set or not. But in current code, > while generating the response for the query, it is not checked if the result > set field is set or not. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-20985) If select operator inputs are temporary columns vectorization may reuse some of them as output
[ https://issues.apache.org/jira/browse/HIVE-20985?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16704296#comment-16704296 ] Ashutosh Chauhan commented on HIVE-20985: - There are couple of junit test failures. Do those tests need to be adjusted? If so, please update those so that we get a green run. > If select operator inputs are temporary columns vectorization may reuse some > of them as output > -- > > Key: HIVE-20985 > URL: https://issues.apache.org/jira/browse/HIVE-20985 > Project: Hive > Issue Type: Bug > Components: Vectorization >Reporter: Zoltan Haindrich >Assignee: Zoltan Haindrich >Priority: Major > Attachments: HIVE-20985.01.patch, HIVE-20985.02.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-20987) Split Druid Tests to avoid Timeouts
[
https://issues.apache.org/jira/browse/HIVE-20987?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16704297#comment-16704297
]
Hive QA commented on HIVE-20987:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
28s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m
30s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
51s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
28s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
20s{color} | {color:blue} itests/qtest-druid in master has 6 extant Findbugs
warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
38s{color} | {color:blue} itests/util in master has 48 extant Findbugs
warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
22s{color} | {color:blue} testutils/ptest2 in master has 24 extant Findbugs
warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
38s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
26s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
11s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
49s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
49s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
8s{color} | {color:red} itests/qtest-druid: The patch generated 2 new + 12
unchanged - 2 fixed = 14 total (was 14) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m
26s{color} | {color:red} itests/qtest-druid generated 3 new + 2 unchanged - 4
fixed = 5 total (was 6) {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
37s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} asflicense {color} | {color:red} 0m
13s{color} | {color:red} The patch generated 3 ASF License warnings. {color} |
| {color:black}{color} | {color:black} {color} | {color:black} 17m 28s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| FindBugs | module:itests/qtest-druid |
| | Synchronization performed on java.util.concurrent.atomic.AtomicBoolean in
org.apache.hive.druid.ForkingDruidNode.close() At
ForkingDruidNode.java:org.apache.hive.druid.ForkingDruidNode.close() At
ForkingDruidNode.java:[line 143] |
| | Synchronization performed on java.util.concurrent.atomic.AtomicBoolean in
org.apache.hive.druid.ForkingDruidNode.isAlive() At
ForkingDruidNode.java:org.apache.hive.druid.ForkingDruidNode.isAlive() At
ForkingDruidNode.java:[line 136] |
| | Synchronization performed on java.util.concurrent.atomic.AtomicBoolean in
org.apache.hive.druid.ForkingDruidNode.start() At
ForkingDruidNode.java:org.apache.hive.druid.ForkingDruidNode.start() At
ForkingDruidNode.java:[line 125] |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-15125/dev-support/hive-personality.sh
|
| git revision | master / 8411ea5 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-15125/yetus/diff-checkstyle-itests_qtest-druid.txt
|
| findbugs |
http://104.198.109.242/logs//PreCommit-HIVE-Build-15125/yetus/new-findbugs-itests_qtest-druid.html
|
| asflicense |
http://104.198.109.242/logs//PreCommit-HIVE-Build-15125/yetus/patch-asflicense-problems.txt
|
| modules | C: itests itests/qtest-druid
[jira] [Updated] (HIVE-20897) TestJdbcDriver2#testSelectExecAsync2 fails with result set not present error
[ https://issues.apache.org/jira/browse/HIVE-20897?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sankar Hariappan updated HIVE-20897: Assignee: Sankar Hariappan (was: mahesh kumar behera) Status: Open (was: Patch Available) > TestJdbcDriver2#testSelectExecAsync2 fails with result set not present error > > > Key: HIVE-20897 > URL: https://issues.apache.org/jira/browse/HIVE-20897 > Project: Hive > Issue Type: Bug > Components: Hive >Reporter: mahesh kumar behera >Assignee: Sankar Hariappan >Priority: Major > Labels: pull-request-available > Attachments: HIVE-20897.01.patch, HIVE-20897.02.patch, > HIVE-20897.03.patch, HIVE-20897.04.patch, HIVE-20897.05.patch, > HIVE-20897.06.patch, HIVE-20897.07.patch, HIVE-20897.08.patch > > > if async prepare is enabled, control will be returned to the client before > driver could set of the query has a result set or not. But in current code, > while generating the response for the query, it is not checked if the result > set field is set or not. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-20985) If select operator inputs are temporary columns vectorization may reuse some of them as output
[ https://issues.apache.org/jira/browse/HIVE-20985?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16704292#comment-16704292 ] Teddy Choi commented on HIVE-20985: --- Looks good on correctness. There are some cases that lose memory efficiency more than others. Of course, vector_reuse_scratchcols.q does it on purpose, but CASE and nested math operations are pretty common cases and hard to ignore. Table: the maximum column number in a map or reduce operation. ||File||Before||After|| |vector_case_when_1.q|38|68| |vector_case_when_2.q|17|39| |vector_decimal_aggregate.q|25|46| |vector_decimal_expressions.q|22|38| |vector_decimal_math_funcs.q|31|80| |vector_interval_2.q|32|62| |vector_reuse_scratchcols.q|44|162| Usually, a column vector has 1,024 elements of long or double, which are 8 byte. So a column vector will take 8 KB+. vector_reuse_scratchcols.q will use 118 column vectors more, which is 0.6 MB+. The additional memory use will be on every executor. Even with 32-core machine, it will take 19.2 MB+ more. However, it may lose performance advantage from CPU caches, I guess. > If select operator inputs are temporary columns vectorization may reuse some > of them as output > -- > > Key: HIVE-20985 > URL: https://issues.apache.org/jira/browse/HIVE-20985 > Project: Hive > Issue Type: Bug > Components: Vectorization >Reporter: Zoltan Haindrich >Assignee: Zoltan Haindrich >Priority: Major > Attachments: HIVE-20985.01.patch, HIVE-20985.02.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-20953) Remove a function from function registry when it can not be added to the metastore when creating it.
[ https://issues.apache.org/jira/browse/HIVE-20953?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashutosh Bapat updated HIVE-20953: -- Attachment: HIVE-20953.03 Status: Patch Available (was: In Progress) > Remove a function from function registry when it can not be added to the > metastore when creating it. > > > Key: HIVE-20953 > URL: https://issues.apache.org/jira/browse/HIVE-20953 > Project: Hive > Issue Type: Bug > Components: Tests >Affects Versions: 4.0.0 >Reporter: Ashutosh Bapat >Assignee: Ashutosh Bapat >Priority: Major > Labels: pull-request-available > Fix For: 4.0.0 > > Attachments: HIVE-20953.01, HIVE-20953.02, HIVE-20953.02, > HIVE-20953.03, test_func_load_failure_retry.patch > > Original Estimate: 24h > Remaining Estimate: 24h > > The testcase is intended to test REPL LOAD with retry. The test creates a > partitioned table and a function in the source database and loads those to > the replica. The first attempt to load a dump is intended to fail while > loading one of the partitions. Based on the order in which the objects get > loaded, if the function is queued after the table, it will not be available > in replica after the load failure. But if it's queued before the table, it > will be available in replica even after the load failure. The test assumes > the later case, which may not be true always. > Hence fix the testcase to order the objects by a fixed ordering. By setting > hive.in.repl.test.files.sorted to true, the objects are ordered by the > directory names. This ordering is available with minimal changes for testing, > hence we use it. With this ordering a function gets loaded before a table. So > changed the test to not expect the function to be available after the failed > load, but be available after the retry. > While writing that testcase, I found that even if a function fails to load, > it's visible through show functions and also is available to be called just > as if the failure has not happened. Digging further it was found that when > creating a function we add it to the registry and also to the metastore. If > the later fails, we do not clean it up from the registry and thus it remains > visible after failure. Fixed the same. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-20953) Remove a function from function registry when it can not be added to the metastore when creating it.
[ https://issues.apache.org/jira/browse/HIVE-20953?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashutosh Bapat updated HIVE-20953: -- Status: In Progress (was: Patch Available) > Remove a function from function registry when it can not be added to the > metastore when creating it. > > > Key: HIVE-20953 > URL: https://issues.apache.org/jira/browse/HIVE-20953 > Project: Hive > Issue Type: Bug > Components: Tests >Affects Versions: 4.0.0 >Reporter: Ashutosh Bapat >Assignee: Ashutosh Bapat >Priority: Major > Labels: pull-request-available > Fix For: 4.0.0 > > Attachments: HIVE-20953.01, HIVE-20953.02, HIVE-20953.02, > test_func_load_failure_retry.patch > > Original Estimate: 24h > Remaining Estimate: 24h > > The testcase is intended to test REPL LOAD with retry. The test creates a > partitioned table and a function in the source database and loads those to > the replica. The first attempt to load a dump is intended to fail while > loading one of the partitions. Based on the order in which the objects get > loaded, if the function is queued after the table, it will not be available > in replica after the load failure. But if it's queued before the table, it > will be available in replica even after the load failure. The test assumes > the later case, which may not be true always. > Hence fix the testcase to order the objects by a fixed ordering. By setting > hive.in.repl.test.files.sorted to true, the objects are ordered by the > directory names. This ordering is available with minimal changes for testing, > hence we use it. With this ordering a function gets loaded before a table. So > changed the test to not expect the function to be available after the failed > load, but be available after the retry. > While writing that testcase, I found that even if a function fails to load, > it's visible through show functions and also is available to be called just > as if the failure has not happened. Digging further it was found that when > creating a function we add it to the registry and also to the metastore. If > the later fails, we do not clean it up from the registry and thus it remains > visible after failure. Fixed the same. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-20979) Fix memory leak in hive streaming
[
https://issues.apache.org/jira/browse/HIVE-20979?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16704283#comment-16704283
]
Hive QA commented on HIVE-20979:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12950117/HIVE-20979.5.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 15644 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/15124/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/15124/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-15124/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12950117 - PreCommit-HIVE-Build
> Fix memory leak in hive streaming
> -
>
> Key: HIVE-20979
> URL: https://issues.apache.org/jira/browse/HIVE-20979
> Project: Hive
> Issue Type: Bug
> Components: Streaming
>Affects Versions: 3.1.1
>Reporter: Shubham Chaurasia
>Assignee: Shubham Chaurasia
>Priority: Critical
> Labels: pull-request-available
> Attachments: HIVE-20979.1.patch, HIVE-20979.1.patch,
> HIVE-20979.2.patch, HIVE-20979.3.patch, HIVE-20979.4.patch, HIVE-20979.5.patch
>
>
> {{1) HiveStreamingConnection.Builder#init() adds a shutdown hook handler via
> }}{{ShutdownHookManager.addShutdownHook but it is never removed which causes
> all the handlers to accumulate and hence a memory leak.}}
> 2) AbstractRecordWriter creates an instance of FileSystem but does not close
> it which in turn causes a leak due to accumulation in FileSystem$Cache#map
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-20979) Fix memory leak in hive streaming
[
https://issues.apache.org/jira/browse/HIVE-20979?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16704261#comment-16704261
]
Hive QA commented on HIVE-20979:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
15s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
19s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
10s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
24s{color} | {color:blue} streaming in master has 2 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
11s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
17s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
16s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
16s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
10s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
30s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
12s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
12s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 11m 13s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-15124/dev-support/hive-personality.sh
|
| git revision | master / 8411ea5 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| modules | C: streaming U: streaming |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-15124/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Fix memory leak in hive streaming
> -
>
> Key: HIVE-20979
> URL: https://issues.apache.org/jira/browse/HIVE-20979
> Project: Hive
> Issue Type: Bug
> Components: Streaming
>Affects Versions: 3.1.1
>Reporter: Shubham Chaurasia
>Assignee: Shubham Chaurasia
>Priority: Critical
> Labels: pull-request-available
> Attachments: HIVE-20979.1.patch, HIVE-20979.1.patch,
> HIVE-20979.2.patch, HIVE-20979.3.patch, HIVE-20979.4.patch, HIVE-20979.5.patch
>
>
> {{1) HiveStreamingConnection.Builder#init() adds a shutdown hook handler via
> }}{{ShutdownHookManager.addShutdownHook but it is never removed which causes
> all the handlers to accumulate and hence a memory leak.}}
> 2) AbstractRecordWriter creates an instance of FileSystem but does not close
> it which in turn causes a leak due to accumulation in FileSystem$Cache#map
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-20987) Split Druid Tests to avoid Timeouts
[ https://issues.apache.org/jira/browse/HIVE-20987?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] slim bouguerra updated HIVE-20987: -- Attachment: HIVE-20987.patch > Split Druid Tests to avoid Timeouts > --- > > Key: HIVE-20987 > URL: https://issues.apache.org/jira/browse/HIVE-20987 > Project: Hive > Issue Type: Test >Reporter: slim bouguerra >Assignee: slim bouguerra >Priority: Major > Labels: pull-request-available > Attachments: HIVE-20987.patch > > > Currently Druid Tests fail with Timeout issue. > I am plaining on splitting the test into 2 batches at least to avoid timeouts. > I will tweak the test code to pick random Druid nodes ports like that > minimize the collision issue that we saw before. > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-20987) Split Druid Tests to avoid Timeouts
[ https://issues.apache.org/jira/browse/HIVE-20987?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HIVE-20987: -- Labels: pull-request-available (was: ) > Split Druid Tests to avoid Timeouts > --- > > Key: HIVE-20987 > URL: https://issues.apache.org/jira/browse/HIVE-20987 > Project: Hive > Issue Type: Test >Reporter: slim bouguerra >Assignee: slim bouguerra >Priority: Major > Labels: pull-request-available > > Currently Druid Tests fail with Timeout issue. > I am plaining on splitting the test into 2 batches at least to avoid timeouts. > I will tweak the test code to pick random Druid nodes ports like that > minimize the collision issue that we saw before. > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-20979) Fix memory leak in hive streaming
[
https://issues.apache.org/jira/browse/HIVE-20979?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Shubham Chaurasia updated HIVE-20979:
-
Attachment: HIVE-20979.5.patch
> Fix memory leak in hive streaming
> -
>
> Key: HIVE-20979
> URL: https://issues.apache.org/jira/browse/HIVE-20979
> Project: Hive
> Issue Type: Bug
> Components: Streaming
>Affects Versions: 3.1.1
>Reporter: Shubham Chaurasia
>Assignee: Shubham Chaurasia
>Priority: Critical
> Labels: pull-request-available
> Attachments: HIVE-20979.1.patch, HIVE-20979.1.patch,
> HIVE-20979.2.patch, HIVE-20979.3.patch, HIVE-20979.4.patch, HIVE-20979.5.patch
>
>
> {{1) HiveStreamingConnection.Builder#init() adds a shutdown hook handler via
> }}{{ShutdownHookManager.addShutdownHook but it is never removed which causes
> all the handlers to accumulate and hence a memory leak.}}
> 2) AbstractRecordWriter creates an instance of FileSystem but does not close
> it which in turn causes a leak due to accumulation in FileSystem$Cache#map
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-20987) Split Druid Tests to avoid Timeouts
[ https://issues.apache.org/jira/browse/HIVE-20987?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] slim bouguerra updated HIVE-20987: -- Status: Patch Available (was: Open) > Split Druid Tests to avoid Timeouts > --- > > Key: HIVE-20987 > URL: https://issues.apache.org/jira/browse/HIVE-20987 > Project: Hive > Issue Type: Test >Reporter: slim bouguerra >Assignee: slim bouguerra >Priority: Major > Labels: pull-request-available > > Currently Druid Tests fail with Timeout issue. > I am plaining on splitting the test into 2 batches at least to avoid timeouts. > I will tweak the test code to pick random Druid nodes ports like that > minimize the collision issue that we saw before. > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-20987) Split Druid Tests to avoid Timeouts
[ https://issues.apache.org/jira/browse/HIVE-20987?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16704219#comment-16704219 ] ASF GitHub Bot commented on HIVE-20987: --- GitHub user b-slim opened a pull request: https://github.com/apache/hive/pull/498 HIVE-20987: split Druid Tests and start nodes on random ports Change-Id: I89009fd8a79a85b26bcc080c34a07577125f0110 You can merge this pull request into a Git repository by running: $ git pull https://github.com/b-slim/hive HIVE-20987 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/hive/pull/498.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #498 commit 959438cf82a522f6cf573b8df2b7850762ea6c7b Author: Slim Bouguerra Date: 2018-11-30T03:35:55Z HIVE-20987: split Druid Tests and start nodes on random ports Change-Id: I89009fd8a79a85b26bcc080c34a07577125f0110 > Split Druid Tests to avoid Timeouts > --- > > Key: HIVE-20987 > URL: https://issues.apache.org/jira/browse/HIVE-20987 > Project: Hive > Issue Type: Test >Reporter: slim bouguerra >Assignee: slim bouguerra >Priority: Major > Labels: pull-request-available > > Currently Druid Tests fail with Timeout issue. > I am plaining on splitting the test into 2 batches at least to avoid timeouts. > I will tweak the test code to pick random Druid nodes ports like that > minimize the collision issue that we saw before. > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (HIVE-20987) Split Druid Tests to avoid Timeouts
[ https://issues.apache.org/jira/browse/HIVE-20987?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] slim bouguerra reassigned HIVE-20987: - > Split Druid Tests to avoid Timeouts > --- > > Key: HIVE-20987 > URL: https://issues.apache.org/jira/browse/HIVE-20987 > Project: Hive > Issue Type: Test >Reporter: slim bouguerra >Assignee: slim bouguerra >Priority: Major > > Currently Druid Tests fail with Timeout issue. > I am plaining on splitting the test into 2 batches at least to avoid timeouts. > I will tweak the test code to pick random Druid nodes ports like that > minimize the collision issue that we saw before. > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-20979) Fix memory leak in hive streaming
[
https://issues.apache.org/jira/browse/HIVE-20979?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16704218#comment-16704218
]
Shubham Chaurasia commented on HIVE-20979:
--
Thanks for the review [~prasanth_j] [~ewohlstadter] [~ashutoshc]
These test failures do not seem related. However, as you said [~prasanth_j],
will keep uploading patches until I get green.
> Fix memory leak in hive streaming
> -
>
> Key: HIVE-20979
> URL: https://issues.apache.org/jira/browse/HIVE-20979
> Project: Hive
> Issue Type: Bug
> Components: Streaming
>Affects Versions: 3.1.1
>Reporter: Shubham Chaurasia
>Assignee: Shubham Chaurasia
>Priority: Critical
> Labels: pull-request-available
> Attachments: HIVE-20979.1.patch, HIVE-20979.1.patch,
> HIVE-20979.2.patch, HIVE-20979.3.patch, HIVE-20979.4.patch
>
>
> {{1) HiveStreamingConnection.Builder#init() adds a shutdown hook handler via
> }}{{ShutdownHookManager.addShutdownHook but it is never removed which causes
> all the handlers to accumulate and hence a memory leak.}}
> 2) AbstractRecordWriter creates an instance of FileSystem but does not close
> it which in turn causes a leak due to accumulation in FileSystem$Cache#map
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-20897) TestJdbcDriver2#testSelectExecAsync2 fails with result set not present error
[
https://issues.apache.org/jira/browse/HIVE-20897?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16704199#comment-16704199
]
Hive QA commented on HIVE-20897:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12950107/HIVE-20897.08.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 15644 tests
executed
*Failed tests:*
{noformat}
TestMiniDruidCliDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=197)
[druidmini_masking.q,druidmini_joins.q,druid_timestamptz.q]
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/15123/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/15123/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-15123/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 1 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12950107 - PreCommit-HIVE-Build
> TestJdbcDriver2#testSelectExecAsync2 fails with result set not present error
>
>
> Key: HIVE-20897
> URL: https://issues.apache.org/jira/browse/HIVE-20897
> Project: Hive
> Issue Type: Bug
> Components: Hive
>Reporter: mahesh kumar behera
>Assignee: mahesh kumar behera
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-20897.01.patch, HIVE-20897.02.patch,
> HIVE-20897.03.patch, HIVE-20897.04.patch, HIVE-20897.05.patch,
> HIVE-20897.06.patch, HIVE-20897.07.patch, HIVE-20897.08.patch
>
>
> if async prepare is enabled, control will be returned to the client before
> driver could set of the query has a result set or not. But in current code,
> while generating the response for the query, it is not checked if the result
> set field is set or not.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-20897) TestJdbcDriver2#testSelectExecAsync2 fails with result set not present error
[
https://issues.apache.org/jira/browse/HIVE-20897?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16704155#comment-16704155
]
Hive QA commented on HIVE-20897:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
58s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
23s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
12s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
36s{color} | {color:blue} service in master has 48 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
16s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
24s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
23s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
23s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
12s{color} | {color:green} service: The patch generated 0 new + 59 unchanged -
1 fixed = 59 total (was 60) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
42s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
17s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
12s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 11m 59s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-15123/dev-support/hive-personality.sh
|
| git revision | master / 8411ea5 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| modules | C: service U: service |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-15123/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> TestJdbcDriver2#testSelectExecAsync2 fails with result set not present error
>
>
> Key: HIVE-20897
> URL: https://issues.apache.org/jira/browse/HIVE-20897
> Project: Hive
> Issue Type: Bug
> Components: Hive
>Reporter: mahesh kumar behera
>Assignee: mahesh kumar behera
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-20897.01.patch, HIVE-20897.02.patch,
> HIVE-20897.03.patch, HIVE-20897.04.patch, HIVE-20897.05.patch,
> HIVE-20897.06.patch, HIVE-20897.07.patch, HIVE-20897.08.patch
>
>
> if async prepare is enabled, control will be returned to the client before
> driver could set of the query has a result set or not. But in current code,
> while generating the response for the query, it is not checked if the result
> set field is set or not.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-20982) Avoid the un-needed object creation within hotloop
[
https://issues.apache.org/jira/browse/HIVE-20982?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16704147#comment-16704147
]
Hive QA commented on HIVE-20982:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12950102/HIVE-20982.2.patch
{color:green}SUCCESS:{color} +1 due to 3 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 2 failed/errored test(s), 15629 tests
executed
*Failed tests:*
{noformat}
TestMiniDruidCliDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=197)
[druidmini_masking.q,druidmini_joins.q,druid_timestamptz.q]
TestMiniLlapCliDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=154)
[intersect_all.q,unionDistinct_1.q,table_nonprintable.q,orc_llap_counters1.q,mm_cttas.q,whroot_external1.q,global_limit.q,cte_2.q,rcfile_createas1.q,dynamic_partition_pruning_2.q,intersect_merge.q,results_cache_diff_fs.q,cttl.q,parallel_colstats.q,load_hdfs_file_with_space_in_the_name.q]
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/15122/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/15122/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-15122/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 2 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12950102 - PreCommit-HIVE-Build
> Avoid the un-needed object creation within hotloop
> --
>
> Key: HIVE-20982
> URL: https://issues.apache.org/jira/browse/HIVE-20982
> Project: Hive
> Issue Type: Sub-task
>Reporter: slim bouguerra
>Assignee: slim bouguerra
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-20982.2.patch, HIVE-20982.2.patch, HIVE-20982.patch
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-20897) TestJdbcDriver2#testSelectExecAsync2 fails with result set not present error
[ https://issues.apache.org/jira/browse/HIVE-20897?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] mahesh kumar behera updated HIVE-20897: --- Status: Patch Available (was: Open) > TestJdbcDriver2#testSelectExecAsync2 fails with result set not present error > > > Key: HIVE-20897 > URL: https://issues.apache.org/jira/browse/HIVE-20897 > Project: Hive > Issue Type: Bug > Components: Hive >Reporter: mahesh kumar behera >Assignee: mahesh kumar behera >Priority: Major > Labels: pull-request-available > Attachments: HIVE-20897.01.patch, HIVE-20897.02.patch, > HIVE-20897.03.patch, HIVE-20897.04.patch, HIVE-20897.05.patch, > HIVE-20897.06.patch, HIVE-20897.07.patch, HIVE-20897.08.patch > > > if async prepare is enabled, control will be returned to the client before > driver could set of the query has a result set or not. But in current code, > while generating the response for the query, it is not checked if the result > set field is set or not. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-20897) TestJdbcDriver2#testSelectExecAsync2 fails with result set not present error
[ https://issues.apache.org/jira/browse/HIVE-20897?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] mahesh kumar behera updated HIVE-20897: --- Attachment: (was: HIVE-20897.08.patch) > TestJdbcDriver2#testSelectExecAsync2 fails with result set not present error > > > Key: HIVE-20897 > URL: https://issues.apache.org/jira/browse/HIVE-20897 > Project: Hive > Issue Type: Bug > Components: Hive >Reporter: mahesh kumar behera >Assignee: mahesh kumar behera >Priority: Major > Labels: pull-request-available > Attachments: HIVE-20897.01.patch, HIVE-20897.02.patch, > HIVE-20897.03.patch, HIVE-20897.04.patch, HIVE-20897.05.patch, > HIVE-20897.06.patch, HIVE-20897.07.patch, HIVE-20897.07.patch, > HIVE-20897.08.patch > > > if async prepare is enabled, control will be returned to the client before > driver could set of the query has a result set or not. But in current code, > while generating the response for the query, it is not checked if the result > set field is set or not. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-20982) Avoid the un-needed object creation within hotloop
[
https://issues.apache.org/jira/browse/HIVE-20982?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16704117#comment-16704117
]
Hive QA commented on HIVE-20982:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
44s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m
36s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
17s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
44s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
48s{color} | {color:blue} ql in master has 2312 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
26s{color} | {color:blue} druid-handler in master has 4 extant Findbugs
warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
8s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
26s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
49s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
17s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
17s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
46s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
1s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
53s{color} | {color:green} ql in the patch passed. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
32s{color} | {color:green} druid-handler generated 0 new + 3 unchanged - 1
fixed = 3 total (was 4) {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
7s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
13s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 26m 31s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc xml compile findbugs
checkstyle |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-15122/dev-support/hive-personality.sh
|
| git revision | master / 8411ea5 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| modules | C: ql druid-handler U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-15122/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Avoid the un-needed object creation within hotloop
> --
>
> Key: HIVE-20982
> URL: https://issues.apache.org/jira/browse/HIVE-20982
> Project: Hive
> Issue Type: Sub-task
>Reporter: slim bouguerra
>Assignee: slim bouguerra
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-20982.2.patch, HIVE-20982.2.patch, HIVE-20982.patch
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-20897) TestJdbcDriver2#testSelectExecAsync2 fails with result set not present error
[ https://issues.apache.org/jira/browse/HIVE-20897?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] mahesh kumar behera updated HIVE-20897: --- Attachment: (was: HIVE-20897.07.patch) > TestJdbcDriver2#testSelectExecAsync2 fails with result set not present error > > > Key: HIVE-20897 > URL: https://issues.apache.org/jira/browse/HIVE-20897 > Project: Hive > Issue Type: Bug > Components: Hive >Reporter: mahesh kumar behera >Assignee: mahesh kumar behera >Priority: Major > Labels: pull-request-available > Attachments: HIVE-20897.01.patch, HIVE-20897.02.patch, > HIVE-20897.03.patch, HIVE-20897.04.patch, HIVE-20897.05.patch, > HIVE-20897.06.patch, HIVE-20897.07.patch, HIVE-20897.08.patch > > > if async prepare is enabled, control will be returned to the client before > driver could set of the query has a result set or not. But in current code, > while generating the response for the query, it is not checked if the result > set field is set or not. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-20897) TestJdbcDriver2#testSelectExecAsync2 fails with result set not present error
[ https://issues.apache.org/jira/browse/HIVE-20897?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] mahesh kumar behera updated HIVE-20897: --- Attachment: HIVE-20897.08.patch > TestJdbcDriver2#testSelectExecAsync2 fails with result set not present error > > > Key: HIVE-20897 > URL: https://issues.apache.org/jira/browse/HIVE-20897 > Project: Hive > Issue Type: Bug > Components: Hive >Reporter: mahesh kumar behera >Assignee: mahesh kumar behera >Priority: Major > Labels: pull-request-available > Attachments: HIVE-20897.01.patch, HIVE-20897.02.patch, > HIVE-20897.03.patch, HIVE-20897.04.patch, HIVE-20897.05.patch, > HIVE-20897.06.patch, HIVE-20897.07.patch, HIVE-20897.08.patch > > > if async prepare is enabled, control will be returned to the client before > driver could set of the query has a result set or not. But in current code, > while generating the response for the query, it is not checked if the result > set field is set or not. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-20897) TestJdbcDriver2#testSelectExecAsync2 fails with result set not present error
[ https://issues.apache.org/jira/browse/HIVE-20897?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] mahesh kumar behera updated HIVE-20897: --- Attachment: (was: HIVE-20897.08.patch) > TestJdbcDriver2#testSelectExecAsync2 fails with result set not present error > > > Key: HIVE-20897 > URL: https://issues.apache.org/jira/browse/HIVE-20897 > Project: Hive > Issue Type: Bug > Components: Hive >Reporter: mahesh kumar behera >Assignee: mahesh kumar behera >Priority: Major > Labels: pull-request-available > Attachments: HIVE-20897.01.patch, HIVE-20897.02.patch, > HIVE-20897.03.patch, HIVE-20897.04.patch, HIVE-20897.05.patch, > HIVE-20897.06.patch, HIVE-20897.07.patch, HIVE-20897.08.patch > > > if async prepare is enabled, control will be returned to the client before > driver could set of the query has a result set or not. But in current code, > while generating the response for the query, it is not checked if the result > set field is set or not. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-20897) TestJdbcDriver2#testSelectExecAsync2 fails with result set not present error
[ https://issues.apache.org/jira/browse/HIVE-20897?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] mahesh kumar behera updated HIVE-20897: --- Attachment: (was: HIVE-20897.08.patch) > TestJdbcDriver2#testSelectExecAsync2 fails with result set not present error > > > Key: HIVE-20897 > URL: https://issues.apache.org/jira/browse/HIVE-20897 > Project: Hive > Issue Type: Bug > Components: Hive >Reporter: mahesh kumar behera >Assignee: mahesh kumar behera >Priority: Major > Labels: pull-request-available > Attachments: HIVE-20897.01.patch, HIVE-20897.02.patch, > HIVE-20897.03.patch, HIVE-20897.04.patch, HIVE-20897.05.patch, > HIVE-20897.06.patch, HIVE-20897.07.patch, HIVE-20897.07.patch, > HIVE-20897.08.patch > > > if async prepare is enabled, control will be returned to the client before > driver could set of the query has a result set or not. But in current code, > while generating the response for the query, it is not checked if the result > set field is set or not. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-20897) TestJdbcDriver2#testSelectExecAsync2 fails with result set not present error
[ https://issues.apache.org/jira/browse/HIVE-20897?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] mahesh kumar behera updated HIVE-20897: --- Status: Open (was: Patch Available) > TestJdbcDriver2#testSelectExecAsync2 fails with result set not present error > > > Key: HIVE-20897 > URL: https://issues.apache.org/jira/browse/HIVE-20897 > Project: Hive > Issue Type: Bug > Components: Hive >Reporter: mahesh kumar behera >Assignee: mahesh kumar behera >Priority: Major > Labels: pull-request-available > Attachments: HIVE-20897.01.patch, HIVE-20897.02.patch, > HIVE-20897.03.patch, HIVE-20897.04.patch, HIVE-20897.05.patch, > HIVE-20897.06.patch, HIVE-20897.07.patch, HIVE-20897.07.patch, > HIVE-20897.08.patch > > > if async prepare is enabled, control will be returned to the client before > driver could set of the query has a result set or not. But in current code, > while generating the response for the query, it is not checked if the result > set field is set or not. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-20985) If select operator inputs are temporary columns vectorization may reuse some of them as output
[ https://issues.apache.org/jira/browse/HIVE-20985?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16704095#comment-16704095 ] Eric Wohlstadter commented on HIVE-20985: - [~teddy.choi] Can you please review? Does the current patch use memory efficiently? > If select operator inputs are temporary columns vectorization may reuse some > of them as output > -- > > Key: HIVE-20985 > URL: https://issues.apache.org/jira/browse/HIVE-20985 > Project: Hive > Issue Type: Bug > Components: Vectorization >Reporter: Zoltan Haindrich >Assignee: Zoltan Haindrich >Priority: Major > Attachments: HIVE-20985.01.patch, HIVE-20985.02.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-20982) Avoid the un-needed object creation within hotloop
[ https://issues.apache.org/jira/browse/HIVE-20982?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] slim bouguerra updated HIVE-20982: -- Attachment: HIVE-20982.2.patch > Avoid the un-needed object creation within hotloop > -- > > Key: HIVE-20982 > URL: https://issues.apache.org/jira/browse/HIVE-20982 > Project: Hive > Issue Type: Sub-task >Reporter: slim bouguerra >Assignee: slim bouguerra >Priority: Major > Labels: pull-request-available > Attachments: HIVE-20982.2.patch, HIVE-20982.2.patch, HIVE-20982.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-20982) Avoid the un-needed object creation within hotloop
[
https://issues.apache.org/jira/browse/HIVE-20982?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16704059#comment-16704059
]
Hive QA commented on HIVE-20982:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12950080/HIVE-20982.2.patch
{color:green}SUCCESS:{color} +1 due to 3 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 46 failed/errored test(s), 15639 tests
executed
*Failed tests:*
{noformat}
TestMiniDruidCliDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=196)
[druidmini_dynamic_partition.q,druidmini_test1.q,druidkafkamini_basic.q,druidmini_test_insert.q,druidkafkamini_delimited.q]
org.apache.hadoop.hive.metastore.TestRemoteHiveMetaStoreZK.testAlterPartition
(batchId=230)
org.apache.hadoop.hive.metastore.TestRemoteHiveMetaStoreZK.testAlterTable
(batchId=230)
org.apache.hadoop.hive.metastore.TestRemoteHiveMetaStoreZK.testAlterTableCascade
(batchId=230)
org.apache.hadoop.hive.metastore.TestRemoteHiveMetaStoreZK.testAlterViewParititon
(batchId=230)
org.apache.hadoop.hive.metastore.TestRemoteHiveMetaStoreZK.testColumnStatistics
(batchId=230)
org.apache.hadoop.hive.metastore.TestRemoteHiveMetaStoreZK.testComplexTable
(batchId=230)
org.apache.hadoop.hive.metastore.TestRemoteHiveMetaStoreZK.testComplexTypeApi
(batchId=230)
org.apache.hadoop.hive.metastore.TestRemoteHiveMetaStoreZK.testConcurrentMetastores
(batchId=230)
org.apache.hadoop.hive.metastore.TestRemoteHiveMetaStoreZK.testCreateAndGetTableWithDriver
(batchId=230)
org.apache.hadoop.hive.metastore.TestRemoteHiveMetaStoreZK.testCreateTableSettingId
(batchId=230)
org.apache.hadoop.hive.metastore.TestRemoteHiveMetaStoreZK.testDBLocationChange
(batchId=230)
org.apache.hadoop.hive.metastore.TestRemoteHiveMetaStoreZK.testDBOwner
(batchId=230)
org.apache.hadoop.hive.metastore.TestRemoteHiveMetaStoreZK.testDBOwnerChange
(batchId=230)
org.apache.hadoop.hive.metastore.TestRemoteHiveMetaStoreZK.testDatabase
(batchId=230)
org.apache.hadoop.hive.metastore.TestRemoteHiveMetaStoreZK.testDatabaseLocation
(batchId=230)
org.apache.hadoop.hive.metastore.TestRemoteHiveMetaStoreZK.testDatabaseLocationWithPermissionProblems
(batchId=230)
org.apache.hadoop.hive.metastore.TestRemoteHiveMetaStoreZK.testDropDatabaseCascadeMVMultiDB
(batchId=230)
org.apache.hadoop.hive.metastore.TestRemoteHiveMetaStoreZK.testDropTable
(batchId=230)
org.apache.hadoop.hive.metastore.TestRemoteHiveMetaStoreZK.testFilterLastPartition
(batchId=230)
org.apache.hadoop.hive.metastore.TestRemoteHiveMetaStoreZK.testFilterSinglePartition
(batchId=230)
org.apache.hadoop.hive.metastore.TestRemoteHiveMetaStoreZK.testFunctionWithResources
(batchId=230)
org.apache.hadoop.hive.metastore.TestRemoteHiveMetaStoreZK.testGetConfigValue
(batchId=230)
org.apache.hadoop.hive.metastore.TestRemoteHiveMetaStoreZK.testGetMetastoreUuid
(batchId=230)
org.apache.hadoop.hive.metastore.TestRemoteHiveMetaStoreZK.testGetPartitionsWithSpec
(batchId=230)
org.apache.hadoop.hive.metastore.TestRemoteHiveMetaStoreZK.testGetSchemaWithNoClassDefFoundError
(batchId=230)
org.apache.hadoop.hive.metastore.TestRemoteHiveMetaStoreZK.testGetTableObjects
(batchId=230)
org.apache.hadoop.hive.metastore.TestRemoteHiveMetaStoreZK.testGetUUIDInParallel
(batchId=230)
org.apache.hadoop.hive.metastore.TestRemoteHiveMetaStoreZK.testJDOPersistanceManagerCleanup
(batchId=230)
org.apache.hadoop.hive.metastore.TestRemoteHiveMetaStoreZK.testListPartitionNames
(batchId=230)
org.apache.hadoop.hive.metastore.TestRemoteHiveMetaStoreZK.testListPartitions
(batchId=230)
org.apache.hadoop.hive.metastore.TestRemoteHiveMetaStoreZK.testListPartitionsWihtLimitEnabled
(batchId=230)
org.apache.hadoop.hive.metastore.TestRemoteHiveMetaStoreZK.testNameMethods
(batchId=230)
org.apache.hadoop.hive.metastore.TestRemoteHiveMetaStoreZK.testPartition
(batchId=230)
org.apache.hadoop.hive.metastore.TestRemoteHiveMetaStoreZK.testPartitionFilter
(batchId=230)
org.apache.hadoop.hive.metastore.TestRemoteHiveMetaStoreZK.testRenamePartition
(batchId=230)
org.apache.hadoop.hive.metastore.TestRemoteHiveMetaStoreZK.testRetriableClientWithConnLifetime
(batchId=230)
org.apache.hadoop.hive.metastore.TestRemoteHiveMetaStoreZK.testSimpleFunction
(batchId=230)
org.apache.hadoop.hive.metastore.TestRemoteHiveMetaStoreZK.testSimpleTable
(batchId=230)
org.apache.hadoop.hive.metastore.TestRemoteHiveMetaStoreZK.testSimpleTypeApi
(batchId=230)
org.apache.hadoop.hive.metastore.TestRemoteHiveMetaStoreZK.testStatsFastTrivial
(batchId=230)
org.apache.hadoop.hive.metastore.TestRemoteHiveMetaStoreZK.testSynchronized
(batchId=230)
org.apache.hadoop.hive.metastore.TestRemoteHiveMetaStoreZK.testTableDatabase
(batchId=230)
org.apache.hadoop.hive.metastore.TestRemoteHiveMetaStoreZK.testTableFilter
(batchId=230)
[jira] [Commented] (HIVE-20982) Avoid the un-needed object creation within hotloop
[
https://issues.apache.org/jira/browse/HIVE-20982?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16704043#comment-16704043
]
Hive QA commented on HIVE-20982:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
30s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m
38s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
17s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
46s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
46s{color} | {color:blue} ql in master has 2312 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
26s{color} | {color:blue} druid-handler in master has 4 extant Findbugs
warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
8s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
24s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
53s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
18s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
18s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
45s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
2s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
52s{color} | {color:green} ql in the patch passed. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
32s{color} | {color:green} druid-handler generated 0 new + 3 unchanged - 1
fixed = 3 total (was 4) {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
6s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
13s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 26m 17s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc xml compile findbugs
checkstyle |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-15121/dev-support/hive-personality.sh
|
| git revision | master / 8411ea5 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| modules | C: ql druid-handler U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-15121/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Avoid the un-needed object creation within hotloop
> --
>
> Key: HIVE-20982
> URL: https://issues.apache.org/jira/browse/HIVE-20982
> Project: Hive
> Issue Type: Sub-task
>Reporter: slim bouguerra
>Assignee: slim bouguerra
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-20982.2.patch, HIVE-20982.patch
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-20985) If select operator inputs are temporary columns vectorization may reuse some of them as output
[ https://issues.apache.org/jira/browse/HIVE-20985?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16704012#comment-16704012 ] Zoltan Haindrich commented on HIVE-20985: - yes; it disables reuse inside a select expression; I think we can't reuse possible inputs during expression evaluation. If those columns may contain non-needed reusable parts - then they are not freed when they became un-needed. But I think we need consistency and then fix reuse. note: I think un-needed columns are not transfered - as I think after a vector operator is executed only outputcolumns are taken forward; temporary computations are left out; so even thru higher indexed columns are used; the data volume will not increase. > If select operator inputs are temporary columns vectorization may reuse some > of them as output > -- > > Key: HIVE-20985 > URL: https://issues.apache.org/jira/browse/HIVE-20985 > Project: Hive > Issue Type: Bug > Components: Vectorization >Reporter: Zoltan Haindrich >Assignee: Zoltan Haindrich >Priority: Major > Attachments: HIVE-20985.01.patch, HIVE-20985.02.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-20740) Remove global lock in ObjectStore.setConf method
[
https://issues.apache.org/jira/browse/HIVE-20740?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vihang Karajgaonkar updated HIVE-20740:
---
Resolution: Fixed
Fix Version/s: 3.2.0
4.0.0
Status: Resolved (was: Patch Available)
Patch merged into master and branch-3. Thanks for your reviews [~ngangam]
[~asherman] and [~karthik.manamcheri]
> Remove global lock in ObjectStore.setConf method
>
>
> Key: HIVE-20740
> URL: https://issues.apache.org/jira/browse/HIVE-20740
> Project: Hive
> Issue Type: Improvement
>Reporter: Vihang Karajgaonkar
>Assignee: Vihang Karajgaonkar
>Priority: Major
> Fix For: 4.0.0, 3.2.0
>
> Attachments: HIVE-20740.01.patch, HIVE-20740.02.patch,
> HIVE-20740.04.patch, HIVE-20740.05.patch, HIVE-20740.06.patch,
> HIVE-20740.08.patch, HIVE-20740.09.patch, HIVE-20740.10.patch,
> HIVE-20740.11.patch, HIVE-20740.12.patch, HIVE-20740.13.patch,
> HIVE-20740.14.patch
>
>
> The ObjectStore#setConf method has a global lock which can block other
> clients in concurrent workloads.
> {code}
> @Override
> @SuppressWarnings("nls")
> public void setConf(Configuration conf) {
> // Although an instance of ObjectStore is accessed by one thread, there
> may
> // be many threads with ObjectStore instances. So the static variables
> // pmf and prop need to be protected with locks.
> pmfPropLock.lock();
> try {
> isInitialized = false;
> this.conf = conf;
> this.areTxnStatsSupported = MetastoreConf.getBoolVar(conf,
> ConfVars.HIVE_TXN_STATS_ENABLED);
> configureSSL(conf);
> Properties propsFromConf = getDataSourceProps(conf);
> boolean propsChanged = !propsFromConf.equals(prop);
> if (propsChanged) {
> if (pmf != null){
> clearOutPmfClassLoaderCache(pmf);
> if (!forTwoMetastoreTesting) {
> // close the underlying connection pool to avoid leaks
> pmf.close();
> }
> }
> pmf = null;
> prop = null;
> }
> assert(!isActiveTransaction());
> shutdown();
> // Always want to re-create pm as we don't know if it were created by
> the
> // most recent instance of the pmf
> pm = null;
> directSql = null;
> expressionProxy = null;
> openTrasactionCalls = 0;
> currentTransaction = null;
> transactionStatus = TXN_STATUS.NO_STATE;
> initialize(propsFromConf);
> String partitionValidationRegex =
> MetastoreConf.getVar(this.conf,
> ConfVars.PARTITION_NAME_WHITELIST_PATTERN);
> if (partitionValidationRegex != null &&
> !partitionValidationRegex.isEmpty()) {
> partitionValidationPattern =
> Pattern.compile(partitionValidationRegex);
> } else {
> partitionValidationPattern = null;
> }
> // Note, if metrics have not been initialized this will return null,
> which means we aren't
> // using metrics. Thus we should always check whether this is non-null
> before using.
> MetricRegistry registry = Metrics.getRegistry();
> if (registry != null) {
> directSqlErrors =
> Metrics.getOrCreateCounter(MetricsConstants.DIRECTSQL_ERRORS);
> }
> this.batchSize = MetastoreConf.getIntVar(conf,
> ConfVars.RAWSTORE_PARTITION_BATCH_SIZE);
> if (!isInitialized) {
> throw new RuntimeException(
> "Unable to create persistence manager. Check dss.log for details");
> } else {
> LOG.debug("Initialized ObjectStore");
> }
> } finally {
> pmfPropLock.unlock();
> }
> }
> {code}
> The {{pmfPropLock}} is a static object and it disallows any other new
> connection to HMS which is trying to instantiate ObjectStore. We should
> either remove the lock or reduce the scope of the lock so that it is held for
> a very small amount of time.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-20985) If select operator inputs are temporary columns vectorization may reuse some of them as output
[
https://issues.apache.org/jira/browse/HIVE-20985?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16704007#comment-16704007
]
Hive QA commented on HIVE-20985:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12950076/HIVE-20985.02.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 3 failed/errored test(s), 15638 tests
executed
*Failed tests:*
{noformat}
TestMiniDruidCliDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=196)
[druidmini_dynamic_partition.q,druidmini_test1.q,druidkafkamini_basic.q,druidmini_test_insert.q,druidkafkamini_delimited.q]
org.apache.hadoop.hive.ql.exec.vector.TestVectorizationContext.testArithmeticExpressionVectorization
(batchId=309)
org.apache.hadoop.hive.ql.exec.vector.TestVectorizationContext.testNullExpressions
(batchId=309)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/15120/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/15120/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-15120/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 3 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12950076 - PreCommit-HIVE-Build
> If select operator inputs are temporary columns vectorization may reuse some
> of them as output
> --
>
> Key: HIVE-20985
> URL: https://issues.apache.org/jira/browse/HIVE-20985
> Project: Hive
> Issue Type: Bug
> Components: Vectorization
>Reporter: Zoltan Haindrich
>Assignee: Zoltan Haindrich
>Priority: Major
> Attachments: HIVE-20985.01.patch, HIVE-20985.02.patch
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-20979) Fix memory leak in hive streaming
[
https://issues.apache.org/jira/browse/HIVE-20979?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16703999#comment-16703999
]
Ashutosh Chauhan commented on HIVE-20979:
-
+1
> Fix memory leak in hive streaming
> -
>
> Key: HIVE-20979
> URL: https://issues.apache.org/jira/browse/HIVE-20979
> Project: Hive
> Issue Type: Bug
> Components: Streaming
>Affects Versions: 3.1.1
>Reporter: Shubham Chaurasia
>Assignee: Shubham Chaurasia
>Priority: Critical
> Labels: pull-request-available
> Attachments: HIVE-20979.1.patch, HIVE-20979.1.patch,
> HIVE-20979.2.patch, HIVE-20979.3.patch, HIVE-20979.4.patch
>
>
> {{1) HiveStreamingConnection.Builder#init() adds a shutdown hook handler via
> }}{{ShutdownHookManager.addShutdownHook but it is never removed which causes
> all the handlers to accumulate and hence a memory leak.}}
> 2) AbstractRecordWriter creates an instance of FileSystem but does not close
> it which in turn causes a leak due to accumulation in FileSystem$Cache#map
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-20985) If select operator inputs are temporary columns vectorization may reuse some of them as output
[ https://issues.apache.org/jira/browse/HIVE-20985?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16703927#comment-16703927 ] Jason Dere commented on HIVE-20985: --- Looks like the patch ends up disabling most of the scratch column re-use. I don't have a lot of knowledge of the vectorization details, cc [~teddy.choi] [~gopalv] > If select operator inputs are temporary columns vectorization may reuse some > of them as output > -- > > Key: HIVE-20985 > URL: https://issues.apache.org/jira/browse/HIVE-20985 > Project: Hive > Issue Type: Bug > Components: Vectorization >Reporter: Zoltan Haindrich >Assignee: Zoltan Haindrich >Priority: Major > Attachments: HIVE-20985.01.patch, HIVE-20985.02.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-20985) If select operator inputs are temporary columns vectorization may reuse some of them as output
[
https://issues.apache.org/jira/browse/HIVE-20985?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16703922#comment-16703922
]
Hive QA commented on HIVE-20985:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 9m
4s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
2s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
45s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
43s{color} | {color:blue} ql in master has 2312 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
53s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
20s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
1s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
1s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
46s{color} | {color:green} ql: The patch generated 0 new + 775 unchanged - 1
fixed = 775 total (was 776) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
52s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
53s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
13s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 24m 6s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-15120/dev-support/hive-personality.sh
|
| git revision | master / 75c6ee4 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-15120/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> If select operator inputs are temporary columns vectorization may reuse some
> of them as output
> --
>
> Key: HIVE-20985
> URL: https://issues.apache.org/jira/browse/HIVE-20985
> Project: Hive
> Issue Type: Bug
> Components: Vectorization
>Reporter: Zoltan Haindrich
>Assignee: Zoltan Haindrich
>Priority: Major
> Attachments: HIVE-20985.01.patch, HIVE-20985.02.patch
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-20982) Avoid the un-needed object creation within hotloop
[ https://issues.apache.org/jira/browse/HIVE-20982?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] slim bouguerra updated HIVE-20982: -- Attachment: HIVE-20982.2.patch > Avoid the un-needed object creation within hotloop > -- > > Key: HIVE-20982 > URL: https://issues.apache.org/jira/browse/HIVE-20982 > Project: Hive > Issue Type: Sub-task >Reporter: slim bouguerra >Assignee: slim bouguerra >Priority: Major > Labels: pull-request-available > Attachments: HIVE-20982.2.patch, HIVE-20982.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-20979) Fix memory leak in hive streaming
[
https://issues.apache.org/jira/browse/HIVE-20979?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16703878#comment-16703878
]
Eric Wohlstadter commented on HIVE-20979:
-
[~ShubhamChaurasia]
lgtm
> Fix memory leak in hive streaming
> -
>
> Key: HIVE-20979
> URL: https://issues.apache.org/jira/browse/HIVE-20979
> Project: Hive
> Issue Type: Bug
> Components: Streaming
>Affects Versions: 3.1.1
>Reporter: Shubham Chaurasia
>Assignee: Shubham Chaurasia
>Priority: Critical
> Labels: pull-request-available
> Attachments: HIVE-20979.1.patch, HIVE-20979.1.patch,
> HIVE-20979.2.patch, HIVE-20979.3.patch, HIVE-20979.4.patch
>
>
> {{1) HiveStreamingConnection.Builder#init() adds a shutdown hook handler via
> }}{{ShutdownHookManager.addShutdownHook but it is never removed which causes
> all the handlers to accumulate and hence a memory leak.}}
> 2) AbstractRecordWriter creates an instance of FileSystem but does not close
> it which in turn causes a leak due to accumulation in FileSystem$Cache#map
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-20985) If select operator inputs are temporary columns vectorization may reuse some of them as output
[ https://issues.apache.org/jira/browse/HIVE-20985?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zoltan Haindrich updated HIVE-20985: Attachment: HIVE-20985.02.patch > If select operator inputs are temporary columns vectorization may reuse some > of them as output > -- > > Key: HIVE-20985 > URL: https://issues.apache.org/jira/browse/HIVE-20985 > Project: Hive > Issue Type: Bug > Components: Vectorization >Reporter: Zoltan Haindrich >Assignee: Zoltan Haindrich >Priority: Major > Attachments: HIVE-20985.01.patch, HIVE-20985.02.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-20985) If select operator inputs are temporary columns vectorization may reuse some of them as output
[
https://issues.apache.org/jira/browse/HIVE-20985?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16703816#comment-16703816
]
Hive QA commented on HIVE-20985:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12950044/HIVE-20985.01.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 152 failed/errored test(s), 15643 tests
executed
*Failed tests:*
{noformat}
TestMiniDruidCliDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=197)
[druidmini_masking.q,druidmini_joins.q,druid_timestamptz.q]
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[orc_struct_type_vectorization]
(batchId=74)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[parquet_complex_types_vectorization]
(batchId=77)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[parquet_map_type_vectorization]
(batchId=90)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[parquet_struct_type_vectorization]
(batchId=28)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[parquet_vectorization_0]
(batchId=18)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[parquet_vectorization_10]
(batchId=24)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[parquet_vectorization_11]
(batchId=40)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[parquet_vectorization_12]
(batchId=25)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[parquet_vectorization_13]
(batchId=55)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[parquet_vectorization_14]
(batchId=41)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[parquet_vectorization_15]
(batchId=93)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[parquet_vectorization_17]
(batchId=31)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[parquet_vectorization_1]
(batchId=12)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[parquet_vectorization_2]
(batchId=3)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[parquet_vectorization_3]
(batchId=82)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[parquet_vectorization_7]
(batchId=91)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[parquet_vectorization_8]
(batchId=15)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[parquet_vectorization_div0]
(batchId=83)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[vector_case_when_1]
(batchId=91)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[vector_case_when_2]
(batchId=58)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[vector_coalesce]
(batchId=12)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[vector_coalesce_2]
(batchId=77)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[vector_decimal_aggregate]
(batchId=19)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[vector_decimal_expressions]
(batchId=57)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[vector_decimal_math_funcs]
(batchId=25)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[vector_elt] (batchId=38)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[vector_interval_1]
(batchId=17)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[vector_string_concat]
(batchId=35)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[vectorization_10]
(batchId=27)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[vectorization_11]
(batchId=41)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[vectorization_12]
(batchId=11)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[vectorization_13]
(batchId=54)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[vectorization_14]
(batchId=16)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[vectorization_15]
(batchId=70)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[vectorization_17]
(batchId=95)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[vectorization_1]
(batchId=63)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[vectorization_2]
(batchId=22)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[vectorization_3]
(batchId=83)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[vectorization_7]
(batchId=47)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[vectorization_8]
(batchId=50)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[vectorization_numeric_overflows]
(batchId=74)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[vectorized_case]
(batchId=62)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[vectorized_casts]
(batchId=89)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[vectorized_date_funcs]
(batchId=82)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[vectorized_math_funcs]
(batchId=22)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[vectorized_timestamp_funcs]
(batchId=32)
[jira] [Updated] (HIVE-20981) streaming/AbstractRecordWriter leaks HeapMemoryMonitor
[ https://issues.apache.org/jira/browse/HIVE-20981?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Dere updated HIVE-20981: -- Resolution: Fixed Fix Version/s: 4.0.0 Status: Resolved (was: Patch Available) Committed to master > streaming/AbstractRecordWriter leaks HeapMemoryMonitor > -- > > Key: HIVE-20981 > URL: https://issues.apache.org/jira/browse/HIVE-20981 > Project: Hive > Issue Type: Bug >Reporter: Eric Wohlstadter >Assignee: Eric Wohlstadter >Priority: Major > Fix For: 4.0.0 > > Attachments: HIVE-20981.1.patch > > > Each record writer registers a memory monitor with the MemoryMXBean but they > aren't removed. So the listener objects/lambdas accumulate over time in the > bean. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-20985) If select operator inputs are temporary columns vectorization may reuse some of them as output
[
https://issues.apache.org/jira/browse/HIVE-20985?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16703742#comment-16703742
]
Hive QA commented on HIVE-20985:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
56s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
2s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
44s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
46s{color} | {color:blue} ql in master has 2312 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
54s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
24s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
6s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
6s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
44s{color} | {color:green} ql: The patch generated 0 new + 775 unchanged - 1
fixed = 775 total (was 776) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
0s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
57s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
15s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 23m 18s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-15119/dev-support/hive-personality.sh
|
| git revision | master / 3e98143 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-15119/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> If select operator inputs are temporary columns vectorization may reuse some
> of them as output
> --
>
> Key: HIVE-20985
> URL: https://issues.apache.org/jira/browse/HIVE-20985
> Project: Hive
> Issue Type: Bug
> Components: Vectorization
>Reporter: Zoltan Haindrich
>Assignee: Zoltan Haindrich
>Priority: Major
> Attachments: HIVE-20985.01.patch
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-20740) Remove global lock in ObjectStore.setConf method
[
https://issues.apache.org/jira/browse/HIVE-20740?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16703714#comment-16703714
]
Naveen Gangam commented on HIVE-20740:
--
Looks good to me. +1
> Remove global lock in ObjectStore.setConf method
>
>
> Key: HIVE-20740
> URL: https://issues.apache.org/jira/browse/HIVE-20740
> Project: Hive
> Issue Type: Improvement
>Reporter: Vihang Karajgaonkar
>Assignee: Vihang Karajgaonkar
>Priority: Major
> Attachments: HIVE-20740.01.patch, HIVE-20740.02.patch,
> HIVE-20740.04.patch, HIVE-20740.05.patch, HIVE-20740.06.patch,
> HIVE-20740.08.patch, HIVE-20740.09.patch, HIVE-20740.10.patch,
> HIVE-20740.11.patch, HIVE-20740.12.patch, HIVE-20740.13.patch,
> HIVE-20740.14.patch
>
>
> The ObjectStore#setConf method has a global lock which can block other
> clients in concurrent workloads.
> {code}
> @Override
> @SuppressWarnings("nls")
> public void setConf(Configuration conf) {
> // Although an instance of ObjectStore is accessed by one thread, there
> may
> // be many threads with ObjectStore instances. So the static variables
> // pmf and prop need to be protected with locks.
> pmfPropLock.lock();
> try {
> isInitialized = false;
> this.conf = conf;
> this.areTxnStatsSupported = MetastoreConf.getBoolVar(conf,
> ConfVars.HIVE_TXN_STATS_ENABLED);
> configureSSL(conf);
> Properties propsFromConf = getDataSourceProps(conf);
> boolean propsChanged = !propsFromConf.equals(prop);
> if (propsChanged) {
> if (pmf != null){
> clearOutPmfClassLoaderCache(pmf);
> if (!forTwoMetastoreTesting) {
> // close the underlying connection pool to avoid leaks
> pmf.close();
> }
> }
> pmf = null;
> prop = null;
> }
> assert(!isActiveTransaction());
> shutdown();
> // Always want to re-create pm as we don't know if it were created by
> the
> // most recent instance of the pmf
> pm = null;
> directSql = null;
> expressionProxy = null;
> openTrasactionCalls = 0;
> currentTransaction = null;
> transactionStatus = TXN_STATUS.NO_STATE;
> initialize(propsFromConf);
> String partitionValidationRegex =
> MetastoreConf.getVar(this.conf,
> ConfVars.PARTITION_NAME_WHITELIST_PATTERN);
> if (partitionValidationRegex != null &&
> !partitionValidationRegex.isEmpty()) {
> partitionValidationPattern =
> Pattern.compile(partitionValidationRegex);
> } else {
> partitionValidationPattern = null;
> }
> // Note, if metrics have not been initialized this will return null,
> which means we aren't
> // using metrics. Thus we should always check whether this is non-null
> before using.
> MetricRegistry registry = Metrics.getRegistry();
> if (registry != null) {
> directSqlErrors =
> Metrics.getOrCreateCounter(MetricsConstants.DIRECTSQL_ERRORS);
> }
> this.batchSize = MetastoreConf.getIntVar(conf,
> ConfVars.RAWSTORE_PARTITION_BATCH_SIZE);
> if (!isInitialized) {
> throw new RuntimeException(
> "Unable to create persistence manager. Check dss.log for details");
> } else {
> LOG.debug("Initialized ObjectStore");
> }
> } finally {
> pmfPropLock.unlock();
> }
> }
> {code}
> The {{pmfPropLock}} is a static object and it disallows any other new
> connection to HMS which is trying to instantiate ObjectStore. We should
> either remove the lock or reduce the scope of the lock so that it is held for
> a very small amount of time.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-20897) TestJdbcDriver2#testSelectExecAsync2 fails with result set not present error
[
https://issues.apache.org/jira/browse/HIVE-20897?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16703713#comment-16703713
]
Hive QA commented on HIVE-20897:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12950031/HIVE-20897.08.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 55 failed/errored test(s), 15617 tests
executed
*Failed tests:*
{noformat}
TestHiveKVResultCache - did not produce a TEST-*.xml file (likely timed out)
(batchId=317)
TestHiveSparkClient - did not produce a TEST-*.xml file (likely timed out)
(batchId=317)
TestLocalHiveSparkClient - did not produce a TEST-*.xml file (likely timed out)
(batchId=317)
TestMapRedTask - did not produce a TEST-*.xml file (likely timed out)
(batchId=317)
TestMapredLocalTask - did not produce a TEST-*.xml file (likely timed out)
(batchId=317)
TestSparkInvalidFileFormat - did not produce a TEST-*.xml file (likely timed
out) (batchId=317)
TestSparkJobMonitor - did not produce a TEST-*.xml file (likely timed out)
(batchId=317)
TestSparkPlan - did not produce a TEST-*.xml file (likely timed out)
(batchId=317)
TestSparkSessionManagerImpl - did not produce a TEST-*.xml file (likely timed
out) (batchId=317)
TestSparkTask - did not produce a TEST-*.xml file (likely timed out)
(batchId=317)
org.apache.hadoop.hive.metastore.TestRemoteHiveMetaStoreZKBindHost.testAlterPartition
(batchId=232)
org.apache.hadoop.hive.metastore.TestRemoteHiveMetaStoreZKBindHost.testAlterTable
(batchId=232)
org.apache.hadoop.hive.metastore.TestRemoteHiveMetaStoreZKBindHost.testAlterTableCascade
(batchId=232)
org.apache.hadoop.hive.metastore.TestRemoteHiveMetaStoreZKBindHost.testAlterViewParititon
(batchId=232)
org.apache.hadoop.hive.metastore.TestRemoteHiveMetaStoreZKBindHost.testColumnStatistics
(batchId=232)
org.apache.hadoop.hive.metastore.TestRemoteHiveMetaStoreZKBindHost.testComplexTable
(batchId=232)
org.apache.hadoop.hive.metastore.TestRemoteHiveMetaStoreZKBindHost.testComplexTypeApi
(batchId=232)
org.apache.hadoop.hive.metastore.TestRemoteHiveMetaStoreZKBindHost.testConcurrentMetastores
(batchId=232)
org.apache.hadoop.hive.metastore.TestRemoteHiveMetaStoreZKBindHost.testCreateAndGetTableWithDriver
(batchId=232)
org.apache.hadoop.hive.metastore.TestRemoteHiveMetaStoreZKBindHost.testCreateTableSettingId
(batchId=232)
org.apache.hadoop.hive.metastore.TestRemoteHiveMetaStoreZKBindHost.testDBLocationChange
(batchId=232)
org.apache.hadoop.hive.metastore.TestRemoteHiveMetaStoreZKBindHost.testDBOwner
(batchId=232)
org.apache.hadoop.hive.metastore.TestRemoteHiveMetaStoreZKBindHost.testDBOwnerChange
(batchId=232)
org.apache.hadoop.hive.metastore.TestRemoteHiveMetaStoreZKBindHost.testDatabase
(batchId=232)
org.apache.hadoop.hive.metastore.TestRemoteHiveMetaStoreZKBindHost.testDatabaseLocation
(batchId=232)
org.apache.hadoop.hive.metastore.TestRemoteHiveMetaStoreZKBindHost.testDatabaseLocationWithPermissionProblems
(batchId=232)
org.apache.hadoop.hive.metastore.TestRemoteHiveMetaStoreZKBindHost.testDropDatabaseCascadeMVMultiDB
(batchId=232)
org.apache.hadoop.hive.metastore.TestRemoteHiveMetaStoreZKBindHost.testDropTable
(batchId=232)
org.apache.hadoop.hive.metastore.TestRemoteHiveMetaStoreZKBindHost.testFilterLastPartition
(batchId=232)
org.apache.hadoop.hive.metastore.TestRemoteHiveMetaStoreZKBindHost.testFilterSinglePartition
(batchId=232)
org.apache.hadoop.hive.metastore.TestRemoteHiveMetaStoreZKBindHost.testFunctionWithResources
(batchId=232)
org.apache.hadoop.hive.metastore.TestRemoteHiveMetaStoreZKBindHost.testGetConfigValue
(batchId=232)
org.apache.hadoop.hive.metastore.TestRemoteHiveMetaStoreZKBindHost.testGetMetastoreUuid
(batchId=232)
org.apache.hadoop.hive.metastore.TestRemoteHiveMetaStoreZKBindHost.testGetPartitionsWithSpec
(batchId=232)
org.apache.hadoop.hive.metastore.TestRemoteHiveMetaStoreZKBindHost.testGetSchemaWithNoClassDefFoundError
(batchId=232)
org.apache.hadoop.hive.metastore.TestRemoteHiveMetaStoreZKBindHost.testGetTableObjects
(batchId=232)
org.apache.hadoop.hive.metastore.TestRemoteHiveMetaStoreZKBindHost.testGetUUIDInParallel
(batchId=232)
org.apache.hadoop.hive.metastore.TestRemoteHiveMetaStoreZKBindHost.testJDOPersistanceManagerCleanup
(batchId=232)
org.apache.hadoop.hive.metastore.TestRemoteHiveMetaStoreZKBindHost.testListPartitionNames
(batchId=232)
org.apache.hadoop.hive.metastore.TestRemoteHiveMetaStoreZKBindHost.testListPartitions
(batchId=232)
org.apache.hadoop.hive.metastore.TestRemoteHiveMetaStoreZKBindHost.testListPartitionsWihtLimitEnabled
(batchId=232)
org.apache.hadoop.hive.metastore.TestRemoteHiveMetaStoreZKBindHost.testNameMethods
(batchId=232)
org.apache.hadoop.hive.metastore.TestRemoteHiveMetaStoreZKBindHost.testPartition
(batchId=232)
org.apache.hadoop.hive.metastore.TestRemoteHiveMetaStoreZKBindHost.testPartitionFilter
[jira] [Resolved] (HIVE-20770) Need improvement in hive for ACID properties and tables
[ https://issues.apache.org/jira/browse/HIVE-20770?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Voros resolved HIVE-20770. - Resolution: Invalid > Need improvement in hive for ACID properties and tables > --- > > Key: HIVE-20770 > URL: https://issues.apache.org/jira/browse/HIVE-20770 > Project: Hive > Issue Type: Improvement > Components: Hive >Reporter: pritam >Assignee: Daniel Voros >Priority: Major > > Hello Team, > In current version of Apache Hive if we set ACID properties it can not be > revert as well as > Apache Spark does not support Hive ACID table. If it is possible to revert > ACID properties from hive table and read-write Hive table from spark-scala > please let me know. > If there is no provision for above conditions it is important to add features > and improvement in next Apache Hive version. It will be very helpful for all > in distributed batch processing. > I am eager to hear from you. Thank you all in advance for such a great batch > processing tool. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-18415) Lower "Updating Partition Stats" Logging Level
[
https://issues.apache.org/jira/browse/HIVE-18415?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16703701#comment-16703701
]
Naveen Gangam commented on HIVE-18415:
--
[~belugabehr] Should we merge the 2 separate log messages into a single log
record to reduce the noise and make it INFO instead of DEBUG? If we think
having separate log messages is useful to have some sense of timing on how long
it takes to compute statistics, we can leave them both. Also should we change
the log message to include the partition location as well, instead of just the
table name. This will give us a better sense of timing per partition. Thanks
> Lower "Updating Partition Stats" Logging Level
> --
>
> Key: HIVE-18415
> URL: https://issues.apache.org/jira/browse/HIVE-18415
> Project: Hive
> Issue Type: Improvement
> Components: Metastore
>Affects Versions: 1.2.2, 2.2.0, 3.0.0, 2.3.2
>Reporter: BELUGA BEHR
>Assignee: BELUGA BEHR
>Priority: Trivial
> Attachments: HIVE-18415.1.patch
>
>
> {code:title=org.apache.hadoop.hive.metastore.utils.MetaStoreUtils}
> LOG.warn("Updating partition stats fast for: " + part.getTableName());
> ...
> LOG.warn("Updated size to " + params.get(StatsSetupConst.TOTAL_SIZE));
> {code}
> This logging produces many lines of WARN log messages in my log file and it's
> not clear to me what the issue is here. Why is this a warning and how should
> I respond to address this warning?
> DEBUG is probably more appropriate for a utility class. Please lower.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-20982) Avoid the un-needed object creation within hotloop
[
https://issues.apache.org/jira/browse/HIVE-20982?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16703691#comment-16703691
]
Gopal V commented on HIVE-20982:
LGTM - +1 (heh, {{* @param output Rowboat used to carry columns values.}}).
Minor nit - the null setter can be made default, so that the entry is always
cleared.
{code:java}
final Object value = input.isCompacted() ?
input.getCompactedValue().get(i) : input.getValue().get(columns[i]);
if (value == null) {
-output.add(null);
+output[i] = (null);
continue;
}
{code}
into
{code:java}
final Object value = input.isCompacted() ?
input.getCompactedValue().get(i) : input.getValue().get(columns[i]);
+ output[i] = (null);
if (value == null) {
-output.add(null);
continue;
}
{code}
> Avoid the un-needed object creation within hotloop
> --
>
> Key: HIVE-20982
> URL: https://issues.apache.org/jira/browse/HIVE-20982
> Project: Hive
> Issue Type: Sub-task
>Reporter: slim bouguerra
>Assignee: slim bouguerra
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-20982.patch
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-20897) TestJdbcDriver2#testSelectExecAsync2 fails with result set not present error
[
https://issues.apache.org/jira/browse/HIVE-20897?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16703636#comment-16703636
]
Hive QA commented on HIVE-20897:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
50s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
23s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
12s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
36s{color} | {color:blue} service in master has 48 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
16s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
24s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
22s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
22s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
11s{color} | {color:green} service: The patch generated 0 new + 59 unchanged -
1 fixed = 59 total (was 60) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
45s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
16s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
12s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 11m 56s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-15118/dev-support/hive-personality.sh
|
| git revision | master / 3e98143 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| modules | C: service U: service |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-15118/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> TestJdbcDriver2#testSelectExecAsync2 fails with result set not present error
>
>
> Key: HIVE-20897
> URL: https://issues.apache.org/jira/browse/HIVE-20897
> Project: Hive
> Issue Type: Bug
> Components: Hive
>Reporter: mahesh kumar behera
>Assignee: mahesh kumar behera
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-20897.01.patch, HIVE-20897.02.patch,
> HIVE-20897.03.patch, HIVE-20897.04.patch, HIVE-20897.05.patch,
> HIVE-20897.06.patch, HIVE-20897.07.patch, HIVE-20897.07.patch,
> HIVE-20897.08.patch, HIVE-20897.08.patch, HIVE-20897.08.patch
>
>
> if async prepare is enabled, control will be returned to the client before
> driver could set of the query has a result set or not. But in current code,
> while generating the response for the query, it is not checked if the result
> set field is set or not.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-20891) Call alter_partition in batch when dynamically loading partitions
[
https://issues.apache.org/jira/browse/HIVE-20891?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16703620#comment-16703620
]
Hive QA commented on HIVE-20891:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12950016/HIVE-20891.07.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 15643 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/15117/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/15117/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-15117/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12950016 - PreCommit-HIVE-Build
> Call alter_partition in batch when dynamically loading partitions
> -
>
> Key: HIVE-20891
> URL: https://issues.apache.org/jira/browse/HIVE-20891
> Project: Hive
> Issue Type: Improvement
> Components: Hive
>Affects Versions: 4.0.0
>Reporter: Laszlo Pinter
>Assignee: Laszlo Pinter
>Priority: Major
> Attachments: HIVE-20891.01.patch, HIVE-20891.02.patch,
> HIVE-20891.03.patch, HIVE-20891.04.patch, HIVE-20891.05.patch,
> HIVE-20891.06.patch, HIVE-20891.07.patch
>
>
> When dynamically loading partitions, the setStatsPropAndAlterPartition() is
> called for each partition one by one, resulting in unnecessary calls to the
> metastore client. This whole logic can be changed to just one call.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Assigned] (HIVE-20986) Add TransactionalValidationListener to HMS preListeners only when ACID support is enabled
[
https://issues.apache.org/jira/browse/HIVE-20986?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Karthik Manamcheri reassigned HIVE-20986:
-
> Add TransactionalValidationListener to HMS preListeners only when ACID
> support is enabled
> -
>
> Key: HIVE-20986
> URL: https://issues.apache.org/jira/browse/HIVE-20986
> Project: Hive
> Issue Type: Improvement
>Reporter: Karthik Manamcheri
>Assignee: Adam Holley
>Priority: Major
>
> We add the TransactionalValidationListener to the preListeners in HMS
> unconditionally.
> {code:java}
> public void init() throws MetaException {
> ..
> preListeners.add(0, new TransactionalValidationListener(conf));
> ..
> }{code}
> This causes some performance issues because the listener is called even when
> not needed. Lets add a condition around this and add this listener only if
> the transactional support is enabled.
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-20891) Call alter_partition in batch when dynamically loading partitions
[
https://issues.apache.org/jira/browse/HIVE-20891?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16703548#comment-16703548
]
Hive QA commented on HIVE-20891:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
49s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
59s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
38s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
43s{color} | {color:blue} ql in master has 2312 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
53s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
23s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
59s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
59s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
37s{color} | {color:red} ql: The patch generated 1 new + 218 unchanged - 0
fixed = 219 total (was 218) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
52s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
53s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
13s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 22m 28s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-15117/dev-support/hive-personality.sh
|
| git revision | master / 3e98143 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-15117/yetus/diff-checkstyle-ql.txt
|
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-15117/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Call alter_partition in batch when dynamically loading partitions
> -
>
> Key: HIVE-20891
> URL: https://issues.apache.org/jira/browse/HIVE-20891
> Project: Hive
> Issue Type: Improvement
> Components: Hive
>Affects Versions: 4.0.0
>Reporter: Laszlo Pinter
>Assignee: Laszlo Pinter
>Priority: Major
> Attachments: HIVE-20891.01.patch, HIVE-20891.02.patch,
> HIVE-20891.03.patch, HIVE-20891.04.patch, HIVE-20891.05.patch,
> HIVE-20891.06.patch, HIVE-20891.07.patch
>
>
> When dynamically loading partitions, the setStatsPropAndAlterPartition() is
> called for each partition one by one, resulting in unnecessary calls to the
> metastore client. This whole logic can be changed to just one call.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Issue Comment Deleted] (HIVE-20985) If select operator inputs are temporary columns vectorization may reuse some of them as output
[ https://issues.apache.org/jira/browse/HIVE-20985?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashutosh Chauhan updated HIVE-20985: Comment: was deleted (was: [~kgyrtkirk] Did you forget to attach patch file?) > If select operator inputs are temporary columns vectorization may reuse some > of them as output > -- > > Key: HIVE-20985 > URL: https://issues.apache.org/jira/browse/HIVE-20985 > Project: Hive > Issue Type: Bug > Components: Vectorization >Reporter: Zoltan Haindrich >Assignee: Zoltan Haindrich >Priority: Major > Attachments: HIVE-20985.01.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-20915) Make dynamic sort partition optimization available to HoS and MR
[ https://issues.apache.org/jira/browse/HIVE-20915?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16703511#comment-16703511 ] Yongzhi Chen commented on HIVE-20915: - This patch makes sort dynamic partitions available for MR and HoS by using property hive.optimize.sort.dynamic.partition.threshold It uses the same cost based algorithm introduced by HIVE-20703 . We can improve the algorithm for MR and HoS with other jiras. [~pvary] , [~ngangam] could you review the patch? Thanks > Make dynamic sort partition optimization available to HoS and MR > > > Key: HIVE-20915 > URL: https://issues.apache.org/jira/browse/HIVE-20915 > Project: Hive > Issue Type: Bug > Components: Query Planning >Affects Versions: 4.0.0 >Reporter: Yongzhi Chen >Assignee: Yongzhi Chen >Priority: Major > Attachments: HIVE-20915.1.patch, HIVE-20915.2.patch, > HIVE-20915.3.patch, HIVE-20915.4.patch, HIVE-20915.5.patch > > > HIVE-20703 put dynamic sort partition optimization under cost based decision, > but it also makes the optimizer only available to tez. > hive.optimize.sort.dynamic.partition works with other execution engines for a > long time, we should keep the optimizer available to them. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-20979) Fix memory leak in hive streaming
[
https://issues.apache.org/jira/browse/HIVE-20979?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16703509#comment-16703509
]
Hive QA commented on HIVE-20979:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12950002/HIVE-20979.4.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 46 failed/errored test(s), 15643 tests
executed
*Failed tests:*
{noformat}
TestMiniDruidCliDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=197)
[druidmini_masking.q,druidmini_joins.q,druid_timestamptz.q]
org.apache.hadoop.hive.metastore.TestRemoteHiveMetaStoreZKBindHost.testAlterPartition
(batchId=232)
org.apache.hadoop.hive.metastore.TestRemoteHiveMetaStoreZKBindHost.testAlterTable
(batchId=232)
org.apache.hadoop.hive.metastore.TestRemoteHiveMetaStoreZKBindHost.testAlterTableCascade
(batchId=232)
org.apache.hadoop.hive.metastore.TestRemoteHiveMetaStoreZKBindHost.testAlterViewParititon
(batchId=232)
org.apache.hadoop.hive.metastore.TestRemoteHiveMetaStoreZKBindHost.testColumnStatistics
(batchId=232)
org.apache.hadoop.hive.metastore.TestRemoteHiveMetaStoreZKBindHost.testComplexTable
(batchId=232)
org.apache.hadoop.hive.metastore.TestRemoteHiveMetaStoreZKBindHost.testComplexTypeApi
(batchId=232)
org.apache.hadoop.hive.metastore.TestRemoteHiveMetaStoreZKBindHost.testConcurrentMetastores
(batchId=232)
org.apache.hadoop.hive.metastore.TestRemoteHiveMetaStoreZKBindHost.testCreateAndGetTableWithDriver
(batchId=232)
org.apache.hadoop.hive.metastore.TestRemoteHiveMetaStoreZKBindHost.testCreateTableSettingId
(batchId=232)
org.apache.hadoop.hive.metastore.TestRemoteHiveMetaStoreZKBindHost.testDBLocationChange
(batchId=232)
org.apache.hadoop.hive.metastore.TestRemoteHiveMetaStoreZKBindHost.testDBOwner
(batchId=232)
org.apache.hadoop.hive.metastore.TestRemoteHiveMetaStoreZKBindHost.testDBOwnerChange
(batchId=232)
org.apache.hadoop.hive.metastore.TestRemoteHiveMetaStoreZKBindHost.testDatabase
(batchId=232)
org.apache.hadoop.hive.metastore.TestRemoteHiveMetaStoreZKBindHost.testDatabaseLocation
(batchId=232)
org.apache.hadoop.hive.metastore.TestRemoteHiveMetaStoreZKBindHost.testDatabaseLocationWithPermissionProblems
(batchId=232)
org.apache.hadoop.hive.metastore.TestRemoteHiveMetaStoreZKBindHost.testDropDatabaseCascadeMVMultiDB
(batchId=232)
org.apache.hadoop.hive.metastore.TestRemoteHiveMetaStoreZKBindHost.testDropTable
(batchId=232)
org.apache.hadoop.hive.metastore.TestRemoteHiveMetaStoreZKBindHost.testFilterLastPartition
(batchId=232)
org.apache.hadoop.hive.metastore.TestRemoteHiveMetaStoreZKBindHost.testFilterSinglePartition
(batchId=232)
org.apache.hadoop.hive.metastore.TestRemoteHiveMetaStoreZKBindHost.testFunctionWithResources
(batchId=232)
org.apache.hadoop.hive.metastore.TestRemoteHiveMetaStoreZKBindHost.testGetConfigValue
(batchId=232)
org.apache.hadoop.hive.metastore.TestRemoteHiveMetaStoreZKBindHost.testGetMetastoreUuid
(batchId=232)
org.apache.hadoop.hive.metastore.TestRemoteHiveMetaStoreZKBindHost.testGetPartitionsWithSpec
(batchId=232)
org.apache.hadoop.hive.metastore.TestRemoteHiveMetaStoreZKBindHost.testGetSchemaWithNoClassDefFoundError
(batchId=232)
org.apache.hadoop.hive.metastore.TestRemoteHiveMetaStoreZKBindHost.testGetTableObjects
(batchId=232)
org.apache.hadoop.hive.metastore.TestRemoteHiveMetaStoreZKBindHost.testGetUUIDInParallel
(batchId=232)
org.apache.hadoop.hive.metastore.TestRemoteHiveMetaStoreZKBindHost.testJDOPersistanceManagerCleanup
(batchId=232)
org.apache.hadoop.hive.metastore.TestRemoteHiveMetaStoreZKBindHost.testListPartitionNames
(batchId=232)
org.apache.hadoop.hive.metastore.TestRemoteHiveMetaStoreZKBindHost.testListPartitions
(batchId=232)
org.apache.hadoop.hive.metastore.TestRemoteHiveMetaStoreZKBindHost.testListPartitionsWihtLimitEnabled
(batchId=232)
org.apache.hadoop.hive.metastore.TestRemoteHiveMetaStoreZKBindHost.testNameMethods
(batchId=232)
org.apache.hadoop.hive.metastore.TestRemoteHiveMetaStoreZKBindHost.testPartition
(batchId=232)
org.apache.hadoop.hive.metastore.TestRemoteHiveMetaStoreZKBindHost.testPartitionFilter
(batchId=232)
org.apache.hadoop.hive.metastore.TestRemoteHiveMetaStoreZKBindHost.testRenamePartition
(batchId=232)
org.apache.hadoop.hive.metastore.TestRemoteHiveMetaStoreZKBindHost.testRetriableClientWithConnLifetime
(batchId=232)
org.apache.hadoop.hive.metastore.TestRemoteHiveMetaStoreZKBindHost.testSimpleFunction
(batchId=232)
org.apache.hadoop.hive.metastore.TestRemoteHiveMetaStoreZKBindHost.testSimpleTable
(batchId=232)
org.apache.hadoop.hive.metastore.TestRemoteHiveMetaStoreZKBindHost.testSimpleTypeApi
(batchId=232)
org.apache.hadoop.hive.metastore.TestRemoteHiveMetaStoreZKBindHost.testStatsFastTrivial
(batchId=232)
org.apache.hadoop.hive.metastore.TestRemoteHiveMetaStoreZKBindHost.testSynchronized
(batchId=232)
[jira] [Commented] (HIVE-20985) If select operator inputs are temporary columns vectorization may reuse some of them as output
[ https://issues.apache.org/jira/browse/HIVE-20985?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16703502#comment-16703502 ] Ashutosh Chauhan commented on HIVE-20985: - [~kgyrtkirk] Did you forget to attach patch file? > If select operator inputs are temporary columns vectorization may reuse some > of them as output > -- > > Key: HIVE-20985 > URL: https://issues.apache.org/jira/browse/HIVE-20985 > Project: Hive > Issue Type: Bug > Components: Vectorization >Reporter: Zoltan Haindrich >Assignee: Zoltan Haindrich >Priority: Major > Attachments: HIVE-20985.01.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-20985) If select operator inputs are temporary columns vectorization may reuse some of them as output
[ https://issues.apache.org/jira/browse/HIVE-20985?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zoltan Haindrich updated HIVE-20985: Attachment: HIVE-20985.01.patch > If select operator inputs are temporary columns vectorization may reuse some > of them as output > -- > > Key: HIVE-20985 > URL: https://issues.apache.org/jira/browse/HIVE-20985 > Project: Hive > Issue Type: Bug > Components: Vectorization >Reporter: Zoltan Haindrich >Assignee: Zoltan Haindrich >Priority: Major > Attachments: HIVE-20985.01.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-20985) If select operator inputs are temporary columns vectorization may reuse some of them as output
[ https://issues.apache.org/jira/browse/HIVE-20985?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zoltan Haindrich updated HIVE-20985: Status: Patch Available (was: Open) > If select operator inputs are temporary columns vectorization may reuse some > of them as output > -- > > Key: HIVE-20985 > URL: https://issues.apache.org/jira/browse/HIVE-20985 > Project: Hive > Issue Type: Bug > Components: Vectorization >Reporter: Zoltan Haindrich >Assignee: Zoltan Haindrich >Priority: Major > Attachments: HIVE-20985.01.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-20985) If select operator inputs are temporary columns vectorization may reuse some of them as output
[ https://issues.apache.org/jira/browse/HIVE-20985?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zoltan Haindrich updated HIVE-20985: Attachment: (was: HIVE-20985.01.patch) > If select operator inputs are temporary columns vectorization may reuse some > of them as output > -- > > Key: HIVE-20985 > URL: https://issues.apache.org/jira/browse/HIVE-20985 > Project: Hive > Issue Type: Bug > Components: Vectorization >Reporter: Zoltan Haindrich >Assignee: Zoltan Haindrich >Priority: Major > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-20985) If select operator inputs are temporary columns vectorization may reuse some of them as output
[ https://issues.apache.org/jira/browse/HIVE-20985?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zoltan Haindrich updated HIVE-20985: Attachment: HIVE-20985.01.patch > If select operator inputs are temporary columns vectorization may reuse some > of them as output > -- > > Key: HIVE-20985 > URL: https://issues.apache.org/jira/browse/HIVE-20985 > Project: Hive > Issue Type: Bug > Components: Vectorization >Reporter: Zoltan Haindrich >Assignee: Zoltan Haindrich >Priority: Major > Attachments: HIVE-20985.01.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (HIVE-20985) If select operator inputs are temporary columns vectorization may reuse some of them as output
[ https://issues.apache.org/jira/browse/HIVE-20985?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zoltan Haindrich reassigned HIVE-20985: --- > If select operator inputs are temporary columns vectorization may reuse some > of them as output > -- > > Key: HIVE-20985 > URL: https://issues.apache.org/jira/browse/HIVE-20985 > Project: Hive > Issue Type: Bug > Components: Vectorization >Reporter: Zoltan Haindrich >Assignee: Zoltan Haindrich >Priority: Major > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-20897) TestJdbcDriver2#testSelectExecAsync2 fails with result set not present error
[ https://issues.apache.org/jira/browse/HIVE-20897?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] mahesh kumar behera updated HIVE-20897: --- Status: Patch Available (was: Open) > TestJdbcDriver2#testSelectExecAsync2 fails with result set not present error > > > Key: HIVE-20897 > URL: https://issues.apache.org/jira/browse/HIVE-20897 > Project: Hive > Issue Type: Bug > Components: Hive >Reporter: mahesh kumar behera >Assignee: mahesh kumar behera >Priority: Major > Labels: pull-request-available > Attachments: HIVE-20897.01.patch, HIVE-20897.02.patch, > HIVE-20897.03.patch, HIVE-20897.04.patch, HIVE-20897.05.patch, > HIVE-20897.06.patch, HIVE-20897.07.patch, HIVE-20897.07.patch, > HIVE-20897.08.patch, HIVE-20897.08.patch, HIVE-20897.08.patch > > > if async prepare is enabled, control will be returned to the client before > driver could set of the query has a result set or not. But in current code, > while generating the response for the query, it is not checked if the result > set field is set or not. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-20897) TestJdbcDriver2#testSelectExecAsync2 fails with result set not present error
[ https://issues.apache.org/jira/browse/HIVE-20897?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] mahesh kumar behera updated HIVE-20897: --- Attachment: HIVE-20897.08.patch > TestJdbcDriver2#testSelectExecAsync2 fails with result set not present error > > > Key: HIVE-20897 > URL: https://issues.apache.org/jira/browse/HIVE-20897 > Project: Hive > Issue Type: Bug > Components: Hive >Reporter: mahesh kumar behera >Assignee: mahesh kumar behera >Priority: Major > Labels: pull-request-available > Attachments: HIVE-20897.01.patch, HIVE-20897.02.patch, > HIVE-20897.03.patch, HIVE-20897.04.patch, HIVE-20897.05.patch, > HIVE-20897.06.patch, HIVE-20897.07.patch, HIVE-20897.07.patch, > HIVE-20897.08.patch, HIVE-20897.08.patch, HIVE-20897.08.patch > > > if async prepare is enabled, control will be returned to the client before > driver could set of the query has a result set or not. But in current code, > while generating the response for the query, it is not checked if the result > set field is set or not. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-20897) TestJdbcDriver2#testSelectExecAsync2 fails with result set not present error
[ https://issues.apache.org/jira/browse/HIVE-20897?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] mahesh kumar behera updated HIVE-20897: --- Status: Open (was: Patch Available) > TestJdbcDriver2#testSelectExecAsync2 fails with result set not present error > > > Key: HIVE-20897 > URL: https://issues.apache.org/jira/browse/HIVE-20897 > Project: Hive > Issue Type: Bug > Components: Hive >Reporter: mahesh kumar behera >Assignee: mahesh kumar behera >Priority: Major > Labels: pull-request-available > Attachments: HIVE-20897.01.patch, HIVE-20897.02.patch, > HIVE-20897.03.patch, HIVE-20897.04.patch, HIVE-20897.05.patch, > HIVE-20897.06.patch, HIVE-20897.07.patch, HIVE-20897.07.patch, > HIVE-20897.08.patch, HIVE-20897.08.patch, HIVE-20897.08.patch > > > if async prepare is enabled, control will be returned to the client before > driver could set of the query has a result set or not. But in current code, > while generating the response for the query, it is not checked if the result > set field is set or not. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-20979) Fix memory leak in hive streaming
[
https://issues.apache.org/jira/browse/HIVE-20979?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16703415#comment-16703415
]
Hive QA commented on HIVE-20979:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
47s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
18s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
10s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
26s{color} | {color:blue} streaming in master has 2 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
11s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
19s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
17s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
17s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
10s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
33s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
12s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
12s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 10m 58s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-15116/dev-support/hive-personality.sh
|
| git revision | master / 3e98143 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| modules | C: streaming U: streaming |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-15116/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Fix memory leak in hive streaming
> -
>
> Key: HIVE-20979
> URL: https://issues.apache.org/jira/browse/HIVE-20979
> Project: Hive
> Issue Type: Bug
> Components: Streaming
>Affects Versions: 3.1.1
>Reporter: Shubham Chaurasia
>Assignee: Shubham Chaurasia
>Priority: Critical
> Labels: pull-request-available
> Attachments: HIVE-20979.1.patch, HIVE-20979.1.patch,
> HIVE-20979.2.patch, HIVE-20979.3.patch, HIVE-20979.4.patch
>
>
> {{1) HiveStreamingConnection.Builder#init() adds a shutdown hook handler via
> }}{{ShutdownHookManager.addShutdownHook but it is never removed which causes
> all the handlers to accumulate and hence a memory leak.}}
> 2) AbstractRecordWriter creates an instance of FileSystem but does not close
> it which in turn causes a leak due to accumulation in FileSystem$Cache#map
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-20255) Review LevelOrderWalker.java
[
https://issues.apache.org/jira/browse/HIVE-20255?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16703380#comment-16703380
]
Hive QA commented on HIVE-20255:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12938674/HIVE-20255.5.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 45 failed/errored test(s), 15643 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.metastore.TestRemoteHiveMetaStoreZK.testAlterPartition
(batchId=230)
org.apache.hadoop.hive.metastore.TestRemoteHiveMetaStoreZK.testAlterTable
(batchId=230)
org.apache.hadoop.hive.metastore.TestRemoteHiveMetaStoreZK.testAlterTableCascade
(batchId=230)
org.apache.hadoop.hive.metastore.TestRemoteHiveMetaStoreZK.testAlterViewParititon
(batchId=230)
org.apache.hadoop.hive.metastore.TestRemoteHiveMetaStoreZK.testColumnStatistics
(batchId=230)
org.apache.hadoop.hive.metastore.TestRemoteHiveMetaStoreZK.testComplexTable
(batchId=230)
org.apache.hadoop.hive.metastore.TestRemoteHiveMetaStoreZK.testComplexTypeApi
(batchId=230)
org.apache.hadoop.hive.metastore.TestRemoteHiveMetaStoreZK.testConcurrentMetastores
(batchId=230)
org.apache.hadoop.hive.metastore.TestRemoteHiveMetaStoreZK.testCreateAndGetTableWithDriver
(batchId=230)
org.apache.hadoop.hive.metastore.TestRemoteHiveMetaStoreZK.testCreateTableSettingId
(batchId=230)
org.apache.hadoop.hive.metastore.TestRemoteHiveMetaStoreZK.testDBLocationChange
(batchId=230)
org.apache.hadoop.hive.metastore.TestRemoteHiveMetaStoreZK.testDBOwner
(batchId=230)
org.apache.hadoop.hive.metastore.TestRemoteHiveMetaStoreZK.testDBOwnerChange
(batchId=230)
org.apache.hadoop.hive.metastore.TestRemoteHiveMetaStoreZK.testDatabase
(batchId=230)
org.apache.hadoop.hive.metastore.TestRemoteHiveMetaStoreZK.testDatabaseLocation
(batchId=230)
org.apache.hadoop.hive.metastore.TestRemoteHiveMetaStoreZK.testDatabaseLocationWithPermissionProblems
(batchId=230)
org.apache.hadoop.hive.metastore.TestRemoteHiveMetaStoreZK.testDropDatabaseCascadeMVMultiDB
(batchId=230)
org.apache.hadoop.hive.metastore.TestRemoteHiveMetaStoreZK.testDropTable
(batchId=230)
org.apache.hadoop.hive.metastore.TestRemoteHiveMetaStoreZK.testFilterLastPartition
(batchId=230)
org.apache.hadoop.hive.metastore.TestRemoteHiveMetaStoreZK.testFilterSinglePartition
(batchId=230)
org.apache.hadoop.hive.metastore.TestRemoteHiveMetaStoreZK.testFunctionWithResources
(batchId=230)
org.apache.hadoop.hive.metastore.TestRemoteHiveMetaStoreZK.testGetConfigValue
(batchId=230)
org.apache.hadoop.hive.metastore.TestRemoteHiveMetaStoreZK.testGetMetastoreUuid
(batchId=230)
org.apache.hadoop.hive.metastore.TestRemoteHiveMetaStoreZK.testGetPartitionsWithSpec
(batchId=230)
org.apache.hadoop.hive.metastore.TestRemoteHiveMetaStoreZK.testGetSchemaWithNoClassDefFoundError
(batchId=230)
org.apache.hadoop.hive.metastore.TestRemoteHiveMetaStoreZK.testGetTableObjects
(batchId=230)
org.apache.hadoop.hive.metastore.TestRemoteHiveMetaStoreZK.testGetUUIDInParallel
(batchId=230)
org.apache.hadoop.hive.metastore.TestRemoteHiveMetaStoreZK.testJDOPersistanceManagerCleanup
(batchId=230)
org.apache.hadoop.hive.metastore.TestRemoteHiveMetaStoreZK.testListPartitionNames
(batchId=230)
org.apache.hadoop.hive.metastore.TestRemoteHiveMetaStoreZK.testListPartitions
(batchId=230)
org.apache.hadoop.hive.metastore.TestRemoteHiveMetaStoreZK.testListPartitionsWihtLimitEnabled
(batchId=230)
org.apache.hadoop.hive.metastore.TestRemoteHiveMetaStoreZK.testNameMethods
(batchId=230)
org.apache.hadoop.hive.metastore.TestRemoteHiveMetaStoreZK.testPartition
(batchId=230)
org.apache.hadoop.hive.metastore.TestRemoteHiveMetaStoreZK.testPartitionFilter
(batchId=230)
org.apache.hadoop.hive.metastore.TestRemoteHiveMetaStoreZK.testRenamePartition
(batchId=230)
org.apache.hadoop.hive.metastore.TestRemoteHiveMetaStoreZK.testRetriableClientWithConnLifetime
(batchId=230)
org.apache.hadoop.hive.metastore.TestRemoteHiveMetaStoreZK.testSimpleFunction
(batchId=230)
org.apache.hadoop.hive.metastore.TestRemoteHiveMetaStoreZK.testSimpleTable
(batchId=230)
org.apache.hadoop.hive.metastore.TestRemoteHiveMetaStoreZK.testSimpleTypeApi
(batchId=230)
org.apache.hadoop.hive.metastore.TestRemoteHiveMetaStoreZK.testStatsFastTrivial
(batchId=230)
org.apache.hadoop.hive.metastore.TestRemoteHiveMetaStoreZK.testSynchronized
(batchId=230)
org.apache.hadoop.hive.metastore.TestRemoteHiveMetaStoreZK.testTableDatabase
(batchId=230)
org.apache.hadoop.hive.metastore.TestRemoteHiveMetaStoreZK.testTableFilter
(batchId=230)
org.apache.hadoop.hive.metastore.TestRemoteHiveMetaStoreZK.testUpdatePartitionStat_doesNotUpdateStats
(batchId=230)
org.apache.hadoop.hive.metastore.TestRemoteHiveMetaStoreZK.testValidateTableCols
(batchId=230)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/15115/testReport
Console
[jira] [Updated] (HIVE-20980) Reinstate Parquet timestamp conversion between HS2 time zone and UTC
[ https://issues.apache.org/jira/browse/HIVE-20980?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Karen Coppage updated HIVE-20980: - Status: Open (was: Patch Available) > Reinstate Parquet timestamp conversion between HS2 time zone and UTC > > > Key: HIVE-20980 > URL: https://issues.apache.org/jira/browse/HIVE-20980 > Project: Hive > Issue Type: Improvement > Components: File Formats >Reporter: Karen Coppage >Assignee: Karen Coppage >Priority: Major > Attachments: HIVE-20980.1.patch > > > [WIP] > With HIVE-20007, Parquet timestamps became timezone-agnostic. This means that > timestamps written after the change are read exactly as they were written; > but timestamps stored before this change are effectively converted from the > writing HS2 server time zone to GMT time zone. This patch reinstates the > original behavior: timestamps are converted to UTC before write and from UTC > before read. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-20255) Review LevelOrderWalker.java
[
https://issues.apache.org/jira/browse/HIVE-20255?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16703315#comment-16703315
]
Hive QA commented on HIVE-20255:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
44s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
4s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
35s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
43s{color} | {color:blue} ql in master has 2312 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
56s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
22s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
5s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
5s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
35s{color} | {color:green} ql: The patch generated 0 new + 1 unchanged - 2
fixed = 1 total (was 3) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
53s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
56s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
13s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 22m 34s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-15115/dev-support/hive-personality.sh
|
| git revision | master / 3e98143 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-15115/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Review LevelOrderWalker.java
>
>
> Key: HIVE-20255
> URL: https://issues.apache.org/jira/browse/HIVE-20255
> Project: Hive
> Issue Type: Improvement
> Components: Query Planning
>Affects Versions: 3.0.0, 4.0.0
>Reporter: BELUGA BEHR
>Assignee: BELUGA BEHR
>Priority: Minor
> Attachments: HIVE-20255.1.patch, HIVE-20255.2.patch,
> HIVE-20255.3.patch, HIVE-20255.4.patch, HIVE-20255.5.patch
>
>
> https://github.com/apache/hive/blob/6d890faf22fd1ede3658a5eed097476eab3c67e9/ql/src/java/org/apache/hadoop/hive/ql/lib/LevelOrderWalker.java
> * Make code more concise
> * Fix some check style issues
> {code}
> if (toWalk.get(index).getChildren() != null) {
> for(Node child : toWalk.get(index).getChildren()) {
> {code}
> Actually, the underlying implementation of {{getChildren()}} has to do some
> real work, so do not throw away the work after checking for null. Simply
> call once and store the results.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-20897) TestJdbcDriver2#testSelectExecAsync2 fails with result set not present error
[
https://issues.apache.org/jira/browse/HIVE-20897?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16703274#comment-16703274
]
Hive QA commented on HIVE-20897:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12949987/HIVE-20897.08.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 11 failed/errored test(s), 15643 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.metastore.TestObjectStore.catalogs (batchId=232)
org.apache.hadoop.hive.metastore.TestObjectStore.testDatabaseOps (batchId=232)
org.apache.hadoop.hive.metastore.TestObjectStore.testDirectSQLDropParitionsCleanup
(batchId=232)
org.apache.hadoop.hive.metastore.TestObjectStore.testDirectSQLDropPartitionsCacheCrossSession
(batchId=232)
org.apache.hadoop.hive.metastore.TestObjectStore.testDirectSqlErrorMetrics
(batchId=232)
org.apache.hadoop.hive.metastore.TestObjectStore.testMasterKeyOps (batchId=232)
org.apache.hadoop.hive.metastore.TestObjectStore.testMaxEventResponse
(batchId=232)
org.apache.hadoop.hive.metastore.TestObjectStore.testPartitionOps (batchId=232)
org.apache.hadoop.hive.metastore.TestObjectStore.testQueryCloseOnError
(batchId=232)
org.apache.hadoop.hive.metastore.TestObjectStore.testRoleOps (batchId=232)
org.apache.hadoop.hive.metastore.TestObjectStore.testTableOps (batchId=232)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/15114/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/15114/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-15114/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 11 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12949987 - PreCommit-HIVE-Build
> TestJdbcDriver2#testSelectExecAsync2 fails with result set not present error
>
>
> Key: HIVE-20897
> URL: https://issues.apache.org/jira/browse/HIVE-20897
> Project: Hive
> Issue Type: Bug
> Components: Hive
>Reporter: mahesh kumar behera
>Assignee: mahesh kumar behera
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-20897.01.patch, HIVE-20897.02.patch,
> HIVE-20897.03.patch, HIVE-20897.04.patch, HIVE-20897.05.patch,
> HIVE-20897.06.patch, HIVE-20897.07.patch, HIVE-20897.07.patch,
> HIVE-20897.08.patch, HIVE-20897.08.patch
>
>
> if async prepare is enabled, control will be returned to the client before
> driver could set of the query has a result set or not. But in current code,
> while generating the response for the query, it is not checked if the result
> set field is set or not.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-20891) Call alter_partition in batch when dynamically loading partitions
[ https://issues.apache.org/jira/browse/HIVE-20891?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Laszlo Pinter updated HIVE-20891: - Attachment: HIVE-20891.07.patch > Call alter_partition in batch when dynamically loading partitions > - > > Key: HIVE-20891 > URL: https://issues.apache.org/jira/browse/HIVE-20891 > Project: Hive > Issue Type: Improvement > Components: Hive >Affects Versions: 4.0.0 >Reporter: Laszlo Pinter >Assignee: Laszlo Pinter >Priority: Major > Attachments: HIVE-20891.01.patch, HIVE-20891.02.patch, > HIVE-20891.03.patch, HIVE-20891.04.patch, HIVE-20891.05.patch, > HIVE-20891.06.patch, HIVE-20891.07.patch > > > When dynamically loading partitions, the setStatsPropAndAlterPartition() is > called for each partition one by one, resulting in unnecessary calls to the > metastore client. This whole logic can be changed to just one call. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-20897) TestJdbcDriver2#testSelectExecAsync2 fails with result set not present error
[
https://issues.apache.org/jira/browse/HIVE-20897?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16703185#comment-16703185
]
Hive QA commented on HIVE-20897:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
47s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
23s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
13s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
36s{color} | {color:blue} service in master has 48 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
16s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
23s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
20s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
20s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
12s{color} | {color:green} service: The patch generated 0 new + 59 unchanged -
1 fixed = 59 total (was 60) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
43s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
16s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
12s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 11m 45s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-15114/dev-support/hive-personality.sh
|
| git revision | master / 3e98143 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| modules | C: service U: service |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-15114/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> TestJdbcDriver2#testSelectExecAsync2 fails with result set not present error
>
>
> Key: HIVE-20897
> URL: https://issues.apache.org/jira/browse/HIVE-20897
> Project: Hive
> Issue Type: Bug
> Components: Hive
>Reporter: mahesh kumar behera
>Assignee: mahesh kumar behera
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-20897.01.patch, HIVE-20897.02.patch,
> HIVE-20897.03.patch, HIVE-20897.04.patch, HIVE-20897.05.patch,
> HIVE-20897.06.patch, HIVE-20897.07.patch, HIVE-20897.07.patch,
> HIVE-20897.08.patch, HIVE-20897.08.patch
>
>
> if async prepare is enabled, control will be returned to the client before
> driver could set of the query has a result set or not. But in current code,
> while generating the response for the query, it is not checked if the result
> set field is set or not.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-20915) Make dynamic sort partition optimization available to HoS and MR
[ https://issues.apache.org/jira/browse/HIVE-20915?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16703192#comment-16703192 ] Yongzhi Chen commented on HIVE-20915: - The two whitespace issues are all related to q.out file, we'd better not change them: ./ql/src/test/results/clientpositive/spark/auto_sortmerge_join_16.q.out_spark:445:169 val_169 NULLday11 ./ql/src/test/results/clientpositive/spark/auto_sortmerge_join_16.q.out_spark:450:374 val_374 NULLday11 > Make dynamic sort partition optimization available to HoS and MR > > > Key: HIVE-20915 > URL: https://issues.apache.org/jira/browse/HIVE-20915 > Project: Hive > Issue Type: Bug > Components: Query Planning >Affects Versions: 4.0.0 >Reporter: Yongzhi Chen >Assignee: Yongzhi Chen >Priority: Major > Attachments: HIVE-20915.1.patch, HIVE-20915.2.patch, > HIVE-20915.3.patch, HIVE-20915.4.patch, HIVE-20915.5.patch > > > HIVE-20703 put dynamic sort partition optimization under cost based decision, > but it also makes the optimizer only available to tez. > hive.optimize.sort.dynamic.partition works with other execution engines for a > long time, we should keep the optimizer available to them. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-20984) Hive cannot call MapReduce, please tell me where there is a configuration problem
[
https://issues.apache.org/jira/browse/HIVE-20984?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
yuxuqi updated HIVE-20984:
--
Description:
hive> insert into table fdm_db.mytable values(1,1);
Query ID = root_20181129211919_7dadd9cc-05fa-4220-aa18-6dea4068e1cd
Total jobs = 3
Launching Job 1 out of 3
Number of reduce tasks is set to 0 since there's no reduce operator
Starting Job = job_1543494350883_0003, Tracking URL =
http://master:8088/proxy/application_1543494350883_0003/
Kill Command =
/opt/cloudera/parcels/CDH-5.14.2-1.cdh5.14.2.p0.3/lib/hadoop/bin/hadoop job
-kill job_1543494350883_0003
yarn.acl.enable
true
yarn.admin.acl
*
yarn.resourcemanager.address
master:8032
yarn.resourcemanager.admin.address
master:8033
yarn.resourcemanager.scheduler.address
master:8030
yarn.resourcemanager.resource-tracker.address
master:8031
yarn.resourcemanager.webapp.address
master:8088
yarn.resourcemanager.webapp.https.address
master:8090
yarn.resourcemanager.client.thread-count
50
yarn.resourcemanager.scheduler.client.thread-count
50
yarn.resourcemanager.admin.client.thread-count
1
yarn.scheduler.minimum-allocation-mb
1024
yarn.scheduler.increment-allocation-mb
512
yarn.scheduler.maximum-allocation-mb
31647
yarn.scheduler.minimum-allocation-vcores
1
yarn.scheduler.increment-allocation-vcores
1
yarn.scheduler.maximum-allocation-vcores
48
yarn.resourcemanager.amliveliness-monitor.interval-ms
1000
yarn.am.liveness-monitor.expiry-interval-ms
60
yarn.resourcemanager.am.max-attempts
2
yarn.resourcemanager.container.liveness-monitor.interval-ms
60
yarn.resourcemanager.nm.liveness-monitor.interval-ms
1000
yarn.nm.liveness-monitor.expiry-interval-ms
60
yarn.resourcemanager.resource-tracker.client.thread-count
50
yarn.application.classpath
$HADOOP_CLIENT_CONF_DIR,$HADOOP_CONF_DIR,$HADOOP_COMMON_HOME/*,$HADOOP_COMMON_HOME/lib/*,$HADOOP_HDFS_HOME/*,$HADOOP_HDFS_HOME/lib/*,$HADOOP_YARN_HOME/*,$HADOOP_YARN_HOME/lib/*
yarn.resourcemanager.scheduler.class
org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FairScheduler
yarn.resourcemanager.max-completed-applications
1
yarn.nodemanager.remote-app-log-dir
/tmp/logs
yarn.nodemanager.remote-app-log-dir-suffix
logs
was:
{color:#FF}{color}
yarn.acl.enable
true
yarn.admin.acl
*
yarn.resourcemanager.address
master:8032
yarn.resourcemanager.admin.address
master:8033
yarn.resourcemanager.scheduler.address
master:8030
yarn.resourcemanager.resource-tracker.address
master:8031
yarn.resourcemanager.webapp.address
master:8088
yarn.resourcemanager.webapp.https.address
master:8090
yarn.resourcemanager.client.thread-count
50
yarn.resourcemanager.scheduler.client.thread-count
50
yarn.resourcemanager.admin.client.thread-count
1
yarn.scheduler.minimum-allocation-mb
1024
yarn.scheduler.increment-allocation-mb
512
yarn.scheduler.maximum-allocation-mb
31647
yarn.scheduler.minimum-allocation-vcores
1
yarn.scheduler.increment-allocation-vcores
1
yarn.scheduler.maximum-allocation-vcores
48
yarn.resourcemanager.amliveliness-monitor.interval-ms
1000
yarn.am.liveness-monitor.expiry-interval-ms
60
yarn.resourcemanager.am.max-attempts
2
yarn.resourcemanager.container.liveness-monitor.interval-ms
60
yarn.resourcemanager.nm.liveness-monitor.interval-ms
1000
yarn.nm.liveness-monitor.expiry-interval-ms
60
yarn.resourcemanager.resource-tracker.client.thread-count
50
yarn.application.classpath
$HADOOP_CLIENT_CONF_DIR,$HADOOP_CONF_DIR,$HADOOP_COMMON_HOME/*,$HADOOP_COMMON_HOME/lib/*,$HADOOP_HDFS_HOME/*,$HADOOP_HDFS_HOME/lib/*,$HADOOP_YARN_HOME/*,$HADOOP_YARN_HOME/lib/*
yarn.resourcemanager.scheduler.class
org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FairScheduler
yarn.resourcemanager.max-completed-applications
1
yarn.nodemanager.remote-app-log-dir
/tmp/logs
yarn.nodemanager.remote-app-log-dir-suffix
logs
> Hive cannot call MapReduce, please tell me where there is a configuration
> problem
> -
>
> Key: HIVE-20984
> URL: https://issues.apache.org/jira/browse/HIVE-20984
> Project: Hive
> Issue Type: Bug
> Components: Beeline
>Affects Versions: 1.1.0
> Environment: CDH 5.14.2 hive 1.1.0
>Reporter: yuxuqi
>Priority: Major
> Attachments: image-2018-11-29-19-58-56-026.png
>
>
> hive> insert into table fdm_db.mytable values(1,1);
> Query ID = root_20181129211919_7dadd9cc-05fa-4220-aa18-6dea4068e1cd
> Total jobs = 3
> Launching Job 1 out of 3
> Number of reduce
[jira] [Commented] (HIVE-20891) Call alter_partition in batch when dynamically loading partitions
[
https://issues.apache.org/jira/browse/HIVE-20891?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16703146#comment-16703146
]
Hive QA commented on HIVE-20891:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12949957/HIVE-20891.06.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 3 failed/errored test(s), 15639 tests
executed
*Failed tests:*
{noformat}
org.apache.hive.jdbc.TestTriggersTezSessionPoolManager.testTriggerCustomCreatedDynamicPartitions
(batchId=259)
org.apache.hive.jdbc.TestTriggersTezSessionPoolManager.testTriggerCustomCreatedDynamicPartitionsUnionAll
(batchId=259)
org.apache.hive.jdbc.TestTriggersTezSessionPoolManager.testTriggerHighShuffleBytes
(batchId=259)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/15113/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/15113/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-15113/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 3 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12949957 - PreCommit-HIVE-Build
> Call alter_partition in batch when dynamically loading partitions
> -
>
> Key: HIVE-20891
> URL: https://issues.apache.org/jira/browse/HIVE-20891
> Project: Hive
> Issue Type: Improvement
> Components: Hive
>Affects Versions: 4.0.0
>Reporter: Laszlo Pinter
>Assignee: Laszlo Pinter
>Priority: Major
> Attachments: HIVE-20891.01.patch, HIVE-20891.02.patch,
> HIVE-20891.03.patch, HIVE-20891.04.patch, HIVE-20891.05.patch,
> HIVE-20891.06.patch
>
>
> When dynamically loading partitions, the setStatsPropAndAlterPartition() is
> called for each partition one by one, resulting in unnecessary calls to the
> metastore client. This whole logic can be changed to just one call.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-20891) Call alter_partition in batch when dynamically loading partitions
[
https://issues.apache.org/jira/browse/HIVE-20891?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16703106#comment-16703106
]
Hive QA commented on HIVE-20891:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
45s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
58s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
38s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
39s{color} | {color:blue} ql in master has 2312 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
54s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
22s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
0s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
0s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
37s{color} | {color:red} ql: The patch generated 1 new + 218 unchanged - 0
fixed = 219 total (was 218) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
1s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
50s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
54s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
12s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 22m 18s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-15113/dev-support/hive-personality.sh
|
| git revision | master / 3e98143 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-15113/yetus/diff-checkstyle-ql.txt
|
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-15113/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Call alter_partition in batch when dynamically loading partitions
> -
>
> Key: HIVE-20891
> URL: https://issues.apache.org/jira/browse/HIVE-20891
> Project: Hive
> Issue Type: Improvement
> Components: Hive
>Affects Versions: 4.0.0
>Reporter: Laszlo Pinter
>Assignee: Laszlo Pinter
>Priority: Major
> Attachments: HIVE-20891.01.patch, HIVE-20891.02.patch,
> HIVE-20891.03.patch, HIVE-20891.04.patch, HIVE-20891.05.patch,
> HIVE-20891.06.patch
>
>
> When dynamically loading partitions, the setStatsPropAndAlterPartition() is
> called for each partition one by one, resulting in unnecessary calls to the
> metastore client. This whole logic can be changed to just one call.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-20979) Fix memory leak in hive streaming
[
https://issues.apache.org/jira/browse/HIVE-20979?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Shubham Chaurasia updated HIVE-20979:
-
Attachment: HIVE-20979.4.patch
> Fix memory leak in hive streaming
> -
>
> Key: HIVE-20979
> URL: https://issues.apache.org/jira/browse/HIVE-20979
> Project: Hive
> Issue Type: Bug
> Components: Streaming
>Affects Versions: 3.1.1
>Reporter: Shubham Chaurasia
>Assignee: Shubham Chaurasia
>Priority: Critical
> Labels: pull-request-available
> Attachments: HIVE-20979.1.patch, HIVE-20979.1.patch,
> HIVE-20979.2.patch, HIVE-20979.3.patch, HIVE-20979.4.patch
>
>
> {{1) HiveStreamingConnection.Builder#init() adds a shutdown hook handler via
> }}{{ShutdownHookManager.addShutdownHook but it is never removed which causes
> all the handlers to accumulate and hence a memory leak.}}
> 2) AbstractRecordWriter creates an instance of FileSystem but does not close
> it which in turn causes a leak due to accumulation in FileSystem$Cache#map
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-20983) Vectorization: Scale up small hashtables, when collisions are detected
[
https://issues.apache.org/jira/browse/HIVE-20983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16703080#comment-16703080
]
Hive QA commented on HIVE-20983:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12949955/HIVE-20983.1.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 7 failed/errored test(s), 15643 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.ql.exec.vector.mapjoin.fast.TestVectorMapJoinFastBytesHashMap.testExpand
(batchId=315)
org.apache.hadoop.hive.ql.exec.vector.mapjoin.fast.TestVectorMapJoinFastBytesHashMultiSet.testExpand
(batchId=315)
org.apache.hadoop.hive.ql.exec.vector.mapjoin.fast.TestVectorMapJoinFastBytesHashSet.testExpand
(batchId=314)
org.apache.hadoop.hive.ql.exec.vector.mapjoin.fast.TestVectorMapJoinFastLongHashMap.testExpand
(batchId=314)
org.apache.hadoop.hive.ql.exec.vector.mapjoin.fast.TestVectorMapJoinFastLongHashMapNonMatched.testExpand
(batchId=314)
org.apache.hadoop.hive.ql.exec.vector.mapjoin.fast.TestVectorMapJoinFastLongHashMultiSet.testExpand
(batchId=315)
org.apache.hadoop.hive.ql.exec.vector.mapjoin.fast.TestVectorMapJoinFastLongHashSet.testExpand
(batchId=315)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/15112/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/15112/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-15112/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 7 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12949955 - PreCommit-HIVE-Build
> Vectorization: Scale up small hashtables, when collisions are detected
> --
>
> Key: HIVE-20983
> URL: https://issues.apache.org/jira/browse/HIVE-20983
> Project: Hive
> Issue Type: Bug
>Reporter: Gopal V
>Assignee: Gopal V
>Priority: Major
> Attachments: HIVE-20983.1.patch
>
>
> Hive's hashtable estimates are getting better with HyperLogLog stats in
> place, but an accurate estimate does not always result in a low number of
> collisions.
> The hashtables which contain a very small number of items tend to lose their
> O(1) lookup performance where there are collisions. Since collisions are easy
> to detect within the fast hashtable implementation, a rehashing to a higher
> size will help these small hashtables avoid collisions and go back to O(1)
> perf.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-18902) Lower Logging Level for Cleaning Up "local RawStore"
[
https://issues.apache.org/jira/browse/HIVE-18902?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Peter Vary updated HIVE-18902:
--
Resolution: Fixed
Fix Version/s: 4.0.0
Status: Resolved (was: Patch Available)
Pushed to master.
Thanks for the patch [~bchupika] and for the review [~belugabehr]!
> Lower Logging Level for Cleaning Up "local RawStore"
>
>
> Key: HIVE-18902
> URL: https://issues.apache.org/jira/browse/HIVE-18902
> Project: Hive
> Issue Type: Improvement
> Components: Standalone Metastore
>Affects Versions: 3.0.0, 2.4.0
>Reporter: BELUGA BEHR
>Assignee: Bohdan Chupika
>Priority: Trivial
> Labels: noob
> Fix For: 4.0.0
>
> Attachments: HIVE-18902.01.patch, HIVE-18902.02.patch,
> HIVE-18902.03.patch, HIVE-18902.04.patch, HIVE-18902.5.patch
>
>
> [https://github.com/apache/hive/blob/7c22d74c8d0eb0650adf6e84e0536127c103e46c/standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java#L7756-L7768]
>
> {code:java}
> private static void cleanupRawStore() {
> try {
> RawStore rs = HMSHandler.getRawStore();
> if (rs != null) {
> HMSHandler.logInfo("Cleaning up thread local RawStore...");
> rs.shutdown();
> }
> } finally {
> HMSHandler handler = HMSHandler.threadLocalHMSHandler.get();
> if (handler != null) {
> handler.notifyMetaListenersOnShutDown();
> }
> HMSHandler.threadLocalHMSHandler.remove();
> HMSHandler.threadLocalConf.remove();
> HMSHandler.threadLocalModifiedConfig.remove();
> HMSHandler.removeRawStore();
> HMSHandler.logInfo("Done cleaning up thread local RawStore");
> }
> }
> {code}
> {code}
> 2018-03-03 17:21:49,832 INFO
> org.apache.hadoop.hive.metastore.HiveMetaStore: [pool-4-thread-21]: 19:
> Cleaning up thread local RawStore...
> 2018-03-03 17:21:49,834 INFO
> org.apache.hadoop.hive.metastore.HiveMetaStore: [pool-4-thread-21]: 19: Done
> cleaning up thread local RawStore
> {code}
> Not very helpful logging. Please change logging levels to _debug_ or even
> _trace_
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-18415) Lower "Updating Partition Stats" Logging Level
[
https://issues.apache.org/jira/browse/HIVE-18415?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16703061#comment-16703061
]
Peter Vary commented on HIVE-18415:
---
[~karthik.manamcheri]: What do you think of this change? Do you want to review?
> Lower "Updating Partition Stats" Logging Level
> --
>
> Key: HIVE-18415
> URL: https://issues.apache.org/jira/browse/HIVE-18415
> Project: Hive
> Issue Type: Improvement
> Components: Metastore
>Affects Versions: 1.2.2, 2.2.0, 3.0.0, 2.3.2
>Reporter: BELUGA BEHR
>Assignee: BELUGA BEHR
>Priority: Trivial
> Attachments: HIVE-18415.1.patch
>
>
> {code:title=org.apache.hadoop.hive.metastore.utils.MetaStoreUtils}
> LOG.warn("Updating partition stats fast for: " + part.getTableName());
> ...
> LOG.warn("Updated size to " + params.get(StatsSetupConst.TOTAL_SIZE));
> {code}
> This logging produces many lines of WARN log messages in my log file and it's
> not clear to me what the issue is here. Why is this a warning and how should
> I respond to address this warning?
> DEBUG is probably more appropriate for a utility class. Please lower.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-19403) Demote 'Pattern' Logging
[
https://issues.apache.org/jira/browse/HIVE-19403?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Peter Vary updated HIVE-19403:
--
Resolution: Fixed
Fix Version/s: 4.0.0
Status: Resolved (was: Patch Available)
Pushed to master.
Thanks for the patch [~gonglinglei] and for all the reviewers!
> Demote 'Pattern' Logging
>
>
> Key: HIVE-19403
> URL: https://issues.apache.org/jira/browse/HIVE-19403
> Project: Hive
> Issue Type: Improvement
> Components: HiveServer2
>Affects Versions: 3.0.0, 2.4.0
>Reporter: BELUGA BEHR
>Assignee: gonglinglei
>Priority: Trivial
> Labels: noob
> Fix For: 4.0.0
>
> Attachments: HIVE-19403.1.patch
>
>
> In the {{DDLTask}} class, there is some logging that is not helpful to a
> cluster admin and should be demoted to _debug_ level logging. In fact, in
> one place in the code, it already is.
> {code}
> LOG.info("pattern: {}", showDatabasesDesc.getPattern());
> LOG.debug("pattern: {}", pattern);
> LOG.info("pattern: {}", showFuncs.getPattern());
> LOG.info("pattern: {}", showTblStatus.getPattern());
> {code}
> Here is an example... as an admin, I can already see what the pattern is, I
> do not need this extra logging. It provides no additional context.
> {code:java|title=Example}
> 2018-05-03 03:08:26,354 INFO org.apache.hadoop.hive.ql.Driver:
> [HiveServer2-Background-Pool: Thread-101980]: Executing
> command(queryId=hive_20180503030808_e53c26ef-2280-4eca-929b-668503105e2e):
> SHOW TABLE EXTENDED FROM my_db LIKE '*'
> 2018-05-03 03:08:26,355 INFO hive.ql.exec.DDLTask:
> [HiveServer2-Background-Pool: Thread-101980]: pattern: *
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-20160) Do Not Print StackTraces to STDERR in OperatorFactory
[
https://issues.apache.org/jira/browse/HIVE-20160?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16703047#comment-16703047
]
Peter Vary commented on HIVE-20160:
---
[~belugabehr]: Please get a green preCommit run.
Thanks,
Peter
> Do Not Print StackTraces to STDERR in OperatorFactory
> -
>
> Key: HIVE-20160
> URL: https://issues.apache.org/jira/browse/HIVE-20160
> Project: Hive
> Issue Type: Improvement
> Components: Query Processor
>Affects Versions: 3.0.0, 4.0.0
>Reporter: BELUGA BEHR
>Assignee: BELUGA BEHR
>Priority: Minor
> Labels: newbie, noob
> Attachments: HIVE-20160.1.patch
>
>
> https://github.com/apache/hive/blob/ac6b2a3fb195916e22b2e5f465add2ffbcdc7430/ql/src/java/org/apache/hadoop/hive/ql/exec/OperatorFactory.java#L158
> {code}
> } catch (Exception e) {
> e.printStackTrace();
> throw new HiveException(...
> {code}
> Do not print the stack trace. The error is being wrapped in a HiveException.
> Allow the code catching this exception to print the error to a logger
> instead of dumping it here to STDERR. There are several instances of this in
> the class.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-20161) Do Not Print StackTraces to STDERR in ParseDriver
[
https://issues.apache.org/jira/browse/HIVE-20161?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Peter Vary updated HIVE-20161:
--
Resolution: Fixed
Fix Version/s: 4.0.0
Status: Resolved (was: Patch Available)
Pushed to master.
Thanks for the patch [~belugabehr] and [~prasanth_j] for the review!
> Do Not Print StackTraces to STDERR in ParseDriver
> -
>
> Key: HIVE-20161
> URL: https://issues.apache.org/jira/browse/HIVE-20161
> Project: Hive
> Issue Type: Improvement
> Components: Query Planning
>Affects Versions: 3.0.0, 4.0.0
>Reporter: BELUGA BEHR
>Assignee: BELUGA BEHR
>Priority: Minor
> Labels: newbie, noob
> Fix For: 4.0.0
>
> Attachments: HIVE-20161.1.patch
>
>
> https://github.com/apache/hive/blob/6d890faf22fd1ede3658a5eed097476eab3c67e9/ql/src/java/org/apache/hadoop/hive/ql/exec/JoinOperator.java
> {code}
> // Do not print stack trace to STDERR - remove this, just throw the
> HiveException
> } catch (Exception e) {
> e.printStackTrace();
> throw new HiveException(e);
> }
> ...
> // Do not log and throw. log *or* throw. In this case, just throw. Remove
> logging.
> // Remove explicit 'return' call. No need for it.
> try {
> skewJoinKeyContext.endGroup();
> } catch (IOException e) {
> LOG.error(e.getMessage(), e);
> throw new HiveException(e);
> }
> return;
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-19846) Removed Deprecated Calls From FileUtils-getJarFilesByPath
[ https://issues.apache.org/jira/browse/HIVE-19846?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Peter Vary updated HIVE-19846: -- Resolution: Fixed Fix Version/s: 4.0.0 Status: Resolved (was: Patch Available) Pushed to master. Thanks for the patch [~belugabehr]! > Removed Deprecated Calls From FileUtils-getJarFilesByPath > - > > Key: HIVE-19846 > URL: https://issues.apache.org/jira/browse/HIVE-19846 > Project: Hive > Issue Type: Improvement > Components: HiveServer2 >Affects Versions: 3.0.0, 4.0.0 >Reporter: BELUGA BEHR >Assignee: BELUGA BEHR >Priority: Minor > Fix For: 4.0.0 > > Attachments: HIVE-19846.1.patch, HIVE-19846.2.patch, > HIVE-19846.3.patch, HIVE-19846.4.patch, HIVE-19846.5.patch.txt, > HIVE-19846.6.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-20983) Vectorization: Scale up small hashtables, when collisions are detected
[
https://issues.apache.org/jira/browse/HIVE-20983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16703041#comment-16703041
]
Hive QA commented on HIVE-20983:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
47s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
0s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
35s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
40s{color} | {color:blue} ql in master has 2312 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
53s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
22s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
0s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
0s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
37s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
44s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
52s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
13s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 22m 13s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-15112/dev-support/hive-personality.sh
|
| git revision | master / a217b81 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-15112/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Vectorization: Scale up small hashtables, when collisions are detected
> --
>
> Key: HIVE-20983
> URL: https://issues.apache.org/jira/browse/HIVE-20983
> Project: Hive
> Issue Type: Bug
>Reporter: Gopal V
>Assignee: Gopal V
>Priority: Major
> Attachments: HIVE-20983.1.patch
>
>
> Hive's hashtable estimates are getting better with HyperLogLog stats in
> place, but an accurate estimate does not always result in a low number of
> collisions.
> The hashtables which contain a very small number of items tend to lose their
> O(1) lookup performance where there are collisions. Since collisions are easy
> to detect within the fast hashtable implementation, a rehashing to a higher
> size will help these small hashtables avoid collisions and go back to O(1)
> perf.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
