[jira] [Commented] (HIVE-19430) ObjectStore.cleanNotificationEvents OutOfMemory on large number of pending events
[ https://issues.apache.org/jira/browse/HIVE-19430?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16770933#comment-16770933 ] Ashutosh Bapat commented on HIVE-19430: --- Thanks [~sankarh] > ObjectStore.cleanNotificationEvents OutOfMemory on large number of pending > events > - > > Key: HIVE-19430 > URL: https://issues.apache.org/jira/browse/HIVE-19430 > Project: Hive > Issue Type: Bug > Components: HCatalog, repl >Affects Versions: 3.0.0 >Reporter: Thejas M Nair >Assignee: Ashutosh Bapat >Priority: Major > Labels: pull-request-available > Fix For: 4.0.0 > > Attachments: HIVE-19430.01.patch, HIVE-19430.02.patch, > HIVE-19430.03.patch, HIVE-19430.04.patch > > Time Spent: 1h 10m > Remaining Estimate: 0h > > If there are large number of events that haven't been cleaned up for some > reason, then ObjectStore.cleanNotificationEvents() can run out of memory > while it loads all the events to be deleted. > It should fetch events in batches. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-19430) ObjectStore.cleanNotificationEvents OutOfMemory on large number of pending events
[
https://issues.apache.org/jira/browse/HIVE-19430?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16770816#comment-16770816
]
Hive QA commented on HIVE-19430:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12959044/HIVE-19430.04.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 15797 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/16121/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/16121/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-16121/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12959044 - PreCommit-HIVE-Build
> ObjectStore.cleanNotificationEvents OutOfMemory on large number of pending
> events
> -
>
> Key: HIVE-19430
> URL: https://issues.apache.org/jira/browse/HIVE-19430
> Project: Hive
> Issue Type: Bug
>Reporter: Thejas M Nair
>Assignee: Ashutosh Bapat
>Priority: Major
> Labels: pull-request-available
> Fix For: 4.0.0
>
> Attachments: HIVE-19430.01.patch, HIVE-19430.02.patch,
> HIVE-19430.03.patch, HIVE-19430.04.patch
>
> Time Spent: 1h 10m
> Remaining Estimate: 0h
>
> If there are large number of events that haven't been cleaned up for some
> reason, then ObjectStore.cleanNotificationEvents() can run out of memory
> while it loads all the events to be deleted.
> It should fetch events in batches.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-19430) ObjectStore.cleanNotificationEvents OutOfMemory on large number of pending events
[
https://issues.apache.org/jira/browse/HIVE-19430?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16770803#comment-16770803
]
Hive QA commented on HIVE-19430:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
37s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m
59s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
15s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
41s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 2m
34s{color} | {color:blue} standalone-metastore/metastore-common in master has
29 extant Findbugs warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 1m
6s{color} | {color:blue} standalone-metastore/metastore-server in master has
181 extant Findbugs warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
27s{color} | {color:blue} hcatalog/server-extensions in master has 3 extant
Findbugs warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
33s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
28s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
21s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
19s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
19s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
20s{color} | {color:red} standalone-metastore/metastore-server: The patch
generated 1 new + 403 unchanged - 0 fixed = 404 total (was 403) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
25s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
25s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
12s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 26m 56s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-16121/dev-support/hive-personality.sh
|
| git revision | master / aaf01ae |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-16121/yetus/diff-checkstyle-standalone-metastore_metastore-server.txt
|
| modules | C: standalone-metastore/metastore-common
standalone-metastore/metastore-server hcatalog/server-extensions U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-16121/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> ObjectStore.cleanNotificationEvents OutOfMemory on large number of pending
> events
> -
>
> Key: HIVE-19430
> URL: https://issues.apache.org/jira/browse/HIVE-19430
> Project: Hive
> Issue Type: Bug
>Reporter: Thejas M Nair
>Assignee: Ashutosh Bapat
>Priority: Major
> Labels: pull-request-available
> Fix For: 4.0.0
>
> Attachments: HIVE-19430.01.patch, HIVE-19430.02.patch,
> HIVE-19430.03.patch, HIVE-19430.04.patch
>
> Time Spent: 1h 10m
> Remaining Estimate: 0h
>
> If there are
[jira] [Commented] (HIVE-19430) ObjectStore.cleanNotificationEvents OutOfMemory on large number of pending events
[
https://issues.apache.org/jira/browse/HIVE-19430?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16769776#comment-16769776
]
Hive QA commented on HIVE-19430:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12958907/HIVE-19430.03.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 15797 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/16100/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/16100/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-16100/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12958907 - PreCommit-HIVE-Build
> ObjectStore.cleanNotificationEvents OutOfMemory on large number of pending
> events
> -
>
> Key: HIVE-19430
> URL: https://issues.apache.org/jira/browse/HIVE-19430
> Project: Hive
> Issue Type: Bug
>Reporter: Thejas M Nair
>Assignee: Ashutosh Bapat
>Priority: Major
> Labels: pull-request-available
> Fix For: 4.0.0
>
> Attachments: HIVE-19430.01.patch, HIVE-19430.02.patch,
> HIVE-19430.03.patch
>
> Time Spent: 1h 10m
> Remaining Estimate: 0h
>
> If there are large number of events that haven't been cleaned up for some
> reason, then ObjectStore.cleanNotificationEvents() can run out of memory
> while it loads all the events to be deleted.
> It should fetch events in batches.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-19430) ObjectStore.cleanNotificationEvents OutOfMemory on large number of pending events
[
https://issues.apache.org/jira/browse/HIVE-19430?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16769737#comment-16769737
]
Hive QA commented on HIVE-19430:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
44s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
8s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
17s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
43s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 2m
40s{color} | {color:blue} standalone-metastore/metastore-common in master has
29 extant Findbugs warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 1m
13s{color} | {color:blue} standalone-metastore/metastore-server in master has
181 extant Findbugs warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
28s{color} | {color:blue} hcatalog/server-extensions in master has 3 extant
Findbugs warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
33s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
29s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
32s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
22s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
22s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
23s{color} | {color:red} standalone-metastore/metastore-server: The patch
generated 1 new + 403 unchanged - 0 fixed = 404 total (was 403) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
40s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
30s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
14s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 28m 19s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-16100/dev-support/hive-personality.sh
|
| git revision | master / 4cf320f |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-16100/yetus/diff-checkstyle-standalone-metastore_metastore-server.txt
|
| modules | C: standalone-metastore/metastore-common
standalone-metastore/metastore-server hcatalog/server-extensions U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-16100/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> ObjectStore.cleanNotificationEvents OutOfMemory on large number of pending
> events
> -
>
> Key: HIVE-19430
> URL: https://issues.apache.org/jira/browse/HIVE-19430
> Project: Hive
> Issue Type: Bug
>Reporter: Thejas M Nair
>Assignee: Ashutosh Bapat
>Priority: Major
> Labels: pull-request-available
> Fix For: 4.0.0
>
> Attachments: HIVE-19430.01.patch, HIVE-19430.02.patch,
> HIVE-19430.03.patch
>
> Time Spent: 1h 10m
> Remaining Estimate: 0h
>
> If there are large number of
[jira] [Commented] (HIVE-19430) ObjectStore.cleanNotificationEvents OutOfMemory on large number of pending events
[
https://issues.apache.org/jira/browse/HIVE-19430?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16769595#comment-16769595
]
Vihang Karajgaonkar commented on HIVE-19430:
Thanks for providing the patch. Looks good to me. One suggestion related to the
log message.
"LOG.warn("Exception received while cleaning notifications. More details can be
found in debug mode", ex);"
The log message above is not super helpful, since it is not necessary that
logging level is in debug mode when the exception occurred. Since you already
are adding the exception trace in the warning, may be you can remove the "More
details can be found ..." statement.
> ObjectStore.cleanNotificationEvents OutOfMemory on large number of pending
> events
> -
>
> Key: HIVE-19430
> URL: https://issues.apache.org/jira/browse/HIVE-19430
> Project: Hive
> Issue Type: Bug
>Reporter: Thejas M Nair
>Assignee: Ashutosh Bapat
>Priority: Major
> Labels: pull-request-available
> Fix For: 4.0.0
>
> Attachments: HIVE-19430.01.patch, HIVE-19430.02.patch,
> HIVE-19430.03.patch
>
> Time Spent: 1h 10m
> Remaining Estimate: 0h
>
> If there are large number of events that haven't been cleaned up for some
> reason, then ObjectStore.cleanNotificationEvents() can run out of memory
> while it loads all the events to be deleted.
> It should fetch events in batches.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-19430) ObjectStore.cleanNotificationEvents OutOfMemory on large number of pending events
[ https://issues.apache.org/jira/browse/HIVE-19430?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16769472#comment-16769472 ] Sankar Hariappan commented on HIVE-19430: - +1 for 03.patch, pending tests. > ObjectStore.cleanNotificationEvents OutOfMemory on large number of pending > events > - > > Key: HIVE-19430 > URL: https://issues.apache.org/jira/browse/HIVE-19430 > Project: Hive > Issue Type: Bug >Reporter: Thejas M Nair >Assignee: Ashutosh Bapat >Priority: Major > Labels: pull-request-available > Fix For: 4.0.0 > > Attachments: HIVE-19430.01.patch, HIVE-19430.02.patch, > HIVE-19430.03.patch > > Time Spent: 1h 10m > Remaining Estimate: 0h > > If there are large number of events that haven't been cleaned up for some > reason, then ObjectStore.cleanNotificationEvents() can run out of memory > while it loads all the events to be deleted. > It should fetch events in batches. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-19430) ObjectStore.cleanNotificationEvents OutOfMemory on large number of pending events
[
https://issues.apache.org/jira/browse/HIVE-19430?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16769419#comment-16769419
]
Hive QA commented on HIVE-19430:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12958879/HIVE-19430.02.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 2 failed/errored test(s), 15767 tests
executed
*Failed tests:*
{noformat}
TestMiniLlapLocalCliDriver - did not produce a TEST-*.xml file (likely timed
out) (batchId=181)
[vector_windowing_expressions.q,tez_union_group_by.q,materialized_view_rewrite_no_join_opt_2.q,vector_like_2.q,llap_acid.q,sqlmerge.q,tez_dynpart_hashjoin_1.q,schema_evol_orc_acid_part_update_llap_io.q,vector_windowing_gby.q,vector_binary_join_groupby.q,runtime_stats_hs2.q,lateral_view.q,optimize_nullscan.q,vectorization_decimal_date.q,schema_evol_orc_nonvec_table_llap_io.q,udaf_all_keyword.q,acid_vectorization_original.q,tez_fsstat.q,vector_fullouter_mapjoin_1_optimized_passthru.q,stats11.q,vector_mapjoin_reduce.q,join_acid_non_acid.q,empty_join.q,vector_groupby_grouping_window.q,auto_join21.q,tez_input_counters.q,vector_groupby_sort_11.q,schema_evol_orc_nonvec_part_all_complex_llap_io.q,orc_ppd_timestamp.q,vector_decimal_1.q]
org.apache.hadoop.hive.metastore.TestObjectStore.testMasterKeyOps (batchId=230)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/16094/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/16094/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-16094/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 2 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12958879 - PreCommit-HIVE-Build
> ObjectStore.cleanNotificationEvents OutOfMemory on large number of pending
> events
> -
>
> Key: HIVE-19430
> URL: https://issues.apache.org/jira/browse/HIVE-19430
> Project: Hive
> Issue Type: Bug
>Reporter: Thejas M Nair
>Assignee: Ashutosh Bapat
>Priority: Major
> Labels: pull-request-available
> Fix For: 4.0.0
>
> Attachments: HIVE-19430.01.patch, HIVE-19430.02.patch
>
> Time Spent: 1h 10m
> Remaining Estimate: 0h
>
> If there are large number of events that haven't been cleaned up for some
> reason, then ObjectStore.cleanNotificationEvents() can run out of memory
> while it loads all the events to be deleted.
> It should fetch events in batches.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-19430) ObjectStore.cleanNotificationEvents OutOfMemory on large number of pending events
[
https://issues.apache.org/jira/browse/HIVE-19430?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16769384#comment-16769384
]
Hive QA commented on HIVE-19430:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
52s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
30s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
22s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
43s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 2m
33s{color} | {color:blue} standalone-metastore/metastore-common in master has
29 extant Findbugs warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 1m
7s{color} | {color:blue} standalone-metastore/metastore-server in master has
181 extant Findbugs warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
28s{color} | {color:blue} hcatalog/server-extensions in master has 3 extant
Findbugs warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
27s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
29s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
23s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
18s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
18s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
20s{color} | {color:red} standalone-metastore/metastore-server: The patch
generated 1 new + 403 unchanged - 0 fixed = 404 total (was 403) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
31s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
31s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
14s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 28m 7s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-16094/dev-support/hive-personality.sh
|
| git revision | master / 34db82b |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-16094/yetus/diff-checkstyle-standalone-metastore_metastore-server.txt
|
| modules | C: standalone-metastore/metastore-common
standalone-metastore/metastore-server hcatalog/server-extensions U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-16094/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> ObjectStore.cleanNotificationEvents OutOfMemory on large number of pending
> events
> -
>
> Key: HIVE-19430
> URL: https://issues.apache.org/jira/browse/HIVE-19430
> Project: Hive
> Issue Type: Bug
>Reporter: Thejas M Nair
>Assignee: Ashutosh Bapat
>Priority: Major
> Labels: pull-request-available
> Fix For: 4.0.0
>
> Attachments: HIVE-19430.01.patch, HIVE-19430.02.patch
>
> Time Spent: 1h 10m
> Remaining Estimate: 0h
>
> If there are large number of events that haven't been
[jira] [Commented] (HIVE-19430) ObjectStore.cleanNotificationEvents OutOfMemory on large number of pending events
[
https://issues.apache.org/jira/browse/HIVE-19430?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16769288#comment-16769288
]
Hive QA commented on HIVE-19430:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12958867/HIVE-19430.01.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 15 failed/errored test(s), 15797 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[timestamptz_2]
(batchId=86)
org.apache.hadoop.hive.metastore.TestObjectStore.catalogs (batchId=230)
org.apache.hadoop.hive.metastore.TestObjectStore.testDatabaseOps (batchId=230)
org.apache.hadoop.hive.metastore.TestObjectStore.testDeprecatedConfigIsOverwritten
(batchId=230)
org.apache.hadoop.hive.metastore.TestObjectStore.testDirectSQLDropParitionsCleanup
(batchId=230)
org.apache.hadoop.hive.metastore.TestObjectStore.testDirectSQLDropPartitionsCacheCrossSession
(batchId=230)
org.apache.hadoop.hive.metastore.TestObjectStore.testDirectSqlErrorMetrics
(batchId=230)
org.apache.hadoop.hive.metastore.TestObjectStore.testEmptyTrustStoreProps
(batchId=230)
org.apache.hadoop.hive.metastore.TestObjectStore.testMasterKeyOps (batchId=230)
org.apache.hadoop.hive.metastore.TestObjectStore.testMaxEventResponse
(batchId=230)
org.apache.hadoop.hive.metastore.TestObjectStore.testPartitionOps (batchId=230)
org.apache.hadoop.hive.metastore.TestObjectStore.testQueryCloseOnError
(batchId=230)
org.apache.hadoop.hive.metastore.TestObjectStore.testRoleOps (batchId=230)
org.apache.hadoop.hive.metastore.TestObjectStore.testTableOps (batchId=230)
org.apache.hadoop.hive.metastore.TestObjectStore.testUseSSLProperty
(batchId=230)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/16091/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/16091/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-16091/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 15 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12958867 - PreCommit-HIVE-Build
> ObjectStore.cleanNotificationEvents OutOfMemory on large number of pending
> events
> -
>
> Key: HIVE-19430
> URL: https://issues.apache.org/jira/browse/HIVE-19430
> Project: Hive
> Issue Type: Bug
>Reporter: Thejas M Nair
>Assignee: Ashutosh Bapat
>Priority: Major
> Labels: pull-request-available
> Fix For: 4.0.0
>
> Attachments: HIVE-19430.01.patch, HIVE-19430.02.patch
>
> Time Spent: 1h 10m
> Remaining Estimate: 0h
>
> If there are large number of events that haven't been cleaned up for some
> reason, then ObjectStore.cleanNotificationEvents() can run out of memory
> while it loads all the events to be deleted.
> It should fetch events in batches.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-19430) ObjectStore.cleanNotificationEvents OutOfMemory on large number of pending events
[
https://issues.apache.org/jira/browse/HIVE-19430?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16769260#comment-16769260
]
Hive QA commented on HIVE-19430:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
34s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
1s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
17s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
42s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 2m
28s{color} | {color:blue} standalone-metastore/metastore-common in master has
29 extant Findbugs warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 1m
7s{color} | {color:blue} standalone-metastore/metastore-server in master has
181 extant Findbugs warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
27s{color} | {color:blue} hcatalog/server-extensions in master has 3 extant
Findbugs warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
28s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
27s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
18s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
19s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
19s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
20s{color} | {color:red} standalone-metastore/metastore-server: The patch
generated 1 new + 403 unchanged - 0 fixed = 404 total (was 403) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
19s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
31s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
13s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 26m 44s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-16091/dev-support/hive-personality.sh
|
| git revision | master / 34db82b |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-16091/yetus/diff-checkstyle-standalone-metastore_metastore-server.txt
|
| modules | C: standalone-metastore/metastore-common
standalone-metastore/metastore-server hcatalog/server-extensions U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-16091/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> ObjectStore.cleanNotificationEvents OutOfMemory on large number of pending
> events
> -
>
> Key: HIVE-19430
> URL: https://issues.apache.org/jira/browse/HIVE-19430
> Project: Hive
> Issue Type: Bug
>Reporter: Thejas M Nair
>Assignee: Ashutosh Bapat
>Priority: Major
> Labels: pull-request-available
> Fix For: 4.0.0
>
> Attachments: HIVE-19430.01.patch
>
> Time Spent: 1h
> Remaining Estimate: 0h
>
> If there are large number of events that haven't been cleaned up for some
>
