[jira] [Commented] (HIVE-15541) Hive OOM when ATSHook enabled and ATS goes down
[ https://issues.apache.org/jira/browse/HIVE-15541?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15837487#comment-15837487 ] Lefty Leverenz commented on HIVE-15541: --- Doc note: This adds *hive.ats.hook.queue.capacity* to HiveConf.java, so it needs to be documented in the wiki for release 2.2.0. * [Configuration Properties -- Query and DDL Execution | https://cwiki.apache.org/confluence/display/Hive/Configuration+Properties#ConfigurationProperties-QueryandDDLExecution] Added a TODOC2.2 label. > Hive OOM when ATSHook enabled and ATS goes down > --- > > Key: HIVE-15541 > URL: https://issues.apache.org/jira/browse/HIVE-15541 > Project: Hive > Issue Type: Bug > Components: Hooks >Reporter: Jason Dere >Assignee: Jason Dere > Labels: TODOC2.2 > Fix For: 2.2.0 > > Attachments: HIVE-15541.1.patch, HIVE-15541.2.patch, > HIVE-15541.3.patch, HIVE-15541.4.patch > > > The ATS API used by the Hive ATSHook is a blocking call, if ATS goes down > this can block the ATSHook executor, while the hook continues to submit work > to the executor with each query. > Over time the buildup of queued items can cause OOM. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-15541) Hive OOM when ATSHook enabled and ATS goes down
[ https://issues.apache.org/jira/browse/HIVE-15541?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15836467#comment-15836467 ] Jason Dere commented on HIVE-15541: --- Test were broken by HIVE-15297. About fireAndForget() and conf, that variable is not final so it does not get used by the Runnable. Will remove. > Hive OOM when ATSHook enabled and ATS goes down > --- > > Key: HIVE-15541 > URL: https://issues.apache.org/jira/browse/HIVE-15541 > Project: Hive > Issue Type: Bug > Components: Hooks >Reporter: Jason Dere >Assignee: Jason Dere > Attachments: HIVE-15541.1.patch, HIVE-15541.2.patch, > HIVE-15541.3.patch, HIVE-15541.4.patch > > > The ATS API used by the Hive ATSHook is a blocking call, if ATS goes down > this can block the ATSHook executor, while the hook continues to submit work > to the executor with each query. > Over time the buildup of queued items can cause OOM. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-15541) Hive OOM when ATSHook enabled and ATS goes down
[
https://issues.apache.org/jira/browse/HIVE-15541?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15830942#comment-15830942
]
Hive QA commented on HIVE-15541:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12848181/HIVE-15541.4.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 50 failed/errored test(s), 10964 tests
executed
*Failed tests:*
{noformat}
TestDerbyConnector - did not produce a TEST-*.xml file (likely timed out)
(batchId=235)
org.apache.hadoop.hive.cli.TestAccumuloCliDriver.testCliDriver[accumulo_joins]
(batchId=219)
org.apache.hadoop.hive.cli.TestAccumuloCliDriver.testCliDriver[accumulo_predicate_pushdown]
(batchId=219)
org.apache.hadoop.hive.cli.TestAccumuloCliDriver.testCliDriver[accumulo_single_sourced_multi_insert]
(batchId=219)
org.apache.hadoop.hive.cli.TestBlobstoreCliDriver.testCliDriver[ctas]
(batchId=231)
org.apache.hadoop.hive.cli.TestBlobstoreCliDriver.testCliDriver[insert_into_dynamic_partitions]
(batchId=231)
org.apache.hadoop.hive.cli.TestBlobstoreCliDriver.testCliDriver[insert_into_table]
(batchId=231)
org.apache.hadoop.hive.cli.TestBlobstoreCliDriver.testCliDriver[insert_overwrite_directory]
(batchId=231)
org.apache.hadoop.hive.cli.TestBlobstoreCliDriver.testCliDriver[insert_overwrite_dynamic_partitions]
(batchId=231)
org.apache.hadoop.hive.cli.TestBlobstoreCliDriver.testCliDriver[insert_overwrite_table]
(batchId=231)
org.apache.hadoop.hive.cli.TestBlobstoreCliDriver.testCliDriver[write_final_output_blobstore]
(batchId=231)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[specialChar] (batchId=22)
org.apache.hadoop.hive.cli.TestContribCliDriver.testCliDriver[dboutput]
(batchId=222)
org.apache.hadoop.hive.cli.TestContribCliDriver.testCliDriver[fileformat_base64]
(batchId=222)
org.apache.hadoop.hive.cli.TestContribCliDriver.testCliDriver[udf_row_sequence]
(batchId=222)
org.apache.hadoop.hive.cli.TestContribCliDriver.testCliDriver[url_hook]
(batchId=222)
org.apache.hadoop.hive.cli.TestContribNegativeCliDriver.testCliDriver[case_with_row_sequence]
(batchId=225)
org.apache.hadoop.hive.cli.TestContribNegativeCliDriver.testCliDriver[invalid_row_sequence]
(batchId=225)
org.apache.hadoop.hive.cli.TestContribNegativeCliDriver.testCliDriver[serde_regex]
(batchId=225)
org.apache.hadoop.hive.cli.TestEncryptedHDFSCliDriver.testCliDriver[encryption_insert_partition_dynamic]
(batchId=158)
org.apache.hadoop.hive.cli.TestEncryptedHDFSCliDriver.testCliDriver[encryption_insert_partition_static]
(batchId=156)
org.apache.hadoop.hive.cli.TestEncryptedHDFSCliDriver.testCliDriver[encryption_insert_values]
(batchId=157)
org.apache.hadoop.hive.cli.TestEncryptedHDFSCliDriver.testCliDriver[encryption_join_unencrypted_tbl]
(batchId=159)
org.apache.hadoop.hive.cli.TestEncryptedHDFSCliDriver.testCliDriver[encryption_join_with_different_encryption_keys]
(batchId=159)
org.apache.hadoop.hive.cli.TestEncryptedHDFSCliDriver.testCliDriver[encryption_load_data_to_encrypted_tables]
(batchId=157)
org.apache.hadoop.hive.cli.TestEncryptedHDFSCliDriver.testCliDriver[encryption_move_tbl]
(batchId=157)
org.apache.hadoop.hive.cli.TestEncryptedHDFSCliDriver.testCliDriver[encryption_select_read_only_encrypted_tbl]
(batchId=159)
org.apache.hadoop.hive.cli.TestEncryptedHDFSCliDriver.testCliDriver[encryption_select_read_only_unencrypted_tbl]
(batchId=159)
org.apache.hadoop.hive.cli.TestEncryptedHDFSCliDriver.testCliDriver[encryption_unencrypted_nonhdfs_external_tables]
(batchId=157)
org.apache.hadoop.hive.cli.TestHBaseNegativeCliDriver.testCliDriver[cascade_dbdrop]
(batchId=226)
org.apache.hadoop.hive.cli.TestHBaseNegativeCliDriver.testCliDriver[generatehfiles_require_family_path]
(batchId=226)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[orc_ppd_basic]
(batchId=135)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[orc_ppd_schema_evol_3a]
(batchId=136)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[escape1]
(batchId=139)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[escape2]
(batchId=154)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[offset_limit_ppd_optimizer]
(batchId=151)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[schema_evol_text_vec_part]
(batchId=149)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[vector_if_expr]
(batchId=140)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[vector_varchar_simple]
(batchId=152)
org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver[bucket_num_reducers2]
(batchId=84)
org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver[bucket_num_reducers]
(batchId=84)
org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver[index_bitmap3]
(batchId=83)
org.apache.hadoop.hive.cli.Te
[jira] [Commented] (HIVE-15541) Hive OOM when ATSHook enabled and ATS goes down
[ https://issues.apache.org/jira/browse/HIVE-15541?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15830639#comment-15830639 ] Sergey Shelukhin commented on HIVE-15541: - fireAndForget does not need conf. Not sure if Java is smart enough to remove it from Runnable capture, I hope so (since it's not final and works anyway), but removing it would be harmless +1 can be fixed on commit > Hive OOM when ATSHook enabled and ATS goes down > --- > > Key: HIVE-15541 > URL: https://issues.apache.org/jira/browse/HIVE-15541 > Project: Hive > Issue Type: Bug > Components: Hooks >Reporter: Jason Dere >Assignee: Jason Dere > Attachments: HIVE-15541.1.patch, HIVE-15541.2.patch, > HIVE-15541.3.patch, HIVE-15541.4.patch > > > The ATS API used by the Hive ATSHook is a blocking call, if ATS goes down > this can block the ATSHook executor, while the hook continues to submit work > to the executor with each query. > Over time the buildup of queued items can cause OOM. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-15541) Hive OOM when ATSHook enabled and ATS goes down
[ https://issues.apache.org/jira/browse/HIVE-15541?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15829077#comment-15829077 ] Jason Dere commented on HIVE-15541: --- [~sershe], can you take a look one more time? > Hive OOM when ATSHook enabled and ATS goes down > --- > > Key: HIVE-15541 > URL: https://issues.apache.org/jira/browse/HIVE-15541 > Project: Hive > Issue Type: Bug > Components: Hooks >Reporter: Jason Dere >Assignee: Jason Dere > Attachments: HIVE-15541.1.patch, HIVE-15541.2.patch, > HIVE-15541.3.patch, HIVE-15541.4.patch > > > The ATS API used by the Hive ATSHook is a blocking call, if ATS goes down > this can block the ATSHook executor, while the hook continues to submit work > to the executor with each query. > Over time the buildup of queued items can cause OOM. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-15541) Hive OOM when ATSHook enabled and ATS goes down
[ https://issues.apache.org/jira/browse/HIVE-15541?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15829051#comment-15829051 ] Jason Dere commented on HIVE-15541: --- Per [~hagleitn], Creating the ATS event on the main query thread is expensive and should be done on a separate thread. Will try creating a separate thread pool for sending the event to ATS, so that the existing executor does not get tied up in blocking calls. > Hive OOM when ATSHook enabled and ATS goes down > --- > > Key: HIVE-15541 > URL: https://issues.apache.org/jira/browse/HIVE-15541 > Project: Hive > Issue Type: Bug > Components: Hooks >Reporter: Jason Dere >Assignee: Jason Dere > Attachments: HIVE-15541.1.patch, HIVE-15541.2.patch, > HIVE-15541.3.patch > > > The ATS API used by the Hive ATSHook is a blocking call, if ATS goes down > this can block the ATSHook executor, while the hook continues to submit work > to the executor with each query. > Over time the buildup of queued items can cause OOM. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-15541) Hive OOM when ATSHook enabled and ATS goes down
[ https://issues.apache.org/jira/browse/HIVE-15541?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15819393#comment-15819393 ] Jason Dere commented on HIVE-15541: --- When the issue was first found, I think something like 100 queries was enough to hit OOM, so was trying to default this to a conservative number. > Hive OOM when ATSHook enabled and ATS goes down > --- > > Key: HIVE-15541 > URL: https://issues.apache.org/jira/browse/HIVE-15541 > Project: Hive > Issue Type: Bug > Components: Hooks >Reporter: Jason Dere >Assignee: Jason Dere > Attachments: HIVE-15541.1.patch, HIVE-15541.2.patch, > HIVE-15541.3.patch > > > The ATS API used by the Hive ATSHook is a blocking call, if ATS goes down > this can block the ATSHook executor, while the hook continues to submit work > to the executor with each query. > Over time the buildup of queued items can cause OOM. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-15541) Hive OOM when ATSHook enabled and ATS goes down
[ https://issues.apache.org/jira/browse/HIVE-15541?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15819078#comment-15819078 ] Sergey Shelukhin commented on HIVE-15541: - nit: whitespace in case PRE_EXEC_HOOK: is weird (statements are on different levels than other cases). Why 64 and not 128 or 10240? :) What I had in mind was a single thread drawing items out of the queue, w/o runnables. But this also work. +1, above can be fixed on commit > Hive OOM when ATSHook enabled and ATS goes down > --- > > Key: HIVE-15541 > URL: https://issues.apache.org/jira/browse/HIVE-15541 > Project: Hive > Issue Type: Bug > Components: Hooks >Reporter: Jason Dere >Assignee: Jason Dere > Attachments: HIVE-15541.1.patch, HIVE-15541.2.patch, > HIVE-15541.3.patch > > > The ATS API used by the Hive ATSHook is a blocking call, if ATS goes down > this can block the ATSHook executor, while the hook continues to submit work > to the executor with each query. > Over time the buildup of queued items can cause OOM. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-15541) Hive OOM when ATSHook enabled and ATS goes down
[
https://issues.apache.org/jira/browse/HIVE-15541?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15816891#comment-15816891
]
Hive QA commented on HIVE-15541:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12846668/HIVE-15541.3.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 9 failed/errored test(s), 10949 tests
executed
*Failed tests:*
{noformat}
TestDerbyConnector - did not produce a TEST-*.xml file (likely timed out)
(batchId=233)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[case_sensitivity]
(batchId=61)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[input_testxpath]
(batchId=28)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[udf_coalesce]
(batchId=75)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[orc_ppd_basic]
(batchId=134)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[orc_ppd_schema_evol_3a]
(batchId=135)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[offset_limit_ppd_optimizer]
(batchId=150)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[schema_evol_text_vec_part]
(batchId=148)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[vector_if_expr]
(batchId=139)
{noformat}
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/2870/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/2870/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-2870/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 9 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12846668 - PreCommit-HIVE-Build
> Hive OOM when ATSHook enabled and ATS goes down
> ---
>
> Key: HIVE-15541
> URL: https://issues.apache.org/jira/browse/HIVE-15541
> Project: Hive
> Issue Type: Bug
> Components: Hooks
>Reporter: Jason Dere
>Assignee: Jason Dere
> Attachments: HIVE-15541.1.patch, HIVE-15541.2.patch,
> HIVE-15541.3.patch
>
>
> The ATS API used by the Hive ATSHook is a blocking call, if ATS goes down
> this can block the ATSHook executor, while the hook continues to submit work
> to the executor with each query.
> Over time the buildup of queued items can cause OOM.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-15541) Hive OOM when ATSHook enabled and ATS goes down
[ https://issues.apache.org/jira/browse/HIVE-15541?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15803004#comment-15803004 ] Sergey Shelukhin commented on HIVE-15541: - Hmm. Why not change the queue to reference requests created at enqueue time, instead of capturing huge amount of context in the runnable (unless I'm reading this wrong)? In addition to queue limit. That way queue limit could probably be set to a larger value. > Hive OOM when ATSHook enabled and ATS goes down > --- > > Key: HIVE-15541 > URL: https://issues.apache.org/jira/browse/HIVE-15541 > Project: Hive > Issue Type: Bug > Components: Hooks >Reporter: Jason Dere >Assignee: Jason Dere > Attachments: HIVE-15541.1.patch, HIVE-15541.2.patch > > > The ATS API used by the Hive ATSHook is a blocking call, if ATS goes down > this can block the ATSHook executor, while the hook continues to submit work > to the executor with each query. > Over time the buildup of queued items can cause OOM. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-15541) Hive OOM when ATSHook enabled and ATS goes down
[ https://issues.apache.org/jira/browse/HIVE-15541?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15802965#comment-15802965 ] Jason Dere commented on HIVE-15541: --- [~sershe] review? > Hive OOM when ATSHook enabled and ATS goes down > --- > > Key: HIVE-15541 > URL: https://issues.apache.org/jira/browse/HIVE-15541 > Project: Hive > Issue Type: Bug > Components: Hooks >Reporter: Jason Dere >Assignee: Jason Dere > Attachments: HIVE-15541.1.patch, HIVE-15541.2.patch > > > The ATS API used by the Hive ATSHook is a blocking call, if ATS goes down > this can block the ATSHook executor, while the hook continues to submit work > to the executor with each query. > Over time the buildup of queued items can cause OOM. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-15541) Hive OOM when ATSHook enabled and ATS goes down
[
https://issues.apache.org/jira/browse/HIVE-15541?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15802346#comment-15802346
]
Hive QA commented on HIVE-15541:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12845838/HIVE-15541.2.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 7 failed/errored test(s), 10915 tests
executed
*Failed tests:*
{noformat}
TestDerbyConnector - did not produce a TEST-*.xml file (likely timed out)
(batchId=233)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[case_sensitivity]
(batchId=61)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[input_testxpath]
(batchId=28)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[udf_coalesce]
(batchId=75)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[orc_ppd_basic]
(batchId=134)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[orc_ppd_schema_evol_3a]
(batchId=135)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[offset_limit_ppd_optimizer]
(batchId=150)
{noformat}
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/2802/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/2802/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-2802/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 7 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12845838 - PreCommit-HIVE-Build
> Hive OOM when ATSHook enabled and ATS goes down
> ---
>
> Key: HIVE-15541
> URL: https://issues.apache.org/jira/browse/HIVE-15541
> Project: Hive
> Issue Type: Bug
> Components: Hooks
>Reporter: Jason Dere
>Assignee: Jason Dere
> Attachments: HIVE-15541.1.patch, HIVE-15541.2.patch
>
>
> The ATS API used by the Hive ATSHook is a blocking call, if ATS goes down
> this can block the ATSHook executor, while the hook continues to submit work
> to the executor with each query.
> Over time the buildup of queued items can cause OOM.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-15541) Hive OOM when ATSHook enabled and ATS goes down
[
https://issues.apache.org/jira/browse/HIVE-15541?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15800278#comment-15800278
]
Hive QA commented on HIVE-15541:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12845686/HIVE-15541.1.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 5 failed/errored test(s), 10914 tests
executed
*Failed tests:*
{noformat}
TestDerbyConnector - did not produce a TEST-*.xml file (likely timed out)
(batchId=233)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[case_sensitivity]
(batchId=61)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[input_testxpath]
(batchId=28)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[udf_coalesce]
(batchId=75)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[orc_ppd_basic]
(batchId=134)
{noformat}
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/2789/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/2789/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-2789/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 5 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12845686 - PreCommit-HIVE-Build
> Hive OOM when ATSHook enabled and ATS goes down
> ---
>
> Key: HIVE-15541
> URL: https://issues.apache.org/jira/browse/HIVE-15541
> Project: Hive
> Issue Type: Bug
> Components: Hooks
>Reporter: Jason Dere
>Assignee: Jason Dere
> Attachments: HIVE-15541.1.patch
>
>
> The ATS API used by the Hive ATSHook is a blocking call, if ATS goes down
> this can block the ATSHook executor, while the hook continues to submit work
> to the executor with each query.
> Over time the buildup of queued items can cause OOM.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-15541) Hive OOM when ATSHook enabled and ATS goes down
[ https://issues.apache.org/jira/browse/HIVE-15541?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15800034#comment-15800034 ] Jason Dere commented on HIVE-15541: --- One solution is to create the ATSHook executor with a bounded queue capacity so the hook does not submit new requests if it gets too full. > Hive OOM when ATSHook enabled and ATS goes down > --- > > Key: HIVE-15541 > URL: https://issues.apache.org/jira/browse/HIVE-15541 > Project: Hive > Issue Type: Bug > Components: Hooks >Reporter: Jason Dere >Assignee: Jason Dere > > The ATS API used by the Hive ATSHook is a blocking call, if ATS goes down > this can block the ATSHook executor, while the hook continues to submit work > to the executor with each query. > Over time the buildup of queued items can cause OOM. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
