[
https://issues.apache.org/jira/browse/HIVE-24590?focusedWorklogId=669778&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-669778
]
ASF GitHub Bot logged work on HIVE-24590:
-----------------------------------------
Author: ASF GitHub Bot
Created on: 25/Oct/21 20:23
Start Date: 25/Oct/21 20:23
Worklog Time Spent: 10m
Work Description: zabetak commented on pull request #2432:
URL: https://github.com/apache/hive/pull/2432#issuecomment-951288032
> Ya, nothing about this solution feels ideal. How was the default of 60s
chosen? That seems pretty short to me if there's a stall in the query engine
(or anywhere else).
Even if there is a big delay and the 60sec window passes there is no
problem. The appender will be closed and it will reopen again when the next log
event arrives. Having a purge policy guarantees that there will be no leak no
matter what happens.
Something that may become problematic is the arrival of many (in the order
of thousands) queries in the 60 sec window. This will lead to the creation of
many appenders and usage of many file descriptors. However, this might never be
a problem for Hive, at least not before other parts of the system need to be
fixed first.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 669778)
Time Spent: 3h 40m (was: 3.5h)
> Operation Logging still leaks the log4j Appenders
> -------------------------------------------------
>
> Key: HIVE-24590
> URL: https://issues.apache.org/jira/browse/HIVE-24590
> Project: Hive
> Issue Type: Bug
> Components: Logging
> Reporter: Eugene Chung
> Assignee: Stamatis Zampetakis
> Priority: Major
> Labels: pull-request-available
> Attachments: Screen Shot 2021-01-06 at 18.42.05.png, Screen Shot
> 2021-01-06 at 18.42.24.png, Screen Shot 2021-01-06 at 18.42.55.png, Screen
> Shot 2021-01-06 at 21.38.32.png, Screen Shot 2021-01-06 at 21.47.28.png,
> Screen Shot 2021-01-08 at 21.01.40.png, add_debug_log_and_trace.patch
>
> Time Spent: 3h 40m
> Remaining Estimate: 0h
>
> I'm using Hive 3.1.2 with options below.
> * hive.server2.logging.operation.enabled=true
> * hive.server2.logging.operation.level=VERBOSE
> * hive.async.log.enabled=false
> I already know the ticket, https://issues.apache.org/jira/browse/HIVE-17128
> but HS2 still leaks log4j RandomAccessFileManager.
> !Screen Shot 2021-01-06 at 18.42.05.png|width=756,height=197!
> I checked the operation log file which is not closed/deleted properly.
> !Screen Shot 2021-01-06 at 18.42.24.png|width=603,height=272!
> Then there's the log,
> {code:java}
> client.TezClient: Shutting down Tez Session, sessionName= ....{code}
> !Screen Shot 2021-01-06 at 18.42.55.png|width=1372,height=26!
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
