[
https://issues.apache.org/jira/browse/HIVE-24590?focusedWorklogId=674241&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-674241
]
ASF GitHub Bot logged work on HIVE-24590:
-----------------------------------------
Author: ASF GitHub Bot
Created on: 02/Nov/21 22:40
Start Date: 02/Nov/21 22:40
Worklog Time Spent: 10m
Work Description: zabetak commented on pull request #2432:
URL: https://github.com/apache/hive/pull/2432#issuecomment-958309007
> What is responsible for cleaning up these files? Could there be a case
where the cleanup process happens and then the file is re-created and never
scheduled for cleanup again?
@belugabehr It's been a while since I have worked on this code but if I
remember well the operation log file deletion is handled by
`OperationLogCleaner`. When the query/operation ends it is registered for
cleanup (see
https://github.com/apache/hive/blob/6e152aa28bc5116bf9210f9deb0f95d2d73183f7/service/src/java/org/apache/hive/service/cli/operation/Operation.java#L323)
. The deletion may happen immediately or with some delay as specified by the
properties.
If for some reason there is a log event for the **same** query after the
cleanup then I assume the file will be recreated and it will never be removed
from the file-system. This could happen even without the changes introduced in
this PR.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 674241)
Time Spent: 4h 50m (was: 4h 40m)
> Operation Logging still leaks the log4j Appenders
> -------------------------------------------------
>
> Key: HIVE-24590
> URL: https://issues.apache.org/jira/browse/HIVE-24590
> Project: Hive
> Issue Type: Bug
> Components: Logging
> Reporter: Eugene Chung
> Assignee: Stamatis Zampetakis
> Priority: Major
> Labels: pull-request-available
> Attachments: Screen Shot 2021-01-06 at 18.42.05.png, Screen Shot
> 2021-01-06 at 18.42.24.png, Screen Shot 2021-01-06 at 18.42.55.png, Screen
> Shot 2021-01-06 at 21.38.32.png, Screen Shot 2021-01-06 at 21.47.28.png,
> Screen Shot 2021-01-08 at 21.01.40.png, add_debug_log_and_trace.patch
>
> Time Spent: 4h 50m
> Remaining Estimate: 0h
>
> I'm using Hive 3.1.2 with options below.
> * hive.server2.logging.operation.enabled=true
> * hive.server2.logging.operation.level=VERBOSE
> * hive.async.log.enabled=false
> I already know the ticket, https://issues.apache.org/jira/browse/HIVE-17128
> but HS2 still leaks log4j RandomAccessFileManager.
> !Screen Shot 2021-01-06 at 18.42.05.png|width=756,height=197!
> I checked the operation log file which is not closed/deleted properly.
> !Screen Shot 2021-01-06 at 18.42.24.png|width=603,height=272!
> Then there's the log,
> {code:java}
> client.TezClient: Shutting down Tez Session, sessionName= ....{code}
> !Screen Shot 2021-01-06 at 18.42.55.png|width=1372,height=26!
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
