Junfan Zhang created OOZIE-3396:
-----------------------------------
Summary: AuthOozieClient leak memeory
Key: OOZIE-3396
URL: https://issues.apache.org/jira/browse/OOZIE-3396
Project: Oozie
Issue Type: Bug
Affects Versions: 5.0.0
Reporter: Junfan Zhang
Assignee: Junfan Zhang
Phenomenon:
Recently we upgraded the AuthOozieClient and added the OOZIE-2447 (Illegal
character 0x0 oozie client) patch. When we were running online, the server
memory usage continued to increase. Through the jmap and jhat command analysis,
it was found that the deleteOnExist method in the writeAuthToken method under
the AuthOozieClient class occupies a large amount of memory, which is caused by
oozie-2447(https://issues.apache.org/jira/browse/OOZIE-2447).
Also, because I have applied the oozie-3379 patch in the production
environment. I still find that the .oozie-auth-token-xxxx file update time has
changed, but the file content has not changed.
After reviewing the code, I found that currentToken.equals(readToken) in the
AuthOozieClient class is always false because the AuthenticatedURL.token does
not have an equals method.
The reason is because currentToken.equals(readToken) is always false, causing
the tmp file to be created continuously, but there is a memory leak due to the
deleteOnExist method
(https://stackoverflow.com/questions/40119188/memory-leak-on-deleteonexithook)
Solution:
1. In the AuthOozieClient, change to
currentToken.toString().equals(readToken.toString())
2. In addition, the deleteOnExist method should be changed to the delete() in
AuthOozieClient class.
Doubt:
Why delete the equals method in the AuthenticatedURL.Token class in Hadoop?
See: https://issues.apache.org/jira/browse/HADOOP-10771
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
