Richard Zowalla created STORM-4052:
--------------------------------------
Summary: Simplify/Remove double delete/lookup in heartbeat cleanup
code
Key: STORM-4052
URL: https://issues.apache.org/jira/browse/STORM-4052
Project: Apache Storm
Issue Type: Bug
Affects Versions: 2.6.2, 2.6.1
Reporter: Richard Zowalla
Fix For: 2.6.3
This changes slightly simplifies the heartbeat cleanup code so it no longer
tries to delete the heartbeat files twice. It also removes an unneeded
directory listing (and possible race) by truncating the versions list and using
it for removal instead of for keeping.
Removing the double delete attempt is important because it removes a lookup for
now non-existent files. Looking up non existent files, especially highly unique
(like timestamped) ones can adversely affect many operating systems as these
lookups are cached as negative dentries.
[https://lwn.net/Articles/814535/]
When cleanup runs, it iterates over the heartbeat directory that contains a
token and version file for each heartbeat. It calls deleteVersion for each file
in the directory which attempts to delete both files associated with the
heartbeat. As deleteVersion already deletes both when it first iterates over
the token file, the iteration for the version file has nothing to do.
Before removing, the deleteVersion code checks for the existence of these now
non existent files. On linux (and other OSs) a lookup for a non-existent path
will create a negative dentry in the operating system's cache. On some
configurations this cache can grow effectively unbounded leading to performance
issues. On newer systems this cache is better managed, but this will still
dilute an otherwise useful OS cache with useless entries.
Copied from [https://github.com/apache/storm/pull/3635] (Author: sammac)
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
