[
https://issues.apache.org/jira/browse/YARN-9080?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16808714#comment-16808714
]
Szilard Nemeth commented on YARN-9080:
--------------------------------------
Hi [~Prabhu Joseph]!
Here are my comments:
1. The depth of the nested while / if statements makes to code very hard to
read and increase cyclomatic complexity
(https://en.wikipedia.org/wiki/Cyclomatic_complexity)
First of all, I would extract the logic into some private methods.
Essentially, the pseudo-code of the algorithm is this:
1. Loop over list of files under dirPath
2. if file is a directory, we should do something with the dir, let's call this
"dir1"
3. We loop over files under "dir1" (bucket1Iter)
4. If file is a directory and it matches bucket1Regex, we iterate over files
under the file (bucket2Iter)
5. If the file matches bucket2Regex then we have a valid dir
6. If we have files under this dir, we loop over those
7. If we find a directory and it's a valid applicationId, we invoke delete.
Please try to come up with something more readable, more easy to understand.
I would try to extract methods first along with the while-loops then go until
you have reasonable chunks of code.
2. I was wondering what is the meaning of "clusterts" and only realized from
the tests that is clusterTimeStamp. You should either use this latter name or
use clusterTs, but I prefer clusterTimeStamp.
3. Please extract the condition of the if-statement into a method from here:
{code:java}
if ((fs.listStatus(bucket2Path).length != 0) || (now
- bucket2Stat.getModificationTime() <= retainMillis)) {
{code}
Please let me know if you are ready and I will check again, thanks!
> Bucket Directories as part of ATS done accumulates
> --------------------------------------------------
>
> Key: YARN-9080
> URL: https://issues.apache.org/jira/browse/YARN-9080
> Project: Hadoop YARN
> Issue Type: Bug
> Reporter: Prabhu Joseph
> Assignee: Prabhu Joseph
> Priority: Major
> Attachments: 0001-YARN-9080.patch, 0002-YARN-9080.patch,
> 0003-YARN-9080.patch, YARN-9080-004.patch, YARN-9080-005.patch,
> YARN-9080-006.patch
>
>
> Have observed older bucket directories cluster_timestamp, bucket1 and bucket2
> as part of ATS done accumulates. The cleanLogs part of EntityLogCleaner
> removes only the app directories and not the bucket directories.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
