[ 
https://issues.apache.org/jira/browse/YARN-9080?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16808747#comment-16808747
 ] 

Peter Bacsko commented on YARN-9080:
------------------------------------

I'd like to join Szilard in suggesting making the code more readable.

I can imagine sth like:

{code}
            RemoteIterator<FileStatus> clustertsIter = list(dirpath);
            while (clustertsIter.hasNext()) {
               FileStatus clustertsStat = clustertsIter.next();
               MutableBoolean toBeRemoved = new MutableBoolean();
               MutableBoolean isValid = new MutableBoolean();

               if (clustertsStat.isDirectory()) {
                processClusterTsDir(clustertsStat, toBeRemoved, isValid);
             }
           ..

private void processClusterTsDir(FileStatus fs, MutableBoolean toBeRemoved, 
MutableBoolean isValid) {
  Path clustertsPath = fs.getPath();
  RemoteIterator<FileStatus> bucket1Iter = list(dir);
  
  while (bucket1Iter.hasNext()) {
    FileStatus bucket1Stat = bucket1Iter.next();
    Path bucket1Path = bucket1Stat.getPath();
    if (bucket1Stat.isDirectory() &&
    bucket1Path.getName().matches(bucket1Regex)) {
      processBucket1Dir(bucket1Stat, toBeRemoved, isValid);
   }
}

private void processBucket1Dir(FileStatus fs, MutableBoolean toBeRemoved, 
MutableBoolean isValid) {
// walk through directories, check condition, go down to processBucket2Dir if 
it's true
...
}

private void processBucket2Dir(FileStatus fs, MutableBoolean toBeRemoved, 
MutableBoolean isValid) {
...
// walk through directories, check condition, go down to processAppDir if it's 
true
}

private void processAppDir(FileStatus fs, MutableBoolean toBeRemoved, 
MutableBoolean isValid) {
...
}
{code}

So basically each time you descend down the hierarchy, you enter a new method 
and pass around fields that you need later - then changes will be reflected in 
the outermost call.

> Bucket Directories as part of ATS done accumulates
> --------------------------------------------------
>
>                 Key: YARN-9080
>                 URL: https://issues.apache.org/jira/browse/YARN-9080
>             Project: Hadoop YARN
>          Issue Type: Bug
>            Reporter: Prabhu Joseph
>            Assignee: Prabhu Joseph
>            Priority: Major
>         Attachments: 0001-YARN-9080.patch, 0002-YARN-9080.patch, 
> 0003-YARN-9080.patch, YARN-9080-004.patch, YARN-9080-005.patch, 
> YARN-9080-006.patch
>
>
> Have observed older bucket directories cluster_timestamp, bucket1 and bucket2 
> as part of ATS done accumulates. The cleanLogs part of EntityLogCleaner 
> removes only the app directories and not the bucket directories.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org

Reply via email to