[
https://issues.apache.org/jira/browse/AMBARI-17834?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15387762#comment-15387762
]
Hemanth Yamijala commented on AMBARI-17834:
-------------------------------------------
Another impact I see is that the code to stop this thread would not work
because it waits until the size of the data structure is 0.
> HDFS copy thread in OutputHDFSFile is not removing files that are processed
> ----------------------------------------------------------------------------
>
> Key: AMBARI-17834
> URL: https://issues.apache.org/jira/browse/AMBARI-17834
> Project: Ambari
> Issue Type: Bug
> Components: ambari-logsearch
> Reporter: Hemanth Yamijala
>
> Consider the HDFS copy thread here:
> https://github.com/apache/ambari/blob/trunk/ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/output/OutputHDFSFile.java#L177
> It uses a data structure localReadyFiles that has the list of files to copy
> to HDFS. But AFAICS, nowhere are processed files removed from this data
> structure. This could result in multiple incorrect behaviours:
> * Over a period of time, this structure could keep growing indefinitely.
> While the check for existence of files will allow progress, it will take
> longer and longer to run.
> * The check to wait when there are no files to process will never pass
> because the size of the data structure will never be 0. This could lead to
> tight looped checks and some utilization issues.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
