Raghu Angadi wrote:
Keith Fisher wrote:
I realized that this could be one alternative. But what if the process
writing to HDFS is a daemon that's designed to run 24X7? In that
scenario,
will it eventually, over time, use all the open files?
Or will the DFSClient periodically close resources that it no longer
needs?
It immediately closes the resources it does not need. Namely if you
don't have any files open, it would not hold any file descriptors.
Roughly speaking. If you had 100 threads doing I/O on the client and if
you close all the files at once, it might take about 10 seconds for some
internal file descriptors to close. These internal descriptors are
proportional to number of active threads blocked on I/O and not to
number of threads.
Raghu.
Raghu.
Thanks again.
