bhavinthaker commented on issue #8904: fix multiprocessing too many open files URL: https://github.com/apache/incubator-mxnet/pull/8904#issuecomment-349420885 I have documented the following in the MXNet 1.0 Release Notes here: https://cwiki.apache.org/confluence/display/MXNET/Apache+MXNet+%28incubating%29+1.0+Release+Notes (see this Release notes for up-to-date information) Problem: The data-loader has a file-descriptor leak when the number of files being opened exceeds the ulimit of the user session. Workaround: Increase the ulimit of the user session or at the OS level using any of the following steps: $ ulimit -n unlimited (to make the fd limit as unlimited), OR $ ulimit -n 60000 (to make the fd limit as 60,000 files assuming you have around but less than 60,000 files to be opened) OR $ echo 65535 > /proc/sys/fs/file-max (to increase the system-wide file limit) along with the ulimit command. Further details: https://docs.oracle.com/cd/E19623-01/820-6168/file-descriptor-requirements.html
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services
