Issue #13097 has been reported by Evan Mezeske. ---------------------------------------- Bug #13097: Filebucket reads entire files into memory https://projects.puppetlabs.com/issues/13097
Author: Evan Mezeske Status: Unreviewed Priority: Normal Assignee: Category: filebucket Target version: Affected Puppet version: Keywords: Branch: During backup/restore, Puppet::FileBucket::Dipper uses IO.binread() with no length parameter to read the entire file resource in question into memory at one time. This is very simple, but it causes problems when Puppet is used to manage large files. To compound the problem, Puppet::FileBucketFile.verify_identical_file! also reads any backed-up copy of the file fully into memory. Thus, in the worst case, Puppet might read two full copies of a large file into memory at one time. This problem can result in Puppet daemon processes taking up large amounts of RAM. Even though this RAM may be reused by the Ruby interpreter, it does not seem to be released back to the OS. I propose that it would be better to operate on the files in question in small chunks, only reading a couple kilobytes into memory at once. This is a common practice. For instance, Puppet::Util::Checksums.checksum_file reads the file in small chunks for performing the MD5. -- You have received this notification because you have either subscribed to it, or are involved in it. To change your notification preferences, please click here: http://projects.puppetlabs.com/my/account -- You received this message because you are subscribed to the Google Groups "Puppet Bugs" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/puppet-bugs?hl=en.
