Jira (PUP-1208) md5lite, mtime not honoured for file type/provider

[Charlie Sharpsteen (JIRA)]

Title: Message Title

Charlie Sharpsteen commented on an issue   Re: md5lite, mtime not honoured for file type/provider I did some more poking last evening and this morning, here are the results: Ken's patch must be expanded to the find method in order for the server to honor the checksum_type parameter in file_metadata requests: diff --git a/lib/puppet/indirector/file_server.rb b/lib/puppet/indirector/file_server.rb index 389951d..c152892 100644 --- a/lib/puppet/indirector/file_server.rb +++ b/lib/puppet/indirector/file_server.rb @@ -29,6 +29,7 @@ class Puppet::Indirector::FileServer < Puppet::Indirector::Terminus return nil unless path = mount.find(relative_path, request) result = model.new(path) result.links = request.options[:links] if request.options[:links] + result.checksum_type = request.options[:checksum_type] if request.options[:checksum_type] result.collect(request.options[:source_permissions]) result end @@ -51,6 +52,7 @@ class Puppet::Indirector::FileServer < Puppet::Indirector::Terminus Puppet::FileServing::Fileset.merge(*filesets).collect do |file, base_path| inst = model.new(base_path, :relative_path => file) inst.links = request.options[:links] if request.options[:links] + inst.checksum_type = request.options[:checksum_type] if request.options[:checksum_type] inst.collect inst end Additionally, the File type must be modified so that the agent actually includes the checksum type in metadata requests: --- a/lib/puppet/type/file/source.rb +++ b/lib/puppet/type/file/source.rb @@ -171,6 +171,7 @@ module Puppet options = { :environment => resource.catalog.environment, :links => resource[:links], + :checksum_type => resource[:checksum], :source_permissions => resource[:source_permissions] } This appears to be enough to get the agent and master talking about file metadata using a checksum other than MD5. However, since these code paths have not been exercised in a while, and new bugs are being exposed. The issues which I ran into yesterday has to do with checksums used when Puppet updates the contents of a file. Puppet updates files by writing chunks to a temporary file wile performing a "stream" checksum. After all content has been flushed, a "file" checksum is executed on the temp file before it is moved into position. Unfortunately for mtime, the "stream" checksum returns a dummy value. The md5lite and sha256lite checksums also perform a full hash of the stream instead of just the first chunk of bytes and are thus equivalent to a full MD5 or SHA256 hash. This behavior results in a checksum mismatch which fails write operations for files using one of these lightweight checksums. So, fixing this will take more than one or three lines of changes and probably a few days of work. Eric Sorenson, Andy Parker, Kylo Ginsberg: It looks like there is more work here than there is time left before 3.7.3 ships, should we move this to 4.0? Add Comment   Puppet / PUP-1208 md5lite, mtime not honoured for file type/provider It seems I can't get the puppetmaster to honour the checksum => mtime setting, or the md5lite setting. So the following example has no performance improvement over just using md5: {code} file { "/testtransfer":   ensure => directory,   recurse => remote,   purge => true,   checksum => mtime,   source => "puppet:///modules/${module_name}/bigfilesh... This message was sent by Atlassian JIRA (v6.1.4#6159-sha1:44eaede) -- You received this message because you are subscribed to the Google Groups "Puppet Bugs" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To post to this group, send email to puppet-bugs@googlegroups.com. Visit this group at http://groups.google.com/group/puppet-bugs. For more options, visit https://groups.google.com/d/optout.