Jira (PUP-7951) agent - server setup - file bucketing for binary files fails in case same file is bucketed for 2 different paths
Title: Message Title Standa eM commented on PUP-7951 Re: agent - server setup - file bucketing for binary files fails in case same file is bucketed for 2 different paths thank you Add Comment This message was sent by Atlassian JIRA (v6.4.14#64029-sha1:ae256fe) -- You received this message because you are subscribed to the Google Groups "Puppet Bugs" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To post to this group, send email to puppet-bugs@googlegroups.com. Visit this group at https://groups.google.com/group/puppet-bugs. For more options, visit https://groups.google.com/d/optout.
Jira (PUP-7951) agent - server setup - file bucketing for binary files fails in case same file is bucketed for 2 different paths
Title: Message Title Moses Mendoza commented on PUP-7951 Re: agent - server setup - file bucketing for binary files fails in case same file is bucketed for 2 different paths passed CI, here: https://jenkins-master-prod-1.delivery.puppetlabs.net/view/puppet-agent/view/1.10.x/view/Suite/job/platform_puppet-agent_intn-van-sys_suite-daily-puppet-1.10.x/ Add Comment This message was sent by Atlassian JIRA (v6.4.14#64029-sha1:ae256fe) -- You received this message because you are subscribed to the Google Groups "Puppet Bugs" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To post to this group, send email to puppet-bugs@googlegroups.com. Visit this group at https://groups.google.com/group/puppet-bugs. For more options, visit https://groups.google.com/d/optout.
Jira (PUP-7951) agent - server setup - file bucketing for binary files fails in case same file is bucketed for 2 different paths
Title: Message Title Moses Mendoza updated an issue Puppet / PUP-7951 agent - server setup - file bucketing for binary files fails in case same file is bucketed for 2 different paths Change By: Moses Mendoza Release Notes Summary: Prior to PUP-7951, when running the master via puppet server, backing up the same file content to filebucket more than once could result in a mistaken error warning that while the files had the same checksum value, they were different (ie a potential hash collision). PUP-7951 addresses this issue so that this is correctly treated as a duplicate file by filebucket. Add Comment This message was sent by Atlassian JIRA (v6.4.14#64029-sha1:ae256fe) -- You received this message because you are subscribed to the Google Groups "Puppet Bugs" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To post to this group, send email to puppet-bugs@googlegroups.com. Visit this group at https://groups.google.com/group/puppet-bugs. For more options, visit https://groups.google.com/d/optout.
Jira (PUP-7951) agent - server setup - file bucketing for binary files fails in case same file is bucketed for 2 different paths
Title: Message Title Moses Mendoza updated an issue Puppet / PUP-7951 agent - server setup - file bucketing for binary files fails in case same file is bucketed for 2 different paths Change By: Moses Mendoza Release Notes: Bug Fix Add Comment This message was sent by Atlassian JIRA (v6.4.14#64029-sha1:ae256fe) -- You received this message because you are subscribed to the Google Groups "Puppet Bugs" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To post to this group, send email to puppet-bugs@googlegroups.com. Visit this group at https://groups.google.com/group/puppet-bugs. For more options, visit https://groups.google.com/d/optout.
Jira (PUP-7951) agent - server setup - file bucketing for binary files fails in case same file is bucketed for 2 different paths
Title: Message Title Adrien Thebo assigned an issue to Unassigned Puppet / PUP-7951 agent - server setup - file bucketing for binary files fails in case same file is bucketed for 2 different paths Change By: Adrien Thebo Assignee: Adrien Thebo Add Comment This message was sent by Atlassian JIRA (v6.4.14#64029-sha1:ae256fe) -- You received this message because you are subscribed to the Google Groups "Puppet Bugs" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To post to this group, send email to puppet-bugs@googlegroups.com. Visit this group at https://groups.google.com/group/puppet-bugs. For more options, visit https://groups.google.com/d/optout.
Jira (PUP-7951) agent - server setup - file bucketing for binary files fails in case same file is bucketed for 2 different paths
Title: Message Title Eric Delaney updated an issue Puppet / PUP-7951 agent - server setup - file bucketing for binary files fails in case same file is bucketed for 2 different paths Change By: Eric Delaney QA Risk Assessment Reason: updated tests to cover QA Risk Assessment: Needs Assessment No Action QA Contact: Eric Delaney Add Comment This message was sent by Atlassian JIRA (v6.4.14#64029-sha1:ae256fe) -- You received this message because you are subscribed to the Google Groups "Puppet Bugs" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To post to this group, send email to puppet-bugs@googlegroups.com. Visit this group at https://groups.google.com/group/puppet-bugs. For more options, visit https://groups.google.com/d/optout.
Jira (PUP-7951) agent - server setup - file bucketing for binary files fails in case same file is bucketed for 2 different paths
Title: Message Title Moses Mendoza commented on PUP-7951 Re: agent - server setup - file bucketing for binary files fails in case same file is bucketed for 2 different paths PR raised at https://github.com/puppetlabs/puppet/pull/6261 Add Comment This message was sent by Atlassian JIRA (v6.4.14#64029-sha1:ae256fe) -- You received this message because you are subscribed to the Google Groups "Puppet Bugs" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To post to this group, send email to puppet-bugs@googlegroups.com. Visit this group at https://groups.google.com/group/puppet-bugs. For more options, visit https://groups.google.com/d/optout.
Jira (PUP-7951) agent - server setup - file bucketing for binary files fails in case same file is bucketed for 2 different paths
Title: Message Title Moses Mendoza assigned an issue to Adrien Thebo Puppet / PUP-7951 agent - server setup - file bucketing for binary files fails in case same file is bucketed for 2 different paths Change By: Moses Mendoza Assignee: Moses Mendoza Adrien Thebo Add Comment This message was sent by Atlassian JIRA (v6.4.14#64029-sha1:ae256fe) -- You received this message because you are subscribed to the Google Groups "Puppet Bugs" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To post to this group, send email to puppet-bugs@googlegroups.com. Visit this group at https://groups.google.com/group/puppet-bugs. For more options, visit https://groups.google.com/d/optout.
Jira (PUP-7951) agent - server setup - file bucketing for binary files fails in case same file is bucketed for 2 different paths
Title: Message Title Moses Mendoza assigned an issue to Moses Mendoza Puppet / PUP-7951 agent - server setup - file bucketing for binary files fails in case same file is bucketed for 2 different paths Change By: Moses Mendoza Assignee: Moses Mendoza Add Comment This message was sent by Atlassian JIRA (v6.4.14#64029-sha1:ae256fe) -- You received this message because you are subscribed to the Google Groups "Puppet Bugs" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To post to this group, send email to puppet-bugs@googlegroups.com. Visit this group at https://groups.google.com/group/puppet-bugs. For more options, visit https://groups.google.com/d/optout.
Jira (PUP-7951) agent - server setup - file bucketing for binary files fails in case same file is bucketed for 2 different paths
Title: Message Title Moses Mendoza updated an issue Puppet / PUP-7951 agent - server setup - file bucketing for binary files fails in case same file is bucketed for 2 different paths Change By: Moses Mendoza Sprint: Platform Core KANBAN Add Comment This message was sent by Atlassian JIRA (v6.4.14#64029-sha1:ae256fe) -- You received this message because you are subscribed to the Google Groups "Puppet Bugs" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To post to this group, send email to puppet-bugs@googlegroups.com. Visit this group at https://groups.google.com/group/puppet-bugs. For more options, visit https://groups.google.com/d/optout.
Jira (PUP-7951) agent - server setup - file bucketing for binary files fails in case same file is bucketed for 2 different paths
Title: Message Title Moses Mendoza updated an issue Puppet / PUP-7951 agent - server setup - file bucketing for binary files fails in case same file is bucketed for 2 different paths Change By: Moses Mendoza Priority: Normal Major Add Comment This message was sent by Atlassian JIRA (v6.4.14#64029-sha1:ae256fe) -- You received this message because you are subscribed to the Google Groups "Puppet Bugs" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To post to this group, send email to puppet-bugs@googlegroups.com. Visit this group at https://groups.google.com/group/puppet-bugs. For more options, visit https://groups.google.com/d/optout.
Jira (PUP-7951) agent - server setup - file bucketing for binary files fails in case same file is bucketed for 2 different paths
Title: Message Title Moses Mendoza updated an issue Puppet / PUP-7951 agent - server setup - file bucketing for binary files fails in case same file is bucketed for 2 different paths Change By: Moses Mendoza Fix Version/s: PUP 5.3.1 Add Comment This message was sent by Atlassian JIRA (v6.4.14#64029-sha1:ae256fe) -- You received this message because you are subscribed to the Google Groups "Puppet Bugs" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To post to this group, send email to puppet-bugs@googlegroups.com. Visit this group at https://groups.google.com/group/puppet-bugs. For more options, visit https://groups.google.com/d/optout.
Jira (PUP-7951) agent - server setup - file bucketing for binary files fails in case same file is bucketed for 2 different paths
Title: Message Title Moses Mendoza updated an issue Puppet / PUP-7951 agent - server setup - file bucketing for binary files fails in case same file is bucketed for 2 different paths Change By: Moses Mendoza Fix Version/s: PUP 4.10.9 Add Comment This message was sent by Atlassian JIRA (v6.4.14#64029-sha1:ae256fe) -- You received this message because you are subscribed to the Google Groups "Puppet Bugs" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To post to this group, send email to puppet-bugs@googlegroups.com. Visit this group at https://groups.google.com/group/puppet-bugs. For more options, visit https://groups.google.com/d/optout.
Jira (PUP-7951) agent - server setup - file bucketing for binary files fails in case same file is bucketed for 2 different paths
Title: Message Title Moses Mendoza commented on PUP-7951 Re: agent - server setup - file bucketing for binary files fails in case same file is bucketed for 2 different paths I've reproduced this issue given the original scenario. I can confirm it appears that rather than being in binary, the content of the bucket_file, a Puppet::FileBucket::File instance, is encoded in UTF-8: code [21] pry(#)> bucket_file.to_binary.encoding bucket_file.to_binary.encoding => # code Since this is an instance over the wire, we're dealing with a StringContents implementation, and #to_binary just returns the content of the object. Add Comment This message was sent by Atlassian JIRA (v6.4.14#64029-sha1:ae256fe) -- You received this message because you are subscribed to the Google Groups "Puppet Bugs" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To post to this group, send email to puppet-bugs@googlegroups.com. Visit this group at https://groups.google.com/group/puppet-bugs. For more options, visit https://groups.google.com/d/optout.
Jira (PUP-7951) agent - server setup - file bucketing for binary files fails in case same file is bucketed for 2 different paths
Title: Message Title Moses Mendoza updated an issue Puppet / PUP-7951 agent - server setup - file bucketing for binary files fails in case same file is bucketed for 2 different paths Change By: Moses Mendoza Story Points: 1 Add Comment This message was sent by Atlassian JIRA (v6.4.14#64029-sha1:ae256fe) -- You received this message because you are subscribed to the Google Groups "Puppet Bugs" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To post to this group, send email to puppet-bugs@googlegroups.com. Visit this group at https://groups.google.com/group/puppet-bugs. For more options, visit https://groups.google.com/d/optout.
Jira (PUP-7951) agent - server setup - file bucketing for binary files fails in case same file is bucketed for 2 different paths
Title: Message Title Moses Mendoza commented on PUP-7951 Re: agent - server setup - file bucketing for binary files fails in case same file is bucketed for 2 different paths Thanks for filing Standa eM. In Agent => Server filebucket communication the data is supposed to be sent over the network as binary with content type application/octet-stream. If it's being transcoded as UTF-8 somewhere that's a problem. Add Comment This message was sent by Atlassian JIRA (v6.4.14#64029-sha1:ae256fe) -- You received this message because you are subscribed to the Google Groups "Puppet Bugs" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To post to this group, send email to puppet-bugs@googlegroups.com. Visit this group at https://groups.google.com/group/puppet-bugs. For more options, visit https://groups.google.com/d/optout.
Jira (PUP-7951) agent - server setup - file bucketing for binary files fails in case same file is bucketed for 2 different paths
Title: Message Title
Eric Delaney updated an issue
Puppet / PUP-7951
agent - server setup - file bucketing for binary files fails in case same file is bucketed for 2 different paths
Change By:
Eric Delaney
Scenario: Puppet Agent is bucketing file to PuppetserverFollowing scenario (but in agent-server) configuration should show the error: {noformat} puppet apply -e 'file { "/tmp/d1/file1": source => "/usr/bin/at" }'puppet apply -e 'file { "/tmp/d1/file1": source => "/usr/bin/bash" }'# file1 already exists, so it is bucketed before rewritten by contents of 'bash'puppet apply -e 'file { "/tmp/d2/file2": source => "/usr/bin/at" }'puppet apply -e 'file { "/tmp/d2/file2": source => "/usr/bin/bash" }'# file2 already exists, so it is bucketed before rewritten by contents of 'bash'. It is bucketed to same directory on server because md5 hash of file2 is same as md5 hash of bucketed file1. And bucketing fails. {noformat} In theory this should work. And for local bucketing it works. There are 2 artefacts on server for each bucketed file - contents and paths. Paths contains all paths where contents was stored on origin system.When file is bucketed there are multiple checks - size, md5 hash and file to be bucketed is byte-to-byte compared with already bucketed file.In case that given file is bucketed (file with md5sum hash exists on server) and path on origin system is different to path stored in 'paths' than there are 2 checks - size of backeted bucketed file and chacksum checksum . Size is compared between data of file received over network from agent and existing 'contents' on server.The problem is, that this comparison usually fails because encoding of data received over network are 'UTF-8' encoded and contents is "like" plain bytes. So size doesn't match and it is considered error and processing of file resource fails.For puppet-agent-1.10.4-1.el7.x86_64 rpm:Comparison is in: method Puppet::FileBucketFile::File.verify_identical_file()bucket_file is: Puppet::FileBucket::File::StringContentsSolution should be to ensure size of data received over network to be converted to something like "ASCII-8BIT" encoding even for 'size' operation.
Add Comment
Jira (PUP-7951) agent - server setup - file bucketing for binary files fails in case same file is bucketed for 2 different paths
Title: Message Title
Adrien Thebo updated an issue
Puppet / PUP-7951
agent - server setup - file bucketing for binary files fails in case same file is bucketed for 2 different paths
Change By:
Adrien Thebo
Scenario: Puppet Agent is bucketing file to PuppetserverFollowing scenario (but in agent-server) configuration should show the error: {noformat} puppet apply -e 'file { "/tmp/d1/file1": source => "/usr/bin/at" }'puppet apply -e 'file { "/tmp/d1/file1": source => "/usr/bin/bash" }'# file1 already exists, so it is bucketed before rewritten by contents of 'bash'puppet apply -e 'file { "/tmp/d2/file2": source => "/usr/bin/at" }'puppet apply -e 'file { "/tmp/d2/file2": source => "/usr/bin/bash" }'# file2 already exists, so it is bucketed before rewritten by contents of 'bash'. It is bucketed to same directory on server because md5 hash of file2 is same as md5 hash of bucketed file1. And bucketing fails. {noformat} In theory this should work. And for local bucketing it works. There are 2 artefacts on server for each bucketed file - contents and paths. Paths contains all paths where contents was stored on origin system.When file is bucketed there are multiple checks - size, md5 hash and file to be bucketed is byte-to-byte compared with already bucketed file.In case that given file is bucketed (file with md5sum hash exists on server) and path on origin system is different to path stored in 'paths' than there are 2 checks - size of backeted file and chacksum. Size is compared between data of file received over network from agent and existing 'contents' on server.The problem is, that this comparison usually fails because encoding of data received over network are 'UTF-8' encoded and contents is "like" plain bytes. So size doesn't match and it is considered error and processing of file resource fails.For puppet-agent-1.10.4-1.el7.x86_64 rpm:Comparison is in: method Puppet::FileBucketFile::File.verify_identical_file()bucket_file is: Puppet::FileBucket::File::StringContentsSolution should be to ensure size of data received over network to be converted to something like "ASCII-8BIT" encoding even for 'size' operation.
Add Comment
Jira (PUP-7951) agent - server setup - file bucketing for binary files fails in case same file is bucketed for 2 different paths
Title: Message Title Eric Delaney updated an issue Puppet / PUP-7951 agent - server setup - file bucketing for binary files fails in case same file is bucketed for 2 different paths Change By: Eric Delaney Team: Platform Core Add Comment This message was sent by Atlassian JIRA (v6.4.14#64029-sha1:ae256fe) -- You received this message because you are subscribed to the Google Groups "Puppet Bugs" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To post to this group, send email to puppet-bugs@googlegroups.com. Visit this group at https://groups.google.com/group/puppet-bugs. For more options, visit https://groups.google.com/d/optout.
Jira (PUP-7951) agent - server setup - file bucketing for binary files fails in case same file is bucketed for 2 different paths
Title: Message Title
Eric Delaney updated an issue
Puppet / PUP-7951
agent - server setup - file bucketing for binary files fails in case same file is bucketed for 2 different paths
Change By:
Eric Delaney
Scenario: Puppet Agent is bucketing file to PuppetserverFollowing scenario (but in agent-server) configuration should show the error:puppet apply -e 'file { "/tmp/d1/file1": source => "/usr/bin/at" }'puppet apply -e 'file { "/tmp/d1/file1": source => "/usr/bin/bash" }'# file1 already exists, so it is bucketed before rewritten by contents of 'bash'puppet apply -e 'file { "/tmp/d2/file2": source => "/usr/bin/at" }'puppet apply -e 'file { "/tmp/d2/file2": source => "/usr/bin/bash" }'# file2 already exists, so it is bucketed before rewritten by contents of 'bash'. It is bucketed to same directory on server because md5 hash of file2 is same as md5 hash of bucketed file1. And bucketing fails.In theory this should work. And for local bucketing it works. There are 2 artefacts on server for each bucketed file - contents and paths. Paths contains all paths where contents was stored on origin system.When file is bucketed there are multiple checks - size, md5 hash and file to be backeted bucketed is byte-to-byte compared with already bucketed file.In case that given file is bucketed (file with md5sum hash exists on server) and path on origin system is different to path stored in 'paths' than there are 2 checks - size of backeted file and chacksum. Size is compared between data of file received over network from agent and existing 'contents' on server.The problem is, that this comparison usually fails because encoding of data received over network are 'UTF-8' encoded and contents is "like" plain bytes. So size doesn't match and it is considered error and processing of file resource fails.For puppet-agent-1.10.4-1.el7.x86_64 rpm:Comparison is in: method Puppet::FileBucketFile::File.verify_identical_file()bucket_file is: Puppet::FileBucket::File::StringContentsSolution should be to ensure size of data received over network to be converted to something like "ASCII-8BIT" encoding even for 'size' operation.
Add Comment
Jira (PUP-7951) agent - server setup - file bucketing for binary files fails in case same file is bucketed for 2 different paths
Title: Message Title Eric Delaney updated an issue Puppet / PUP-7951 agent - server setup - file bucketing for binary files fails in case same file is bucketed for 2 different paths Change By: Eric Delaney Summary: agent - server setup - file backeting bucketing for binary files fails in case same file is bucketed for 2 different paths Add Comment This message was sent by Atlassian JIRA (v6.4.14#64029-sha1:ae256fe) -- You received this message because you are subscribed to the Google Groups "Puppet Bugs" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To post to this group, send email to puppet-bugs@googlegroups.com. Visit this group at https://groups.google.com/group/puppet-bugs. For more options, visit https://groups.google.com/d/optout.
