Jira (PUP-3551) Sanitize non-UTF-8 data in run reports

[Josh Cooper (JIRA)]

Title: Message Title

Josh Cooper commented on PUP-3551   Re: Sanitize non-UTF-8 data in run reports Now that we're sanitizing log entries, I agree that it's unlikely for binary data to get into a report. One remaining issue is whether the previous_value or desired_value event fields can contain binary data? In the case of the file type, it always emits md5 checksums for previous/desired, so that isn't an issue. The only other place I think of is if a custom type/provider happens to manage a property whose value is binary data, such as a kerberos keytab file or java keystore. Using the host resource's comment property, I was able to demonstrate the problem, though I had to delete the property's validate method: Error: Could not convert change 'comment' to string: invalid byte sequence in UTF-8 /Users/josh/work/puppet/lib/puppet/pops/types/string_converter.rb:842:in `each_codepoint' /Users/josh/work/puppet/lib/puppet/pops/types/string_converter.rb:842:in `puppet_quote' /Users/josh/work/puppet/lib/puppet/pops/types/string_converter.rb:802:in `string_PStringType' /Users/josh/work/puppet/lib/puppet/pops/visitor.rb:110:in `visit_this_3' /Users/josh/work/puppet/lib/puppet/pops/types/string_converter.rb:516:in `_convert' /Users/josh/work/puppet/lib/puppet/pops/types/string_converter.rb:503:in `convert' /Users/josh/work/puppet/lib/puppet/pops/types/string_converter.rb:242:in `convert' /Users/josh/work/puppet/lib/puppet/parameter.rb:553:in `format_value_for_display' /Users/josh/work/puppet/lib/puppet/property.rb:555:in `should_to_s' /Users/josh/work/puppet/lib/puppet/property.rb:209:in `change_to_s' /Users/josh/work/puppet/lib/puppet/transaction/resource_harness.rb:242:in `sync'   ....   Error: /Stage[main]/Main/Host[foo.example.com]: Could not evaluate: Puppet::Util::Log requires a message /Users/josh/work/puppet/lib/puppet/util/log.rb:355:in `message=' /Users/josh/work/puppet/lib/puppet/util/log.rb:289:in `initialize' /Users/josh/work/puppet/lib/puppet/util/log.rb:87:in `new' /Users/josh/work/puppet/lib/puppet/util/log.rb:87:in `create' /Users/josh/work/puppet/lib/puppet/util/logging.rb:12:in `send_log' /Users/josh/work/puppet/lib/puppet/transaction/event.rb:96:in `send_log' /Users/josh/work/puppet/lib/puppet/transaction/resource_harness.rb:163:in `ensure in sync_if_needed' /Users/josh/work/puppet/lib/puppet/transaction/resource_harness.rb:164:in `sync_if_needed'   ...   Error: Could not send report: Could not render to json: source sequence is illegal/malformed utf-8 /Users/josh/work/puppet/lib/puppet/network/format_support.rb:100:in `to_json' /Users/josh/work/puppet/lib/puppet/network/format_support.rb:100:in `to_json' /Users/josh/work/puppet/lib/puppet/network/format.rb:65:in `render' /Users/josh/work/puppet/lib/puppet/network/format_support.rb:106:in `render' /Users/josh/work/puppet/lib/puppet/indirector/rest.rb:238:in `block in save' /Users/josh/work/puppet/lib/puppet/indirector/rest.rb:256:in `block in do_request' /Users/josh/work/puppet/lib/puppet/indirector/request.rb:220:in `do_request' /Users/josh/work/puppet/lib/puppet/indirector/rest.rb:256:in `do_request' /Users/josh/work/puppet/lib/puppet/indirector/rest.rb:237:in `save' /Users/josh/work/puppet/lib/puppet/indirector/indirection.rb:286:in `save' /Users/josh/work/puppet/lib/puppet/configurer.rb:392:in `send_report' /Users/josh/work/puppet/lib/puppet/configurer.rb:355:in `run_internal' Interestingly it caused 3 different failures. One failure when trying to generate the change_to_s message, e.g. "comment changed from ...", another failure because the "Puppet::Util::Log requires a message" (which there's another ticket floating around about that), and finally it failed serializing the report to JSON. So I think the question is how likely it is for a type/provider to have a binary property? Even if that did happen, I think many other parts of puppet would break if given arbitrary binary data like that. For example, when trying to emit a "changed from" message to the console, when trying to apply a regexp to the value for validation. Since I was able to reproduce the problem (in a hacky way), I think it would be good to investigate more, but I don't feel like this is necessary for 5.0, so feel free to bump to a later 5.y release. Add Comment   This message was sent by Atlassian JIRA (v6.4.14#64029-sha1:ae256fe) -- You received this message because you are subscribed to the Google Groups "Puppet Bugs" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To post to this group, send email to puppet-bugs@googlegroups.com. Visit this group at https://groups.google.com/group/puppet-bugs. For more options, visit https://groups.google.com/d/optout.