Jira (PUP-10096) Parameters of resources are base64 encoded in puppetdb if created as raw byte strings by ruby functions

[Josh Cooper (JIRA)]

Title: Message Title

Josh Cooper commented on PUP-10096   Re: Parameters of resources are base64 encoded in puppetdb if created as raw byte strings by ruby functions The problem stems from ruby's Resolv.getname function which returns a ruby ASCII-8BIT string: irb(main):001:0> require 'resolv' => true irb(main):002:0> name = Resolv.getname("172.217.14.196") => "sea30s01-in-f4.1e100.net" irb(main):003:0> name.encoding => #<Encoding:ASCII-8BIT> Note it's the same as a ruby binary string: irb(main):008:0> Encoding::ASCII_8BIT == Encoding::BINARY => true Puppet preserves the binary-ness of the string throughout the catalog compilation process, and converts it to base64 when storing in puppetdb. IIRC postgres can't store null bytes specifically. Nick Lewis pointed out: Internally, programs that manipulate domain names should represent them as sequences of labels, where each label is a length octet followed by an octet string. By convention, domain names can be stored with arbitrary case, but domain name comparisons for all present domain functions are done in a case-insensitive manner, assuming an ASCII character set, and a high order zero bit. So it is reasonable for the ruby API to return a binary string. However, the function should coerce the string to UTF-8 prior to returning. This can be accomplished via: result = Resolv.new.getname(args[0]) result.force_encoding(Encoding::UTF_8) result.scrub! Ideally it shouldn't be so hard to debug this issue. We can't prevent or warn if any function returns binary data. For example, the binary_file is supposed to return a Binary data type. But perhaps we can leverage the function's return_type. If a function doesn't declare that it returns a Binary string, but returns a String with encoding Encoding::ASCII_8BIT then warn (maybe in strict mode only?), as it should be returning a Binary instead? That way it's clear the function needs to return a String with a non-binary encoding or a Binary value. Add Comment   This message was sent by Atlassian JIRA (v7.7.1#77002-sha1:e75ca93) -- You received this message because you are subscribed to the Google Groups "Puppet Bugs" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-bugs/JIRA.328499.1570559378000.135469.1570583040242%40Atlassian.JIRA.