Jira (FACT-1902) Confirm Facter 4 validates that external/custom/executable facts output proper UTF-8

[Josh Cooper (Jira)]

Title: Message Title

Josh Cooper updated an issue   Facter / FACT-1902 Confirm Facter 4 validates that external/custom/executable facts output proper UTF-8 Change By: Josh Cooper Modern versions of Puppet require that the data they serialize to JSON is proper UTF-8. Since facter collects data from different external sources, it's possible for facter data to be incorrectly encoded. Examples include: * Unicode code points are encoded as String contains a valid UTF-16LE byte sequence, but the string's "encoding" method returns UTF-8 (Windows Registry) * String contains binary data, but "encoding" returns UTF-8 (EC2 userdata) * String contains the start of a valid multibyte UTF-8 sequnce, e.g.   hen facts have an incorrect encoding ( either "\xc3\x28") * String contains embedded nulls. Strictly speaking the encoding "\u0000" code point is mislabeled/doesn't match the  match the underlying valid and is encoded as a single null byte sequence or the byte sequence , this currently does not raise an error until but it is serialized, at which point it is far too late, surprising and can't be stored in Postgres. * String was generated by a child process based on the error message active code page (Windows CP1252), but the output is not helpful. interpreted as UTF-8 Instead, Facter itself 's normalization should encode the ensure: * All fact data as contains valid UTF-8 data * If the string data is not valid , replacing the invalid byte sequences sequence should be replaced with the unicode replacement character . And issue a warning , so that it is valid * Same for embedded null values * A warning should be generated specifying the fact key or value for debugging. with invalid data Add Comment   This message was sent by Atlassian Jira (v8.20.11#820011-sha1:0629dd8) -- You received this message because you are subscribed to the Google Groups "Puppet Bugs" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-bugs/JIRA.290197.1546471789000.8166.1682440560043%40Atlassian.JIRA.