Jira (PDB-3058) Report not being stored in PuppetDB due to Unicode Error
Title: Message Title Russell Mull updated an issue PuppetDB / PDB-3058 Report not being stored in PuppetDB due to Unicode Error Change By: Russell Mull Story Points: 3 Add Comment This message was sent by Atlassian JIRA (v6.4.14#64029-sha1:ae256fe) -- You received this message because you are subscribed to the Google Groups "Puppet Bugs" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To post to this group, send email to puppet-bugs@googlegroups.com. Visit this group at https://groups.google.com/group/puppet-bugs. For more options, visit https://groups.google.com/d/optout.
Jira (PDB-3058) Report not being stored in PuppetDB due to Unicode Error
Title: Message Title Kenn Hussey updated an issue PuppetDB / PDB-3058 Report not being stored in PuppetDB due to Unicode Error Change By: Kenn Hussey Team: Systems Engineering Data Platform Add Comment This message was sent by Atlassian JIRA (v6.4.14#64029-sha1:ae256fe) -- You received this message because you are subscribed to the Google Groups "Puppet Bugs" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To post to this group, send email to puppet-bugs@googlegroups.com. Visit this group at https://groups.google.com/group/puppet-bugs. For more options, visit https://groups.google.com/d/optout.
Jira (PDB-3058) Report not being stored in PuppetDB due to Unicode Error
Title: Message Title Russell Mull updated an issue PuppetDB / PDB-3058 Report not being stored in PuppetDB due to Unicode Error Change By: Russell Mull Fix Version/s: PDB 5.1.0 Fix Version/s: PDB 4.4.2 Add Comment This message was sent by Atlassian JIRA (v6.4.14#64029-sha1:ae256fe) -- You received this message because you are subscribed to the Google Groups "Puppet Bugs" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To post to this group, send email to puppet-bugs@googlegroups.com. Visit this group at https://groups.google.com/group/puppet-bugs. For more options, visit https://groups.google.com/d/optout.
Jira (PDB-3058) Report not being stored in PuppetDB due to Unicode Error
Title: Message Title Russell Mull updated an issue PuppetDB / PDB-3058 Report not being stored in PuppetDB due to Unicode Error Change By: Russell Mull Sprint: FF 2016-10-19, FF 2016-11-02, PuppetDB 2017-04-19 , Data Platform 2017-09-06 Add Comment This message was sent by Atlassian JIRA (v6.4.14#64029-sha1:ae256fe) -- You received this message because you are subscribed to the Google Groups "Puppet Bugs" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To post to this group, send email to puppet-bugs@googlegroups.com. Visit this group at https://groups.google.com/group/puppet-bugs. For more options, visit https://groups.google.com/d/optout.
Jira (PDB-3058) Report not being stored in PuppetDB due to Unicode Error
Title: Message Title Russell Mull updated an issue PuppetDB / PDB-3058 Report not being stored in PuppetDB due to Unicode Error Change By: Russell Mull Story Points: 2 Add Comment This message was sent by Atlassian JIRA (v6.4.14#64029-sha1:ae256fe) -- You received this message because you are subscribed to the Google Groups "Puppet Bugs" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To post to this group, send email to puppet-bugs@googlegroups.com. Visit this group at https://groups.google.com/group/puppet-bugs. For more options, visit https://groups.google.com/d/optout.
Jira (PDB-3058) Report not being stored in PuppetDB due to Unicode Error
Title: Message Title Charlie Sharpsteen updated an issue PuppetDB / PDB-3058 Report not being stored in PuppetDB due to Unicode Error Change By: Charlie Sharpsteen Fix Version/s: PDB 4.4.1 Add Comment This message was sent by Atlassian JIRA (v6.4.14#64029-sha1:ae256fe) -- You received this message because you are subscribed to the Google Groups "Puppet Bugs" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To post to this group, send email to puppet-bugs@googlegroups.com. Visit this group at https://groups.google.com/group/puppet-bugs. For more options, visit https://groups.google.com/d/optout.
Jira (PDB-3058) Report not being stored in PuppetDB due to Unicode Error
Title: Message Title
Andreas Lambrecht commented on PDB-3058
Re: Report not being stored in PuppetDB due to Unicode Error
I've experience the same issue on PE2017.2.1 today, with a profile which includes the following:
registry_value { 'HKEY_LOCAL_MACHINE\Software\Microsoft\Windows\CurrentVersion\Policies\System\LegalNoticeText': type => string, data ="" 'This Legal Notice Text is Managed By Puppet', }
registry_value { KEY_LOCAL_MACHINE\Software\Microsoft\Windows\CurrentVersion\Policies\System\LegalNoticeCaption': type => string, data ="" 'This Legal Notice Caption is Managed By Puppet', }
According to "Fixed version" this should be fixed in PE2017.2.2+. However I have updated my PE2017.2.1 -> PE2017.2.3 and still have the same issue.
Add Comment
This message was sent by Atlassian JIRA (v6.4.14#64029-sha1:ae256fe)
--
You received this message because you are subscribed to the Google Groups "Puppet Bugs" group.
To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com.
To post to this group, send email to puppet-bugs@googlegroups.com.
Visit this group at https://groups.google.com/group/puppet-bugs.
For more options, visit https://groups.google.com/d/optout.
Jira (PDB-3058) Report not being stored in PuppetDB due to Unicode Error
Title: Message Title Russell Mull commented on PDB-3058 Re: Report not being stored in PuppetDB due to Unicode Error Isaac Eldridge the fixversion field is accurate Add Comment This message was sent by Atlassian JIRA (v6.4.14#64029-sha1:ae256fe) -- You received this message because you are subscribed to the Google Groups "Puppet Bugs" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To post to this group, send email to puppet-bugs@googlegroups.com. Visit this group at https://groups.google.com/group/puppet-bugs. For more options, visit https://groups.google.com/d/optout.
Jira (PDB-3058) Report not being stored in PuppetDB due to Unicode Error
Title: Message Title Isaac Eldridge updated an issue PuppetDB / PDB-3058 Report not being stored in PuppetDB due to Unicode Error Change By: Isaac Eldridge Labels: docs_reviewed maintenance Add Comment This message was sent by Atlassian JIRA (v6.4.14#64029-sha1:ae256fe) -- You received this message because you are subscribed to the Google Groups "Puppet Bugs" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To post to this group, send email to puppet-bugs@googlegroups.com. Visit this group at https://groups.google.com/group/puppet-bugs. For more options, visit https://groups.google.com/d/optout.
Jira (PDB-3058) Report not being stored in PuppetDB due to Unicode Error
Title: Message Title Isaac Eldridge commented on PDB-3058 Re: Report not being stored in PuppetDB due to Unicode Error Which PE versions will is this fixed in? 2016.4.4 and Glisan? Add Comment This message was sent by Atlassian JIRA (v6.4.14#64029-sha1:ae256fe) -- You received this message because you are subscribed to the Google Groups "Puppet Bugs" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To post to this group, send email to puppet-bugs@googlegroups.com. Visit this group at https://groups.google.com/group/puppet-bugs. For more options, visit https://groups.google.com/d/optout.
Jira (PDB-3058) Report not being stored in PuppetDB due to Unicode Error
Title: Message Title Russell Mull updated an issue PuppetDB / PDB-3058 Report not being stored in PuppetDB due to Unicode Error Change By: Russell Mull Release Notes Summary: PuppetDB now filters null bytes in Resource Events fields; this fixes a problem when dealing with certain entries in the Windows registry which contain the value '\0' by default. Add Comment This message was sent by Atlassian JIRA (v6.4.14#64029-sha1:ae256fe) -- You received this message because you are subscribed to the Google Groups "Puppet Bugs" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To post to this group, send email to puppet-bugs@googlegroups.com. Visit this group at https://groups.google.com/group/puppet-bugs. For more options, visit https://groups.google.com/d/optout.
Jira (PDB-3058) Report not being stored in PuppetDB due to Unicode Error
Title: Message Title Russell Mull updated an issue PuppetDB / PDB-3058 Report not being stored in PuppetDB due to Unicode Error Change By: Russell Mull Release Notes: Bug Fix Add Comment This message was sent by Atlassian JIRA (v6.4.14#64029-sha1:ae256fe) -- You received this message because you are subscribed to the Google Groups "Puppet Bugs" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To post to this group, send email to puppet-bugs@googlegroups.com. Visit this group at https://groups.google.com/group/puppet-bugs. For more options, visit https://groups.google.com/d/optout.
Jira (PDB-3058) Report not being stored in PuppetDB due to Unicode Error
Title: Message Title Kenn Hussey updated an issue PuppetDB / PDB-3058 Report not being stored in PuppetDB due to Unicode Error Change By: Kenn Hussey Team: Foundational Features Systems Engineering Add Comment This message was sent by Atlassian JIRA (v6.4.14#64029-sha1:ae256fe) -- You received this message because you are subscribed to the Google Groups "Puppet Bugs" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To post to this group, send email to puppet-bugs@googlegroups.com. Visit this group at https://groups.google.com/group/puppet-bugs. For more options, visit https://groups.google.com/d/optout.
Jira (PDB-3058) Report not being stored in PuppetDB due to Unicode Error
Title: Message Title Susan McNerney commented on PDB-3058 Re: Report not being stored in PuppetDB due to Unicode Error We've pushed 4.2.3.2 release out to 18th to give best chance of this releasing. Add Comment This message was sent by Atlassian JIRA (v6.4.14#64029-sha1:ae256fe) -- You received this message because you are subscribed to the Google Groups "Puppet Bugs" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To post to this group, send email to puppet-bugs@googlegroups.com. Visit this group at https://groups.google.com/group/puppet-bugs. For more options, visit https://groups.google.com/d/optout.
Jira (PDB-3058) Report not being stored in PuppetDB due to Unicode Error
Title: Message Title Susan McNerney updated an issue PuppetDB / PDB-3058 Report not being stored in PuppetDB due to Unicode Error Change By: Susan McNerney Priority: Normal Critical Add Comment This message was sent by Atlassian JIRA (v6.4.14#64029-sha1:ae256fe) -- You received this message because you are subscribed to the Google Groups "Puppet Bugs" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To post to this group, send email to puppet-bugs@googlegroups.com. Visit this group at https://groups.google.com/group/puppet-bugs. For more options, visit https://groups.google.com/d/optout.
Jira (PDB-3058) Report not being stored in PuppetDB due to Unicode Error
Title: Message Title Russell Mull updated an issue PuppetDB / PDB-3058 Report not being stored in PuppetDB due to Unicode Error Change By: Russell Mull Fix Version/s: PDB 4.3.0 Fix Version/s: PDB 4.4.1 Add Comment This message was sent by Atlassian JIRA (v6.4.14#64029-sha1:ae256fe) -- You received this message because you are subscribed to the Google Groups "Puppet Bugs" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To post to this group, send email to puppet-bugs@googlegroups.com. Visit this group at https://groups.google.com/group/puppet-bugs. For more options, visit https://groups.google.com/d/optout.
Jira (PDB-3058) Report not being stored in PuppetDB due to Unicode Error
Title: Message Title Russell Mull assigned an issue to Russell Mull PuppetDB / PDB-3058 Report not being stored in PuppetDB due to Unicode Error Change By: Russell Mull Assignee: Wyatt Alt Russell Mull Add Comment This message was sent by Atlassian JIRA (v6.4.14#64029-sha1:ae256fe) -- You received this message because you are subscribed to the Google Groups "Puppet Bugs" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To post to this group, send email to puppet-bugs@googlegroups.com. Visit this group at https://groups.google.com/group/puppet-bugs. For more options, visit https://groups.google.com/d/optout.
Jira (PDB-3058) Report not being stored in PuppetDB due to Unicode Error
Title: Message Title Russell Mull updated an issue PuppetDB / PDB-3058 Report not being stored in PuppetDB due to Unicode Error Change By: Russell Mull Sprint: FF 2016-10-19, FF 2016-11-02 , PuppetDB 2017-04-19 Add Comment This message was sent by Atlassian JIRA (v6.4.14#64029-sha1:ae256fe) -- You received this message because you are subscribed to the Google Groups "Puppet Bugs" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To post to this group, send email to puppet-bugs@googlegroups.com. Visit this group at https://groups.google.com/group/puppet-bugs. For more options, visit https://groups.google.com/d/optout.
Jira (PDB-3058) Report not being stored in PuppetDB due to Unicode Error
Title: Message Title Susan McNerney updated an issue PuppetDB / PDB-3058 Report not being stored in PuppetDB due to Unicode Error Change By: Susan McNerney Labels: davis-triage maintenance Add Comment This message was sent by Atlassian JIRA (v6.4.14#64029-sha1:ae256fe) -- You received this message because you are subscribed to the Google Groups "Puppet Bugs" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To post to this group, send email to puppet-bugs@googlegroups.com. Visit this group at https://groups.google.com/group/puppet-bugs. For more options, visit https://groups.google.com/d/optout.
Jira (PDB-3058) Report not being stored in PuppetDB due to Unicode Error
Title: Message Title Isaac Eldridge commented on PDB-3058 Re: Report not being stored in PuppetDB due to Unicode Error please use the docs tab to indicate if this needs a release note or not Add Comment This message was sent by Atlassian JIRA (v6.4.14#64029-sha1:ae256fe) -- You received this message because you are subscribed to the Google Groups "Puppet Bugs" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To post to this group, send email to puppet-bugs@googlegroups.com. Visit this group at https://groups.google.com/group/puppet-bugs. For more options, visit https://groups.google.com/d/optout.
Jira (PDB-3058) Report not being stored in PuppetDB due to Unicode Error
Title: Message Title Susan McNerney commented on PDB-3058 Re: Report not being stored in PuppetDB due to Unicode Error Approved for PDB 4.3.2.1 / PE 2016.4.4 by davis leads. Add Comment This message was sent by Atlassian JIRA (v6.4.14#64029-sha1:ae256fe) -- You received this message because you are subscribed to the Google Groups "Puppet Bugs" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To post to this group, send email to puppet-bugs@googlegroups.com. Visit this group at https://groups.google.com/group/puppet-bugs. For more options, visit https://groups.google.com/d/optout.
Jira (PDB-3058) Report not being stored in PuppetDB due to Unicode Error
Title: Message Title Susan McNerney updated an issue PuppetDB / PDB-3058 Report not being stored in PuppetDB due to Unicode Error Change By: Susan McNerney Fix Version/s: PDB 4.2.3.2 Add Comment This message was sent by Atlassian JIRA (v6.4.14#64029-sha1:ae256fe) -- You received this message because you are subscribed to the Google Groups "Puppet Bugs" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To post to this group, send email to puppet-bugs@googlegroups.com. Visit this group at https://groups.google.com/group/puppet-bugs. For more options, visit https://groups.google.com/d/optout.
Jira (PDB-3058) Report not being stored in PuppetDB due to Unicode Error
Title: Message Title Jacky Leung commented on PDB-3058 Re: Report not being stored in PuppetDB due to Unicode Error After i run this SQL it fixed my problems puppetdb-# SET puppetdb-# new_value = replace(new_value, '\u', ''), puppetdb-# old_value = replace(old_value, '\u', ''); UPDATE 652444 Add Comment This message was sent by Atlassian JIRA (v6.4.14#64029-sha1:ae256fe) -- You received this message because you are subscribed to the Google Groups "Puppet Bugs" group. To unsubscribe from this group and stop receiving emails from it, send an
Jira (PDB-3058) Report not being stored in PuppetDB due to Unicode Error
Title: Message Title Jacky Leung commented on PDB-3058 Re: Report not being stored in PuppetDB due to Unicode Error I found another area that caused the issue. The offending SQL is: SELECT reports.receive_time AS receive_time, reports.corrective_change AS corrective_change, reports.code_id AS code_id, reports.noop AS noop, reports.certname AS certname, reports.puppet_version AS puppet_version, encode(reports.hash::bytea, 'hex') AS hash, reports.cached_catalog_status AS cached_catalog_status, reports.transaction_uuid::text AS transaction_uuid, reports.configuration_version AS configuration_version,
Jira (PDB-3058) Report not being stored in PuppetDB due to Unicode Error
Title: Message Title Susan McNerney updated an issue PuppetDB / PDB-3058 Report not being stored in PuppetDB due to Unicode Error Change By: Susan McNerney Labels: davis-triage maintenance Add Comment This message was sent by Atlassian JIRA (v6.4.14#64029-sha1:ae256fe) -- You received this message because you are subscribed to the Google Groups "Puppet Bugs" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To post to this group, send email to puppet-bugs@googlegroups.com. Visit this group at https://groups.google.com/group/puppet-bugs. For more options, visit https://groups.google.com/d/optout.
Jira (PDB-3058) Report not being stored in PuppetDB due to Unicode Error
Title: Message Title
Charlie Sharpsteen commented on PDB-3058
Re: Report not being stored in PuppetDB due to Unicode Error
Also looks like this can happen when setting registry values on windows, as the default value for some is a null byte.
For example the legal notice that can be set on Windows:
# The default value on 2012R2 is a null byte so this resource will produce a failing report.
registry_value { 'HKLM\Software\Microsoft\Windows\CurrentVersion\Policies\System\legalnoticetext':
ensure => present,
data ="" "test string",
type => 'string',
}
# Using a unicode escape to re-set the default value will succeed, but also produces a failing report.
registry_value { 'HKLM\Software\Microsoft\Windows\CurrentVersion\Policies\System\legalnoticetext':
Jira (PDB-3058) Report not being stored in PuppetDB due to Unicode Error
Title: Message Title Jacky Leung commented on PDB-3058 Re: Report not being stored in PuppetDB due to Unicode Error Wyatt Alt hi thanks. i have no discarded catalog or report but i have lot for discarded facts. it seem i have some encoded data in ec2_user_data and the facter picked it up. is there any way to disable certain facters on the machine? as ec2_user_data only being used when server launch, it never need by puppet itself. i think this issue start happening to us may related to puppet4 deprecated stringify_facts. beside that facts i don't see any binary data anywhere and being discarded Add Comment This message was sent by Atlassian JIRA (v6.4.14#64029-sha1:ae256fe) -- You received this message because you are subscribed to the Google Groups "Puppet Bugs" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To post to this group, send email to puppet-bugs@googlegroups.com. Visit this group at https://groups.google.com/group/puppet-bugs. For more options, visit https://groups.google.com/d/optout.
Jira (PDB-3058) Report not being stored in PuppetDB due to Unicode Error
Title: Message Title Charlie Sharpsteen commented on PDB-3058 Re: Report not being stored in PuppetDB due to Unicode Error Turns out that the nulls are present in discarded commands, they just show up as the unicode escape sequence \u rather than literal null values. Add Comment This message was sent by Atlassian JIRA (v6.4.14#64029-sha1:ae256fe) -- You received this message because you are subscribed to the Google Groups "Puppet Bugs" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To post to this group, send email to puppet-bugs@googlegroups.com. Visit this group at https://groups.google.com/group/puppet-bugs. For more options, visit https://groups.google.com/d/optout.
Jira (PDB-3058) Report not being stored in PuppetDB due to Unicode Error
Title: Message Title Charlie Sharpsteen commented on PDB-3058 Re: Report not being stored in PuppetDB due to Unicode Error Started poking around in the discarded directory for another ticket where this came up. It appears that whatever process dumps out the discarded command is scrubbing null bytes, so we'll need to figure out another way of tracking these down. Add Comment This message was sent by Atlassian JIRA (v6.4.14#64029-sha1:ae256fe) -- You received this message because you are subscribed to the Google Groups "Puppet Bugs" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To post to this group, send email to puppet-bugs@googlegroups.com. Visit this group at https://groups.google.com/group/puppet-bugs. For more options, visit https://groups.google.com/d/optout.
Jira (PDB-3058) Report not being stored in PuppetDB due to Unicode Error
Title: Message Title Wyatt Alt commented on PDB-3058 Re: Report not being stored in PuppetDB due to Unicode Error Jacky Leung right – sorry if it wasn't clear but my implication there was that important things may be being lost. If I were you I would try and tie the null byte to a specific node and figure out why it's there/how to get rid of it. The usual cause is binary file content in a manifest. If you're getting log messages like the one in the ticket description (e.g "Retrying after attempt xxx") then you should be able to find the failing command in your dead letter office and inspect it for a null byte. The dead letter office should be in /var/lib/puppetdb/mq/discarded assuming you're on the open source defaults. Add Comment This message was sent by Atlassian JIRA (v6.4.14#64029-sha1:ae256fe) -- You received this message because you are subscribed to the Google Groups "Puppet Bugs" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To post to this group, send email to puppet-bugs@googlegroups.com. Visit this group at https://groups.google.com/group/puppet-bugs. For more options, visit https://groups.google.com/d/optout.
Jira (PDB-3058) Report not being stored in PuppetDB due to Unicode Error
Title: Message Title Jacky Leung commented on PDB-3058 Re: Report not being stored in PuppetDB due to Unicode Error hm... about the "nothing import is getting lost" the problem is that it also refuse record exported resource. which mean all code/module related to exported resources is not operational Add Comment This message was sent by Atlassian JIRA (v6.4.14#64029-sha1:ae256fe) -- You received this message because you are subscribed to the Google Groups "Puppet Bugs" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To post to this group, send email to puppet-bugs@googlegroups.com. Visit this group at https://groups.google.com/group/puppet-bugs. For more options, visit https://groups.google.com/d/optout.
Jira (PDB-3058) Report not being stored in PuppetDB due to Unicode Error
Title: Message Title Wyatt Alt commented on PDB-3058 Re: Report not being stored in PuppetDB due to Unicode Error Jacky Leung I saw the comment you left – no immediate timeline on the fix, but next release I'd assume. If you're getting the error in the ticket description now, does the PuppetDB log contain the name of the node with the null byte like it does in the description? If so, it shouldn't be too hard to track down the source of it (the report is cached on your master probably). Since the "fix" is just to replace those bytes with empty strings on ingestion (likely corrupting whatever binary data was there) it's probably worth understanding where it's coming from anyway so you know nothing important is getting lost. Add Comment This message was sent by Atlassian JIRA (v6.4.14#64029-sha1:ae256fe) -- You received this message because you are subscribed to the Google Groups "Puppet Bugs" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To post to this group, send email to puppet-bugs@googlegroups.com. Visit this group at https://groups.google.com/group/puppet-bugs. For more options, visit https://groups.google.com/d/optout.
Jira (PDB-3058) Report not being stored in PuppetDB due to Unicode Error
Title: Message Title Jacky Leung updated an issue PuppetDB / PDB-3058 Report not being stored in PuppetDB due to Unicode Error Change By: Jacky Leung Comment: i done another wipe on my puppetdb. it seem the issue caused if i run puppet3 agent with puppet4 master. the data can get insert. now i am having issue of puppet report can't store into puppetdb as the same as the ticket description.is there any ETA this fix can get release? Add Comment This message was sent by Atlassian JIRA (v6.4.14#64029-sha1:ae256fe) -- You received this message because you are subscribed to the Google Groups "Puppet Bugs" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To post to this group, send email to puppet-bugs@googlegroups.com. Visit this group at https://groups.google.com/group/puppet-bugs. For more options, visit https://groups.google.com/d/optout.
Jira (PDB-3058) Report not being stored in PuppetDB due to Unicode Error
Title: Message Title Jacky Leung commented on PDB-3058 Re: Report not being stored in PuppetDB due to Unicode Error i done another wipe on my puppetdb. it seem the issue caused if i run puppet3 agent with puppet4 master. the data can get insert. now i am having issue of puppet report can't store into puppetdb as the same as the ticket description. is there any ETA this fix can get release? Add Comment This message was sent by Atlassian JIRA (v6.4.14#64029-sha1:ae256fe) -- You received this message because you are subscribed to the Google Groups "Puppet Bugs" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To post to this group, send email to puppet-bugs@googlegroups.com. Visit this group at https://groups.google.com/group/puppet-bugs. For more options, visit https://groups.google.com/d/optout.
Jira (PDB-3058) Report not being stored in PuppetDB due to Unicode Error
Title: Message Title Jacky Leung commented on PDB-3058 Re: Report not being stored in PuppetDB due to Unicode Error no binary data from dump. there are "binary" but it is part of the text content of a file so i don't think that is related Add Comment This message was sent by Atlassian JIRA (v6.4.14#64029-sha1:ae256fe) -- You received this message because you are subscribed to the Google Groups "Puppet Bugs" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To post to this group, send email to puppet-bugs@googlegroups.com. Visit this group at https://groups.google.com/group/puppet-bugs. For more options, visit https://groups.google.com/d/optout.
Jira (PDB-3058) Report not being stored in PuppetDB due to Unicode Error
Title: Message Title Wyatt Alt commented on PDB-3058 Re: Report not being stored in PuppetDB due to Unicode Error Jacky Leung considering this a bit, the easiest way to find the null byte might be to create a text dump of the database using pg_dump, and then use grep on it. Add Comment This message was sent by Atlassian JIRA (v6.4.14#64029-sha1:ae256fe) -- You received this message because you are subscribed to the Google Groups "Puppet Bugs" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To post to this group, send email to puppet-bugs@googlegroups.com. Visit this group at https://groups.google.com/group/puppet-bugs. For more options, visit https://groups.google.com/d/optout.
Jira (PDB-3058) Report not being stored in PuppetDB due to Unicode Error
Title: Message Title Wyatt Alt commented on PDB-3058 Re: Report not being stored in PuppetDB due to Unicode Error Jacky Leung Okay, thanks for confirming that. I'll pick this up again tomorrow (I'm US pacific) and get back to you with some more targeted queries. Add Comment This message was sent by Atlassian JIRA (v6.4.14#64029-sha1:ae256fe) -- You received this message because you are subscribed to the Google Groups "Puppet Bugs" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To post to this group, send email to puppet-bugs@googlegroups.com. Visit this group at https://groups.google.com/group/puppet-bugs. For more options, visit https://groups.google.com/d/optout.
Jira (PDB-3058) Report not being stored in PuppetDB due to Unicode Error
Title: Message Title Jacky Leung commented on PDB-3058 Re: Report not being stored in PuppetDB due to Unicode Error the stored reports don't have the binary data Add Comment This message was sent by Atlassian JIRA (v6.4.14#64029-sha1:ae256fe) -- You received this message because you are subscribed to the Google Groups "Puppet Bugs" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To post to this group, send email to puppet-bugs@googlegroups.com. Visit this group at https://groups.google.com/group/puppet-bugs. For more options, visit https://groups.google.com/d/optout.
Jira (PDB-3058) Report not being stored in PuppetDB due to Unicode Error
Title: Message Title Jacky Leung commented on PDB-3058 Re: Report not being stored in PuppetDB due to Unicode Error hm.. they both return zero puppetdb=> select count(*) from reports where metrics_json is not null puppetdb-> ; count --- 0 (1 row) puppetdb=> select count(*) from reports where logs_json is not null; count --- 0
Jira (PDB-3058) Report not being stored in PuppetDB due to Unicode Error
Title: Message Title Wyatt Alt commented on PDB-3058 Re: Report not being stored in PuppetDB due to Unicode Error Jacky Leung I spent some time trying to reproduce your issue this morning but haven't been able to yet. Can you verify that the two queries here return counts of zero? psql -d puppetdb -c 'select count(*) from reports where metrics_json is not null' psql -d puppetdb -c 'select count(*) from reports where logs_json is not null' If either of those return nonzero counts, it means the database wasn't actually cleared on the move to 4.2.0. I think the PR linked in this ticket should fix the problem, though I can't say how quickly it'll be released. To get around this you'll need to figure out where the null byte is coming from. You might be able to enable the yaml store for reports (https://docs.puppet.com/puppet/latest/configuration.html#reports), and then once your nodes have checked in, grep through the directory for the word "binary". This will indicate where you've got binary data in the report. When I stick a null byte in one of my reports, I get data that looks like this: https://gist.github.com/wkalt/db5aa9b9d16a5ca4e9f3f873b8b4886d You may be able to grep for that base64 encoded string directly and get lucky, though I could imagine that depending on the representation. Once you find the null byte, it should be pretty easy to backtrack it to a manifest. Add Comment
Jira (PDB-3058) Report not being stored in PuppetDB due to Unicode Error
Title: Message Title Jacky Leung commented on PDB-3058 Re: Report not being stored in PuppetDB due to Unicode Error Wyatt Alt actually forget what i say in last comment. it is still happening in new flash data Add Comment This message was sent by Atlassian JIRA (v6.4.14#64029-sha1:ae256fe) -- You received this message because you are subscribed to the Google Groups "Puppet Bugs" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To post to this group, send email to puppet-bugs@googlegroups.com. Visit this group at https://groups.google.com/group/puppet-bugs. For more options, visit https://groups.google.com/d/optout.
Jira (PDB-3058) Report not being stored in PuppetDB due to Unicode Error
Title: Message Title Jacky Leung commented on PDB-3058 Re: Report not being stored in PuppetDB due to Unicode Error Wyatt Alt I am recently starting to migrate from puppet 3.8 to puppet 4.8, the code side seem to be compatible and puppetdb/postgresql both seem to be upgraded without errors. but it seem the existing data is causing problem. I am not sure will new data cause problem and i just truncated my whole puppetdb and start from scratch. so far after the truncate i don't seem to have any more errors, but this mean i also need to re-run puppet on all servers to regenerate all exported resources and all historical puppet reports. Add Comment This message was sent by Atlassian JIRA (v6.4.14#64029-sha1:ae256fe) -- You received this message because you are subscribed to the Google Groups "Puppet Bugs" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To post to this group, send email to puppet-bugs@googlegroups.com. Visit this group at https://groups.google.com/group/puppet-bugs. For more options, visit https://groups.google.com/d/optout.
Jira (PDB-3058) Report not being stored in PuppetDB due to Unicode Error
Title: Message Title Wyatt Alt commented on PDB-3058 Re: Report not being stored in PuppetDB due to Unicode Error Jacky Leung thanks for the note – that's really annoying. I can see how it might happen if the null byte were in an event, possibly in a message or resource type/title-- those are serialized to JSON on the way out of the database, but stored as text inside. I've been slowly picking away at this – planning to do some more work on it today. I think the solution is ultimately going to be to strip the null bytes out on the way in. I'm not sure there's a good way to fix cases where the data is already there short of manual updates to the database. If this is causing you problems I might be able to help with that. Add Comment This message was sent by Atlassian JIRA (v6.4.14#64029-sha1:ae256fe) -- You received this message because you are subscribed to the Google Groups "Puppet Bugs" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To post to this group, send email to puppet-bugs@googlegroups.com. Visit this group at https://groups.google.com/group/puppet-bugs. For more options, visit https://groups.google.com/d/optout.
Jira (PDB-3058) Report not being stored in PuppetDB due to Unicode Error
Title: Message Title Jacky Leung commented on PDB-3058 Re: Report not being stored in PuppetDB due to Unicode Error Puppetdb version is 4.2.0 Add Comment This message was sent by Atlassian JIRA (v6.4.14#64029-sha1:ae256fe) -- You received this message because you are subscribed to the Google Groups "Puppet Bugs" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To post to this group, send email to puppet-bugs@googlegroups.com. Visit this group at https://groups.google.com/group/puppet-bugs. For more options, visit https://groups.google.com/d/optout.
Jira (PDB-3058) Report not being stored in PuppetDB due to Unicode Error
Title: Message Title Jacky Leung commented on PDB-3058 Re: Report not being stored in PuppetDB due to Unicode Error I have a similar issue but it seem i get this when attempt to query it not storing it. so i think the data is already exist inside the database. puppetdb | 172.18.0.5 - - [18/Jan/2017:07:04:13 +] "GET /metrics/v1/mbeans/puppetlabs.puppetdb.population:name=num-nodes HTTP/1.1" 200 19 "https://puppetboard.fishpond.co.nz/" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:50.0) Gecko/20100101 Firefox/50.0" puppetexplorer| 172.18.0.1 - [18/Jan/2017:07:04:13 +] "GET /metrics/v1/mbeans/puppetlabs.puppetdb.population:name=num-nodes HTTP/1.0" 200 19 puppetdb | 172.18.0.5 - - [18/Jan/2017:07:04:13 +] "GET /metrics/v1/mbeans/puppetlabs.puppetdb.population:name=num-resources HTTP/1.1" 200 21 "https://puppetboard.fishpond.co.nz/" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:50.0) Gecko/20100101 Firefox/50.0" puppetexplorer| 172.18.0.1 - [18/Jan/2017:07:04:13 +] "GET /metrics/v1/mbeans/puppetlabs.puppetdb.population:name=num-resources HTTP/1.0" 200 21 puppetexplorer| 172.18.0.1 - [18/Jan/2017:07:04:13 +] "GET /metrics/v1/mbeans/puppetlabs.puppetdb.population:name=avg-resources-per-node HTTP/1.0" 200 33 puppetdb | 172.18.0.5 - - [18/Jan/2017:07:04:13 +] "GET /metrics/v1/mbeans/puppetlabs.puppetdb.population:name=avg-resources-per-node HTTP/1.1" 200 33 "https://puppetboard.fishpond.co.nz/" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:50.0) Gecko/20100101 Firefox/50.0" puppetexplorer| 172.18.0.1 - [18/Jan/2017:07:04:13 +] "GET
Jira (PDB-3058) Report not being stored in PuppetDB due to Unicode Error
Title: Message Title Wyatt Alt updated an issue PuppetDB / PDB-3058 Report not being stored in PuppetDB due to Unicode Error Change By: Wyatt Alt Sprint: FF 2016-10-19, FF 2016-11-02 , FF 2016-11-16 Add Comment This message was sent by Atlassian JIRA (v6.4.14#64029-sha1:ae256fe) -- You received this message because you are subscribed to the Google Groups "Puppet Bugs" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To post to this group, send email to puppet-bugs@googlegroups.com. Visit this group at https://groups.google.com/group/puppet-bugs. For more options, visit https://groups.google.com/d/optout.
Jira (PDB-3058) Report not being stored in PuppetDB due to Unicode Error
Title: Message Title Rob Browning updated an issue PuppetDB / PDB-3058 Report not being stored in PuppetDB due to Unicode Error Change By: Rob Browning Fix Version/s: PDB 4.3.0 Add Comment This message was sent by Atlassian JIRA (v6.4.14#64029-sha1:ae256fe) -- You received this message because you are subscribed to the Google Groups "Puppet Bugs" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To post to this group, send email to puppet-bugs@googlegroups.com. Visit this group at https://groups.google.com/group/puppet-bugs. For more options, visit https://groups.google.com/d/optout.
Jira (PDB-3058) Report not being stored in PuppetDB due to Unicode Error
Title: Message Title Susan McNerney updated an issue PuppetDB / PDB-3058 Report not being stored in PuppetDB due to Unicode Error Change By: Susan McNerney Sprint: FF 2016-10-19, FF 2016-11-02 , FF 2016-11-16 Add Comment This message was sent by Atlassian JIRA (v6.4.14#64029-sha1:ae256fe) -- You received this message because you are subscribed to the Google Groups "Puppet Bugs" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To post to this group, send email to puppet-bugs@googlegroups.com. Visit this group at https://groups.google.com/group/puppet-bugs. For more options, visit https://groups.google.com/d/optout.
Jira (PDB-3058) Report not being stored in PuppetDB due to Unicode Error
Title: Message Title Susan McNerney commented on PDB-3058 Re: Report not being stored in PuppetDB due to Unicode Error probably will not move in 11/15 sprint due to release obligations for PDB. Add Comment This message was sent by Atlassian JIRA (v6.4.14#64029-sha1:ae256fe) -- You received this message because you are subscribed to the Google Groups "Puppet Bugs" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To post to this group, send email to puppet-bugs@googlegroups.com. Visit this group at https://groups.google.com/group/puppet-bugs. For more options, visit https://groups.google.com/d/optout.
Jira (PDB-3058) Report not being stored in PuppetDB due to Unicode Error
Title: Message Title Wyatt Alt assigned an issue to Wyatt Alt PuppetDB / PDB-3058 Report not being stored in PuppetDB due to Unicode Error Change By: Wyatt Alt Assignee: Wyatt Alt Add Comment This message was sent by Atlassian JIRA (v6.4.14#64029-sha1:ae256fe) -- You received this message because you are subscribed to the Google Groups "Puppet Bugs" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To post to this group, send email to puppet-bugs@googlegroups.com. Visit this group at https://groups.google.com/group/puppet-bugs. For more options, visit https://groups.google.com/d/optout.
Jira (PDB-3058) Report not being stored in PuppetDB due to Unicode Error
Title: Message Title
Wyatt Alt commented on PDB-3058
Re: Report not being stored in PuppetDB due to Unicode Error
Nicolas Corrarello Russell Mull I had a minute to try this out. I'm unable to reproduce the issue with that file and the manifest
file {"/home/wyatt/Desktop/test.bin":
source => "file:///home/wyatt/Desktop/default(1).bgi",
ensure => file,
}
on linux running against postgres 9.5. We can revisit next week with Nico's environment to see if we can pull the problem message out of the DLO. Let me know if the manifest has an obvious issue.
Add Comment
This message was sent by Atlassian JIRA (v6.4.14#64029-sha1:ae256fe)
Jira (PDB-3058) Report not being stored in PuppetDB due to Unicode Error
Title: Message Title
Wyatt Alt commented on PDB-3058
Re: Report not being stored in PuppetDB due to Unicode Error
@reid yeah that's correct. It's fixed in one manifestation of the issue (insertion of a null byte into jsonb) but not another (insertion into text). Trivial to reproduce with
Facter.add("mynilfact") do
setcode do
{"foo" => "\u"}
end
end
I've got a fix in progress but since it's a maintenance issue progress has been a bit slow. If you're troubleshooting it with a customer the usual cause is binary file content FWIW.
Add Comment
This message was sent by Atlassian JIRA (v6.4.14#64029-sha1:ae256fe)
Jira (PDB-3058) Report not being stored in PuppetDB due to Unicode Error
Title: Message Title David Kramer updated an issue PuppetDB / PDB-3058 Report not being stored in PuppetDB due to Unicode Error Change By: David Kramer Sprint: FF 2016-10-19 , FF 2016-11-02 Add Comment This message was sent by Atlassian JIRA (v6.4.14#64029-sha1:ae256fe) -- You received this message because you are subscribed to the Google Groups "Puppet Bugs" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To post to this group, send email to puppet-bugs@googlegroups.com. Visit this group at https://groups.google.com/group/puppet-bugs. For more options, visit https://groups.google.com/d/optout.
Jira (PDB-3058) Report not being stored in PuppetDB due to Unicode Error
Title: Message Title Wyatt Alt updated an issue PuppetDB / PDB-3058 Report not being stored in PuppetDB due to Unicode Error Change By: Wyatt Alt Labels: maintenance Story Points: 2 Team: Foundational Features Security: Internal Sprint: Hopper FF 2016-10-19 Add Comment This message was sent by Atlassian JIRA (v6.4.14#64029-sha1:ae256fe) -- You received this message because you are subscribed to the Google Groups "Puppet Bugs" group. To unsubscribe from this group and stop receiving emails from it,
