Jira (PDB-3824) Duplicate unordered structured facts violates unique constraint
Title: Message Title Owen Rodabaugh updated an issue PuppetDB / PDB-3824 Duplicate unordered structured facts violates unique constraint Change By: Owen Rodabaugh CS Priority: Needs Priority Normal CS Impact: At it's worst this could be a silent data corruption. For example if That said this seems like a user had a set structured very rare case, having two facts that were impacted by this with different names but didn't realize it they may not be running code against all the nodes they should, or might hit nodes that should no longer get it because of the same fact_values. This will only happen in situations which questionable custom facts not updating . CS Severity: 4 3 - Major Serious CS Business Value: 4 3 - $ CS Frequency: 2 1 - 5 1 - 25 5 % of Customers Add Comment This message was sent by Atlassian JIRA (v7.5.1#75006-sha1:7df2574)
Jira (PDB-3824) Duplicate unordered structured facts violates unique constraint
Title: Message Title Owen Rodabaugh updated an issue PuppetDB / PDB-3824 Duplicate unordered structured facts violates unique constraint Change By: Owen Rodabaugh CS Priority: Major Needs Priority Add Comment This message was sent by Atlassian JIRA (v7.0.2#70111-sha1:88534db) -- You received this message because you are subscribed to the Google Groups "Puppet Bugs" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To post to this group, send email to puppet-bugs@googlegroups.com. Visit this group at https://groups.google.com/group/puppet-bugs. For more options, visit https://groups.google.com/d/optout.
Jira (PDB-3824) Duplicate unordered structured facts violates unique constraint
Title: Message Title Jarret Lavallee updated an issue PuppetDB / PDB-3824 Duplicate unordered structured facts violates unique constraint Change By: Jarret Lavallee Environment: Puppet 5.3.3 with PuppetDB 5.1.3 and duplicate Duplicate structured facts in the same agent replace_facts payload with a different order. Add Comment This message was sent by Atlassian JIRA (v7.0.2#70111-sha1:88534db) -- You received this message because you are subscribed to the Google Groups "Puppet Bugs" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To post to this group, send email to puppet-bugs@googlegroups.com. Visit this group at https://groups.google.com/group/puppet-bugs. For more options, visit https://groups.google.com/d/optout.
Jira (PDB-3824) Duplicate unordered structured facts violates unique constraint
Title: Message Title Jarret Lavallee updated an issue PuppetDB / PDB-3824 Duplicate unordered structured facts violates unique constraint Change By: Jarret Lavallee Affects Version/s: PDB 4.2.3.7 Add Comment This message was sent by Atlassian JIRA (v7.0.2#70111-sha1:88534db) -- You received this message because you are subscribed to the Google Groups "Puppet Bugs" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To post to this group, send email to puppet-bugs@googlegroups.com. Visit this group at https://groups.google.com/group/puppet-bugs. For more options, visit https://groups.google.com/d/optout.
Jira (PDB-3824) Duplicate unordered structured facts violates unique constraint
Title: Message Title
Jarret Lavallee updated an issue
PuppetDB / PDB-3824
Duplicate unordered structured facts violates unique constraint
Change By:
Jarret Lavallee
Duplicate custom structured facts that in the same payload which are not ordered cause replace_facts to fail with the following error message. {code}2018-01-30 16:10:11,844 ERROR [p.p.command] [10,612] [replace facts] Retrying after attempt 0 for testing.example.com, due to: org.postgresql.util.PSQLException: ERROR: duplicate key value violates unique constraint "fact_values_value_hash_key" Detail: Key (value_hash)=(\x88680eb39ee03249046efa1c51598d865afc7519) already exists.org.postgresql.util.PSQLException: ERROR: duplicate key value violates unique constraint "fact_values_value_hash_key" Detail: Key (value_hash)=(\x88680eb39ee03249046efa1c51598d865afc7519) already exists.{code}The hash is not currently in the database. {code}pe-puppetdb=# select * from fact_values where value_hash = '\x88680eb39ee03249046efa1c51598d865afc7519'; id | value_hash | value_type_id | value_integer | value_float | value_string | value_boolean | value++---+---+-+--+---+---(0 rows){code}This occurs when there are two structured facts in the payload with the same contents, but not in the same order. Below is a basic facts payload to reproduce the issue. {code:json}{ "certname": "testing.example.com", "values": {"a_structured_fact": { "element2": {"somevalue": "2" }, "element1": {"somevalue": "1" }},"another_structured_fact": { "element1": {"somevalue": "1" }, "element2": {"somevalue": "2" }} }, "environment": "production", "producer_timestamp": "2018-01-29T15:34:59.792-08:00", "producer": "somemaster.example.com"}{code} To simulate this, one can run the following command after populating payload.json with the content above. {code}curl -X POST -H 'Content-Type:application/json' -H 'Accept:application/json' -d @payload.json "http://localhost:8080/pdb/cmd/v1?command=replace_facts=5=testing.example.com"{code} If I put both facts in the same order, we do not encounter the error. It sounds like the fix is to sort the structured facts when checking if they already exist. The end result of this issue is that we get continual retries and discards. The facts are never updated in PuppetDB for these nodes, unless the value_hash already exists in the database. _Workaround_Sort the hash prior to setting the fact. Hash[myhash.sort] worked in testing.
Add Comment
Jira (PDB-3824) Duplicate unordered structured facts violates unique constraint
Title: Message Title
Jarret Lavallee updated an issue
PuppetDB / PDB-3824
Duplicate unordered structured facts violates unique constraint
Change By:
Jarret Lavallee
Duplicate custom structured facts that are not ordered cause replace_facts to fail with the following error message. {code}2018-01-30 16:10:11,844 ERROR [p.p.command] [10,612] [replace facts] Retrying after attempt 0 for testing.example.com, due to: org.postgresql.util.PSQLException: ERROR: duplicate key value violates unique constraint "fact_values_value_hash_key" Detail: Key (value_hash)=(\x88680eb39ee03249046efa1c51598d865afc7519) already exists.org.postgresql.util.PSQLException: ERROR: duplicate key value violates unique constraint "fact_values_value_hash_key" Detail: Key (value_hash)=(\x88680eb39ee03249046efa1c51598d865afc7519) already exists.{code}The hash is not currently in the database. {code}pe-puppetdb=# select * from fact_values where value_hash = '\x88680eb39ee03249046efa1c51598d865afc7519'; id | value_hash | value_type_id | value_integer | value_float | value_string | value_boolean | value++---+---+-+--+---+---(0 rows){code}This occurs when there are two structured facts with the same contents, but not in the same order. Below is a basic facts payload to reproduce the issue. {code:json}{ "certname": "testing.example.com", "values": {"a_structured_fact": { "element2": {"somevalue": "2" }, "element1": {"somevalue": "1" }},"another_structured_fact": { "element1": {"somevalue": "1" }, "element2": {"somevalue": "2" }} }, "environment": "production", "producer_timestamp": "2018-01-29T15:34:59.792-08:00", "producer": "somemaster.example.com"}{code} To simulate this, one can run the following command after populating payload.json with the content above. {code}curl -X POST -H 'Content-Type:application/json' -H 'Accept:application/json' -d @payload.json "http://localhost:8080/pdb/cmd/v1?command=replace_facts=5=testing.example.com"{code} If I put both facts in the same order, we do not encounter the error. It sounds like the fix is to sort the structured facts when checking if they already exist. The end result of this issue is that we get continual retries and discards. The facts are never updated in PuppetDB for these nodes, unless the value_hash already exists in the database. _Workaround_Sort the hash prior to setting the fact. Hash[myhash.sort] worked in testing.
Add Comment
Jira (PDB-3824) Duplicate unordered structured facts violates unique constraint
Title: Message Title Owen Rodabaugh updated an issue PuppetDB / PDB-3824 Duplicate unordered structured facts violates unique constraint Change By: Owen Rodabaugh Team: Data Platform Add Comment This message was sent by Atlassian JIRA (v7.0.2#70111-sha1:88534db) -- You received this message because you are subscribed to the Google Groups "Puppet Bugs" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To post to this group, send email to puppet-bugs@googlegroups.com. Visit this group at https://groups.google.com/group/puppet-bugs. For more options, visit https://groups.google.com/d/optout.
Jira (PDB-3824) Duplicate unordered structured facts violates unique constraint
Title: Message Title
Jarret Lavallee updated an issue
PuppetDB / PDB-3824
Duplicate unordered structured facts violates unique constraint
Change By:
Jarret Lavallee
Duplicate custom structured facts that are not ordered cause replace_facts to fail with the following error message. {code}2018-01-30 16:10:11,844 ERROR [p.p.command] [10,612] [replace facts] Retrying after attempt 0 for testing.example.com, due to: org.postgresql.util.PSQLException: ERROR: duplicate key value violates unique constraint "fact_values_value_hash_key" Detail: Key (value_hash)=(\x88680eb39ee03249046efa1c51598d865afc7519) already exists.org.postgresql.util.PSQLException: ERROR: duplicate key value violates unique constraint "fact_values_value_hash_key" Detail: Key (value_hash)=(\x88680eb39ee03249046efa1c51598d865afc7519) already exists.{code}The hash is not currently in the database. {code}pe-puppetdb=# select * from fact_values where value_hash = '\x88680eb39ee03249046efa1c51598d865afc7519'; id | value_hash | value_type_id | value_integer | value_float | value_string | value_boolean | value++---+---+-+--+---+---(0 rows){code}This occurs when there are two structured facts with the same contents, but not in the same order. Below is a basic facts payload to reproduce the issue. {code:json}{ "certname": "testing.example.com", "values": {"a_structured_fact": { "element2": {"somevalue": "2" }, "element1": {"somevalue": "1" }},"another_structured_fact": { "element1": {"somevalue": "1" }, "element2": {"somevalue": "2" }} }, "environment": "production", "producer_timestamp": "2018-01-29T15:34:59.792-08:00", "producer": "somemaster.example.com"}{code}If I put both facts in the same order, we do not encounter the error. It sounds like the fix is to sort the structured facts when checking if they already exist. The end result of this issue is that we get continual retries and discards. The facts are never updated in PuppetDB for these nodes, unless the value_hash already exists in the database. _Workaround_Sort the hash prior to setting the fact. { Hash[myhash.sort] } worked in testing.
Add Comment
Jira (PDB-3824) Duplicate unordered structured facts violates unique constraint
Title: Message Title
Jarret Lavallee updated an issue
PuppetDB / PDB-3824
Duplicate unordered structured facts violates unique constraint
Change By:
Jarret Lavallee
Duplicate custom structured facts that are not ordered cause replace_facts to fail with the following error message. {code}2018-01-30 16:10:11,844 ERROR [p.p.command] [10,612] [replace facts] Retrying after attempt 0 for testing.example.com, due to: org.postgresql.util.PSQLException: ERROR: duplicate key value violates unique constraint "fact_values_value_hash_key" Detail: Key (value_hash)=(\x88680eb39ee03249046efa1c51598d865afc7519) already exists.org.postgresql.util.PSQLException: ERROR: duplicate key value violates unique constraint "fact_values_value_hash_key" Detail: Key (value_hash)=(\x88680eb39ee03249046efa1c51598d865afc7519) already exists.{code}The hash is not currently in the database. {code}pe-puppetdb=# select * from fact_values where value_hash = '\x88680eb39ee03249046efa1c51598d865afc7519'; id | value_hash | value_type_id | value_integer | value_float | value_string | value_boolean | value++---+---+-+--+---+---(0 rows){code}This occurs when there are two structured facts with the same contents, but not in the same order. Below is a basic facts payload to reproduce the issue. {code:json}{ "certname": "testing.example.com", "values": {"a_structured_fact": { "element2": {"somevalue": "2" }, "element1": {"somevalue": "1" }},"another_structured_fact": { "element1": {"somevalue": "1" }, "element2": {"somevalue": "2" }} }, "environment": "production", "producer_timestamp": "2018-01-29T15:34:59.792-08:00", "producer": "somemaster.example.com"}{code}If I put both facts in the same order, we do not encounter the error. It sounds like the fix is to sort the structured facts when checking if they already exist. The end result of this issue is that we get continual retries and discards. The facts are never updated in PuppetDB for these nodes, unless the value_hash already exists in the database. _Workaround_Sort the hash prior to setting the fact. {Hash[myhash.sort]} worked in testing.
Add Comment
Jira (PDB-3824) Duplicate unordered structured facts violates unique constraint
Title: Message Title
Jarret Lavallee updated an issue
PuppetDB / PDB-3824
Duplicate unordered structured facts violates unique constraint
Change By:
Jarret Lavallee
Duplicate custom structured facts that are not ordered cause replace_facts to fail with the following error message. {code}2018-01-30 16:10:11,844 ERROR [p.p.command] [10,612] [replace facts] Retrying after attempt 0 for testing.example.com, due to: org.postgresql.util.PSQLException: ERROR: duplicate key value violates unique constraint "fact_values_value_hash_key" Detail: Key (value_hash)=(\x88680eb39ee03249046efa1c51598d865afc7519) already exists.org.postgresql.util.PSQLException: ERROR: duplicate key value violates unique constraint "fact_values_value_hash_key" Detail: Key (value_hash)=(\x88680eb39ee03249046efa1c51598d865afc7519) already exists.{code}The hash is not currently in the database. {code}pe-puppetdb=# select * from fact_values where value_hash = '\x88680eb39ee03249046efa1c51598d865afc7519'; id | value_hash | value_type_id | value_integer | value_float | value_string | value_boolean | value++---+---+-+--+---+---(0 rows){code}This occurs when there are two structured facts with the same contents, but not in the same order. Below is a basic facts payload to be sent into replace_facts reproduce the issue . {code:json}{ "certname": "testing.example.com", "values": {"a_structured_fact": { "element2": {"somevalue": "2" }, "element1": {"somevalue": "1" }},"another_structured_fact": { "element1": {"somevalue": "1" }, "element2": {"somevalue": "2" }} }, "environment": "production", "producer_timestamp": "2018-01-29T15:34:59.792-08:00", "producer": "somemaster.example.com"}{code}If I put both facts in the same order, we do not encounter the error. It sounds like the fix is to sort the structured facts when checking if they already exist. The end result of this issue is that we get many continual retries and discards. The facts are never updated in PuppetDB for these nodes, unless the value_hash already exists in the database.
Add Comment
Jira (PDB-3824) Duplicate unordered structured facts violates unique constraint
Title: Message Title Jarret Lavallee updated an issue PuppetDB / PDB-3824 Duplicate unordered structured facts violates unique constraint Change By: Jarret Lavallee CS Priority: Needs Priority Add Comment This message was sent by Atlassian JIRA (v7.0.2#70111-sha1:88534db) -- You received this message because you are subscribed to the Google Groups "Puppet Bugs" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To post to this group, send email to puppet-bugs@googlegroups.com. Visit this group at https://groups.google.com/group/puppet-bugs. For more options, visit https://groups.google.com/d/optout.
Jira (PDB-3824) Duplicate unordered structured facts violates unique constraint
Title: Message Title
Jarret Lavallee updated an issue
PuppetDB / PDB-3824
Duplicate unordered structured facts violates unique constraint
Change By:
Jarret Lavallee
Duplicate custom structured facts that are not ordered , cause replace_facts fails due to fail with the following error message. {code}2018-01-30 16:10:11,844 ERROR [p.p.command] [10,612] [replace facts] Retrying after attempt 0 for testing.example.com, due to: org.postgresql.util.PSQLException: ERROR: duplicate key value violates unique constraint "fact_values_value_hash_key" Detail: Key (value_hash)=(\x88680eb39ee03249046efa1c51598d865afc7519) already exists.org.postgresql.util.PSQLException: ERROR: duplicate key value violates unique constraint "fact_values_value_hash_key" Detail: Key (value_hash)=(\x88680eb39ee03249046efa1c51598d865afc7519) already exists.{code}The hash is not currently in the database. {code}pe-puppetdb=# select * from fact_values where value_hash = '\x88680eb39ee03249046efa1c51598d865afc7519'; id | value_hash | value_type_id | value_integer | value_float | value_string | value_boolean | value++---+---+-+--+---+---(0 rows){code}This occurs when there are two structured facts with the same contents, but not in the same order. Below is a basic facts payload to be sent into replace_facts. {code:json}{ "certname": "testing.example.com", "values": {"a_structured_fact": { "element2": {"somevalue": "2" }, "element1": {"somevalue": "1" }},"another_structured_fact": { "element1": {"somevalue": "1" }, "element2": {"somevalue": "2" }} }, "environment": "production", "producer_timestamp": "2018-01-29T15:34:59.792-08:00", "producer": "somemaster.example.com"}{code}If I put both facts in the same order, we do encounter the error. It sounds like the fix is to sort the structured facts when checking if they already exist. The end result of this issue is that we get many retries and discards. The facts are never updated in PuppetDB for these nodes, unless the value_hash already exists in the database.
Add Comment
Jira (PDB-3824) Duplicate unordered structured facts violates unique constraint
Title: Message Title Jarret Lavallee updated an issue PuppetDB / PDB-3824 Duplicate unordered structured facts violates unique constraint Change By: Jarret Lavallee Acceptance Criteria: Structured facts in any order should not cause failures in replace_facts. Add Comment This message was sent by Atlassian JIRA (v7.0.2#70111-sha1:88534db) -- You received this message because you are subscribed to the Google Groups "Puppet Bugs" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To post to this group, send email to puppet-bugs@googlegroups.com. Visit this group at https://groups.google.com/group/puppet-bugs. For more options, visit https://groups.google.com/d/optout.
Jira (PDB-3824) Duplicate unordered structured facts violates unique constraint
Title: Message Title Jarret Lavallee created an issue PuppetDB / PDB-3824 Duplicate unordered structured facts violates unique constraint Issue Type: Bug Affects Versions: PDB 5.1.3 Assignee: Unassigned Components: PuppetDB Created: 2018/01/30 4:25 PM Environment: Puppet 5.3.3 with PuppetDB 5.1.3 and duplicate structured facts Priority: Normal Reporter: Jarret Lavallee Duplicate custom structured facts that are not ordered, replace_facts fails due to the following error message. 2018-01-30 16:10:11,844 ERROR [p.p.command] [10,612] [replace facts] Retrying after attempt 0 for testing.example.com, due to: org.postgresql.util.PSQL
