Jira (PDB-5135) PuppetDB should warn about resource titles that exceed Postgres index sizes

[Charlie Sharpsteen (Jira)]

Title: Message Title

Charlie Sharpsteen created an issue   PuppetDB / PDB-5135 PuppetDB should warn about resource titles that exceed Postgres index sizes Issue Type: Improvement Assignee: Unassigned Created: 2021/05/20 8:50 AM Priority: Normal Reporter: Charlie Sharpsteen When PuppetDB inserts data into Postgres, the constraints of the database can cause errors to be raised. A typical example is that Postgres disallows the use of the null byte, "\0", in strings while UTF-8 generally tolerates it. Another constraint comes into play when large data values are inserted into columns that have database indexes: Any data type that can be sorted into a well-defined linear order can be indexed by a btree index. The only limitation is that an index entry cannot exceed approximately one-third of a page (after TOAST compression, if applicable). https://www.postgresql.org/docs/11/btree-intro.html This constraint is typically encountered with resource title values in catalogs or reports and results in an error similar to the following being raised from the storage attempt: 2021-05-11T23:59:47.225Z ERROR [p.p.command] [14,654,263] [store report] Retrying after attempt 0 for node.hostname.example, due to: org.postgresql.util.PSQLException: ERROR: index row size 2720 exceeds maximum 2712 for index "resource_events_resource_timestamp_20210511z" This error is somewhat useful in that it indicates which node tripped the condition. But the error does not help identify which data value needs to be corrected. For resource titles, PuppetDB should check input lengths against the 2712 character maximum and emit a warning or error that includes: The certname of the node that produced the data The type of the resource The manifest file and line number where the resource was defined Add Comment   This message was sent by Atlassian Jira (v8.13.2#813002-sha1:c495a97) -- You received this message because you are subscribed to the Google Groups "Puppet Bugs" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-bugs/JIRA.399989.1621525850000.37539.1621525860118%40Atlassian.JIRA.