Jira (PDB-3211) Collect/expose metrics on vacuum stats
Title: Message Title Claudia Petty updated an issue PuppetDB / PDB-3211 Collect/expose metrics on vacuum stats Change By: Claudia Petty Labels: new-feature tsr-pdb-backlog Add Comment This message was sent by Atlassian Jira (v8.20.21#820021-sha1:38274c8) -- You received this message because you are subscribed to the Google Groups "Puppet Bugs" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-bugs/JIRA.165199.1481036879000.2430.1687359427937%40Atlassian.JIRA.
Jira (PDB-3211) Collect/expose metrics on vacuum stats
Title: Message Title Bogdan Irimie updated an issue PuppetDB / PDB-3211 Collect/expose metrics on vacuum stats Change By: Bogdan Irimie Attachment: vacuum-metrics Add Comment This message was sent by Atlassian Jira (v8.13.2#813002-sha1:c495a97) -- You received this message because you are subscribed to the Google Groups "Puppet Bugs" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-bugs/JIRA.165199.1481036879000.6204.1618235760149%40Atlassian.JIRA.
Jira (PDB-3211) Collect/expose metrics on vacuum stats
Title: Message Title Bogdan Irimie updated an issue PuppetDB / PDB-3211 Collect/expose metrics on vacuum stats Change By: Bogdan Irimie Attachment: vacuum-metrics-1 Add Comment This message was sent by Atlassian Jira (v8.13.2#813002-sha1:c495a97) -- You received this message because you are subscribed to the Google Groups "Puppet Bugs" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-bugs/JIRA.165199.1481036879000.6201.1618235760069%40Atlassian.JIRA.
Jira (PDB-3211) Collect/expose metrics on vacuum stats
Title: Message Title Bogdan Irimie updated an issue PuppetDB / PDB-3211 Collect/expose metrics on vacuum stats Change By: Bogdan Irimie Attachment: vacuum-metrics Add Comment This message was sent by Atlassian Jira (v8.13.2#813002-sha1:c495a97) -- You received this message because you are subscribed to the Google Groups "Puppet Bugs" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-bugs/JIRA.165199.1481036879000.6203.1618235760141%40Atlassian.JIRA.
Jira (PDB-3211) Collect/expose metrics on vacuum stats
Title: Message Title Bogdan Irimie updated an issue PuppetDB / PDB-3211 Collect/expose metrics on vacuum stats Change By: Bogdan Irimie Attachment: vacuum-metrics-1 Add Comment This message was sent by Atlassian Jira (v8.13.2#813002-sha1:c495a97) -- You received this message because you are subscribed to the Google Groups "Puppet Bugs" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-bugs/JIRA.165199.1481036879000.6202.1618235760089%40Atlassian.JIRA.
Jira (PDB-3211) Collect/expose metrics on vacuum stats
Title: Message Title Bogdan Irimie commented on PDB-3211 Re: Collect/expose metrics on vacuum stats We should look over auto vacuum throttling parameters as they can play a bigger role in large deployments where the default values might be to small and auto vacuum will not be able to keep up with the bloat. Add Comment This message was sent by Atlassian Jira (v8.13.2#813002-sha1:c495a97) -- You received this message because you are subscribed to the Google Groups "Puppet Bugs" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-bugs/JIRA.165199.1481036879000.3720.1617822720023%40Atlassian.JIRA.
Jira (PDB-3211) Collect/expose metrics on vacuum stats
Title: Message Title Bogdan Irimie commented on PDB-3211 Re: Collect/expose metrics on vacuum stats `vacuum-metrics` contains SQL queries that we can use for monitoring the bloat and some possible reasons. The puppetlabs-puppet_metrics_collector module collects bloat and vacuum data from `pg_stat_all_tables`. Attached is an example of a Grafana dashboard that can be created with puppet_metrics_dashboard that shows the evolution of dead tuples over time. One of the main advantages of puppetlabs-puppet_metrics_collector module is that it saves data every 5 minutes and can push it to third party monitoring systems and databases like splunk, influxdb and graphite. The module will be included in the next versions of PE. Add Comment This message was sent by Atlassian Jira (v8.13.2#813002-sha1:c495a97) -- You received this message because you are subscribed to the Google Groups "Puppet Bugs" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-bugs/JIRA.165199.1481036879000.3698.1617822180149%40Atlassian.JIRA.
Jira (PDB-3211) Collect/expose metrics on vacuum stats
Title: Message Title Bogdan Irimie updated an issue PuppetDB / PDB-3211 Collect/expose metrics on vacuum stats Change By: Bogdan Irimie Attachment: vacuum-metrics Add Comment This message was sent by Atlassian Jira (v8.13.2#813002-sha1:c495a97) -- You received this message because you are subscribed to the Google Groups "Puppet Bugs" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-bugs/JIRA.165199.1481036879000.3667.1617820920133%40Atlassian.JIRA.
Jira (PDB-3211) Collect/expose metrics on vacuum stats
Title: Message Title Bogdan Irimie updated an issue PuppetDB / PDB-3211 Collect/expose metrics on vacuum stats Change By: Bogdan Irimie Attachment: puppet-metrics-collector-20210401T142001Z.json Add Comment This message was sent by Atlassian Jira (v8.13.2#813002-sha1:c495a97) -- You received this message because you are subscribed to the Google Groups "Puppet Bugs" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-bugs/JIRA.165199.1481036879000.3663.1617820800097%40Atlassian.JIRA.
Jira (PDB-3211) Collect/expose metrics on vacuum stats
Title: Message Title Bogdan Irimie updated an issue PuppetDB / PDB-3211 Collect/expose metrics on vacuum stats Change By: Bogdan Irimie Sprint: ghost-7.04.2021 , HAHA/Grooming 2 Add Comment This message was sent by Atlassian Jira (v8.13.2#813002-sha1:c495a97) -- You received this message because you are subscribed to the Google Groups "Puppet Bugs" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-bugs/JIRA.165199.1481036879000.3125.1617800640136%40Atlassian.JIRA.
Jira (PDB-3211) Collect/expose metrics on vacuum stats
Title: Message Title Bogdan Irimie updated an issue PuppetDB / PDB-3211 Collect/expose metrics on vacuum stats Change By: Bogdan Irimie Attachment: Screenshot 2021-04-02 at 18.06.03.png Add Comment This message was sent by Atlassian Jira (v8.13.2#813002-sha1:c495a97) -- You received this message because you are subscribed to the Google Groups "Puppet Bugs" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-bugs/JIRA.165199.1481036879000.71.1617376140108%40Atlassian.JIRA.
Jira (PDB-3211) Collect/expose metrics on vacuum stats
Title: Message Title Bogdan Irimie assigned an issue to Bogdan Irimie PuppetDB / PDB-3211 Collect/expose metrics on vacuum stats Change By: Bogdan Irimie Assignee: Bogdan Irimie Add Comment This message was sent by Atlassian Jira (v8.5.2#805002-sha1:a66f935) -- You received this message because you are subscribed to the Google Groups "Puppet Bugs" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-bugs/JIRA.165199.1481036879000.176979.1617090900028%40Atlassian.JIRA.
Jira (PDB-3211) Collect/expose metrics on vacuum stats
Title: Message Title Bogdan Irimie updated an issue PuppetDB / PDB-3211 Collect/expose metrics on vacuum stats Change By: Bogdan Irimie Sprint: ready for triage 3 Add Comment This message was sent by Atlassian Jira (v8.5.2#805002-sha1:a66f935) -- You received this message because you are subscribed to the Google Groups "Puppet Bugs" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-bugs/JIRA.165199.1481036879000.172839.1616596500296%40Atlassian.JIRA.
Jira (PDB-3211) Collect/expose metrics on vacuum stats
Title: Message Title Bogdan Irimie updated an issue PuppetDB / PDB-3211 Collect/expose metrics on vacuum stats Change By: Bogdan Irimie Story Points: 5 Add Comment This message was sent by Atlassian Jira (v8.5.2#805002-sha1:a66f935) -- You received this message because you are subscribed to the Google Groups "Puppet Bugs" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-bugs/JIRA.165199.1481036879000.172838.1616596500253%40Atlassian.JIRA.
Jira (PDB-3211) Collect/expose metrics on vacuum stats
Title: Message Title Bogdan Irimie updated an issue PuppetDB / PDB-3211 Collect/expose metrics on vacuum stats Change By: Bogdan Irimie Team: Ghost Add Comment This message was sent by Atlassian Jira (v8.5.2#805002-sha1:a66f935) -- You received this message because you are subscribed to the Google Groups "Puppet Bugs" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-bugs/JIRA.165199.1481036879000.149174.1614085680128%40Atlassian.JIRA.
Jira (PDB-3211) Collect/expose metrics on vacuum stats
Title: Message Title Zachary Kent commented on PDB-3211 Re: Collect/expose metrics on vacuum stats See PDB-4941 and related links in the comments there for more background on times we've seen autovacuum issues with customers and how they were resolved. The information there isn't necessary to complete this ticket, but is useful to gain some context around past issues. Add Comment This message was sent by Atlassian Jira (v8.5.2#805002-sha1:a66f935) -- You received this message because you are subscribed to the Google Groups "Puppet Bugs" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-bugs/JIRA.165199.1481036879000.148725.1614034440085%40Atlassian.JIRA.
Jira (PDB-3211) Collect/expose metrics on vacuum stats
Title: Message Title Zachary Kent updated an issue PuppetDB / PDB-3211 Collect/expose metrics on vacuum stats Change By: Zachary Kent Labels: tsr-pdb-backlog Add Comment This message was sent by Atlassian Jira (v8.5.2#805002-sha1:a66f935) -- You received this message because you are subscribed to the Google Groups "Puppet Bugs" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-bugs/JIRA.165199.1481036879000.148660.1614033060057%40Atlassian.JIRA.
Jira (PDB-3211) Collect/expose metrics on vacuum stats
Title: Message Title Zachary Kent updated an issue PuppetDB / PDB-3211 Collect/expose metrics on vacuum stats Change By: Zachary Kent Acceptance Criteria: * A candidate list of autovacuum related metrics we would want to track in PDB with a brief explanation of why each is valuable* A list of queries that can be used to gather information for each metric Add Comment This message was sent by Atlassian Jira (v8.5.2#805002-sha1:a66f935) -- You received this message because you are subscribed to the Google Groups "Puppet Bugs" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-bugs/JIRA.165199.1481036879000.148360.1614023160044%40Atlassian.JIRA.
Jira (PDB-3211) Collect/expose metrics on vacuum stats
Title: Message Title Zachary Kent updated an issue PuppetDB / PDB-3211 Collect/expose metrics on vacuum stats Change By: Zachary Kent Story Points: 2 Add Comment This message was sent by Atlassian Jira (v8.5.2#805002-sha1:a66f935) -- You received this message because you are subscribed to the Google Groups "Puppet Bugs" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-bugs/JIRA.165199.1481036879000.148361.1614023160089%40Atlassian.JIRA.
Jira (PDB-3211) Collect/expose metrics on vacuum stats
Title: Message Title Zachary Kent commented on PDB-3211 Re: Collect/expose metrics on vacuum stats The main focus of this ticket should be learning about the postgres autovacuum process and identifying what things we would want to track related to autovacuum in PDB. If we can identify a few core metrics we care about we could add a periodic task to check these and log warnings if PDB detects that things aren't healthy. For example, one thing we'll want to check is overall table bloat. This can be accomplished by running the following query: select n_dead_tup, n_live_tup, last_autovacuum from pg_stat_user_tables; Some metrics for postgres are already collected in the puppetlabs-puppet_metrics_collector module. Recently changes were made to this module in PR 71 to gather more postgres metrics. The changes in the psql_metrics file of that PR could be a useful starting place when looking for the types of things we might want to gather in PDB periodically. Add Comment This message was sent by Atlassian Jira (v8.5.2#805002-sha1:a66f935)
Jira (PDB-3211) Collect/expose metrics on vacuum stats
Title: Message Title Ryan Senior updated an issue PuppetDB / PDB-3211 Collect/expose metrics on vacuum stats Change By: Ryan Senior Sprint: Hopper Add Comment This message was sent by Atlassian JIRA (v6.4.14#64029-sha1:ae256fe) -- You received this message because you are subscribed to the Google Groups "Puppet Bugs" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To post to this group, send email to puppet-bugs@googlegroups.com. Visit this group at https://groups.google.com/group/puppet-bugs. For more options, visit https://groups.google.com/d/optout.
Jira (PDB-3211) Collect/expose metrics on vacuum stats
Title: Message Title Ryan Senior updated an issue PuppetDB / PDB-3211 Collect/expose metrics on vacuum stats Change By: Ryan Senior Sprint: SE 2016-01-11 Hopper Add Comment This message was sent by Atlassian JIRA (v6.4.14#64029-sha1:ae256fe) -- You received this message because you are subscribed to the Google Groups "Puppet Bugs" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To post to this group, send email to puppet-bugs@googlegroups.com. Visit this group at https://groups.google.com/group/puppet-bugs. For more options, visit https://groups.google.com/d/optout.
Jira (PDB-3211) Collect/expose metrics on vacuum stats
Title: Message Title Karen Van der Veer updated an issue PuppetDB / PDB-3211 Collect/expose metrics on vacuum stats Change By: Karen Van der Veer Sprint: Hopper SE 2016-01-11 Add Comment This message was sent by Atlassian JIRA (v6.4.14#64029-sha1:ae256fe) -- You received this message because you are subscribed to the Google Groups "Puppet Bugs" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To post to this group, send email to puppet-bugs@googlegroups.com. Visit this group at https://groups.google.com/group/puppet-bugs. For more options, visit https://groups.google.com/d/optout.
Jira (PDB-3211) Collect/expose metrics on vacuum stats
Title: Message Title Ryan Senior created an issue PuppetDB / PDB-3211 Collect/expose metrics on vacuum stats Issue Type: New Feature Assignee: Unassigned Created: 2016/12/06 7:07 AM Priority: Normal Reporter: Ryan Senior PuppetDB has a potential to generate excessive garbage for PostgreSQL depending on usage patterns. We don't currently collect or monitor this metric which is a gap in our performance and scalability. It's likely that this ticket will need to be more of a design/investigation sort to ticket to figure out how we want to monitor this over time (and what historic information is available in PostgreSQL on vacuuming). A second ticket can be created/estimated for implementing this. The end result should be the ability to monitor, assess the impact of PuppetDB usage on PostgreSQL's vacuuming. This should allow us to make measurable improvements over time. Add Comment
