Ubud, Bali - September 18th, 2019 **pgCluu 3.0 released**
PostgreSQL Cluster Utilization ---------------------------------------- pgCluu is a Perl program used to perform a full audit of a PostgreSQL Cluster performances. It is divided in two parts, a collector used to grab statistics on the PostgreSQL server using psql and sar, a reports builder that will generate all HTML and charts output. This release is a major release that adds lot of improvement and new reports. It also improve performances and fully support a CGI mode with dynamic content for realtime reports. New or enhanced reports/features: * The CGI mode to provide dynamic reports on time selection is now production ready and has left the beta stage. See documentation on how to enable it. Debian packaging enable this mode by default. * Installation and configuration is fully managed from Makefile.PL script. This will help distribution packaging with additional configuration directives to control the behavior of pgcluu scripts with generic systemd service and timer files. * Add incremental mode to pgcluu. That mean that you don't have to remove the content of the report directory anymore before running pgcluu again and when pgcluu detect that you are running in this mode. The incremental mode is detected from statistic directory when you have used the --rotate-daily or --rotate-hourly option to pgcluu_collectd. * Rewrite Stats I/O reports to display read/hit per table or indexes during the audit period reported. * Move Xlog (Wal bytes written) report under replication menu as data comes from pg_stat_replication. * Explode System and Cluster reports into several reports dedicated files to limit the size of the HTML files in incremental mode. * Add collect and report of vm.nr_overcommit_hugepages. * Add report on pages scanned to report number of pages scanned by the kswapd daemon and number of pages scanned directly per second (pgscank/s and pgscand/s). It also reports the number of pages the system has reclaimed from cache (pagecache and swapcache) per second to satisfy its memory demands (pgsteal/s). On a second axis %vmeffto reports the the efficiency of page reclaim. * All pages reports have been moved as sub menu of the Page menu and pgfree has been removed from the page fault report. * Limit system cache statistics report to pgpgin/s and pgpgout/s and a new report is dedicated to Page fault statistics with majflt/s, minflt/s and pgfree/s dataset. Dataset minflt/s is calculated from sar output as result of "fault/s - majflt/s". * All statistics about pg_stat_user_* and pg_stat_xact_user_* views are now stored as a snapshot at start and at end of the audit period and each hour if you are running in incremental mode. The old behavior was to append the snapshot at each running interval to the data file but this is not necessary and this can use huge disk space if you have lot of table and indexes in your database. Backward compatibility is preserved. * New report of json versus jsonb columns in each database. * Add keepalive kernel configuration in SysInfo report. * Separate bgwriter "allocated" buffers in a dedicated report for better reading. * Add new report about estimated memory workload based on the value of Committed_AS from /proc/meminfo. Committed_AS is the total amount of memory estimated to complete the workload. This value represents the worst case scenario value, and also includes swap memory. The report show the memory workload aver the time. New command line options: * Add option -x, --external-menu to pgcluu to save the common menu in menu.html and load it into each report using w3-include-html attribute from w3.js. This will only work if access to HTML report is through a Web server, not using the file:// protocol. * Add new option --retention to pgcluu to set number of rolling days to keep in report directory. Default is to store indefinitely. * Add option --retention NDAYS to pgcluu_collectd to set the number of rolling days to keep in data directory in incremental mode. Default is to store indefinitely. New configuration directives : * Add configuration directive STATS_REPORT_CACHING to pgcluu.conf file. This directive must be enabled in CGI mode. In this case the pgcluu script must be run periodically at least each 5 minutes to build and update cached statistics. In cache mode pgcluu generate binary file in the statistics directory and nothing in the report directory. The pgcluu.service and pgcluu.timer can be use, pgcluu will read the configuration file and switch automatically to cache mode if STATS_REPORT_CACHING is enabled. * Add MAX_RENDERED_DAYS configuration directive to set the maximum number of days in a graph. Using default pgcluu_collectd 1 minute interval each, daily graph will have 1440 points. Having too much data to render is not a good point for performances with the current code. The user can set is own graph period, default is to limit to the first seven days of the selected period. This mean a maximum of 10080 points for one week, this seems a safe limit. It also fixes some issues reported by users since last ten months. -- Gilles Darold http://www.darold.net/
