We've discussed previously the negative impact of large bulk operations, especially wrt WAL writes. Patch here allows maintenance operations to have their WAL generation slowed down as a replication lag prevention feature.
I believe there was originally intended to be some work on I/O rate limiting, but that hasn't happened and is in some ways orthogonal to this patch and we will likely eventually want both. Single new parameter works very similarly to vacuum_cost_delay wal_rate_limit_delay = Xms slows down CLUSTER, VACUUM FULL, ALTER TABLE (rewrite & set tablespace), CREATE INDEX so basically same things we optimise WAL for and the same places where we honour maintenance_work_mem (discuss: should we add COPY, CTAS etc also?) (discuss: do we need another parameter to specify "cost"? Currently patch uses "sleep every 64kB of WAL") VACUUM is not included, since we already have controls for that - honouring two controls would be complex and weird. Uses GetCurrentTransactionWALVolume() patch, which is included within the patch to make it easier to review as a whole. Technically, we can't simply wait before/after WAL inserts because these typically occur while holding buffer locks. So we need to put the waits at a higher level, notably in safe places that currently do CHECK_FOR_INTERRUPTS(). Doing that during query execution might make locking of blocks for nested loops joins much worse. -- Simon Riggs http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services
wal_rate_limiting.v1.patch
Description: Binary data
-- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers