On Mon, Apr 15, 2019 at 4:31 PM Bruce Momjian <br...@momjian.us> wrote: > Can I throw out a simple idea? What if, when we finish writing a WAL > file, we create a new file 000000010000000000000001.modblock which > lists all the heap/index files and block numbers modified in that WAL > file? How much does that help with the list I posted earlier? > > I think there is some interesting complexity brought up in this > thread. > Which options are going to minimize storage I/O, network I/O, have > only > background overhead, allow parallel operation, integrate with > pg_basebackup. Eventually we will need to evaluate the incremental > backup options against these criteria. > > I am thinking tools could retain modblock files along with WAL, could > pull full-page-writes from WAL, or from PGDATA. It avoids the need to > scan 16MB WAL files, and the WAL files and modblock files could be > expired independently.
That is pretty much exactly what I was intending to propose. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company
