Re: [HACKERS] [DESIGN] Incremental checksums

2015-07-16 Thread David Christensen
> > > > - pg_disable_checksums(void) => turn checksums off for a cluster. > > > > Sets the state to "disabled", which means bg_worker will not do > > > > anything. > > > > > > > > - pg_request_checksum_cycle(void) => if checksums are "enabled", > > > > increment the data_checksum_cycle coun

Re: [HACKERS] [DESIGN] Incremental checksums

2015-07-15 Thread Amit Kapila
On Wed, Jul 15, 2015 at 9:13 PM, David Christensen wrote: > > > > On Jul 15, 2015, at 3:18 AM, Amit Kapila wrote: > > > > > - pg_disable_checksums(void) => turn checksums off for a cluster. Sets the state to "disabled", which means bg_worker will not do anything. > > > > > > - pg_request_chec

Re: [HACKERS] [DESIGN] Incremental checksums

2015-07-15 Thread David Christensen
> On Jul 15, 2015, at 3:18 AM, Amit Kapila wrote: > > > - pg_disable_checksums(void) => turn checksums off for a cluster. Sets > > the state to "disabled", which means bg_worker will not do anything. > > > > - pg_request_checksum_cycle(void) => if checksums are "enabled", > > increment th

Re: [HACKERS] [DESIGN] Incremental checksums

2015-07-15 Thread Andres Freund
On 2015-07-15 12:48:40 +0530, Amit Kapila wrote: > If during scan of a relation, after doing checksum for half of the > blocks in relation, system crashes, then in the above scheme a > restart would need to again read all the blocks even though some > of the blocks are already checksummed in previo

Re: [HACKERS] [DESIGN] Incremental checksums

2015-07-15 Thread Amit Kapila
On Tue, Jul 14, 2015 at 1:56 AM, David Christensen wrote: > > For any relation that it finds in the database which is not checksummed, it starts an actual worker to handle the checksum process for this table. Since the state of the cluster is already either "enforcing" or "revalidating", any bloc

Re: [HACKERS] [DESIGN] Incremental checksums

2015-07-14 Thread Jim Nasby
On 7/13/15 4:02 PM, David Christensen wrote: On Jul 13, 2015, at 3:50 PM, Jim Nasby wrote: On 7/13/15 3:26 PM, David Christensen wrote: * Incremental Checksums PostgreSQL users should have a way up upgrading their cluster to use data checksums without having to do a costly pg_dump/pg_resto

Re: [HACKERS] [DESIGN] Incremental checksums

2015-07-13 Thread Andres Freund
On 2015-07-13 15:50:44 -0500, Jim Nasby wrote: > Another possibility is some kind of a page-level indicator of what binary > format is in use on a given page. For checksums maybe a single bit would > suffice (indicating that you should verify the page checksum). Another use > case is using this to

Re: [HACKERS] [DESIGN] Incremental checksums

2015-07-13 Thread David Christensen
> On Jul 13, 2015, at 3:50 PM, Jim Nasby wrote: > > On 7/13/15 3:26 PM, David Christensen wrote: >> * Incremental Checksums >> >> PostgreSQL users should have a way up upgrading their cluster to use data >> checksums without having to do a costly pg_dump/pg_restore; in particular, >> checksum

Re: [HACKERS] [DESIGN] Incremental checksums

2015-07-13 Thread Jim Nasby
On 7/13/15 3:26 PM, David Christensen wrote: * Incremental Checksums PostgreSQL users should have a way up upgrading their cluster to use data checksums without having to do a costly pg_dump/pg_restore; in particular, checksums should be able to be enabled/disabled at will, with the database

[HACKERS] [DESIGN] Incremental checksums

2015-07-13 Thread David Christensen
pgsql-hackers, So I’ve put some time into a design for the incremental checksum feature and wanted to get some feedback from the group: * Incremental Checksums PostgreSQL users should have a way up upgrading their cluster to use data checksums without having to do a costly pg_dump/pg_restore;