Re: [HACKERS] Checksums by default?

On Fri, Feb 24, 2017 at 10:30 PM, Bruce Momjian wrote: > On Fri, Feb 24, 2017 at 10:09:50PM +0200, Ants Aasma wrote: >> On Fri, Feb 24, 2017 at 9:37 PM, Bruce Momjian wrote: >> > Oh, that's why we will hopefully eventually change the page checksum >> >

Re: [HACKERS] Checksums by default?

On Fri, Feb 24, 2017 at 10:09:50PM +0200, Ants Aasma wrote: > On Fri, Feb 24, 2017 at 9:37 PM, Bruce Momjian wrote: > > Oh, that's why we will hopefully eventually change the page checksum > > algorithm to use the special CRC32 instruction, and set a new checksum > > version ---

Re: [HACKERS] Checksums by default?

On Fri, Feb 24, 2017 at 9:49 PM, Jim Nasby wrote: > On 2/24/17 12:30 PM, Tomas Vondra wrote: >> >> In any case, we can't just build x86-64 packages with compile-time >> SSE4.1 checks. > > > Dumb question... since we're already discussing llvm for the executor, would >

Re: [HACKERS] Checksums by default?

On Fri, Feb 24, 2017 at 9:37 PM, Bruce Momjian wrote: > Oh, that's why we will hopefully eventually change the page checksum > algorithm to use the special CRC32 instruction, and set a new checksum > version --- got it. I assume there is currently no compile-time way to > do

Re: [HACKERS] Checksums by default?

On Fri, Feb 24, 2017 at 10:02 PM, Bruce Momjian wrote: > Uh, as far as I know, the best you are going to get from llvm is > standard assembly, while the SSE4.1 instructions use special assembly > instructions, so they would be faster, and in a way they are a GPU built > into

Re: [HACKERS] Checksums by default?

On Fri, Feb 24, 2017 at 01:49:07PM -0600, Jim Nasby wrote: > On 2/24/17 12:30 PM, Tomas Vondra wrote: > >In any case, we can't just build x86-64 packages with compile-time > >SSE4.1 checks. > > Dumb question... since we're already discussing llvm for the executor, would > that potentially be an

Re: [HACKERS] Checksums by default?

On 2/24/17 12:30 PM, Tomas Vondra wrote: In any case, we can't just build x86-64 packages with compile-time SSE4.1 checks. Dumb question... since we're already discussing llvm for the executor, would that potentially be an option here? AIUI that also opens the possibility of using the GPU as

Re: [HACKERS] Checksums by default?

On Fri, Feb 24, 2017 at 08:31:09PM +0200, Ants Aasma wrote: > >> We looked at that when picking the algorithm. At that point it seemed > >> that CRC CPU instructions were not universal enough to rely on them. > >> The algorithm we ended up on was designed to be fast on SIMD hardware. > >>

Re: [HACKERS] Checksums by default?

On Fri, Feb 24, 2017 at 7:47 PM, Bruce Momjian wrote: > On Sat, Jan 21, 2017 at 09:02:25PM +0200, Ants Aasma wrote: >> > It might be worth looking into using the CRC CPU instruction to reduce this >> > overhead, like we do for the WAL checksums. Since that is a different >> >

Re: [HACKERS] Checksums by default?

On 02/24/2017 06:47 PM, Bruce Momjian wrote: On Sat, Jan 21, 2017 at 09:02:25PM +0200, Ants Aasma wrote: It might be worth looking into using the CRC CPU instruction to reduce this overhead, like we do for the WAL checksums. Since that is a different algorithm it would be a compatibility break

Re: [HACKERS] Checksums by default?

On Sat, Jan 21, 2017 at 09:02:25PM +0200, Ants Aasma wrote: > > It might be worth looking into using the CRC CPU instruction to reduce this > > overhead, like we do for the WAL checksums. Since that is a different > > algorithm it would be a compatibility break and we would need to support the > >

Re: [HACKERS] Checksums by default?

On Thu, Feb 23, 2017 at 10:37 PM, Bruce Momjian wrote: > On Sat, Jan 21, 2017 at 12:46:05PM -0500, Stephen Frost wrote: > > * Petr Jelinek (petr.jeli...@2ndquadrant.com) wrote: > > > As we don't know the performance impact is (there was no benchmark done > > > on reasonably

Re: [HACKERS] Checksums by default?

On Sat, Jan 21, 2017 at 12:46:05PM -0500, Stephen Frost wrote: > * Petr Jelinek (petr.jeli...@2ndquadrant.com) wrote: > > As we don't know the performance impact is (there was no benchmark done > > on reasonably current code base) I really don't understand how you can > > judge if it's worth it or

Re: [HACKERS] Checksums by default?

On 02/11/2017 01:38 AM, Tomas Vondra wrote: Incidentally, I've been dealing with a checksum failure reported by a customer last week, and based on the experience I tend to agree that we don't have the tools needed to deal with checksum failures. I think such tooling should be a 'must have' for

Re: [HACKERS] Checksums by default?

On 02/13/2017 02:29 AM, Jim Nasby wrote: On 2/10/17 6:38 PM, Tomas Vondra wrote: And no, backups may not be a suitable solution - the failure happens on a standby, and the page (luckily) is not corrupted on the master. Which means that perhaps the standby got corrupted by a WAL, which would

Re: [HACKERS] Checksums by default?

On 2/10/17 6:38 PM, Tomas Vondra wrote: And no, backups may not be a suitable solution - the failure happens on a standby, and the page (luckily) is not corrupted on the master. Which means that perhaps the standby got corrupted by a WAL, which would affect the backups too. I can't verify this,

Re: [HACKERS] Checksums by default?

On Fri, Feb 10, 2017 at 7:38 PM, Tomas Vondra wrote: > Incidentally, I've been dealing with a checksum failure reported by a > customer last week, and based on the experience I tend to agree that we > don't have the tools needed to deal with checksum failures. I

Re: [HACKERS] Checksums by default?

Hi, I've repeated those benchmarks on a much smaller/older machine, with only minimal changes (mostly related to RAM and cores available). I've expected to see more significant differences, assuming that newer CPUs will handle the checksumming better, but to my surprise the impact of

Re: [HACKERS] Checksums by default?

On 2/3/17 5:31 PM, Andres Freund wrote: You can't really see things from other databases that way tho. So you need to write a tool that iterates all databases and such. Not that that's a huge problem, but it doesn't make things easier at least. True. Not terribly hard to iterate though, and

Re: [HACKERS] Checksums by default?

On Mon, Jan 30, 2017 at 12:29 PM, David Steele wrote: > The solution was to simply ignore the checksums of any pages with an LSN >>= the LSN returned by pg_start_backup(). This means that hot blocks > may never be checked during backup, but if they are active then any >

Re: [HACKERS] Checksums by default?

On 2017-02-03 17:23:15 -0600, Jim Nasby wrote: > On 1/25/17 6:40 PM, Stephen Frost wrote: > > Obviously, having to bring up a full database is an extra step (one we > > try to make easy to do), but, sadly, we don't have any way to ask PG to > > verify all the checksums with released versions, so

Re: [HACKERS] Checksums by default?

On 1/25/17 6:40 PM, Stephen Frost wrote: Obviously, having to bring up a full database is an extra step (one we try to make easy to do), but, sadly, we don't have any way to ask PG to verify all the checksums with released versions, so that's what we're working with. Wouldn't it be fairly

Re: [HACKERS] Checksums by default?

On 1/25/17 10:38 PM, Stephen Frost wrote: > * Robert Haas (robertmh...@gmail.com) wrote: >> On Wed, Jan 25, 2017 at 7:37 PM, Andres Freund wrote: >>> On 2017-01-25 19:30:08 -0500, Stephen Frost wrote: * Peter Geoghegan (p...@heroku.com) wrote: > On Wed, Jan 25, 2017

Re: [HACKERS] Checksums by default?

On 01/25/2017 05:25 PM, Peter Geoghegan wrote: On Wed, Jan 25, 2017 at 1:22 PM, Peter Geoghegan wrote: I understand that my experience with storage devices is unusually narrow compared to everyone else here. That's why I remain neutral on the high level question of whether or

Re: [HACKERS] Checksums by default?

* Robert Haas (robertmh...@gmail.com) wrote: > On Wed, Jan 25, 2017 at 7:37 PM, Andres Freund wrote: > > On 2017-01-25 19:30:08 -0500, Stephen Frost wrote: > >> * Peter Geoghegan (p...@heroku.com) wrote: > >> > On Wed, Jan 25, 2017 at 3:30 PM, Stephen Frost

Re: [HACKERS] Checksums by default?

On 26 January 2017 at 01:58, Thomas Munro wrote: > > I don't know how comparable it is to our checksum technology, but > MySQL seems to have some kind of checksums on table data, and you can > find public emails, blogs etc lamenting corrupted databases by >

Re: [HACKERS] Checksums by default?

On Wed, Jan 25, 2017 at 7:37 PM, Andres Freund wrote: > On 2017-01-25 19:30:08 -0500, Stephen Frost wrote: >> * Peter Geoghegan (p...@heroku.com) wrote: >> > On Wed, Jan 25, 2017 at 3:30 PM, Stephen Frost wrote: >> > > As it is, there are backup solutions

Re: [HACKERS] Checksums by default?

On Thu, Jan 26, 2017 at 2:28 PM, Stephen Frost wrote: > Sadly, without having them enabled by default, there's not a huge corpus > of example cases to draw from. > > There have been a few examples already posted about corruption failures > with PG, but one can't say with

Re: [HACKERS] Checksums by default?

Peter, * Peter Geoghegan (p...@heroku.com) wrote: > On Wed, Jan 25, 2017 at 1:22 PM, Peter Geoghegan wrote: > > I understand that my experience with storage devices is unusually > > narrow compared to everyone else here. That's why I remain neutral on > > the high level question

Re: [HACKERS] Checksums by default?

On Wed, Jan 25, 2017 at 1:22 PM, Peter Geoghegan wrote: > I understand that my experience with storage devices is unusually > narrow compared to everyone else here. That's why I remain neutral on > the high level question of whether or not we ought to enable checksums > by

Re: [HACKERS] Checksums by default?

* Robert Haas (robertmh...@gmail.com) wrote: > On Wed, Jan 25, 2017 at 6:30 PM, Stephen Frost wrote: > > I hope to discuss it further after we have the ability to turn it off > > easily. > > I think we should have the ability to flip it in BOTH directions easily. Presumably

Re: [HACKERS] Checksums by default?

Michael, * Michael Paquier (michael.paqu...@gmail.com) wrote: > That would be enough. It should also be rare enough that there would > not be that many pages to track when looking at records from the > backup start position to minimum recovery point. It could be also > simpler, though more

Re: [HACKERS] Checksums by default?

* Andres Freund (and...@anarazel.de) wrote: > On 2017-01-25 19:30:08 -0500, Stephen Frost wrote: > > * Peter Geoghegan (p...@heroku.com) wrote: > > > On Wed, Jan 25, 2017 at 3:30 PM, Stephen Frost wrote: > > > > As it is, there are backup solutions which *do* check the

Re: [HACKERS] Checksums by default?

On Thu, Jan 26, 2017 at 9:32 AM, Stephen Frost wrote: > * Robert Haas (robertmh...@gmail.com) wrote: >> On Wed, Jan 25, 2017 at 7:19 PM, Michael Paquier >> wrote: >> > On Thu, Jan 26, 2017 at 9:14 AM, Peter Geoghegan wrote: >> >>

Re: [HACKERS] Checksums by default?

On 2017-01-25 19:30:08 -0500, Stephen Frost wrote: > * Peter Geoghegan (p...@heroku.com) wrote: > > On Wed, Jan 25, 2017 at 3:30 PM, Stephen Frost wrote: > > > As it is, there are backup solutions which *do* check the checksum when > > > backing up PG. This is no longer,

Re: [HACKERS] Checksums by default?

* Robert Haas (robertmh...@gmail.com) wrote: > On Wed, Jan 25, 2017 at 7:19 PM, Michael Paquier > wrote: > > On Thu, Jan 26, 2017 at 9:14 AM, Peter Geoghegan wrote: > >> On Wed, Jan 25, 2017 at 3:30 PM, Stephen Frost wrote: > >>>

Re: [HACKERS] Checksums by default?

* Michael Paquier (michael.paqu...@gmail.com) wrote: > On Thu, Jan 26, 2017 at 9:14 AM, Peter Geoghegan wrote: > > On Wed, Jan 25, 2017 at 3:30 PM, Stephen Frost wrote: > >> As it is, there are backup solutions which *do* check the checksum when > >> backing

Re: [HACKERS] Checksums by default?

* Peter Geoghegan (p...@heroku.com) wrote: > On Wed, Jan 25, 2017 at 3:30 PM, Stephen Frost wrote: > > As it is, there are backup solutions which *do* check the checksum when > > backing up PG. This is no longer, thankfully, some hypothetical thing, > > but something which

Re: [HACKERS] Checksums by default?

On Thu, Jan 26, 2017 at 9:22 AM, Andres Freund wrote: > On 2017-01-26 09:19:28 +0900, Michael Paquier wrote: >> On Thu, Jan 26, 2017 at 9:14 AM, Peter Geoghegan wrote: >> > On Wed, Jan 25, 2017 at 3:30 PM, Stephen Frost wrote: >> >> As it

Re: [HACKERS] Checksums by default?

On Wed, Jan 25, 2017 at 7:19 PM, Michael Paquier wrote: > On Thu, Jan 26, 2017 at 9:14 AM, Peter Geoghegan wrote: >> On Wed, Jan 25, 2017 at 3:30 PM, Stephen Frost wrote: >>> As it is, there are backup solutions which *do* check

Re: [HACKERS] Checksums by default?

On Wed, Jan 25, 2017 at 6:30 PM, Stephen Frost wrote: > I hope to discuss it further after we have the ability to turn it off > easily. I think we should have the ability to flip it in BOTH directions easily. >> Second, really hard to enable is a relative term. I accept

Re: [HACKERS] Checksums by default?

On 2017-01-26 09:19:28 +0900, Michael Paquier wrote: > On Thu, Jan 26, 2017 at 9:14 AM, Peter Geoghegan wrote: > > On Wed, Jan 25, 2017 at 3:30 PM, Stephen Frost wrote: > >> As it is, there are backup solutions which *do* check the checksum when > >> backing

Re: [HACKERS] Checksums by default?

On Thu, Jan 26, 2017 at 9:14 AM, Peter Geoghegan wrote: > On Wed, Jan 25, 2017 at 3:30 PM, Stephen Frost wrote: >> As it is, there are backup solutions which *do* check the checksum when >> backing up PG. This is no longer, thankfully, some hypothetical

Re: [HACKERS] Checksums by default?

On Wed, Jan 25, 2017 at 3:30 PM, Stephen Frost wrote: > As it is, there are backup solutions which *do* check the checksum when > backing up PG. This is no longer, thankfully, some hypothetical thing, > but something which really exists and will hopefully keep users from >

Re: [HACKERS] Checksums by default?

Robert, * Robert Haas (robertmh...@gmail.com) wrote: > On Wed, Jan 25, 2017 at 2:23 PM, Stephen Frost wrote: > > Yet, our default is to have them disabled and *really* hard to enable. > > First of all, that could be fixed by further development. I'm certainly all for doing

Re: [HACKERS] Checksums by default?

On Wed, Jan 25, 2017 at 12:23 PM, Robert Haas wrote: > Also, I think that one of the big problems with the way checksums work > is that you don't find problems with your archived data until it's too > late. Suppose that in February bits get flipped in a block. You > don't

Re: [HACKERS] Checksums by default?

On Wed, Jan 25, 2017 at 2:23 PM, Stephen Frost wrote: >> Sure. If the database runs fast enough with checksums enabled, >> there's basically no reason to have them turned off. The issue is >> when it doesn't. > > I don't believe we're talking about forcing every user to have

Re: [HACKERS] Checksums by default?

On Wed, Jan 25, 2017 at 8:18 PM, Robert Haas wrote: > Also, it's not as if there are no other ways of checking whether your > disks are failing. SMART, for example, is supposed to tell you about > incipient hardware failures before PostgreSQL ever sees a bit flip. > Surely

Re: [HACKERS] Checksums by default?

On 01/25/2017 11:41 AM, Tom Lane wrote: Stephen Frost writes: Would you say that most user's databases run fast enough with checksums enabled? Or more than most, maybe 70%? 80%? In today's environment, I'd probably say that it's more like 90+%. It would be nice if

Re: [HACKERS] Checksums by default?

Stephen Frost writes: > Would you say that most user's databases run fast enough with checksums > enabled? Or more than most, maybe 70%? 80%? In today's environment, > I'd probably say that it's more like 90+%. It would be nice if there were some actual evidence about

Re: [HACKERS] Checksums by default?

* Peter Geoghegan (p...@heroku.com) wrote: > On Wed, Jan 25, 2017 at 10:18 AM, Robert Haas wrote: > > Trying to force those people to use checksums is just masterminding; > > they've made their own decision that it's not worth bothering with. > > When something goes wrong,

Re: [HACKERS] Checksums by default?

Robert, * Robert Haas (robertmh...@gmail.com) wrote: > On Wed, Jan 25, 2017 at 12:02 AM, Jim Nasby wrote: > > I'm not completely grokking your second paragraph, but I would think that an > > average user would love got get a heads-up that their hardware is failing. > >

Re: [HACKERS] Checksums by default?

On Wed, Jan 25, 2017 at 1:37 PM, Peter Geoghegan wrote: > On Wed, Jan 25, 2017 at 10:18 AM, Robert Haas wrote: >> Trying to force those people to use checksums is just masterminding; >> they've made their own decision that it's not worth bothering with. >>

Re: [HACKERS] Checksums by default?

On Wed, Jan 25, 2017 at 10:18 AM, Robert Haas wrote: > Trying to force those people to use checksums is just masterminding; > they've made their own decision that it's not worth bothering with. > When something goes wrong, WE still care about distinguishing hardware >

Re: [HACKERS] Checksums by default?

On Wed, Jan 25, 2017 at 12:02 AM, Jim Nasby wrote: > I'm not completely grokking your second paragraph, but I would think that an > average user would love got get a heads-up that their hardware is failing. Sure. If the database runs fast enough with checksums enabled,

Re: [HACKERS] Checksums by default?

On Sat, Jan 21, 2017 at 11:57 AM, Andres Freund wrote: > On 2017-01-21 11:39:18 +0100, Magnus Hagander wrote: >> Is it time to enable checksums by default, and give initdb a switch to turn >> it off instead? > > -1 - the WAL overhead is quite massive, and in contrast to the

Re: [HACKERS] Checksums by default?

On 1/24/17 10:30 AM, Joshua D. Drake wrote: Tom is correct here. They are not a net win for the average user. We tend to forget that although we collectively have a lot of enterprise installs where this does matter, we collectively do not equal near the level of average user installs. From an

Re: [HACKERS] Checksums by default?

On 01/21/2017 09:09 AM, Tom Lane wrote: Stephen Frost writes: As for checksums, I do see value in them and I'm pretty sure that the author of that particular feature did as well, or we wouldn't even have it as an option. You seem to be of the opinion that we might as well

Re: [HACKERS] Checksums by default?

On 21.01.2017 19:35, Tom Lane wrote: Andres Freund writes: Sure, it might be easy, but we don't have it. Personally I think checksums just aren't even ready for prime time. If we had: - ability to switch on/off at runtime (early patches for that have IIRC been posted) -

Re: [HACKERS] Checksums by default?

On 21.01.2017 19:37, Stephen Frost wrote: * Tom Lane (t...@sss.pgh.pa.us) wrote: Stephen Frost writes: Because I see having checksums as, frankly, something we always should have had (as most other databases do, for good reason...) and because they will hopefully prevent

Re: [HACKERS] Checksums by default?

Greetings, * Ants Aasma (ants.aa...@eesti.ee) wrote: > On Tue, Jan 24, 2017 at 4:07 AM, Tom Lane wrote: > > Peter Geoghegan writes: > >> I thought that checksums went in in part because we thought that there > >> was some chance that they'd find bugs in

Re: [HACKERS] Checksums by default?

On Tue, Jan 24, 2017 at 4:07 AM, Tom Lane wrote: > Peter Geoghegan writes: >> I thought that checksums went in in part because we thought that there >> was some chance that they'd find bugs in Postgres. > > Not really. AFAICS the only point is to catch

Re: [HACKERS] Checksums by default?

On 2017-01-23 21:11:37 -0600, Merlin Moncure wrote: > End user data damage ought to prevented at all costs IMO. Really, really, really not. We should do a lot, but if that'd be the only priority we'd enable all the expensive as shit stuff and be so slow that there'd be no users. We'd never add

Re: [HACKERS] Checksums by default?

Merlin Moncure writes: > Hm, but at least in some cases wouldn't it protect people from further > damage? End user data damage ought to prevented at all costs IMO. Well ... not directly. Disallowing you from accessing busted block A doesn't in itself prevent the same thing

Re: [HACKERS] Checksums by default?

On Mon, Jan 23, 2017 at 8:07 PM, Tom Lane wrote: > Peter Geoghegan writes: >> I thought that checksums went in in part because we thought that there >> was some chance that they'd find bugs in Postgres. > > Not really. AFAICS the only point is to catch

Re: [HACKERS] Checksums by default?

On 2017-01-23 21:40:53 -0500, Stephen Frost wrote: > Perhaps I'm missing something here, but with checksums enabled, a hint > bit update is going to dirty the page (and we're going to write it into > the WAL and write it out to the heap), no? No. We only WAL log hint bits the first time a page

Re: [HACKERS] Checksums by default?

Stephen Frost writes: > * Tom Lane (t...@sss.pgh.pa.us) wrote: >> But we don't maintain the checksum of a page while it sits in shared >> buffers. Trying to do so would break, eg, concurrent hint-bit updates. > Hence why I said 'clean' pages.. When we write out a page, we

Re: [HACKERS] Checksums by default?

* Tom Lane (t...@sss.pgh.pa.us) wrote: > Jim Nasby writes: > > On 1/23/17 7:47 PM, Stephen Frost wrote: > >> It might be interesting to consider checking them in 'clean' pages in > >> shared_buffers in a background process, as that, presumably, *would* > >> detect shared

Re: [HACKERS] Checksums by default?

On 1/23/17 8:24 PM, Tom Lane wrote: Jim Nasby writes: On 1/23/17 7:47 PM, Stephen Frost wrote: It might be interesting to consider checking them in 'clean' pages in shared_buffers in a background process, as that, presumably, *would* detect shared buffers corruption.

Re: [HACKERS] Checksums by default?

Jim Nasby writes: > On 1/23/17 7:47 PM, Stephen Frost wrote: >> It might be interesting to consider checking them in 'clean' pages in >> shared_buffers in a background process, as that, presumably, *would* >> detect shared buffers corruption. > Hmm... that would be

Re: [HACKERS] Checksums by default?

On 1/23/17 7:47 PM, Stephen Frost wrote: It might be interesting to consider checking them in 'clean' pages in shared_buffers in a background process, as that, presumably, *would* detect shared buffers corruption. Hmm... that would be interesting. Assuming the necessary functions are exposed

Re: [HACKERS] Checksums by default?

Peter Geoghegan writes: > I thought that checksums went in in part because we thought that there > was some chance that they'd find bugs in Postgres. Not really. AFAICS the only point is to catch storage-system malfeasance. It's barely possible that checksumming would help

Re: [HACKERS] Checksums by default?

On Mon, Jan 23, 2017 at 6:01 PM, Tom Lane wrote: > Maybe this is a terminology problem. I'm taking "false positive" to mean > "checksum reports a failure, but in fact there is no observable data > corruption". Depending on why the false positive occurred, that might > help

Re: [HACKERS] Checksums by default?

Peter Geoghegan writes: > Perhaps I've missed the point entirely, but, I have to ask: How could > there ever be false positives? Bugs. For example, checksum is computed while somebody else is setting a hint bit in the page, so that what is written out is completely valid except

Re: [HACKERS] Checksums by default?

On Mon, Jan 23, 2017 at 5:47 PM, Stephen Frost wrote: >> Perhaps I've missed the point entirely, but, I have to ask: How could >> there ever be false positives? With checksums, false positives are >> simply not allowed. Therefore, there cannot be a false positive, >> unless we

Re: [HACKERS] Checksums by default?

On 24/01/17 02:39, Michael Paquier wrote: > On Tue, Jan 24, 2017 at 10:26 AM, Stephen Frost wrote: >> * Tom Lane (t...@sss.pgh.pa.us) wrote: > I don't recall ever seeing a checksum failure on a Heroku Postgres > database, >> >> Not sure how this part of that sentence

Re: [HACKERS] Checksums by default?

* Peter Geoghegan (p...@heroku.com) wrote: > On Mon, Jan 23, 2017 at 5:26 PM, Stephen Frost wrote: > > Not sure how this part of that sentence was missed: > > > > - > > ... even though they were enabled as soon as the feature became > > available. > > - > > > > Which

Re: [HACKERS] Checksums by default?

On Tue, Jan 24, 2017 at 10:26 AM, Stephen Frost wrote: > * Tom Lane (t...@sss.pgh.pa.us) wrote: >> >> I don't recall ever seeing a checksum failure on a Heroku Postgres >> >> database, > > Not sure how this part of that sentence was missed: > > - > ... even though they

Re: [HACKERS] Checksums by default?

On Mon, Jan 23, 2017 at 5:26 PM, Stephen Frost wrote: > Not sure how this part of that sentence was missed: > > - > ... even though they were enabled as soon as the feature became > available. > - > > Which would seem to me to say "the code's been running for a long

Re: [HACKERS] Checksums by default?

On 1/23/17 7:15 PM, Tom Lane wrote: Uhm, Peter G just said that Heroku enables this on all their databases and have yet to see a false-positive report or an issue with having it enabled. That, plus the reports and evidence we've seen in the past couple days, look like a pretty ringing

Re: [HACKERS] Checksums by default?

Jim, * Jim Nasby (jim.na...@bluetreble.com) wrote: > On 1/23/17 6:55 PM, Stephen Frost wrote: > >* Jim Nasby (jim.na...@bluetreble.com) wrote: > >>As others have mentioned, right now practically no one enables this, > >>so we've got zero data on how useful it might actually be. > >Uhm, Peter G

Re: [HACKERS] Checksums by default?

* Tom Lane (t...@sss.pgh.pa.us) wrote: > Stephen Frost writes: > > * Jim Nasby (jim.na...@bluetreble.com) wrote: > >> As others have mentioned, right now practically no one enables this, > >> so we've got zero data on how useful it might actually be. > > > Uhm, Peter G just

Re: [HACKERS] Checksums by default?

On 1/23/17 6:55 PM, Stephen Frost wrote: * Jim Nasby (jim.na...@bluetreble.com) wrote: As others have mentioned, right now practically no one enables this, so we've got zero data on how useful it might actually be. Uhm, Peter G just said that Heroku enables this on all their databases and have

Re: [HACKERS] Checksums by default?

Stephen Frost writes: > * Jim Nasby (jim.na...@bluetreble.com) wrote: >> As others have mentioned, right now practically no one enables this, >> so we've got zero data on how useful it might actually be. > Uhm, Peter G just said that Heroku enables this on all their databases

Re: [HACKERS] Checksums by default?

Jim, * Jim Nasby (jim.na...@bluetreble.com) wrote: > As others have mentioned, right now practically no one enables this, > so we've got zero data on how useful it might actually be. Uhm, Peter G just said that Heroku enables this on all their databases and have yet to see a false-positive

Re: [HACKERS] Checksums by default?

On 1/23/17 6:14 PM, Peter Geoghegan wrote: In practice, Postgres checksums do *not* seem to catch problems. That's been my experience, at least. For someone running on a bunch of AWS hardware that doesn't really surprise me. Presumably, anyone operating at that scale would be quickly

Re: [HACKERS] Checksums by default?

On 1/23/17 1:30 AM, Amit Kapila wrote: On Sun, Jan 22, 2017 at 3:43 PM, Tomas Vondra wrote: That being said, I'm ready to do some benchmarking on this, so that we have at least some numbers to argue about. Can we agree on a set of workloads that we want to

Re: [HACKERS] Checksums by default?

On Sat, Jan 21, 2017 at 12:35 PM, Tom Lane wrote: > Andres Freund writes: >> Sure, it might be easy, but we don't have it. Personally I think >> checksums just aren't even ready for prime time. If we had: >> - ability to switch on/off at runtime (early

Re: [HACKERS] Checksums by default?

On Sat, Jan 21, 2017 at 9:09 AM, Tom Lane wrote: > Not at all; I just think that it's not clear that they are a net win > for the average user, and so I'm unconvinced that turning them on by > default is a good idea. I could be convinced otherwise by suitable > evidence.

Re: [HACKERS] Checksums by default?

On Mon, Jan 23, 2017 at 6:57 PM, Tomas Vondra wrote: > On 01/23/2017 01:40 PM, Amit Kapila wrote: >> >> On Mon, Jan 23, 2017 at 3:56 PM, Tomas Vondra >> wrote: >>> >>> On 01/23/2017 09:57 AM, Amit Kapila wrote: On Mon,

Re: [HACKERS] Checksums by default?

On 01/23/2017 01:40 PM, Amit Kapila wrote: On Mon, Jan 23, 2017 at 3:56 PM, Tomas Vondra wrote: On 01/23/2017 09:57 AM, Amit Kapila wrote: On Mon, Jan 23, 2017 at 1:18 PM, Tomas Vondra wrote: On 01/23/2017 08:30 AM, Amit Kapila

Re: [HACKERS] Checksums by default?

On Mon, Jan 23, 2017 at 3:56 PM, Tomas Vondra wrote: > On 01/23/2017 09:57 AM, Amit Kapila wrote: >> >> On Mon, Jan 23, 2017 at 1:18 PM, Tomas Vondra >> wrote: >>> >>> On 01/23/2017 08:30 AM, Amit Kapila wrote: I

Re: [HACKERS] Checksums by default?

On 01/23/2017 09:57 AM, Amit Kapila wrote: On Mon, Jan 23, 2017 at 1:18 PM, Tomas Vondra wrote: On 01/23/2017 08:30 AM, Amit Kapila wrote: I think if we can get data for pgbench read-write workload when data doesn't fit in shared buffers but fit in RAM, that

Re: [HACKERS] Checksums by default?

On Mon, Jan 23, 2017 at 1:18 PM, Tomas Vondra wrote: > On 01/23/2017 08:30 AM, Amit Kapila wrote: >> >> >> I think if we can get data for pgbench read-write workload when data >> doesn't fit in shared buffers but fit in RAM, that can give us some >> indication. We

Re: [HACKERS] Checksums by default?

On 01/23/2017 08:30 AM, Amit Kapila wrote: On Sun, Jan 22, 2017 at 3:43 PM, Tomas Vondra wrote: That being said, I'm ready to do some benchmarking on this, so that we have at least some numbers to argue about. Can we agree on a set of workloads that we want to

Re: [HACKERS] Checksums by default?

On Sun, Jan 22, 2017 at 3:43 PM, Tomas Vondra wrote: > > That being said, I'm ready to do some benchmarking on this, so that we have > at least some numbers to argue about. Can we agree on a set of workloads > that we want to benchmark in the first round? > I think

Re: [HACKERS] Checksums by default?

> 21 янв. 2017 г., в 18:18, Petr Jelinek > написал(а): > > On 21/01/17 11:39, Magnus Hagander wrote: >> Is it time to enable checksums by default, and give initdb a switch to >> turn it off instead? +1 > > I'd like to see benchmark first, both in terms of CPU

Re: [HACKERS] Checksums by default?

From: pgsql-hackers-ow...@postgresql.org > [mailto:pgsql-hackers-ow...@postgresql.org] On Behalf Of Magnus Hagander > Is it time to enable checksums by default, and give initdb a switch to turn > it off instead? > > I keep running into situations where people haven't enabled it, because > (a)

Re: [HACKERS] Checksums by default?

On Sat, Jan 21, 2017 at 8:16 PM, Ants Aasma wrote: > On Sat, Jan 21, 2017 at 7:39 PM, Petr Jelinek > wrote: > > So in summary "postgresql.conf options are easy to change" while "initdb > > options are hard to change", I can see this argument

Re: [HACKERS] Checksums by default?

On 01/21/2017 04:18 PM, Petr Jelinek wrote: On 21/01/17 11:39, Magnus Hagander wrote: Is it time to enable checksums by default, and give initdb a switch to turn it off instead? I'd like to see benchmark first, both in terms of CPU and in terms of produced WAL (=network traffic) given that it

  1   2   >