Re: [HACKERS] Avoid full page images in streaming replication?

--On 22. Oktober 2015 22:23:58 -0300 Alvaro Herrera wrote: >> You can? The xlog format between 9.4 and 9.5 changed, so I can't see how >> that'd work? > > Oh, crap. Must have been some other cross-version trial run I did, > then. I would hope it's at least not

Re: [HACKERS] Avoid full page images in streaming replication?

--On 23. Oktober 2015 00:03:30 +0200 Andres Freund wrote: > > Note that FPIs are often pretty good for replay performance, avoiding > lots of synchronous random reads. That's a very import argument, i think. The difference can be significant, even if you have a decent

Re: [HACKERS] Avoid full page images in streaming replication?

On 10/22/15 4:42 PM, Magnus Hagander wrote: > How are you going to make that work without LSNs in the WAL received by > the replica diverging from those in the master's WAL? > We could in theory send a "this would be been a fpi but it's skipped" record which would only exist in streaming and

Re: [HACKERS] Avoid full page images in streaming replication?

On 2015-10-22 16:34:38 -0500, Jim Nasby wrote: > ISTM it should be possible to avoid sending full page writes to a streaming > replica once the replica has reached a consistent state. I assume that the > replica would still need to write full pages to it's disk in case of a > crash, but the sender

Re: [HACKERS] Avoid full page images in streaming replication?

Jim Nasby writes: > ISTM it should be possible to avoid sending full page writes to a > streaming replica once the replica has reached a consistent state. I > assume that the replica would still need to write full pages to it's > disk in case of a crash, but the

Re: [HACKERS] Avoid full page images in streaming replication?

On Oct 22, 2015 23:38, "Tom Lane" wrote: > > Jim Nasby writes: > > ISTM it should be possible to avoid sending full page writes to a > > streaming replica once the replica has reached a consistent state. I > > assume that the replica would still need

Re: [HACKERS] Avoid full page images in streaming replication?

On Thu, Oct 22, 2015 at 5:57 PM, Jim Nasby wrote: >> We could in theory send a "this would be been a fpi but it's skipped" >> record which would only exist in streaming and just make the standby >> write a noop of some kind? It would still be on the standby but it would

Re: [HACKERS] Avoid full page images in streaming replication?

On 10/22/15 5:03 PM, Andres Freund wrote: On 2015-10-22 16:34:38 -0500, Jim Nasby wrote: ISTM it should be possible to avoid sending full page writes to a streaming replica once the replica has reached a consistent state. I assume that the replica would still need to write full pages to it's

Re: [HACKERS] Avoid full page images in streaming replication?

On 10/22/15 5:11 PM, Robert Haas wrote: It's true that if the standby didn't have the master's FPIs, it could generate its own in some side location that behaves like a separate WAL stream or a double-write buffer. But that would be a heck of a lot of work to implement for an uncertain benefit.

Re: [HACKERS] Avoid full page images in streaming replication?

On 10/22/15 5:53 PM, Alvaro Herrera wrote: Jim Nasby wrote: But yes, this is all very hand-wavy without any actual data on what percentage of the WAL stream is FPIs. Looks like pageinspect doesn't work for WAL... does anyone have a script/tool that breaks out what percentage of a WAL file is

Re: [HACKERS] Avoid full page images in streaming replication?

Jim Nasby wrote: > On 10/22/15 5:53 PM, Alvaro Herrera wrote: > >Jim Nasby wrote: > > > >>But yes, this is all very hand-wavy without any actual data on what > >>percentage of the WAL stream is FPIs. Looks like pageinspect doesn't work > >>for WAL... does anyone have a script/tool that breaks out

Re: [HACKERS] Avoid full page images in streaming replication?

On 2015-10-22 20:27:20 -0300, Alvaro Herrera wrote: > You can run the new one in old pg_xlog ... You can? The xlog format between 9.4 and 9.5 changed, so I can't see how that'd work? -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription:

Re: [HACKERS] Avoid full page images in streaming replication?

On Fri, Oct 23, 2015 at 7:37 AM, Jim Nasby wrote: > What I'm wondering is how compressible a 'normal' FPI is. Certainly if the > hole is zero'd out and the page is mostly empty you'll get great > compression. What about other workloads? For reference, if a 'FPI > placeholder' WAL record is 16

Re: [HACKERS] Avoid full page images in streaming replication?

On Fri, Oct 23, 2015 at 8:42 AM, Andres Freund wrote: > On 2015-10-22 20:27:20 -0300, Alvaro Herrera wrote: >> You can run the new one in old pg_xlog ... > > You can? The xlog format between 9.4 and 9.5 changed, so I can't see how > that'd work? That's not going to work. --

Re: [HACKERS] Avoid full page images in streaming replication?

On 2015-10-22 17:47:01 -0500, Jim Nasby wrote: > On 10/22/15 5:11 PM, Robert Haas wrote: > >It's true that if the standby didn't have the master's FPIs, it could > >generate its own in some side location that behaves like a separate > >WAL stream or a double-write buffer. But that would be a heck

Re: [HACKERS] Avoid full page images in streaming replication?

Jim Nasby wrote: > But yes, this is all very hand-wavy without any actual data on what > percentage of the WAL stream is FPIs. Looks like pageinspect doesn't work > for WAL... does anyone have a script/tool that breaks out what percentage of > a WAL file is FPIs? pg_xlogdump -z -- Álvaro

Re: [HACKERS] Avoid full page images in streaming replication?

On 10/22/15 5:52 PM, Andres Freund wrote: If the receiver didn't write the WAL before processing it then it can just >stick the page image into the WAL it's writing for itself. Probably not good >for syncrep, but I don't think you'd want this on for syncrep anyway. To me this sounds like a

Re: [HACKERS] Avoid full page images in streaming replication?

Andres Freund wrote: > On 2015-10-22 20:27:20 -0300, Alvaro Herrera wrote: > > You can run the new one in old pg_xlog ... > > You can? The xlog format between 9.4 and 9.5 changed, so I can't see how > that'd work? Oh, crap. Must have been some other cross-version trial run I did, then. I would

Re: [HACKERS] Avoid full page images in streaming replication?

On 2015-10-22 17:59:06 -0500, Jim Nasby wrote: > The WAL would *not* differ. It would. Hint bits and all. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers