Re: [HACKERS] WIP: long transactions on hot standby feedback replica / proof of concept

2019-04-22 Thread Alexander Korotkov
On Thu, Nov 29, 2018 at 3:44 PM Dmitry Dolgov <9erthali...@gmail.com> wrote: > > > On Fri, Aug 24, 2018 at 5:53 PM Alexander Korotkov > > wrote: > > > > Given I've no feedback on this idea yet, I'll try to implement a PoC > > patch for that. It doesn't look to be difficult. And we'll see how >

Re: [HACKERS] WIP: long transactions on hot standby feedback replica / proof of concept

2019-01-31 Thread Andres Freund
Hi, On 2018-11-29 13:45:15 +0100, Dmitry Dolgov wrote: > > On Fri, Aug 24, 2018 at 5:53 PM Alexander Korotkov > > wrote: > > > > Given I've no feedback on this idea yet, I'll try to implement a PoC > > patch for that. It doesn't look to be difficult. And we'll see how > > does it work. > > Un

Re: [HACKERS] WIP: long transactions on hot standby feedback replica / proof of concept

2018-11-29 Thread Dmitry Dolgov
> On Fri, Aug 24, 2018 at 5:53 PM Alexander Korotkov > wrote: > > Given I've no feedback on this idea yet, I'll try to implement a PoC > patch for that. It doesn't look to be difficult. And we'll see how > does it work. Unfortunately, current version of the patch doesn't pass the tests and fai

Re: [HACKERS] WIP: long transactions on hot standby feedback replica / proof of concept

2018-09-02 Thread Robert Haas
On Mon, Aug 27, 2018 at 11:38 AM, Alexander Korotkov wrote: > The aspect I'm more concerned here about is whether we miss ability > for detecting some of IO errors, if we don't distinguish new pages > from pages whose tuples were removed by vacuum. My main concern is correctness. If we ever have

Re: [HACKERS] WIP: long transactions on hot standby feedback replica / proof of concept

2018-08-27 Thread Alexander Korotkov
Hi! Thank you for feedback. On Sun, Aug 26, 2018 at 4:09 AM Robert Haas wrote: > On Tue, Aug 21, 2018 at 9:10 AM, Alexander Korotkov > wrote: > > After heap truncation using this algorithm, shared buffers may contain > > past-OEF buffers. But those buffers are empty (no used items) and > > cle

Re: [HACKERS] WIP: long transactions on hot standby feedback replica / proof of concept

2018-08-25 Thread Robert Haas
On Tue, Aug 21, 2018 at 9:10 AM, Alexander Korotkov wrote: > After heap truncation using this algorithm, shared buffers may contain > past-OEF buffers. But those buffers are empty (no used items) and > clean. So, real-only queries shouldn't hint those buffers dirty > because there are no used it

Re: [HACKERS] WIP: long transactions on hot standby feedback replica / proof of concept

2018-08-24 Thread Alexander Korotkov
On Tue, Aug 21, 2018 at 4:10 PM Alexander Korotkov wrote: > After reading [1] and [2] I got that there are at least 3 different > issues with heap truncation: > 1) Data corruption on file truncation error (explained in [1]). > 2) Expensive scanning of the whole shared buffers before file truncatio

Re: [HACKERS] WIP: long transactions on hot standby feedback replica / proof of concept

2018-08-21 Thread Alexander Korotkov
On Fri, Aug 17, 2018 at 9:55 PM Tom Lane wrote: > But then you are injecting bad pages into the shared buffer arena. > In any case, depending on that behavior seems like a bad idea, because > it's a pretty questionable kluge in itself. > > Another point is that the truncation code attempts to remo

Re: [HACKERS] WIP: long transactions on hot standby feedback replica / proof of concept

2018-08-17 Thread Tom Lane
Alexander Korotkov writes: > On Fri, Aug 17, 2018 at 9:55 PM Tom Lane wrote: >> Another point is that the truncation code attempts to remove all >> to-be-truncated-away pages from the shared buffer arena, but that only >> works if nobody else is loading such pages into shared buffers >> concurren

Re: [HACKERS] WIP: long transactions on hot standby feedback replica / proof of concept

2018-08-17 Thread Alexander Korotkov
On Fri, Aug 17, 2018 at 9:55 PM Tom Lane wrote: > Alexander Korotkov writes: > > On Fri, Aug 17, 2018 at 8:38 PM Tom Lane wrote: > >> Alexander Korotkov writes: > >>> Yes, that's correct. On standby read-only queries can tolerate > >>> concurrent heap truncation. > > >> Uh, what??? > > > VACUU

Re: [HACKERS] WIP: long transactions on hot standby feedback replica / proof of concept

2018-08-17 Thread Tom Lane
Alexander Korotkov writes: > On Fri, Aug 17, 2018 at 8:38 PM Tom Lane wrote: >> Alexander Korotkov writes: >>> Yes, that's correct. On standby read-only queries can tolerate >>> concurrent heap truncation. >> Uh, what??? > VACUUM truncates heap relation only after deletion of all the tuples >

Re: [HACKERS] WIP: long transactions on hot standby feedback replica / proof of concept

2018-08-17 Thread Alexander Korotkov
On Fri, Aug 17, 2018 at 8:38 PM Tom Lane wrote: > Alexander Korotkov writes: > > On Fri, Aug 17, 2018 at 6:41 PM Andres Freund wrote: > >> There's another patch, which I thought Alexander was referring to, that > >> does something a bit smarger. On a super short skim it seems to > >> introduce

Re: [HACKERS] WIP: long transactions on hot standby feedback replica / proof of concept

2018-08-17 Thread Tom Lane
Alexander Korotkov writes: > On Fri, Aug 17, 2018 at 6:41 PM Andres Freund wrote: >> There's another patch, which I thought Alexander was referring to, that >> does something a bit smarger. On a super short skim it seems to >> introduce a separate type of AEL lock that's not replicated, by my >>

Re: [HACKERS] WIP: long transactions on hot standby feedback replica / proof of concept

2018-08-17 Thread Alexander Korotkov
On Fri, Aug 17, 2018 at 6:41 PM Andres Freund wrote: > On 2018-08-17 11:35:40 -0400, Tom Lane wrote: > > Andres Freund writes: > > > On 2018-08-17 18:00:20 +0300, Alexander Korotkov wrote: > > >> So, do we have any objections to committing this? > > > > > I think this needs more review by other s

Re: [HACKERS] WIP: long transactions on hot standby feedback replica / proof of concept

2018-08-17 Thread Andres Freund
On 2018-08-17 11:35:40 -0400, Tom Lane wrote: > Andres Freund writes: > > On 2018-08-17 18:00:20 +0300, Alexander Korotkov wrote: > >> So, do we have any objections to committing this? > > > I think this needs more review by other senior hackers in the community. > > TBH it sounds like a horribl

Re: [HACKERS] WIP: long transactions on hot standby feedback replica / proof of concept

2018-08-17 Thread Tom Lane
Andres Freund writes: > On 2018-08-17 18:00:20 +0300, Alexander Korotkov wrote: >> So, do we have any objections to committing this? > I think this needs more review by other senior hackers in the community. TBH it sounds like a horrible hack. Disable vacuum truncation? That can't be a good ide

Re: [HACKERS] WIP: long transactions on hot standby feedback replica / proof of concept

2018-08-17 Thread Andres Freund
Hi, On 2018-08-17 18:00:20 +0300, Alexander Korotkov wrote: > So, do we have any objections to committing this? I think this needs more review by other senior hackers in the community. Greetings, Andres Freund

Re: [HACKERS] WIP: long transactions on hot standby feedback replica / proof of concept

2018-08-17 Thread Alexander Korotkov
On Thu, Aug 16, 2018 at 2:16 PM Alexander Korotkov wrote: > On Tue, Aug 14, 2018 at 12:05 PM Masahiko Sawada > wrote: > > > > On Wed, Feb 28, 2018 at 11:24 PM, Ivan Kartyshov > > wrote: > > > The main goal of my changes is to let long read-only transactions run on > > > replica if hot_standby_f

Re: [HACKERS] WIP: long transactions on hot standby feedback replica / proof of concept

2018-08-16 Thread Alexander Korotkov
Hi! On Tue, Aug 14, 2018 at 12:05 PM Masahiko Sawada wrote: > > On Wed, Feb 28, 2018 at 11:24 PM, Ivan Kartyshov > wrote: > > The main goal of my changes is to let long read-only transactions run on > > replica if hot_standby_feedback is turned on. > > FWIW the idea proposed on the thread[1], wh

Re: [HACKERS] WIP: long transactions on hot standby feedback replica / proof of concept

2018-08-14 Thread Masahiko Sawada
On Wed, Feb 28, 2018 at 11:24 PM, Ivan Kartyshov wrote: > Thank you for your valuable comments. I've made a few adjustments. > Thank you for working on this! > The main goal of my changes is to let long read-only transactions run on > replica if hot_standby_feedback is turned on. FWIW the idea

Re: [HACKERS] WIP: long transactions on hot standby feedback replica / proof of concept

2018-08-11 Thread Alexander Korotkov
Hi! On Fri, Aug 10, 2018 at 5:07 PM Alexander Korotkov wrote: > On Thu, Aug 9, 2018 at 11:26 PM Ivan Kartyshov > wrote: > > Alexander Korotkov писал 2018-06-20 20:54: > > > Thinking about that more I found that adding vacuum mark as an extra > > > argument to LockAcquireExtended is also wrong.

Re: [HACKERS] WIP: long transactions on hot standby feedback replica / proof of concept

2018-08-10 Thread Alexander Korotkov
Hi! On Thu, Aug 9, 2018 at 11:26 PM Ivan Kartyshov wrote: > Alexander Korotkov писал 2018-06-20 20:54: > > Thinking about that more I found that adding vacuum mark as an extra > > argument to LockAcquireExtended is also wrong. It would be still hard > > to determine if we should log the lock in

Re: [HACKERS] WIP: long transactions on hot standby feedback replica / proof of concept

2018-08-09 Thread Ivan Kartyshov
Hello, thank you for your review. Alexander Korotkov писал 2018-06-20 20:54: Thinking about that more I found that adding vacuum mark as an extra argument to LockAcquireExtended is also wrong. It would be still hard to determine if we should log the lock in LogStandbySnapshot(). We'll have to

Re: [HACKERS] WIP: long transactions on hot standby feedback replica / proof of concept

2018-07-29 Thread Thomas Munro
On Thu, Mar 1, 2018 at 3:24 AM, Ivan Kartyshov wrote: > Could you give me your ideas over these patches. Hi Ivan, Not sure if this is expected at this stage or not, but just in case you didn't know... the tests under src/test/subscription seem to be broken: https://travis-ci.org/postgresql-cfbo

Re: [HACKERS] WIP: long transactions on hot standby feedback replica / proof of concept

2018-06-20 Thread Alexander Korotkov
Hi! Thank you for your work on this issue. On Wed, Feb 28, 2018 at 5:25 PM Ivan Kartyshov wrote: > Thank you for your valuable comments. I've made a few adjustments. > > The main goal of my changes is to let long read-only transactions run on > replica if hot_standby_feedback is turned on. > > >

Re: [HACKERS] WIP: long transactions on hot standby feedback replica / proof of concept

2018-02-28 Thread Ivan Kartyshov
Thank you for your valuable comments. I've made a few adjustments. The main goal of my changes is to let long read-only transactions run on replica if hot_standby_feedback is turned on. Patch1 - hsfeedback_av_truncate.patch is made to stop ResolveRecoveryConflictWithLock occurs on replica, a