Re: Issue with server side statement-level rollback

2020-11-22 Thread Gilles Darold
ommand() and AbortCurrentTransaction() to implement the statement-level rollback feature entirely driven at serverside. It require that the patch [2]  I've provided be applied on PostgreSQL source first. Here is what can be achieved with this patch: LOAD 'pg_statement_rollback.so&#x

Re: Issue with server side statement-level rollback

2020-11-20 Thread Gilles Darold
hooks / integration you'd like would look like, it's hard to comment usefully here. We have implemented an extension to allow server side "statement-level rollback" with what is possible to do now with PG but the objective was to do the same thing that what was proposed

Re: Issue with server side statement-level rollback

2020-11-19 Thread Andres Freund
Hi, On 2020-11-12 11:40:22 +0100, Gilles Darold wrote: > The problem we are encountering is when PostgreSQL is compiled in debug > mode with --enable-cassert. At line 1327 of src/backend/tcop/pquery.c > the following assert fail: > >     /* >      * Clear subsidiary contexts to recover temporary

Issue with server side statement-level rollback

2020-11-12 Thread Gilles Darold
id I am aware that this is clearly not a standard PostgreSQL use but we have no choice. We are emulating DB2 statement-level rollback behavior and we have chosen to not create a new fork of PostgreSQL and only work with extensions. As there is no hook or API that could allow a perfect server side

Re: Statement-level rollback

2019-02-12 Thread Michael Paquier
On Tue, Feb 12, 2019 at 07:13:45PM -0300, Alvaro Herrera wrote: > It was reasonable to close this patch as returned-with-feedback in > January commitfest, but what you did here was first move it to the March > commitfest and then close it as returned-with-feedback in the March > commitfest, which h

Re: Statement-level rollback

2019-02-12 Thread Alvaro Herrera
On 2019-Feb-04, Michael Paquier wrote: > On Thu, Jan 31, 2019 at 04:38:27AM -0800, Andres Freund wrote: > > Is this still in development? Or should we mark this as returned with > > feedback? > > Marked as returned with feedback, as it has already been two months. > If you have an updated patch s

Re: Statement-level rollback

2019-02-03 Thread Michael Paquier
On Thu, Jan 31, 2019 at 04:38:27AM -0800, Andres Freund wrote: > Is this still in development? Or should we mark this as returned with > feedback? Marked as returned with feedback, as it has already been two months. If you have an updated patch set, please feel free to resubmit. -- Michael signa

Re: Statement-level rollback

2019-01-31 Thread Alvaro Herrera
On 2019-Jan-31, Greg Stark wrote: > This looks like repeating the autocommit mistakes of the past. Yeah, this argument has been made before. Peter E argued against that: https://postgr.es/m/ea395aa8-5ac4-6bcd-366d-aab2ff2b0...@2ndquadrant.com > And based on that experience wouldn't the replacem

Re: Statement-level rollback

2019-01-31 Thread Greg Stark
On Fri 7 Dec 2018, 21:43 Robert Haas On Fri, Dec 7, 2018 at 3:34 PM Alvaro Herrera > wrote: > > Yeah, I agree that this downside is real. I think our only protection > > against that is to say "don't do that". Like any other tool, it has > > upsides and downsides; we shouldn't keep it away from

Re: Statement-level rollback

2019-01-31 Thread Andres Freund
On 2018-12-08 17:55:03 -0300, Alvaro Herrera wrote: > On 2018-Dec-08, Andres Freund wrote: > > > On 2018-12-08 17:10:27 -0300, Alvaro Herrera wrote: > > > > This is what patch 0001 does -- it's only allowed in the connection > > > string, or on ALTER USER / ALTER DATABASE. Setting it in > > > po

RE: Statement-level rollback

2018-12-09 Thread Tsunakawa, Takayuki
From: Andres Freund [mailto:and...@anarazel.de] > Isn't the realistic scenario for migrations that people will turn this > feature on on a more global basis? If they need to do per transaction choices, > that makes this much less useful for easy migrations. Agreed. My approach of per transaction

Re: Statement-level rollback

2018-12-08 Thread Alvaro Herrera
On 2018-Dec-08, Andres Freund wrote: > On 2018-12-08 17:10:27 -0300, Alvaro Herrera wrote: > > This is what patch 0001 does -- it's only allowed in the connection > > string, or on ALTER USER / ALTER DATABASE. Setting it in > > postgresql.conf is forbidden, as well as changing from transaction t

Re: Statement-level rollback

2018-12-08 Thread Andres Freund
Hi, On 2018-12-08 17:10:27 -0300, Alvaro Herrera wrote: > On 2018-Dec-07, Andres Freund wrote: > > > I think it could partially be addressed by not allowing to set it on the > > commandline, server config, etc. So the user would have to set it on a > > per-connection basis, potentially via the co

Re: Statement-level rollback

2018-12-08 Thread Alvaro Herrera
On 2018-Dec-07, Andres Freund wrote: > I think it could partially be addressed by not allowing to set it on the > commandline, server config, etc. So the user would have to set it on a > per-connection basis, potentially via the connection string. This is what patch 0001 does -- it's only allowed

Re: Statement-level rollback

2018-12-07 Thread Robert Haas
On Fri, Dec 7, 2018 at 3:50 PM Alvaro Herrera wrote: > On 2018-Dec-07, Robert Haas wrote: > > More generally, whether or not we should "keep something away from our > > users" really depends on how likely the upsides are to occur relative > > to the downsides. We don't try to keep users from runn

Re: Statement-level rollback

2018-12-07 Thread Robert Haas
On Fri, Dec 7, 2018 at 7:25 PM Alexander Korotkov wrote: > The first thing, which comes to the mind is undo log. When you have > undo log, then on new subtransaction you basically memorize the > current undo log position. If subtransaction rollbacks, you have to > just play back undo log until r

Re: Statement-level rollback

2018-12-07 Thread Alexander Korotkov
es one subtransaction per statement; > Andres says per-row referring to the case of one INSERT per row, so it's > still one statement. I'd like to add my 2 cents. I think usage of subtransaction per statement for statement level rollback is fair. It's unlikely we're goin

Re: Statement-level rollback

2018-12-07 Thread Andres Freund
Hi, On 2018-12-07 16:02:53 -0500, Tom Lane wrote: > Alvaro Herrera writes: > > Well, look at this from this point of view: EnterpriseDB implemented > > this because of customer demand (presumably). Fujitsu also implemented > > this for customers. The pgjdbc driver implemented this for its users

Re: Statement-level rollback

2018-12-07 Thread Tom Lane
Alvaro Herrera writes: > Well, look at this from this point of view: EnterpriseDB implemented > this because of customer demand (presumably). Fujitsu also implemented > this for customers. The pgjdbc driver implemented this for its users. > Now 2ndQuadrant also implemented this, and not out of t

Re: Statement-level rollback

2018-12-07 Thread Tom Lane
Robert Haas writes: > I have a hard time arguing against that given that EDB has this thing > in our bag of tricks, but if it weren't for that I'd be fighting > against this tooth and nail. Behavior-changing GUCs sck. Uh, we're not seriously considering a GUC that changes transactional behav

Re: Statement-level rollback

2018-12-07 Thread Joshua D. Drake
On 12/7/18 12:50 PM, Alvaro Herrera wrote: On 2018-Dec-07, Robert Haas wrote: More generally, whether or not we should "keep something away from our users" really depends on how likely the upsides are to occur relative to the downsides. We don't try to keep users from running DELETE because th

Re: Statement-level rollback

2018-12-07 Thread Alvaro Herrera
On 2018-Dec-07, Robert Haas wrote: > More generally, whether or not we should "keep something away from our > users" really depends on how likely the upsides are to occur relative > to the downsides. We don't try to keep users from running DELETE > because they might delete data they want; that w

Re: Statement-level rollback

2018-12-07 Thread Robert Haas
On Fri, Dec 7, 2018 at 3:34 PM Alvaro Herrera wrote: > Yeah, I agree that this downside is real. I think our only protection > against that is to say "don't do that". Like any other tool, it has > upsides and downsides; we shouldn't keep it away from users only because > they might misuse it. I

Re: Statement-level rollback

2018-12-07 Thread Alvaro Herrera
On 2018-Dec-07, Robert Haas wrote: > Full disclosure: EDB has a feature like this and has for years, but it > uses a subtransaction per statement, not a subtransaction per row. Well, this implementation only uses one subtransaction per statement; Andres says per-row referring to the case of one I

Re: Statement-level rollback

2018-12-07 Thread Robert Haas
On Fri, Dec 7, 2018 at 2:57 PM Alvaro Herrera wrote: > The way I envision this to be used in practice is that it's enabled > globally when the migration effort starts, then gradually disabled as > parts of applications affected by these downsides are taught how to deal > with the other behavior.

Re: Statement-level rollback

2018-12-07 Thread Alvaro Herrera
On 2018-Dec-07, Andres Freund wrote: > On December 7, 2018 11:56:55 AM PST, Alvaro Herrera > wrote: > >BTW, a couple of months ago I measured the performance implications for > >a single update under pgbench and it represented a decrease of about > >3%-5%. Side-effects such as xid consumption h

Re: Statement-level rollback

2018-12-07 Thread Andres Freund
On December 7, 2018 11:56:55 AM PST, Alvaro Herrera wrote: >BTW, a couple of months ago I measured the performance implications for >a single update under pgbench and it represented a decrease of about >3%-5%. Side-effects such as xid consumption have worse implications, >but as far as perfor

Re: Statement-level rollback

2018-12-07 Thread Alvaro Herrera
Hi On 2018-Dec-07, Andres Freund wrote: > Isn't the realistic scenario for migrations that people will turn this > feature on on a more global basis? If they need to do per transaction > choices, that makes this much less useful for easy migrations. The way I envision this to be used in practice

Re: Statement-level rollback

2018-12-07 Thread Andres Freund
Hi, On December 7, 2018 11:44:17 AM PST, Alvaro Herrera wrote: >On 2018-Dec-07, Andres Freund wrote: > >> Hi, >> >> On 2018-12-07 16:20:06 -0300, Alvaro Herrera wrote: >> >case TBLOCK_BEGIN: >> > + s->rollbackScope = XactRollbackScope; >> >s->blo

Re: Statement-level rollback

2018-12-07 Thread Alvaro Herrera
On 2018-Dec-07, Andres Freund wrote: > Hi, > > On 2018-12-07 16:20:06 -0300, Alvaro Herrera wrote: > > case TBLOCK_BEGIN: > > + s->rollbackScope = XactRollbackScope; > > s->blockState = TBLOCK_INPROGRESS; > > + if (s->rollbackSco

Re: Statement-level rollback

2018-12-07 Thread Andres Freund
Hi, On 2018-12-07 16:20:06 -0300, Alvaro Herrera wrote: > case TBLOCK_BEGIN: > + s->rollbackScope = XactRollbackScope; > s->blockState = TBLOCK_INPROGRESS; > + if (s->rollbackScope == XACT_ROLLBACK_SCOPE_STMT) > +

Statement-level rollback

2018-12-07 Thread Alvaro Herrera
I would like to bring up again the topic of statement-level rollback. This was discussed in some depth at [1]. This patch is not based on Tsunakawa-san's patch submitted in that thread; although I started from it, I eventually removed almost everything and replaced with a completely diff

Re: [HACKERS] Statement-level rollback

2018-06-18 Thread Robert Haas
On Mon, Jun 18, 2018 at 4:51 PM, Alvaro Herrera wrote: > On 2018-Jun-16, Robert Haas wrote: >> I'm not sure that really solves the problem, because changing the GUC >> in either direction causes the system to behave differently. I don't >> see any particular reason to believe that changing the be

Re: [HACKERS] Statement-level rollback

2018-06-18 Thread Alvaro Herrera
On 2018-Jun-16, Robert Haas wrote: > I'm not sure that really solves the problem, because changing the GUC > in either direction causes the system to behave differently. I don't > see any particular reason to believe that changing the behavior from A > to B is any more or less likely to break app

Re: [HACKERS] Statement-level rollback

2018-06-16 Thread Simon Riggs
On 15 June 2018 at 21:23, Alvaro Herrera wrote: > I think the main objectionable point is that of making servers behave in > a way that could lose data, if applications assume that transactions > behave in the way they do today. I propose that we solve this by > allowing this feature to be enabl

Re: [HACKERS] Statement-level rollback

2018-06-15 Thread Robert Haas
On Fri, Jun 15, 2018 at 4:23 PM, Alvaro Herrera wrote: > I've been looking at re-implementing this feature recently, using > Tsunakawa's proposed UI of a GUC transaction_rollback_scope that can > take values "transaction" (default, current behavior) and "statement". > I didn't take other parts of

Re: [HACKERS] Statement-level rollback

2018-06-15 Thread MauMau
s allowed (if app is unsure > whether env is unsafe, it can set the scope to "transaction" to ensure > it's safe from that point onwards). Changing the scope in > postgresql.conf is forbidden, so a server is never unsafe as a whole. Would it be dangerous to allow both ena

Re: [HACKERS] Statement-level rollback

2018-06-15 Thread Alvaro Herrera
On 2017-Nov-06, Tsunakawa, Takayuki wrote: > From: Simon Riggs > > A backend-based solution is required for PL procedures and functions. > > > > We could put this as an option into PL/pgSQL, but it seems like it is > > a function of the transaction manager rather than the driver. > > Exactly. T

Re: [HACKERS] Statement-level rollback

2018-03-01 Thread Andres Freund
Hi, On 2018-01-09 08:21:33 +, Tsunakawa, Takayuki wrote: > From: Simon Riggs [mailto:si...@2ndquadrant.com] > > When will the next version be posted? > > I'm very sorry I haven't submitted anything. I'd like to address this during > this CF. Thanks for remembering this. Given that no new

RE: [HACKERS] Statement-level rollback

2018-01-09 Thread Tsunakawa, Takayuki
From: Simon Riggs [mailto:si...@2ndquadrant.com] > When will the next version be posted? I'm very sorry I haven't submitted anything. I'd like to address this during this CF. Thanks for remembering this. Regards Takayuki Tsunakawa

Re: [HACKERS] Statement-level rollback

2018-01-07 Thread Simon Riggs
On 6 November 2017 at 12:36, MauMau wrote: > when I submit the next revision of my patch. When will the next version be posted? -- Simon Riggshttp://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Re: [HACKERS] Statement-level rollback

2017-11-28 Thread Michael Paquier
On Mon, Nov 6, 2017 at 9:36 PM, MauMau wrote: > From: Thomas Munro > With your v2 patch "make docs" fails. Here is a small patch to apply > on top of yours to fix that and some small copy/paste errors, if I > understood correctly. > > Ouch, thanks. I'd like to merge your fix when I submit the ne