Re: [HACKERS] WIP Patch: Pgbench Serialization and deadlock errors

2021-05-23 Thread Yugo NAGATA
Hi hackers, On Tue, 10 Mar 2020 09:48:23 +1300 Thomas Munro wrote: > On Tue, Mar 10, 2020 at 8:43 AM Fabien COELHO wrote: > > >> Thank you very much! I'm going to send a new patch set until the end of > > >> this week (I'm sorry I was very busy in the release of Postgres Pro > > >> 11...). > >

Re: [HACKERS] WIP Patch: Pgbench Serialization and deadlock errors

2020-03-09 Thread Thomas Munro
On Tue, Mar 10, 2020 at 8:43 AM Fabien COELHO wrote: > >> Thank you very much! I'm going to send a new patch set until the end of > >> this week (I'm sorry I was very busy in the release of Postgres Pro > >> 11...). > > > > Is anyone interested in rebasing this, and summarising what needs to > >

Re: [HACKERS] WIP Patch: Pgbench Serialization and deadlock errors

2020-03-09 Thread Fabien COELHO
Hello Thomas, Thank you very much! I'm going to send a new patch set until the end of this week (I'm sorry I was very busy in the release of Postgres Pro 11...). Is anyone interested in rebasing this, and summarising what needs to be done to get it in? It's arguably a bug or at least quite

Re: [HACKERS] WIP Patch: Pgbench Serialization and deadlock errors

2020-03-08 Thread Thomas Munro
On Mon, Mar 9, 2020 at 10:00 AM Marina Polyakova wrote: > On 2018-11-16 22:59, Alvaro Herrera wrote: > > On 2018-Sep-05, Marina Polyakova wrote: > > > >> v11-0001-Pgbench-errors-use-the-RandomState-structure-for.patch > >> - a patch for the RandomState structure (this is used to reset a > >>

Re: [HACKERS] WIP Patch: Pgbench Serialization and deadlock errors

2018-11-19 Thread Fabien COELHO
Feel free to update a patch status to "needs review" yourself after submitting a new version that in your opinion respond to a reviewer's comments. Sure, I do that. But I will not switch any of my patch to "Ready". AFAICR the concerns where mostly about imprecise comments in the code, and a

Re: [HACKERS] WIP Patch: Pgbench Serialization and deadlock errors

2018-11-19 Thread Alvaro Herrera
On 2018-Nov-19, Fabien COELHO wrote: > > Hello Alvaro, > > > I also think that the pgbench_error() patch should go in before the main > > one. It seems a bit pointless to introduce code using a bad API only to > > fix the API together with all the new callers immediately afterwards. > > I'm

Re: [HACKERS] WIP Patch: Pgbench Serialization and deadlock errors

2018-11-19 Thread Fabien COELHO
Hello Alvaro, I also think that the pgbench_error() patch should go in before the main one. It seems a bit pointless to introduce code using a bad API only to fix the API together with all the new callers immediately afterwards. I'm not that keen on this part of the patch, because ISTM

Re: [HACKERS] WIP Patch: Pgbench Serialization and deadlock errors

2018-11-19 Thread Alvaro Herrera
On 2018-Nov-19, Marina Polyakova wrote: > On 2018-11-16 22:59, Alvaro Herrera wrote: > > On 2018-Sep-05, Marina Polyakova wrote: > > > > > v11-0001-Pgbench-errors-use-the-RandomState-structure-for.patch > > > - a patch for the RandomState structure (this is used to reset a > > > client's > > >

Re: [HACKERS] WIP Patch: Pgbench Serialization and deadlock errors

2018-11-19 Thread Marina Polyakova
On 2018-11-16 22:59, Alvaro Herrera wrote: On 2018-Sep-05, Marina Polyakova wrote: v11-0001-Pgbench-errors-use-the-RandomState-structure-for.patch - a patch for the RandomState structure (this is used to reset a client's random seed during the repeating of transactions after

Re: [HACKERS] WIP Patch: Pgbench Serialization and deadlock errors

2018-11-16 Thread Alvaro Herrera
On 2018-Sep-05, Marina Polyakova wrote: > v11-0001-Pgbench-errors-use-the-RandomState-structure-for.patch > - a patch for the RandomState structure (this is used to reset a client's > random seed during the repeating of transactions after > serialization/deadlock failures). Pushed this one with

Re: [HACKERS] WIP Patch: Pgbench Serialization and deadlock errors

2018-10-01 Thread Michael Paquier
On Wed, Sep 12, 2018 at 06:12:29PM +0300, Marina Polyakova wrote: > The discussion about this has become entangled from the beginning, because > as I wrote in [1] at first I misread your original proposal... The last emails are about the last reviews of Fabien, which has remained unanswered for

Re: [HACKERS] WIP Patch: Pgbench Serialization and deadlock errors

2018-09-12 Thread Marina Polyakova
On 12-09-2018 17:04, Fabien COELHO wrote: Hello Marina, You can get other errors that cannot happen for only one client if you use shell commands in meta commands: Or if you use untrusted procedural languages in SQL expressions (see the used file in the attachments): Or if you try to

Re: [HACKERS] WIP Patch: Pgbench Serialization and deadlock errors

2018-09-12 Thread Fabien COELHO
Hello Marina, You can get other errors that cannot happen for only one client if you use shell commands in meta commands: Or if you use untrusted procedural languages in SQL expressions (see the used file in the attachments): Or if you try to create a function and perhaps replace an

Re: [HACKERS] WIP Patch: Pgbench Serialization and deadlock errors

2018-09-12 Thread Marina Polyakova
On 11-09-2018 18:29, Fabien COELHO wrote: Hello Marina, Hmm, but we can say the same for serialization or deadlock errors that were not retried (the client test code itself could not run correctly or the SQL sent was somehow wrong, which is also the client's fault), can't we? I think not.

Re: [HACKERS] WIP Patch: Pgbench Serialization and deadlock errors

2018-09-11 Thread Fabien COELHO
Hello Marina, Hmm, but we can say the same for serialization or deadlock errors that were not retried (the client test code itself could not run correctly or the SQL sent was somehow wrong, which is also the client's fault), can't we? I think not. If a client asks for something "legal",

Re: [HACKERS] WIP Patch: Pgbench Serialization and deadlock errors

2018-09-11 Thread Marina Polyakova
On 11-09-2018 16:47, Marina Polyakova wrote: On 08-09-2018 16:03, Fabien COELHO wrote: Hello Marina, I'd insist in a comment that "cnt" does not include "skipped" transactions (anymore). If you mean CState.cnt I'm not sure if this is practically useful because the code uses only the sum of

Re: [HACKERS] WIP Patch: Pgbench Serialization and deadlock errors

2018-09-11 Thread Marina Polyakova
On 08-09-2018 16:03, Fabien COELHO wrote: Hello Marina, v11-0003-Pgbench-errors-and-serialization-deadlock-retrie.patch - the main patch for handling client errors and repetition of transactions with serialization/deadlock failures (see the detailed description in the file). About patch

Re: [HACKERS] WIP Patch: Pgbench Serialization and deadlock errors

2018-09-11 Thread Marina Polyakova
On 08-09-2018 10:17, Fabien COELHO wrote: Hello Marina, Hello, Fabien! About the two first preparatory patches. v11-0001-Pgbench-errors-use-the-RandomState-structure-for.patch - a patch for the RandomState structure (this is used to reset a client's random seed during the repeating of

Re: [HACKERS] WIP Patch: Pgbench Serialization and deadlock errors

2018-09-08 Thread Fabien COELHO
Hello Marina, v11-0003-Pgbench-errors-and-serialization-deadlock-retrie.patch - the main patch for handling client errors and repetition of transactions with serialization/deadlock failures (see the detailed description in the file). About patch v11-3. Patch applies cleanly on top of the

Re: [HACKERS] WIP Patch: Pgbench Serialization and deadlock errors

2018-09-08 Thread Fabien COELHO
Hello Marina, About the two first preparatory patches. v11-0001-Pgbench-errors-use-the-RandomState-structure-for.patch - a patch for the RandomState structure (this is used to reset a client's random seed during the repeating of transactions after serialization/deadlock failures). Same

Re: [HACKERS] WIP Patch: Pgbench Serialization and deadlock errors

2018-08-17 Thread Marina Polyakova
On 17-08-2018 14:04, Fabien COELHO wrote: ... Or perhaps we can use a more detailed failure status so for each type of failure we always know the command name (argument "cmd") and whether the client is aborted. Something like this (but in comparison with the first variant ISTM overly

Re: [HACKERS] WIP Patch: Pgbench Serialization and deadlock errors

2018-08-17 Thread Marina Polyakova
On 17-08-2018 10:49, Fabien COELHO wrote: Hello Marina, Detailed -r report. I understand from the doc that the retry number on the detailed per-statement report is to identify at what point errors occur? Probably this is more or less always at the same point on a given script, so that the

Re: [HACKERS] WIP Patch: Pgbench Serialization and deadlock errors

2018-08-17 Thread Fabien COELHO
Hello Marina, Detailed -r report. I understand from the doc that the retry number on the detailed per-statement report is to identify at what point errors occur? Probably this is more or less always at the same point on a given script, so that the most interesting feature is to report the

Re: [HACKERS] WIP Patch: Pgbench Serialization and deadlock errors

2018-08-16 Thread Marina Polyakova
On 15-08-2018 11:50, Fabien COELHO wrote: Hello Marina, Hello! v10-0004-Pgbench-errors-and-serialization-deadlock-retrie.patch - the main patch for handling client errors and repetition of transactions with serialization/deadlock failures (see the detailed description in the file). Patch

Re: [HACKERS] WIP Patch: Pgbench Serialization and deadlock errors

2018-08-15 Thread Fabien COELHO
Hello Marina, v10-0004-Pgbench-errors-and-serialization-deadlock-retrie.patch - the main patch for handling client errors and repetition of transactions with serialization/deadlock failures (see the detailed description in the file). Patch applies cleanly. It allows retrying a script

Re: [HACKERS] WIP Patch: Pgbench Serialization and deadlock errors

2018-08-13 Thread Marina Polyakova
On 12-08-2018 12:14, Fabien COELHO wrote: HEllo Marina, Hello, Fabien! v10-0003-Pgbench-errors-use-the-Variables-structure-for-c.patch - a patch for the Variables structure (this is used to reset client variables during the repeating of transactions after serialization/deadlock failures).

Re: [HACKERS] WIP Patch: Pgbench Serialization and deadlock errors

2018-08-12 Thread Fabien COELHO
About part 3: Patch applies cleanly, I forgot: compiles, global & local "make check" are ok. -- Fabien.

Re: [HACKERS] WIP Patch: Pgbench Serialization and deadlock errors

2018-08-12 Thread Fabien COELHO
HEllo Marina, v10-0003-Pgbench-errors-use-the-Variables-structure-for-c.patch - a patch for the Variables structure (this is used to reset client variables during the repeating of transactions after serialization/deadlock failures). This patch adds an explicit structure to manage

Re: [HACKERS] WIP Patch: Pgbench Serialization and deadlock errors

2018-08-10 Thread Marina Polyakova
On 10-08-2018 17:19, Arthur Zakirov wrote: On Fri, Aug 10, 2018 at 04:46:04PM +0300, Marina Polyakova wrote: > +1 from me to keep initial name "pgbench_error". "pgbench_log" for new > function looks nice to me. I think it is better than just "log", > because "log" may conflict with natural

Re: [HACKERS] WIP Patch: Pgbench Serialization and deadlock errors

2018-08-10 Thread Arthur Zakirov
On Fri, Aug 10, 2018 at 04:46:04PM +0300, Marina Polyakova wrote: > > +1 from me to keep initial name "pgbench_error". "pgbench_log" for new > > function looks nice to me. I think it is better than just "log", > > because "log" may conflict with natural logarithmic function (see "man 3 > > log").

Re: [HACKERS] WIP Patch: Pgbench Serialization and deadlock errors

2018-08-10 Thread Marina Polyakova
On 10-08-2018 15:53, Arthur Zakirov wrote: On Thu, Aug 09, 2018 at 06:17:22PM +0300, Marina Polyakova wrote: > * ErrorLevel > > If ErrorLevel is used for things which are not errors, its name should > not include "Error"? Maybe "LogLevel"? On the one hand, this sounds better for me too. On the

Re: [HACKERS] WIP Patch: Pgbench Serialization and deadlock errors

2018-08-10 Thread Arthur Zakirov
On Thu, Aug 09, 2018 at 06:17:22PM +0300, Marina Polyakova wrote: > > * ErrorLevel > > > > If ErrorLevel is used for things which are not errors, its name should > > not include "Error"? Maybe "LogLevel"? > > On the one hand, this sounds better for me too. On the other hand, will not > this be

Re: [HACKERS] WIP Patch: Pgbench Serialization and deadlock errors

2018-08-10 Thread Marina Polyakova
On 10-08-2018 11:33, Fabien COELHO wrote: Hello Marina, I'd suggest to let lookupCreateVariable, putVariable* as they are, call pgbench_error with a level which does not stop the execution, and abort if necessary from the callers with a "aborted because of putVariable/eval/... error"

Re: [HACKERS] WIP Patch: Pgbench Serialization and deadlock errors

2018-08-10 Thread Fabien COELHO
Hello Marina, I'd suggest to let lookupCreateVariable, putVariable* as they are, call pgbench_error with a level which does not stop the execution, and abort if necessary from the callers with a "aborted because of putVariable/eval/... error" message, as it was done before. There's one more

Re: [HACKERS] WIP Patch: Pgbench Serialization and deadlock errors

2018-08-09 Thread Marina Polyakova
On 09-08-2018 12:28, Fabien COELHO wrote: Hello Marina, Hello! v10-0002-Pgbench-errors-use-a-separate-function-to-report.patch - a patch for a separate error reporting function (this is used to report client failures that do not cause an aborts and this depends on the level of debugging).

Re: [HACKERS] WIP Patch: Pgbench Serialization and deadlock errors

2018-08-09 Thread Fabien COELHO
Hello Marina, v10-0002-Pgbench-errors-use-a-separate-function-to-report.patch - a patch for a separate error reporting function (this is used to report client failures that do not cause an aborts and this depends on the level of debugging). Patch applies cleanly, compiles, global & local

Re: [HACKERS] WIP Patch: Pgbench Serialization and deadlock errors

2018-08-08 Thread Marina Polyakova
On 07-08-2018 19:21, Fabien COELHO wrote: Hello Marina, Hello, Fabien! v10-0001-Pgbench-errors-use-the-RandomState-structure-for.patch - a patch for the RandomState structure (this is used to reset a client's random seed during the repeating of transactions after serialization/deadlock

Re: [HACKERS] WIP Patch: Pgbench Serialization and deadlock errors

2018-08-07 Thread Fabien COELHO
Hello Marina, v10-0001-Pgbench-errors-use-the-RandomState-structure-for.patch - a patch for the RandomState structure (this is used to reset a client's random seed during the repeating of transactions after serialization/deadlock failures). About this v10 part 1: Patch applies cleanly,

Re: [HACKERS] WIP Patch: Pgbench Serialization and deadlock errors

2018-07-12 Thread Fabien COELHO
The point is to avoid building the message with dynamic allocation and so if in the end it is not used. Ok! About avoidance - I'm afraid there's one more piece of debugging code with the same problem: Indeed. I'd like to avoid all instances, so that PQExpBufferData is not needed

Re: [HACKERS] WIP Patch: Pgbench Serialization and deadlock errors

2018-07-12 Thread Marina Polyakova
On 11-07-2018 22:34, Fabien COELHO wrote: can we try something like this? PGBENCH_ERROR_START(DEBUG_FAIL) { PGBENCH_ERROR("client %d repeats the failed transaction (try %d", Argh, no? I was thinking of something much more trivial: pgbench_error(DEBUG, "message format %d %s...",

Re: [HACKERS] WIP Patch: Pgbench Serialization and deadlock errors

2018-07-12 Thread Marina Polyakova
On 11-07-2018 21:04, Alvaro Herrera wrote: Just a quick skim while refreshing what were those error reporting API changes about ... Thank you! On 2018-May-21, Marina Polyakova wrote: v9-0001-Pgbench-errors-use-the-RandomState-structure-for-.patch - a patch for the RandomState structure

Re: [HACKERS] WIP Patch: Pgbench Serialization and deadlock errors

2018-07-12 Thread Marina Polyakova
On 11-07-2018 20:49, Alvaro Herrera wrote: On 2018-Jul-11, Marina Polyakova wrote: can we try something like this? PGBENCH_ERROR_START(DEBUG_FAIL) { PGBENCH_ERROR("client %d repeats the failed transaction (try %d", st->id, st->retries + 1); if

Re: [HACKERS] WIP Patch: Pgbench Serialization and deadlock errors

2018-07-11 Thread Fabien COELHO
can we try something like this? PGBENCH_ERROR_START(DEBUG_FAIL) { PGBENCH_ERROR("client %d repeats the failed transaction (try %d", Argh, no? I was thinking of something much more trivial: pgbench_error(DEBUG, "message format %d %s...", 12, "hello world"); If you really need

Re: [HACKERS] WIP Patch: Pgbench Serialization and deadlock errors

2018-07-11 Thread Alvaro Herrera
Just a quick skim while refreshing what were those error reporting API changes about ... On 2018-May-21, Marina Polyakova wrote: > v9-0001-Pgbench-errors-use-the-RandomState-structure-for-.patch > - a patch for the RandomState structure (this is used to reset a client's > random seed during the

Re: [HACKERS] WIP Patch: Pgbench Serialization and deadlock errors

2018-07-11 Thread Alvaro Herrera
On 2018-Jul-11, Marina Polyakova wrote: > can we try something like this? > > PGBENCH_ERROR_START(DEBUG_FAIL) > { > PGBENCH_ERROR("client %d repeats the failed transaction (try %d", > st->id, st->retries + 1); > if (max_tries) >

Re: [HACKERS] WIP Patch: Pgbench Serialization and deadlock errors

2018-07-11 Thread Marina Polyakova
On 11-07-2018 16:24, Fabien COELHO wrote: Hello Marina, * -d/--debug: I'm not in favor in requiring a mandatory text argument on this option. As you wrote in [1], adding an additional option is also a bad idea: Hey, I'm entitled to some internal contradictions:-) ... and discussions will

Re: [HACKERS] WIP Patch: Pgbench Serialization and deadlock errors

2018-07-11 Thread Fabien COELHO
Hello Marina, * -d/--debug: I'm not in favor in requiring a mandatory text argument on this option. As you wrote in [1], adding an additional option is also a bad idea: Hey, I'm entitled to some internal contradictions:-) I'm sceptical of the "--debug-fails" options. ISTM that --debug is

Re: [HACKERS] WIP Patch: Pgbench Serialization and deadlock errors

2018-07-11 Thread Marina Polyakova
On 09-07-2018 16:05, Fabien COELHO wrote: Hello Marina, Hello, Fabien! Here is a review for the last part of your v9 version. Thank you very much for this! Patch does not "git apply" (may anymore): error: patch failed: doc/src/sgml/ref/pgbench.sgml:513 error:

Re: [HACKERS] WIP Patch: Pgbench Serialization and deadlock errors

2018-07-09 Thread Fabien COELHO
Hello Marina, v9-0004-Pgbench-errors-and-serialization-deadlock-retries.patch - the main patch for handling client errors and repetition of transactions with serialization/deadlock failures (see the detailed description in the file). Here is a review for the last part of your v9 version.

Re: [HACKERS] WIP Patch: Pgbench Serialization and deadlock errors

2018-07-08 Thread Fabien COELHO
Hello Alvaro, For context: in the backend, elog() is only used for internal messages (i.e. "can't-happen" conditions), and ereport() is used for user-facing messages. There are many things ereport() has that elog() doesn't, such as additional message fields (HINT, DETAIL, etc) that I think

Re: [HACKERS] WIP Patch: Pgbench Serialization and deadlock errors

2018-06-14 Thread Marina Polyakova
On 13-06-2018 22:44, Fabien COELHO wrote: Hello Marina, I suppose that this is related; because of my patch there may be a lot of such code (see v7 in [1]): - fprintf(stderr, - "malformed variable \"%s\" value: \"%s\"\n", -

Re: [HACKERS] WIP Patch: Pgbench Serialization and deadlock errors

2018-06-14 Thread Marina Polyakova
On 13-06-2018 22:59, Alvaro Herrera wrote: For context: in the backend, elog() is only used for internal messages (i.e. "can't-happen" conditions), and ereport() is used for user-facing messages. There are many things ereport() has that elog() doesn't, such as additional message fields (HINT,

Re: [HACKERS] WIP Patch: Pgbench Serialization and deadlock errors

2018-06-13 Thread Alvaro Herrera
On 2018-Jun-13, Fabien COELHO wrote: > > > > With that, is there a need for elog()? In the backend we have > > > > it because $HISTORY but there's no need for that here -- I > > > > propose to lose elog() and use only ereport everywhere. > > See commit 8a07ebb3c172 which turns some ereport into

Re: [HACKERS] WIP Patch: Pgbench Serialization and deadlock errors

2018-06-13 Thread Fabien COELHO
Hello Marina, I suppose that this is related; because of my patch there may be a lot of such code (see v7 in [1]): - fprintf(stderr, - "malformed variable \"%s\" value: \"%s\"\n", - var->name, var->svalue); +

Re: [HACKERS] WIP Patch: Pgbench Serialization and deadlock errors

2018-06-13 Thread Marina Polyakova
On 10-06-2018 10:38, Fabien COELHO wrote: Hello Marina, Hello! v9-0003-Pgbench-errors-use-the-ereport-macro-to-report-de.patch - a patch for the ereport() macro (this is used to report client failures that do not cause an aborts and this depends on the level of debugging). ISTM that

Re: [HACKERS] WIP Patch: Pgbench Serialization and deadlock errors

2018-06-10 Thread Fabien COELHO
Hello Marina, v9-0003-Pgbench-errors-use-the-ereport-macro-to-report-de.patch - a patch for the ereport() macro (this is used to report client failures that do not cause an aborts and this depends on the level of debugging). ISTM that abort() is called under FATAL. - implementation: if

Re: [HACKERS] WIP Patch: Pgbench Serialization and deadlock errors

2018-06-09 Thread Marina Polyakova
On 09-06-2018 16:31, Fabien COELHO wrote: Hello Marina, Hello! v9-0002-Pgbench-errors-use-the-Variables-structure-for-cl.patch - a patch for the Variables structure (this is used to reset client variables during the repeating of transactions after serialization/deadlock failures). About

Re: [HACKERS] WIP Patch: Pgbench Serialization and deadlock errors

2018-06-09 Thread Marina Polyakova
On 09-06-2018 9:55, Fabien COELHO wrote: Hello Marina, Hello! v9-0001-Pgbench-errors-use-the-RandomState-structure-for-.patch - a patch for the RandomState structure (this is used to reset a client's random seed during the repeating of transactions after serialization/deadlock failures).

Re: [HACKERS] WIP Patch: Pgbench Serialization and deadlock errors

2018-06-09 Thread Fabien COELHO
Hello Marina, v9-0002-Pgbench-errors-use-the-Variables-structure-for-cl.patch - a patch for the Variables structure (this is used to reset client variables during the repeating of transactions after serialization/deadlock failures). About this second patch: This extracts the variable

Re: [HACKERS] WIP Patch: Pgbench Serialization and deadlock errors

2018-06-09 Thread Fabien COELHO
Hello Marina, v9-0001-Pgbench-errors-use-the-RandomState-structure-for-.patch - a patch for the RandomState structure (this is used to reset a client's random seed during the repeating of transactions after serialization/deadlock failures). A few comments about this first patch. Patch

Re: [HACKERS] WIP Patch: Pgbench Serialization and deadlock errors

2018-05-08 Thread Alvaro Herrera
Hello, Fabien COELHO wrote: > > Looking over the diff, I find that this patch tries to do too much and > > needs to be split up. > > Yep, I agree that it would help the reviewing process. On the other hand I > have bad memories about maintaining dependent patches which interfere >

Re: [HACKERS] WIP Patch: Pgbench Serialization and deadlock errors

2018-05-08 Thread Fabien COELHO
Hello Alvaro, I think that I'll have time for a round of review in the first half of July. Providing a rebased patch before then would be nice. Note that even in the absence of a rebased patch, you can apply to an older checkout if you have some limited window of time for a review. Yes,

Re: [HACKERS] WIP Patch: Pgbench Serialization and deadlock errors

2018-05-08 Thread Alvaro Herrera
Fabien COELHO wrote: > I think that I'll have time for a round of review in the first half of July. > Providing a rebased patch before then would be nice. Note that even in the absence of a rebased patch, you can apply to an older checkout if you have some limited window of time for a review.

Re: [HACKERS] WIP Patch: Pgbench Serialization and deadlock errors

2018-05-08 Thread Fabien COELHO
Hello Marina, FYI the v8 patch does not apply anymore, mostly because of a recent perl reindentation. I think that I'll have time for a round of review in the first half of July. Providing a rebased patch before then would be nice. -- Fabien.

Re: [HACKERS] WIP Patch: Pgbench Serialization and deadlock errors

2018-04-05 Thread Ildus Kurbangaliev
On Wed, 04 Apr 2018 16:07:25 +0300 Marina Polyakova wrote: > Hello, hackers! > > Here there's a seventh version of the patch for error handling and > retrying of transactions with serialization/deadlock failures in > pgbench (based on the commit >

Re: [HACKERS] WIP Patch: Pgbench Serialization and deadlock errors

2018-03-29 Thread Fabien COELHO
Conception of max-retry option seems strange for me. if number of retries reaches max-retry option, then we just increment counter of failed transaction and try again (possibly, with different random numbers). At the end we should distinguish number of error transaction and failed

Re: [HACKERS] WIP Patch: Pgbench Serialization and deadlock errors

2018-03-29 Thread Teodor Sigaev
Conception of max-retry option seems strange for me. if number of retries reaches max-retry option, then we just increment counter of failed transaction and try again (possibly, with different random numbers). At the end we should distinguish number of error transaction and failed transaction,

Re: [HACKERS] WIP Patch: Pgbench Serialization and deadlock errors

2018-03-26 Thread Fabien COELHO
Hello Marina, Many thanks to both of you! I'm working on a patch in this direction.. I think that the best approach for now is simply to reset (command zero, random generator) and start over the whole script, without attempting to be more intelligent. The limitations should be clearly

Re: [HACKERS] WIP Patch: Pgbench Serialization and deadlock errors

2018-03-26 Thread Marina Polyakova
On 25-03-2018 15:23, Fabien COELHO wrote: Hm, I took a look on both thread about patch and it seems to me now it's overcomplicated. With recently committed enhancements of pgbench (\if, \when) it becomes close to impossible to retry transation in case of failure. So, initial approach just to

Re: [HACKERS] WIP Patch: Pgbench Serialization and deadlock errors

2018-03-25 Thread Fabien COELHO
Hm, I took a look on both thread about patch and it seems to me now it's overcomplicated. With recently committed enhancements of pgbench (\if, \when) it becomes close to impossible to retry transation in case of failure. So, initial approach just to rollback such transaction looks more

Re: [HACKERS] WIP Patch: Pgbench Serialization and deadlock errors

2018-03-23 Thread Teodor Sigaev
I suggest a patch where pgbench client sessions are not disconnected because of serialization or deadlock failures and these failures are mentioned in reports. In details: - transaction with one of these failures continue run normally, but its result is rolled back; - if there were these

Re: WIP Patch: Pgbench Serialization and deadlock errors

2018-03-06 Thread David Steele
Hi Marina, On 3/6/18 4:45 AM, Marina Polyakova wrote: > On 05-03-2018 18:21, David Steele wrote: >> Hello Marina, > > Hello, David! > >> On 1/12/18 12:01 PM, Marina Polyakova wrote: > ... >> >> This patch was marked Waiting on Author on Jan 8 and no new patch was >> submitted before this

Re: WIP Patch: Pgbench Serialization and deadlock errors

2018-03-06 Thread Marina Polyakova
On 05-03-2018 18:21, David Steele wrote: Hello Marina, Hello, David! On 1/12/18 12:01 PM, Marina Polyakova wrote: ... This patch was marked Waiting on Author on Jan 8 and no new patch was submitted before this commitfest. I think we should mark this patch as Returned with Feedback. I'm

Re: Re: WIP Patch: Pgbench Serialization and deadlock errors

2018-03-05 Thread David Steele
Hello Marina, On 1/12/18 12:01 PM, Marina Polyakova wrote: >> >> Yep. I'm trying to suggest an incremental path with simple but yet >> quite useful things first. > > This question ("if there's a failure what savepoint we should rollback > to and start the execution again? ...") mostly concerns

Re: WIP Patch: Pgbench Serialization and deadlock errors

2018-01-12 Thread Fabien COELHO
Hello Marina, If you want 2 transactions, then you have to put them in two scripts, which looks fine with me. Different transactions are expected to be independent, otherwise they should be merged into one transaction. Therefore if the script consists of several single statements (= several

Re: WIP Patch: Pgbench Serialization and deadlock errors

2018-01-12 Thread Marina Polyakova
On 12-01-2018 15:47, Fabien COELHO wrote: Hello Marina, A partial answer, to focus on how the feature may be simplified. Thank you! I apologise as it seems that I overlooked one of your mail. Changing the thread has not helped. I'm wondering whether the whole feature could be simplified

Re: WIP Patch: Pgbench Serialization and deadlock errors

2018-01-12 Thread Fabien COELHO
Hello Marina, A partial answer, to focus on how the feature may be simplified. I apologise as it seems that I overlooked one of your mail. Changing the thread has not helped. I'm wondering whether the whole feature could be simplified by considering that one script is one "transaction" (it

Re: WIP Patch: Pgbench Serialization and deadlock errors

2018-01-11 Thread Marina Polyakova
On 03-01-2018 19:28, Fabien COELHO wrote: Hello Marina, Hello! Some comment about WIP pgbench error handling v4. Many thanks! Patch applies, compiles, global & local "make check" are ok. doc compiles. I'm generally okay with having such a feature, but I'd like it to be *MUCH* simpler,

Re: WIP Patch: Pgbench Serialization and deadlock errors

2018-01-03 Thread Fabien COELHO
Hello Marina, Some comment about WIP pgbench error handling v4. Patch applies, compiles, global & local "make check" are ok. doc compiles. I'm generally okay with having such a feature, but I'd like it to be *MUCH* simpler, otherwise it is going to make pgbench unmaintainable:-( Also,

Re: WIP Patch: Pgbench Serialization and deadlock errors

2017-12-19 Thread Marina Polyakova
On 19-12-2017 17:11, Fabien COELHO wrote: Consider adding the patch to the next commitfest? Hi! Yes, here it is: https://commitfest.postgresql.org/16/1420/ I think you may have to ask the cf app admin to remove the duplicate. https://commitfest.postgresql.org/16/1419/ Thanks, I'm trying

Re: WIP Patch: Pgbench Serialization and deadlock errors

2017-12-19 Thread Fabien COELHO
Consider adding the patch to the next commitfest? Hi! Yes, here it is: https://commitfest.postgresql.org/16/1420/ I think you may have to ask the cf app admin to remove the duplicate. https://commitfest.postgresql.org/16/1419/ -- Fabien.

Re: WIP Patch: Pgbench Serialization and deadlock errors

2017-12-19 Thread Marina Polyakova
On 19-12-2017 16:52, Fabien COELHO wrote: Hello Marina, This is the fourth version of the patch for pgbench. Consider adding the patch to the next commitfest? Hi! Yes, here it is: https://commitfest.postgresql.org/16/1420/ -- Marina Polyakova Postgres Professional:

Re: WIP Patch: Pgbench Serialization and deadlock errors

2017-12-19 Thread Fabien COELHO
Hello Marina, This is the fourth version of the patch for pgbench. Consider adding the patch to the next commitfest? -- Fabien.