Re: Fix around conn_duration in pgbench

2021-08-31 Thread Yugo NAGATA
ould also change the > > behavior of pg13 or later? If so, should we measure disconnection delay even > > when -C is not specified in pg13? > > You mean "pg13 or before"? Sorry, you are right. I mean "pg13 or before". > -- > Tatsuo Ishii > SRA OSS, Inc. Japan > English: http://www.sraoss.co.jp/index_en.php > Japanese:http://www.sraoss.co.jp -- Yugo NAGATA

Re: Fix around conn_duration in pgbench

2021-08-30 Thread Yugo NAGATA
sure connection overhead which > naturally includes disconnection overhead. I think it is better to measure disconnection delays when -C is specified in pg 14. This seems not necessary when -C is not specified because pgbench just reports "initial connection time". However, what abou

Re: Fix around conn_duration in pgbench

2021-08-30 Thread Yugo NAGATA
Hello Fujii-san, On Mon, 30 Aug 2021 23:36:30 +0900 Fujii Masao wrote: > > > On 2021/08/26 12:13, Yugo NAGATA wrote: > > Ok. That makes sense. The output reports "including connections > > establishing" > > and "excluding connections establishing&quo

Re: Fix around conn_duration in pgbench

2021-08-29 Thread Yugo NAGATA
cording to the patch tester, the patch does not apply. Well, that's because both the patch for PG14 and one for PG13 are discussed here. > -- > Tatsuo Ishii > SRA OSS, Inc. Japan > English: http://www.sraoss.co.jp/index_en.php > Japanese:http://www.sraoss.co.jp > > -- Yugo NAGATA

Re: Fix around conn_duration in pgbench

2021-08-25 Thread Yugo NAGATA
ons establishing" and "excluding connections establishing" regardless with -C, so we should measure delays in the same way. I updated the patch for pg13 to measure disconnection delay when -C is not specified. I attached the updated patch for pg13 as well as one for pg14 which is s

Re: Avoid stuck of pbgench due to skipped transactions

2021-08-12 Thread Yugo NAGATA
On Tue, 10 Aug 2021 10:50:20 -0400 Greg Sabino Mullane wrote: > Apologies, just saw this. I found no problems, those "failures" were just > me missing checkboxes on the commitfest interface. +1 on the patch. Thank you! -- Yugo NAGATA

Re: Fix around conn_duration in pgbench

2021-08-05 Thread Yugo NAGATA
Hello Fujii-san, On Thu, 5 Aug 2021 16:16:48 +0900 Fujii Masao wrote: > > > On 2021/08/01 14:50, Yugo NAGATA wrote: > > When -C is not specified, the disconnection time is not measured even in > > the patch for v14+. I don't think it is a problem because the > > dis

Re: Implementing Incremental View Maintenance

2021-08-04 Thread Yugo NAGATA
an index on the materialized view for efficient incremental > maintenance. > ERROR: "foreign_table" is a foreign table > DETAIL: Triggers on foreign tables cannot have transition tables. > > It finally failed to make IVM, but I think it should be checked more early. You are right. We don't support foreign tables as long as we use triggers. I'll fix. Regards, Yugo Nagata -- Yugo NAGATA

Re: Implementing Incremental View Maintenance

2021-08-04 Thread Yugo NAGATA
Hello Zhihong Yu, On Mon, 2 Aug 2021 14:33:46 -0700 Zhihong Yu wrote: > On Sun, Aug 1, 2021 at 11:30 PM Yugo NAGATA wrote: > > > Hi hackers, > > > > On Mon, 19 Jul 2021 09:24:30 +0900 > > Yugo NAGATA wrote: > > > > > On Wed, 14 J

Re: Fix around conn_duration in pgbench

2021-07-31 Thread Yugo NAGATA
On Fri, 30 Jul 2021 15:26:51 +0900 Fujii Masao wrote: > > > On 2021/07/30 14:43, Yugo NAGATA wrote: > > This patch fixes three issues of connection time measurement: > > > > 1. The initial connection time is measured and stored into conn_duration > > bu

Re: Fix around conn_duration in pgbench

2021-07-29 Thread Yugo NAGATA
Hello Fujii-san, On Fri, 30 Jul 2021 02:01:08 +0900 Fujii Masao wrote: > > > On 2021/07/29 2:23, Fujii Masao wrote: > > > > > > On 2021/07/28 16:15, Yugo NAGATA wrote: > >>> I found another disconnect_all(). > >>> > >>> /*

Re: pgbench bug candidate: negative "initial connection time"

2021-07-28 Thread Yugo NAGATA
Hello, On Fri, 18 Jun 2021 15:58:48 +0200 (CEST) Fabien COELHO wrote: > Attached Yugo-san patch with some updates discussed in the previous mails, > so as to move things along. I attached the patch rebased to a change due to 856de3b39cf. Regards, Yugo Nagata -- Yugo NAGATA diff

Re: Fix around conn_duration in pgbench

2021-07-28 Thread Yugo NAGATA
Hello Fujii-san, On Wed, 28 Jul 2021 00:20:21 +0900 Fujii Masao wrote: > > > On 2021/07/27 11:02, Yugo NAGATA wrote: > > Hello Fujii-san, > > > > Thank you for looking at it. > > > > On Tue, 27 Jul 2021 03:04:35 +0900 > > Fujii Masao wrote:

Re: Fix around conn_duration in pgbench

2021-07-26 Thread Yugo NAGATA
, CSTATE_END_TX). We need disconnection here only when we get an error. Therefore, we don't need the measurement here. I attached the updated patch. Regards, Yugo Nagata -- Yugo NAGATA diff --git a/src/bin/pgbench/pgbench.c b/src/bin/pgbench/pgbench.c index 364b5a2e47..ffd74db3ad 100644

Re: Numeric x^y for negative x

2021-07-21 Thread Yugo NAGATA
On Wed, 21 Jul 2021 11:10:16 +0100 Dean Rasheed wrote: > On Tue, 20 Jul 2021 at 10:17, Yugo NAGATA wrote: > > > > This patch fixes numeric_power() to handle negative bases correctly and not > > to raise an error "cannot take logarithm of a negative number", as

Re: pgbench: using prepared BEGIN statement in a pipeline could cause an error

2021-07-20 Thread Yugo NAGATA
On Mon, 19 Jul 2021 10:51:36 +0900 Yugo NAGATA wrote: > Hello Fabien, > > On Sat, 17 Jul 2021 07:03:01 +0200 (CEST) > Fabien COELHO wrote: > > > > > Hello Yugo-san, > > > > > [...] One way to avoid these errors is to send Parse messages before >

Re: Question about non-blocking mode in libpq

2021-07-20 Thread Yugo NAGATA
Hello Alvaro, On Tue, 20 Jul 2021 12:05:11 -0400 Alvaro Herrera wrote: > On 2021-Jul-19, Yugo NAGATA wrote: > > > On Tue, 13 Jul 2021 11:59:49 +0900 > > Yugo NAGATA wrote: > > > > However, looking into the code, PQsendQuery seems not to return an error

Re: Numeric x^y for negative x

2021-07-20 Thread Yugo NAGATA
don't return a * divide-by-zero error code for 0 ^ -1. */ In the original code, two checks that could raise an error of ERRCODE_INVALID_ARGUMENT_FOR_POWER_FUNCTION are following the comment. I think these check codes are placed together under this comment intentionally, so I suggest not to move one of them. Regards, Yugo Nagata -- Yugo NAGATA

Re: Question about non-blocking mode in libpq

2021-07-19 Thread Yugo NAGATA
On Tue, 13 Jul 2021 11:59:49 +0900 Yugo NAGATA wrote: > Hello, > > During reading the documentation of libpq [1] , I found the following > description: > > In the nonblocking state, calls to PQsendQuery, PQputline, PQputnbytes, > PQputCopyData, and PQendcopy will not blo

Re: corruption of WAL page header is never reported

2021-07-19 Thread Yugo NAGATA
On Mon, 19 Jul 2021 06:32:28 -0300 Ranier Vilela wrote: > Em seg., 19 de jul. de 2021 às 06:15, Yugo NAGATA > escreveu: > > > On Mon, 19 Jul 2021 17:47:07 +0900 (JST) > > Kyotaro Horiguchi wrote: > > > > > At Mon, 19 Jul 2021 16:00:39 +0900, Yugo NAG

Re: corruption of WAL page header is never reported

2021-07-19 Thread Yugo NAGATA
On Mon, 19 Jul 2021 17:47:07 +0900 (JST) Kyotaro Horiguchi wrote: > At Mon, 19 Jul 2021 16:00:39 +0900, Yugo NAGATA wrote > in > > Your patch doesn't fix the issue that the error message is never reported in > > standby mode. When a WAL page header is broken, the stan

Re: corruption of WAL page header is never reported

2021-07-19 Thread Yugo NAGATA
On Mon, 19 Jul 2021 15:14:41 +0900 (JST) Kyotaro Horiguchi wrote: > Hello. > > At Sun, 18 Jul 2021 04:55:05 +0900, Yugo NAGATA wrote > in > > Hello, > > > > I found that any corruption of WAL page header found during recovery is > > never > > re

Re: pgbench: using prepared BEGIN statement in a pipeline could cause an error

2021-07-18 Thread Yugo NAGATA
ection may not be established in the CSTATE_CHOOSE_SCRIPT state. > In comments: not yet -> needed. Thanks. I'll fix it. Regards, Yugo Nagata -- Yugo NAGATA

Re: Implementing Incremental View Maintenance

2021-07-18 Thread Yugo NAGATA
On Wed, 14 Jul 2021 21:22:37 +0530 vignesh C wrote: > On Mon, May 17, 2021 at 10:08 AM Yugo NAGATA wrote: > > > > On Fri, 7 May 2021 14:14:16 +0900 > > Yugo NAGATA wrote: > > > > > On Mon, 26 Apr 2021 16:03:48 +0900 > > > Yugo NAGATA wrote: >

Re: corruption of WAL page header is never reported

2021-07-18 Thread Yugo NAGATA
On Sat, 17 Jul 2021 18:40:02 -0300 Ranier Vilela wrote: > Em sáb., 17 de jul. de 2021 às 16:57, Yugo NAGATA > escreveu: > > > Hello, > > > > I found that any corruption of WAL page header found during recovery is > > never > > reported in log

corruption of WAL page header is never reported

2021-07-17 Thread Yugo NAGATA
ion itself, I wander that we could check just xlp_pageaddr instead of calling XLogReaderValidatePageHeader. Regards, Yugo Nagata -- Yugo NAGATA diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c index edb15fe58d..4fdb23f061 100644 --- a/src/backend/access/tran

pgbench: using prepared BEGIN statement in a pipeline could cause an error

2021-07-16 Thread Yugo NAGATA
, noting in the documentation that the first command in a pipeline starts an implicit transaction might be useful for users. What do you think? Regards, Yugo Nagata -- Yugo NAGATA diff --git a/src/bin/pgbench/pgbench.c b/src/bin/pgbench/pgbench.c index 364b5a2e47..56e790fa33 100644 --- a/src/bin/p

Re: [HACKERS] WIP aPatch: Pgbench Serialization and deadlock errors

2021-07-15 Thread Yugo NAGATA
Hello, I attached the updated patch. On Tue, 13 Jul 2021 15:50:52 +0900 Yugo NAGATA wrote: > > >> > I'm a little hesitant about how to count and report such unfinished > > >> > because of bench timeout transactions, though. Not counting them seems > >

Re: [HACKERS] WIP aPatch: Pgbench Serialization and deadlock errors

2021-07-13 Thread Yugo NAGATA
ing transaction is terminated by -T option, or that pgbench reports it as the number of failed transactions? But now, I understood this is the latter that you don't want to count the temination of retrying as failures. Thanks. Regards, Yugo Nagata -- Yugo NAGATA

Re: [HACKERS] WIP aPatch: Pgbench Serialization and deadlock errors

2021-07-12 Thread Yugo NAGATA
on is, I guess it just complained about reporting the termination of retrying as failures. Therefore, I will fix to finish the benchmark when the time is over during retrying, that is, change the state to CSTATE_FINISHED instead of CSTATE_ERROR in such cases. Regards, Yugo Nagata -- Yugo NAGATA

Question about non-blocking mode in libpq

2021-07-12 Thread Yugo NAGATA
need to call PQflush until it returns 0, as documented with regard to PQflush. Do we need to fix the description of PQsetnonblocking? Regards, Yugo Nagata -- Yugo NAGATA

Re: [HACKERS] WIP aPatch: Pgbench Serialization and deadlock errors

2021-07-07 Thread Yugo NAGATA
treated differently than other errors. > >> See the attached files for generating deadlocks reliably (start with 2 > >> clients). What do you think? The PL/pgSQL minimal, it is really > >> client-code oriented. > > > > Sorry, but I cannot find the atta

Re: [HACKERS] WIP aPatch: Pgbench Serialization and deadlock errors

2021-07-07 Thread Yugo NAGATA
for serious users, not for > novice users. Of course, users themselves should be careful of problematic script, but it would be better that pgbench itself avoids problems if pgbench can beforehand. > Or, we should terminate the last cycle of benchmark regardless it is > retrying or not if -T expires. This will make pgbench behaves much > more consistent. Hmmm, indeed this might make the behaviour a bit consistent, but I am not sure such behavioural change benefit users. Regards, Yugo Nagata -- Yugo NAGATA

Re: [HACKERS] WIP aPatch: Pgbench Serialization and deadlock errors

2021-07-07 Thread Yugo NAGATA
f we regard that a transaction is still running even when it is under retrying after an error, terminate of the retry may imply to terminate running the transaction forcibly. > I still don't understand why you think that --max-tries non 0 case > will *certainly* finish in finite time whereas --max-tries=0 case will > not. I just mean that --max-tries greater than zero will prevent pgbench from retrying a transaction forever. Regards, Yugo Nagata -- Yugo NAGATA

Re: [HACKERS] WIP aPatch: Pgbench Serialization and deadlock errors

2021-07-06 Thread Yugo NAGATA
o print the number of transactions failed due to -T, we can fix to forbid to use -T without latency-limit under max-tries=0 for avoiding possible never-ending-benchmark. In this case, users have to limit the number of transaction retry by specifying latency-limit or max-tries (>0). However, if some users would like to benchmark simply allowing unlimited retries, using -T and max-tries=0 seems the most straight way, so I think it is better that they can be used together. Regards, Yugo Nagata -- Yugo NAGATA

Re: [HACKERS] WIP aPatch: Pgbench Serialization and deadlock errors

2021-07-05 Thread Yugo NAGATA
of all transaction tries; more over, you cannot use an unlimited > > number > > of all transaction tries; moreover, you cannot use an unlimited number > Thanks. I'll fix. Regards, Yugo Nagata -- Yugo NAGATA

Re: [HACKERS] WIP aPatch: Pgbench Serialization and deadlock errors

2021-06-30 Thread Yugo NAGATA
ld be devised to trigger > serialization or deadlock errors, eg with a SEQUENCE and an \if. > > See the attached files for generating deadlocks reliably (start with 2 > clients). > What do you think? The PL/pgSQL minimal, it is really client-code > oriented. > > Given th

Re: Fix around conn_duration in pgbench

2021-06-29 Thread Yugo NAGATA
On Wed, 30 Jun 2021 14:35:37 +0900 Yugo NAGATA wrote: > Hello Asif, > > On Tue, 29 Jun 2021 13:21:54 + > Asif Rehman wrote: > > > The following review has been posted through the commitfest application: > > make installcheck-world: tested, passed > >

Re: Fix around conn_duration in pgbench

2021-06-29 Thread Yugo NAGATA
scussed in [1]. [1] https://www.postgresql.org/message-id/alpine.DEB.2.22.394.2106181535400.3146194%40pseudo I attached the updated patach. Regards, Yugo Nagata -- Yugo NAGATA diff --git a/src/bin/pgbench/pgbench.c b/src/bin/pgbench/pgbench.c index d7479925cb..9d2abbfb68 100644 --- a/src/bin/pgbe

Re: [HACKERS] WIP aPatch: Pgbench Serialization and deadlock errors

2021-06-25 Thread Yugo NAGATA
"vars". Could this make a codereader confused? > ## About comments > > Remove the comment on enlargeVariables about "It is assumed …" the issue > of trying MAXINT vars is more than remote and is not worth mentioning. In > the same function, remove the comments about MARGIN, it is already on the > macro declaration, once is enough. Sure. I'll remove them. Regards, Yugo Nagata -- Yugo NAGATA

Re: Avoid stuck of pbgench due to skipped transactions

2021-06-22 Thread Yugo NAGATA
; Documentation:not tested > > Looks fine to me, as a way of catching this edge case. Thank you for looking into this! 'make installcheck-world' and 'Implements feature' are marked "failed", but did you find any problem on this patch? -- Yugo NAGATA

Re: [HACKERS] WIP aPatch: Pgbench Serialization and deadlock errors

2021-06-22 Thread Yugo NAGATA
t.postgresql.org/33/3194/ Thanks! -- Yugo NAGATA

Re: [HACKERS] WIP aPatch: Pgbench Serialization and deadlock errors

2021-06-22 Thread Yugo NAGATA
Hi hackers, On Mon, 24 May 2021 11:29:10 +0900 Yugo NAGATA wrote: > Hi hackers, > > On Tue, 10 Mar 2020 09:48:23 +1300 > Thomas Munro wrote: > > > On Tue, Mar 10, 2020 at 8:43 AM Fabien COELHO wrote: > > > >> Thank you very much! I'm going t

Re: pgbench bug candidate: negative "initial connection time"

2021-06-18 Thread Yugo NAGATA
, I think that error is fine, indeed we do not stop the > process, so "fatal" it is not; I replaced this 'fatal' with 'error' because we are aborting the client instead of exit(1). When pgbench was rewritten to use common logging API by the commit 30a3e772b40, somehow pg_log_fatal was used, but I am wondering it should have be pg_log_error. > Attached Yugo-san patch with some updates discussed in the previous mails, > so as to move things along. Thank you for update. I agree with this fix. Regards, Yugo Nagata -- Yugo NAGATA

Re: pgbench bug candidate: negative "initial connection time"

2021-06-17 Thread Yugo NAGATA
if (thread->state[j].state != CSTATE_FINISHED) + exit_code = 2; + + /* aggregate thread level stats */ Does this make sense? -- Yugo NAGATA diff --git a/src/bin/pgbench/pgbench.c b/src/bin/pgbench/pgbench.c index d7479925cb..901f25aab7 100644 --- a/src/bin/pgbenc

Re: pgbench logging broken by time logic changes

2021-06-17 Thread Yugo NAGATA
84714 60 1398 1345828509 7073 1979779 573489941 236 1411 If we obey the document and keep the back-compatibility, we should fix logAgg(). The attached patch includes these fixes. Regards, Yugo Nagata -- Yugo NAGATA diff --git a/src/bin/pgbench/pgbench.c b/src/bin/pgbench/pgbench.c index

Re: pgbench logging broken by time logic changes

2021-06-17 Thread Yugo NAGATA
On Thu, 17 Jun 2021 14:17:56 +0900 (JST) Kyotaro Horiguchi wrote: > At Thu, 17 Jun 2021 12:23:42 +0900, Yugo NAGATA wrote > in > > On Wed, 16 Jun 2021 21:11:41 +0200 (CEST) > > Fabien COELHO wrote: > > > I cannot say that I'm thrilled by having multiple tv s

Re: pgbench bug candidate: negative "initial connection time"

2021-06-16 Thread Yugo NAGATA
reported in this thread will be fixed by the patch attached in my previous post (a major part are written by you :-) ). Is this acceptable for you? Regards, Yugo Nagata -- Yugo NAGATA

Re: pgbench logging broken by time logic changes

2021-06-16 Thread Yugo NAGATA
ntage of using clock_gettime(). That is, this returns precise results in nanoseconds. However, we can not benefit from it because pg_time_now() converts the value to uint64 by using INSTR_TIME_GET_MICROSEC. So, if we would like more precious statistics in pgbench, it might be better to use INSTR_TIME_GET_M

Re: pgbench logging broken by time logic changes

2021-06-16 Thread Yugo NAGATA
me doesn't return epoch time. Therefore, we can use INSTR_TIME_SET_CURRENT aiming to calculate a duration, but we should not have used this to get the current timestamp. I think we can fix this issue by using gettimeofday for logging as before this was changed. I attached the patch. Regards, Yugo

Re: Avoid stuck of pbgench due to skipped transactions

2021-06-16 Thread Yugo NAGATA
On Mon, 14 Jun 2021 16:06:10 +0900 Yugo NAGATA wrote: > On Mon, 14 Jun 2021 08:47:40 +0200 (CEST) > Fabien COELHO wrote: > > > I attached the fixed patch that uses return instead of break, and I > > > confirmed > > > that this made the progress reporting work

Re: pgbench bug candidate: negative "initial connection time"

2021-06-16 Thread Yugo NAGATA
reattempting connections would make also sense in such case. This might become possible after pgbench gets the feature to retry in deadlock or serialization errors. I am working on rebase of the patch [2] and I will submit this in a few days. [2] https://www.postgresql.org/message-id/2021052

Re: Fix around conn_duration in pgbench

2021-06-15 Thread Yugo NAGATA
inally proposed patch. [1] https://www.postgresql.org/message-id/flat/TYCPR01MB5870057375ACA8A73099C649F5349%40TYCPR01MB5870.jpnprd01.prod.outlook.com. Regards, Yugo Nagata -- Yugo NAGATA

Re: Error on pgbench logs

2021-06-15 Thread Yugo NAGATA
Assert(tx) instead of using "else if (tx)" because we are assuming that tx is always true when agg_interval is not used, right? -- Yugo NAGATA

Re: Error on pgbench logs

2021-06-15 Thread Yugo NAGATA
On Tue, 15 Jun 2021 18:01:18 +0900 Michael Paquier wrote: > On Tue, Jun 15, 2021 at 05:15:14PM +0900, Yugo NAGATA wrote: > > On Tue, 15 Jun 2021 10:05:29 +0200 (CEST) Fabien COELHO > > wrote: > > It was not a problem because --sampling-rate --aggregate-interval cannot be

Re: Error on pgbench logs

2021-06-15 Thread Yugo NAGATA
h this patch. > > I do not get it. It was not a problem because --sampling-rate --aggregate-interval cannot be used at the same time. > > If I am following this code correctly, we don't care about accumStats() > > in the code path of a thread we are done with, right? > >

Re: pgbench bug candidate: negative "initial connection time"

2021-06-14 Thread Yugo NAGATA
t; Otherwise, if we want pgbench to exit immediately when a connection error occurs, we have tocall exit(1) to ensure the exit code is 1, of course. Anyway, it is wrong that thecurrent pgbench exit successfully with exit code 0 when doConnnect fails. Regards, Yugo Nagata -- Yugo NAGATA diff --

Re: pgbench bug candidate: negative "initial connection time"

2021-06-14 Thread Yugo NAGATA
+6419,7 @@ main(int argc, char **argv) initRandomState(>ts_throttle_rs); initRandomState(>ts_sample_rs); thread->logfile = NULL; /* filled in later */ + thread->bench_start = 0; thread->latency_late = 0; initStats(>stats, 0) typo: faild -> failed Regards, Yugo Nagata -- Yugo NAGATA

Re: Avoid stuck of pbgench due to skipped transactions

2021-06-14 Thread Yugo NAGATA
rameters, as pg is unlikely to reach 100 million tps, ever. > It seems to me enough that the command is not blocked in such cases. Sure. The change from "break" to "return" is just for making the progress reporting work in the loop, as you mentioned. However, my original intention is avoiding stuck in a corner-case where a unrealistic parameter was used, and I agree with you that this change is not so necessary for handling such a special situation. Regards, Yugo Nagata -- Yugo NAGATA

Fix around conn_duration in pgbench

2021-06-14 Thread Yugo NAGATA
lso fixed this in the attached patch. Regards, Yugo Nagata -- Yugo NAGATA diff --git a/src/bin/pgbench/pgbench.c b/src/bin/pgbench/pgbench.c index d7479925cb..1ec42a3ba2 100644 --- a/src/bin/pgbench/pgbench.c +++ b/src/bin/pgbench/pgbench.c @@ -3536,8 +3536,10 @@ advanceConnectionState(TState *thre

Re: Avoid stuck of pbgench due to skipped transactions

2021-06-13 Thread Yugo NAGATA
nectionState to threadRun and just break the loop. + /* otherwise loop over PREPARE_THROTTLE */ break; I attached the fixed patch that uses return instead of break, and I confirmed that this made the progress reporting work property. Regards, Y

Avoid stuck of pbgench due to skipped transactions

2021-06-12 Thread Yugo NAGATA
for this fix. Regards, Yugo Nagata -- Yugo NAGATA diff --git a/src/bin/pgbench/pgbench.c b/src/bin/pgbench/pgbench.c index dc84b7b9b7..1aa3e6b7be 100644 --- a/src/bin/pgbench/pgbench.c +++ b/src/bin/pgbench/pgbench.c @@ -3232,7 +3232,8 @@ advanceConnectionState(TState *thread, CState *st, StatsData *agg

Re: Error on pgbench logs

2021-06-12 Thread Yugo NAGATA
onds" to "us" using casting like end_time = threads[0].create_time + (int64) 100 * duration; or next_report = last_report + (int64) 100 * progress; Is there a reason use INT64CONST instead of (int64)? Do these imply the same effect? Sorry, if this is a dumb question... Regards, Yugo Nagata -- Yugo NAGATA

Re: Error on pgbench logs

2021-06-12 Thread Yugo NAGATA
logAgg(thread->logfile, ); I think we don't have to call doLog() before logAgg(). If we call doLog(), we will count an extra transaction that is not actually processed because accumStats() is called in this. Regards, Yugo Nagata -- Yugo NAGATA

Re: [HACKERS] WIP Patch: Pgbench Serialization and deadlock errors

2021-05-23 Thread Yugo NAGATA
> > that a couple of forks already ship Marina's patch set. I got interested in this and now looking into the patch and the past discussion. If anyone other won't do it and there are no objection, I would like to rebase this. Is that okay? Regards, Yugo NAGATA > > >

Re: Implementing Incremental View Maintenance

2021-05-16 Thread Yugo NAGATA
On Fri, 7 May 2021 14:14:16 +0900 Yugo NAGATA wrote: > On Mon, 26 Apr 2021 16:03:48 +0900 > Yugo NAGATA wrote: > > > On Mon, 26 Apr 2021 15:46:21 +0900 > > Yugo NAGATA wrote: > > > > > On Tue, 20 Apr 2021 09:51:34 +0900 > > > Yugo NAGATA wrote

Re: Implementing Incremental View Maintenance

2021-05-06 Thread Yugo NAGATA
On Mon, 26 Apr 2021 16:03:48 +0900 Yugo NAGATA wrote: > On Mon, 26 Apr 2021 15:46:21 +0900 > Yugo NAGATA wrote: > > > On Tue, 20 Apr 2021 09:51:34 +0900 > > Yugo NAGATA wrote: > > > > > On Mon, 19 Apr 2021 17:40:31 -0400 > > > Tom Lane

Re: Implementing Incremental View Maintenance

2021-04-26 Thread Yugo NAGATA
On Mon, 26 Apr 2021 15:46:21 +0900 Yugo NAGATA wrote: > On Tue, 20 Apr 2021 09:51:34 +0900 > Yugo NAGATA wrote: > > > On Mon, 19 Apr 2021 17:40:31 -0400 > > Tom Lane wrote: > > > > > Andrew Dunstan writes: > > > > This patch (v22

Re: Implementing Incremental View Maintenance

2021-04-26 Thread Yugo NAGATA
On Tue, 20 Apr 2021 09:51:34 +0900 Yugo NAGATA wrote: > On Mon, 19 Apr 2021 17:40:31 -0400 > Tom Lane wrote: > > > Andrew Dunstan writes: > > > This patch (v22c) just crashed for me with an assertion failure on > > > Fedora 31. Here's the stack trace:

Re: Implementing Incremental View Maintenance

2021-04-19 Thread Yugo NAGATA
neNumber@entry=199) at > > /home/andrew/pgl/pg_head/src/backend/utils/error/assert.c:69 > > That assert just got added a few days ago, so that's why the patch > seemed OK before. Thank you for letting me know. I'll fix it. -- Yugo NAGATA

Re: Implementing Incremental View Maintenance

2021-04-07 Thread Yugo NAGATA
Hi, I rebased the patch because the cfbot failed. Regards, Yugo Nagata On Tue, 9 Mar 2021 17:27:50 +0900 Yugo NAGATA wrote: > On Tue, 9 Mar 2021 09:20:49 +0900 > Yugo NAGATA wrote: > > > On Mon, 8 Mar 2021 15:42:00 -0500 > > Andrew Dunstan wrote: > > > >

Re: Columns correlation and adaptive query optimization

2021-03-22 Thread Yugo NAGATA
_phone, 2200) already exists. Regards, Yugo Nagata On Sat, 20 Mar 2021 12:41:44 +0300 Konstantin Knizhnik wrote: > > > On 19.03.2021 20:32, Zhihong Yu wrote: > > Hi, > > In AddMultiColumnStatisticsForQual(), > > > > +   /* Loop until we considered all vars */ &g

Re: Columns correlation and adaptive query optimization

2021-03-21 Thread Yugo NAGATA
On Fri, 19 Mar 2021 19:58:27 +0300 Konstantin Knizhnik wrote: > > > On 19.03.2021 12:17, Yugo NAGATA wrote: > > On Wed, 10 Mar 2021 03:00:25 +0100 > > Tomas Vondra wrote: > > > >> What is being proposed here - an extension suggesting which statistics > &

Re: Columns correlation and adaptive query optimization

2021-03-19 Thread Yugo NAGATA
isor. > BTW Why is "qual" in > > static void > AddMultiColumnStatisticsForQual(void* qual, ExplainState *es) > > declared as "void *"? Shouldn't that be "List *"? When I tested this extension using TPC-H queries, it raised segmentation fault in this function. I think the cause would be around this argument. Regards, Yugo Nagata -- Yugo NAGATA

Re: Implementing Incremental View Maintenance

2021-03-09 Thread Yugo NAGATA
On Tue, 9 Mar 2021 09:20:49 +0900 Yugo NAGATA wrote: > On Mon, 8 Mar 2021 15:42:00 -0500 > Andrew Dunstan wrote: > > > > > On 2/18/21 9:01 PM, Yugo NAGATA wrote: > > > On Thu, 18 Feb 2021 19:38:44 +0800 > > > Andy Fan wrote: > > > > >

Re: Implementing Incremental View Maintenance

2021-03-08 Thread Yugo NAGATA
On Mon, 8 Mar 2021 15:42:00 -0500 Andrew Dunstan wrote: > > On 2/18/21 9:01 PM, Yugo NAGATA wrote: > > On Thu, 18 Feb 2021 19:38:44 +0800 > > Andy Fan wrote: > > > >> On Tue, Feb 16, 2021 at 9:33 AM Yugo NAGATA wrote: > >> > >>

Re: Implementing Incremental View Maintenance

2021-02-18 Thread Yugo NAGATA
On Thu, 18 Feb 2021 19:38:44 +0800 Andy Fan wrote: > On Tue, Feb 16, 2021 at 9:33 AM Yugo NAGATA wrote: > > > Hi, > > > > Attached is a rebased patch (v22a). > > > > Thanks for the patch. Will you think posting a patch with the latest commit > at tha

Re: Implementing Incremental View Maintenance

2021-02-15 Thread Yugo NAGATA
Hi, Attached is a rebased patch (v22a). Ragards, Yugo Nagata -- Yugo NAGATA IVM_patches_v22a.tar.gz Description: application/gzip

Re: Is Recovery actually paused?

2021-02-11 Thread Yugo NAGATA
without even changing any > > config parameter), WakeupRecovery gets called. > > > > Thanks for the patch. I tested the new function and it works as > > expected. I have no further comments on the v13 patch. > > Thanks for the review and testing. I have no futher comments on the v13 patch, too. Also, I agree with Robert Haas's suggestions. Regards, Yugo Nagata -- Yugo NAGATA

Re: Is Recovery actually paused?

2021-02-08 Thread Yugo NAGATA
On Tue, 09 Feb 2021 10:58:04 +0900 (JST) Kyotaro Horiguchi wrote: > At Mon, 8 Feb 2021 17:05:52 +0530, Dilip Kumar wrote > in > > On Mon, Feb 8, 2021 at 2:19 PM Yugo NAGATA wrote: > > > > > > On Mon, 08 Feb 2021 17:32:46 +0900 (JST) > > > Kyotaro Horigu

Re: Is Recovery actually paused?

2021-02-08 Thread Yugo NAGATA
On Mon, 08 Feb 2021 17:32:46 +0900 (JST) Kyotaro Horiguchi wrote: > At Mon, 8 Feb 2021 14:12:35 +0900, Yugo NAGATA wrote in > > > > > I think the right fix should be that the state should never go from > > > > > ‘paused’ to ‘pause requested’ so I

Re: Is Recovery actually paused?

2021-02-07 Thread Yugo NAGATA
On Mon, 8 Feb 2021 09:35:00 +0530 Dilip Kumar wrote: > On Mon, Feb 8, 2021 at 8:18 AM Yugo NAGATA wrote: > > > > On Mon, 8 Feb 2021 07:51:22 +0530 > > Dilip Kumar wrote: > > > > > On Mon, 8 Feb 2021 at 6:38 AM, Yugo NAGATA wrote: > > > > >

Re: Is Recovery actually paused?

2021-02-07 Thread Yugo NAGATA
On Mon, 8 Feb 2021 07:51:22 +0530 Dilip Kumar wrote: > On Mon, 8 Feb 2021 at 6:38 AM, Yugo NAGATA wrote: > > > Hi, > > > > On Sun, 7 Feb 2021 19:27:02 +0530 > > Dilip Kumar wrote: > > > > > On Sun, Feb 7, 2021 at 6:44 PM Bharath Rupireddy >

Re: Is Recovery actually paused?

2021-02-07 Thread Yugo NAGATA
andleStartupProcInterrupts(); If a user call pg_wal_replay_pause while waiting in RecoveryRequiresIntParameter, the state become 'pause requested' and this never returns to 'paused'. Should we check recoveryPauseState in this loop as in recoveryPausesHere? Regards, Yugo Nagata -- Yugo NAGATA

Re: [PATCH] Add extra statistics to explain for Nested Loop

2021-02-01 Thread Yugo NAGATA
On Mon, 1 Feb 2021 13:28:45 +0800 Julien Rouhaud wrote: > On Thu, Jan 28, 2021 at 8:38 PM Yugo NAGATA wrote: > > > > postgres=# explain (analyze, verbose) select * from a,b where a.i=b.j; > >

Re: Is Recovery actually paused?

2021-01-29 Thread Yugo NAGATA
On Fri, 29 Jan 2021 16:33:32 +0530 Dilip Kumar wrote: > On Fri, Jan 29, 2021 at 3:25 PM Yugo NAGATA wrote: > > > > On Thu, 28 Jan 2021 09:55:42 +0530 > > Dilip Kumar wrote: > > > > > On Wed, Jan 27, 2021 at 2:28 PM Dilip Kumar wrote: > > > > >

Re: Is Recovery actually paused?

2021-01-29 Thread Yugo NAGATA
On Thu, 28 Jan 2021 09:55:42 +0530 Dilip Kumar wrote: > On Wed, Jan 27, 2021 at 2:28 PM Dilip Kumar wrote: > > > > On Wed, Jan 27, 2021 at 2:06 PM Yugo NAGATA wrote: > > > > > > On Wed, 27 Jan 2021 13:29:23 +0530 > > > Dilip Kumar wrote: > > >

Re: [PATCH] Add extra statistics to explain for Nested Loop

2021-01-28 Thread Yugo NAGATA
it a/src/test/regress/expected/timetz.out b/src/test/regress/expected/timetz.out index 038bb5fa094..5294179aa45 100644 Regards, Yugo Nagata -- Yugo NAGATA

Re: Is Recovery actually paused?

2021-01-27 Thread Yugo NAGATA
with the name you suggest that returns text. > > > That would create less burden for tool authors. > > > > +1 > > > > Yeah, we can do that, I will send an updated patch soon. This means pg_is_wal_replay_paused is left without any change and this returns whether pause is requested or not? If so, it seems good to modify the documentation of this function in order to note that this could not return the actual pause state. Regards, Yugo Nagata -- Yugo NAGATA

Re: Columns correlation and adaptive query optimization

2021-01-26 Thread Yugo NAGATA
ctually task of adaptive query optimization is much bigger. > We have separate AQO extension which tries to use machine learning to > correctly adjust estimations. > This my patch is much simpler and use existed mechanism (extended > statistics) to improve estimations. Well, this patch provide a kind of AQO as auto_explain feature, but this is independent of the AQO extension. Is it right? Anyway, I'm interested in the AQO extension, so I'll look into this, too. Regards, Yugo Nagata -- Yugo NAGATA

Re: Is Recovery actually paused?

2021-01-25 Thread Yugo NAGATA
Here(), but this seems redundant. (2) - while (RecoveryIsPaused()) + while (GetRecoveryPauseState() != RECOVERY_NOT_PAUSED) { + HandleStartupProcInterrupts(); Though it is trivial, I think the new line after the brace is unnecessary. Regards, Yugo Nagata -- Yugo NAGATA

Re: Implementing Incremental View Maintenance

2021-01-22 Thread Yugo NAGATA
Hi, Attached is a revised patch (v22) rebased for the latest master head. Regards, Yugo Nagata -- Yugo NAGATA IVM_patches_v22.tar.gz Description: application/gzip

Re: Is Recovery actually paused?

2021-01-21 Thread Yugo NAGATA
On Tue, 19 Jan 2021 21:32:31 +0530 Dilip Kumar wrote: > On Tue, Jan 19, 2021 at 8:34 AM Dilip Kumar wrote: > > > > On Tue, 19 Jan 2021 at 8:12 AM, Yugo NAGATA wrote: > >> > >> On Sun, 17 Jan 2021 11:33:52 +0530 > >> Dilip Kumar wrote: > >>

Re: Columns correlation and adaptive query optimization

2021-01-21 Thread Yugo NAGATA
and is it useful? (10) To achieve adaptive query optimization (AQO) in PostgreSQL, this patch proposes to use auto_explain for getting feedback from actual results. So, could auto_explain be a infrastructure of AQO in future? Or, do you have any plan or idea to make built-in infrastructure for AQO? Regards, Yugo Nagata -- Yugo NAGATA

Re: Is Recovery actually paused?

2021-01-18 Thread Yugo NAGATA
On Sun, 17 Jan 2021 11:33:52 +0530 Dilip Kumar wrote: > On Thu, Jan 14, 2021 at 6:49 PM Yugo NAGATA wrote: > > > > On Wed, 13 Jan 2021 17:49:43 +0530 > > Dilip Kumar wrote: > > > > > On Wed, Jan 13, 2021 at 3:35 PM Dilip Kumar wrote: > > > > >

Re: Is Recovery actually paused?

2021-01-14 Thread Yugo NAGATA
On Wed, 13 Jan 2021 17:49:43 +0530 Dilip Kumar wrote: > On Wed, Jan 13, 2021 at 3:35 PM Dilip Kumar wrote: > > > > On Wed, Jan 13, 2021 at 3:27 PM Yugo NAGATA wrote: > > > > > > On Thu, 10 Dec 2020 11:25:23 +0530 > > > Dilip Kumar wrote: > > &g

Re: Is Recovery actually paused?

2021-01-13 Thread Yugo NAGATA
ke user wait for a long time, I don't care either. > > > As another comment, while pg_is_wal_replay_paused is blocking, I can not > > > cancel > > > the query. I think CHECK_FOR_INTERRUPTS() is necessary in the waiting > > > loop. How about this fix? I think users may want to cancel pg_is_wal_replay_paused() during this is blocking. Regards, Yugo Nagata -- Yugo NAGATA

Re: Implementing Incremental View Maintenance

2021-01-12 Thread Yugo NAGATA
, Yugo Nagata On Tue, 22 Dec 2020 22:24:22 +0900 Yugo NAGATA wrote: > Hi hackers, > > I heard the opinion that this patch is too big and hard to review. > So, I wander that we should downsize the patch by eliminating some > features and leaving other basic features. > > If th

Re: Implementing Incremental View Maintenance

2020-12-22 Thread Yugo NAGATA
it. If so, we plan to support only selection, projection, inner-join, and some aggregates in the first release and leave sub-queries, outer-join, and CTE supports to the next release. Regards, Yugo Nagata On Tue, 22 Dec 2020 21:51:36 +0900 Yugo NAGATA wrote: > Hi, > > Attached is the revi

Re: Implementing Incremental View Maintenance

2020-12-22 Thread Yugo NAGATA
he effect of the optimization. Regards, Yugo Nagata -- Yugo NAGATA IVM_patches_v20.tar.gz Description: application/gzip

<    1   2   3   4   5   >