Re: Tid scan increments value of pg_stat_all_tables.seq_scan. (but not seq_tup_read)

2020-02-07 Thread Kasahara Tatsuhito
On Fri, Feb 7, 2020 at 10:09 PM Fujii Masao wrote: > Pushed! Thanks! Thanks Fujii. -- Tatsuhito Kasahara kasahara.tatsuhito _at_ gmail.com

Re: Tid scan increments value of pg_stat_all_tables.seq_scan. (but not seq_tup_read)

2020-02-07 Thread Fujii Masao
On 2020/02/07 17:28, Kasahara Tatsuhito wrote: Hi, On Fri, Feb 7, 2020 at 5:02 PM Fujii Masao wrote: BTW, commit 147e3722f7 causing the issue changed currtid_byreloid() and currtid_byrelname() so that they also call table_beginscan(). I'm not sure what those functions are, but probably we

Re: Tid scan increments value of pg_stat_all_tables.seq_scan. (but not seq_tup_read)

2020-02-07 Thread Kyotaro Horiguchi
At Fri, 7 Feb 2020 17:01:59 +0900, Fujii Masao wrote in > > > On 2020/02/07 15:07, Kasahara Tatsuhito wrote: > > Hi, > > On Fri, Feb 7, 2020 at 1:27 PM Kyotaro Horiguchi > > wrote: > >> It seems that nkeys and key are useless. Since every table_beginscan_* > >> functions have distinct

Re: Tid scan increments value of pg_stat_all_tables.seq_scan. (but not seq_tup_read)

2020-02-07 Thread Kasahara Tatsuhito
Hi, On Fri, Feb 7, 2020 at 5:02 PM Fujii Masao wrote: > BTW, commit 147e3722f7 causing the issue changed currtid_byreloid() > and currtid_byrelname() so that they also call table_beginscan(). > I'm not sure what those functions are, but probably we should fix them > so that table_beginscan_tid()

Re: Tid scan increments value of pg_stat_all_tables.seq_scan. (but not seq_tup_read)

2020-02-07 Thread Fujii Masao
On 2020/02/07 15:07, Kasahara Tatsuhito wrote: Hi, On Fri, Feb 7, 2020 at 1:27 PM Kyotaro Horiguchi wrote: It seems that nkeys and key are useless. Since every table_beginscan_* functions have distinct parameter sets, don't we remove them from table_beginscan_tid? Yeah, actually, when

Re: Tid scan increments value of pg_stat_all_tables.seq_scan. (but not seq_tup_read)

2020-02-06 Thread Kasahara Tatsuhito
Hi, On Fri, Feb 7, 2020 at 1:27 PM Kyotaro Horiguchi wrote: > It seems that nkeys and key are useless. Since every table_beginscan_* > functions have distinct parameter sets, don't we remove them from > table_beginscan_tid? Yeah, actually, when calling table_beginscan_tid(), nkeys is set to 0

Re: Tid scan increments value of pg_stat_all_tables.seq_scan. (but not seq_tup_read)

2020-02-06 Thread Kyotaro Horiguchi
At Fri, 7 Feb 2020 12:27:26 +0900, Kasahara Tatsuhito wrote in > > IIUC setting SO_ALLOW_STRAT, SO_ALLOW_SYNC and SO_ALLOW_PAGEMODE has > > no meaning during tid scan. I think v11 also should be the same. > Thanks for your check, and useful advise. > I was wondering if I should keep these

Re: Tid scan increments value of pg_stat_all_tables.seq_scan. (but not seq_tup_read)

2020-02-06 Thread Kasahara Tatsuhito
Hi, On Thu, Feb 6, 2020 at 11:01 PM Masahiko Sawada wrote: > > On Thu, 6 Feb 2020 at 19:12, Kasahara Tatsuhito > wrote: > I've tested predicate locking including promotion cases with v3 patch > and it works fine. > > +table_beginscan_tid(Relation rel, Snapshot snapshot, > + int

Re: Tid scan increments value of pg_stat_all_tables.seq_scan. (but not seq_tup_read)

2020-02-06 Thread Masahiko Sawada
On Thu, 6 Feb 2020 at 19:12, Kasahara Tatsuhito wrote: > > HI, > > On Thu, Feb 6, 2020 at 3:24 PM Fujii Masao > wrote: > > > I added a new (but minimal) isolation test for the case of tid scan. > > > (v12 and HEAD will be failed this test. v11 and HEAD with my patch > > > will be passed) > >

Re: Tid scan increments value of pg_stat_all_tables.seq_scan. (but not seq_tup_read)

2020-02-06 Thread Kasahara Tatsuhito
HI, On Thu, Feb 6, 2020 at 3:24 PM Fujii Masao wrote: > > I added a new (but minimal) isolation test for the case of tid scan. > > (v12 and HEAD will be failed this test. v11 and HEAD with my patch > > will be passed) > > Isn't this test scenario a bit overkill? We can simply test that > as

Re: Tid scan increments value of pg_stat_all_tables.seq_scan. (but not seq_tup_read)

2020-02-05 Thread Fujii Masao
On 2020/02/06 15:04, Kasahara Tatsuhito wrote: Hi, On Thu, Feb 6, 2020 at 11:48 AM Andres Freund wrote: I think it'd be good if we could guard against b) via an isolation test. It's more painful to do that for a), due to the unreliability of stats at the moment (we have some tests, but

Re: Tid scan increments value of pg_stat_all_tables.seq_scan. (but not seq_tup_read)

2020-02-05 Thread Kasahara Tatsuhito
Hi, On Thu, Feb 6, 2020 at 11:48 AM Andres Freund wrote: > I think it'd be good if we could guard against b) via an isolation > test. It's more painful to do that for a), due to the unreliability of > stats at the moment (we have some tests, but they take a long time). Thanks for your advise,

Re: Tid scan increments value of pg_stat_all_tables.seq_scan. (but not seq_tup_read)

2020-02-05 Thread Andres Freund
Hi, On 2020-02-05 16:25:25 +0900, Kasahara Tatsuhito wrote: > On Mon, Feb 3, 2020 at 6:20 PM Kasahara Tatsuhito > wrote: > > Therefore, from v12, Tid scan not only increases the value of > > seq_scan, but also acquires a predicate lock. > > Based on further investigation and Fujii's advice,

Re: Tid scan increments value of pg_stat_all_tables.seq_scan. (but not seq_tup_read)

2020-02-04 Thread Kasahara Tatsuhito
On Mon, Feb 3, 2020 at 6:20 PM Kasahara Tatsuhito wrote: > Therefore, from v12, Tid scan not only increases the value of > seq_scan, but also acquires a predicate lock. Based on further investigation and Fujii's advice, I've summarized this issue as follows. >From commit 147e3722f7, Tid Scan

Re: Tid scan increments value of pg_stat_all_tables.seq_scan. (but not seq_tup_read)

2020-02-03 Thread Kasahara Tatsuhito
On Mon, Feb 3, 2020 at 5:33 PM Fujii Masao wrote: > Thanks for explaining that! But heap_beginscan_internal() was really > called in TidScan case? Oh, you are right. Tid Scan started calling table_beginscan from v12 (commit 147e3722f7). So previously(-v11), Tid Scan might never calls

Re: Tid scan increments value of pg_stat_all_tables.seq_scan. (but not seq_tup_read)

2020-02-03 Thread Fujii Masao
On 2020/02/03 16:39, Kasahara Tatsuhito wrote: On Mon, Feb 3, 2020 at 4:22 PM Fujii Masao wrote: On 2020/02/01 16:05, Kasahara Tatsuhito wrote: if (scan->rs_base.rs_flags & (SO_TYPE_SEQSCAN | SO_TYPE_SAMPLESCAN)) { /* * Ensure a missing snapshot is noticed

Re: Tid scan increments value of pg_stat_all_tables.seq_scan. (but not seq_tup_read)

2020-02-02 Thread Kasahara Tatsuhito
On Mon, Feb 3, 2020 at 4:22 PM Fujii Masao wrote: > On 2020/02/01 16:05, Kasahara Tatsuhito wrote: > > if (scan->rs_base.rs_flags & (SO_TYPE_SEQSCAN | SO_TYPE_SAMPLESCAN)) > > { > > /* > > * Ensure a missing snapshot is noticed reliably, even if the > > *

Re: Tid scan increments value of pg_stat_all_tables.seq_scan. (but not seq_tup_read)

2020-02-02 Thread Fujii Masao
On 2020/02/01 16:05, Kasahara Tatsuhito wrote: On Thu, Jan 30, 2020 at 1:49 PM Kyotaro Horiguchi wrote: At Thu, 30 Jan 2020 13:30:56 +0900, Kasahara Tatsuhito wrote in TID scan : yes , seq_scan, , Here is wrong, because TID scan came to have SO_TYPE_SEQSCAN flags

Re: Tid scan increments value of pg_stat_all_tables.seq_scan. (but not seq_tup_read)

2020-01-31 Thread Kasahara Tatsuhito
On Thu, Jan 30, 2020 at 1:49 PM Kyotaro Horiguchi wrote: > At Thu, 30 Jan 2020 13:30:56 +0900, Kasahara Tatsuhito > wrote in > > > TID scan : yes , seq_scan, , > > Here is wrong, because TID scan came to have SO_TYPE_SEQSCAN flags > > from commit 147e3722f7. > > It is

Re: Tid scan increments value of pg_stat_all_tables.seq_scan. (but not seq_tup_read)

2020-01-29 Thread Kyotaro Horiguchi
At Thu, 30 Jan 2020 13:30:56 +0900, Kasahara Tatsuhito wrote in > > TID scan : yes , seq_scan, , > Here is wrong, because TID scan came to have SO_TYPE_SEQSCAN flags > from commit 147e3722f7. It is reflectings the discussion below, which means TID scan doesn't have

Re: Tid scan increments value of pg_stat_all_tables.seq_scan. (but not seq_tup_read)

2020-01-29 Thread Kasahara Tatsuhito
Hi, On Thu, Jan 30, 2020 at 10:55 AM Kyotaro Horiguchi wrote: > At Wed, 29 Jan 2020 23:24:09 +0900, Fujii Masao > wrote in > > On 2020/01/29 20:06, Kasahara Tatsuhito wrote: > > > Although I'm not sure this behavior is really problem or not, > > > it seems to me that previous behavior is more

Re: Tid scan increments value of pg_stat_all_tables.seq_scan. (but not seq_tup_read)

2020-01-29 Thread Kyotaro Horiguchi
Hello. At Wed, 29 Jan 2020 23:24:09 +0900, Fujii Masao wrote in > On 2020/01/29 20:06, Kasahara Tatsuhito wrote: > > Hi. > > Attached patch solve this problem. > > This patch adds table_beginscan_tid() and call it in TidListEval() > > instead of table_beginscan(). > > table_beginscan_tid() is

Re: Tid scan increments value of pg_stat_all_tables.seq_scan. (but not seq_tup_read)

2020-01-29 Thread Fujii Masao
On 2020/01/29 20:06, Kasahara Tatsuhito wrote: Hi. Attached patch solve this problem. This patch adds table_beginscan_tid() and call it in TidListEval() instead of table_beginscan(). table_beginscan_tid() is the same as table_beginscan() but do not set SO_TYPE_SEQSCAN to flags. Although

Re: Tid scan increments value of pg_stat_all_tables.seq_scan. (but not seq_tup_read)

2020-01-29 Thread Kasahara Tatsuhito
or not, it seems to me that previous behavior is more prefer. Is it worth to apply to HEAD and v12 branch ? Best regards, 2020年1月27日(月) 14:35 Kasahara Tatsuhito : > > Hi, I noticed that since PostgreSQL 12, Tid scan increments value of > pg_stat_all_tables.seq_scan. (but not seq

Tid scan increments value of pg_stat_all_tables.seq_scan. (but not seq_tup_read)

2020-01-26 Thread Kasahara Tatsuhito
Hi, I noticed that since PostgreSQL 12, Tid scan increments value of pg_stat_all_tables.seq_scan. (but not seq_tup_read) The following is an example. CREATE TABLE t (c int); INSERT INTO t SELECT 1; SET enable_seqscan to off; (v12 -) =# EXPLAIN ANALYZE SELECT * FROM t WHERE ctid = '(0,1