Re: Use simplehash.h instead of dynahash in SMgr

2021-04-24 Thread Yura Sokolov
David Rowley писал 2021-04-25 05:23: Thanks for having a look at this. "On Sun, 25 Apr 2021 at 10:27, Yura Sokolov wrote: It is quite interesting result. Simplehash being open-addressing with linear probing is friendly for cpu cache. I'd recommend to define SH_FILLFACTOR with value lower th

Re: Forget close an open relation in ReorderBufferProcessTXN()

2021-04-24 Thread Japin Li
On Fri, 23 Apr 2021 at 22:33, osumi.takami...@fujitsu.com wrote: > On Saturday, April 17, 2021 4:13 PM Amit Kapila > wrote: >> On Sat, Apr 17, 2021 at 12:05 PM Japin Li wrote: >> > >> > On Sat, 17 Apr 2021 at 14:09, Amit Kapila >> wrote: >> > > On Thu, Apr 15, 2021 at 4:53 PM Amit Kapila >

Re: Use simplehash.h instead of dynahash in SMgr

2021-04-24 Thread David Rowley
Thanks for having a look at this. "On Sun, 25 Apr 2021 at 10:27, Yura Sokolov wrote: > > It is quite interesting result. Simplehash being open-addressing with > linear probing is friendly for cpu cache. I'd recommend to define > SH_FILLFACTOR with value lower than default (0.9). I believe 0.75 i

Re: Use simplehash.h instead of dynahash in SMgr

2021-04-24 Thread Yura Sokolov
David Rowley писал 2021-04-24 18:58: Hackers, Last year, when working on making compactify_tuples() go faster for 19c60ad69, I did quite a bit of benchmarking of the recovery process. The next thing that was slow after compactify_tuples() was the hash lookups done in smgropen(). Currently, we u

Re: decoupling table and index vacuum

2021-04-24 Thread Peter Geoghegan
On Sat, Apr 24, 2021 at 1:17 PM Peter Geoghegan wrote: > On Sat, Apr 24, 2021 at 12:56 PM Andres Freund wrote: > > Imo the question isn't really whether criteria will ever do something > > wrong, but how often and how consequential such mistakes will > > be. E.g. unnecessarily vacuuming an index

Re: decoupling table and index vacuum

2021-04-24 Thread Peter Geoghegan
On Sat, Apr 24, 2021 at 12:56 PM Andres Freund wrote: > Did anybody actually argue for using #live entries directly? I think > *dead* entries is more relevant, partiuclarly because various forms of > local cleanup can be taken into account. Live tuples might come in to > put the number of dead tup

Re: decoupling table and index vacuum

2021-04-24 Thread Andres Freund
Hi, On 2021-04-24 11:59:29 -0700, Peter Geoghegan wrote: > The number of live tuples (or even dead tuples) in the whole entire > index is simply not a useful proxy for what actually matters -- this > is 100% clear. Did anybody actually argue for using #live entries directly? I think *dead* entrie

Re: cleaning up PostgresNode.pm

2021-04-24 Thread Alvaro Herrera
On 2021-Apr-24, Andrew Dunstan wrote: > > I would like to undertake some housekeeping on PostgresNode.pm. > > 1. OO modules in perl typically don't export anything. We should remove > the export settings. That would mean that clients would have to call > "PostgresNode->get_new_node()" (but see i

cleaning up PostgresNode.pm

2021-04-24 Thread Andrew Dunstan
I would like to undertake some housekeeping on PostgresNode.pm. 1. OO modules in perl typically don't export anything. We should remove the export settings. That would mean that clients would have to call "PostgresNode->get_new_node()" (but see item 2) and "PostgresNode::get_free_port()" instead

Re: decoupling table and index vacuum

2021-04-24 Thread Peter Geoghegan
On Sat, Apr 24, 2021 at 11:43 AM Andres Freund wrote: > I don't see how that's good enough as a general approach. It won't work > on indexes that insert on one end, delete from the other (think > inserted_at or serial primary keys in many workloads). That can be treated as another extreme that we

Re: decoupling table and index vacuum

2021-04-24 Thread Andres Freund
Hi, On 2021-04-24 11:21:49 -0700, Peter Geoghegan wrote: > To expand on this a bit, my objection to counting the number of live > tuples in the index (as a means to determining how aggressively each > individual index needs to be vacuumed) is this: it's driven by > positive feedback, not negative

Re: decoupling table and index vacuum

2021-04-24 Thread Peter Geoghegan
On Fri, Apr 23, 2021 at 1:04 PM Peter Geoghegan wrote: > I think that a simple heuristic could work very well here, but it > needs to be at least a little sensitive to the extremes. And I mean > all of the extremes, not just the one from my example -- every > variation exists and will cause proble

Re: pg_amcheck option to install extension

2021-04-24 Thread Andrew Dunstan
On 4/18/21 5:58 PM, Mark Dilger wrote: > >> On Apr 18, 2021, at 6:19 AM, Andrew Dunstan wrote: >> >> how about specifying pg_catalog as the schema instead of public? > Done. > Pushed with slight changes. cheers andrew -- Andrew Dunstan EDB: https://www.enterprisedb.com

Re: compute_query_id and pg_stat_statements

2021-04-24 Thread Tom Lane
Bruce Momjian writes: > That's a pretty weird API. I think we just need people to turn it on > like they are doing when the configure pg_stat_statements anyway. > pg_stat_statements already requires configuration anyway. Agreed. If pg_stat_statements were zero-configuration today then this wou

Re: compute_query_id and pg_stat_statements

2021-04-24 Thread Bruce Momjian
On Sat, Apr 24, 2021 at 06:48:53PM +0200, Magnus Hagander wrote: > > I think the query overhead was too high (2%) to enable it by default: > > > > > > https://www.postgresql.org/message-id/20201016160355.GA31474@alvherre.pgsql > > Personally I'd say 2% is not too high to turn it on by def

Re: compute_query_id and pg_stat_statements

2021-04-24 Thread Magnus Hagander
On Sat, Apr 24, 2021 at 5:22 PM Bruce Momjian wrote: > > On Sat, Apr 24, 2021 at 11:54:25PM +0900, Fujii Masao wrote: > > When compute_query_id is not enabled (this is the default setting), > > pg_stat_statements doesn't track any statements. This means that > > we will see no entries in pg_stat_s

Use simplehash.h instead of dynahash in SMgr

2021-04-24 Thread David Rowley
Hackers, Last year, when working on making compactify_tuples() go faster for 19c60ad69, I did quite a bit of benchmarking of the recovery process. The next thing that was slow after compactify_tuples() was the hash lookups done in smgropen(). Currently, we use dynahash hash tables to store the SM

Re: compute_query_id and pg_stat_statements

2021-04-24 Thread Bruce Momjian
On Sat, Apr 24, 2021 at 11:54:25PM +0900, Fujii Masao wrote: > When compute_query_id is not enabled (this is the default setting), > pg_stat_statements doesn't track any statements. This means that > we will see no entries in pg_stat_statements by default. I'm afraid that > users may easily forget

Re: compute_query_id and pg_stat_statements

2021-04-24 Thread Julien Rouhaud
Le sam. 24 avr. 2021 à 22:54, Fujii Masao a écrit : > For example, we already have one report about this issue, in [1]. > this report was only a few days after the patch changing the behavior was committed, unless you've been following the original thread (which has been going on for 2 years), t

compute_query_id and pg_stat_statements

2021-04-24 Thread Fujii Masao
When compute_query_id is not enabled (this is the default setting), pg_stat_statements doesn't track any statements. This means that we will see no entries in pg_stat_statements by default. I'm afraid that users may easily forget to enable compute_query_id when using pg_stat_statements (because th

Re: multi-install PostgresNode fails with older postgres versions

2021-04-24 Thread Andrew Dunstan
On 4/24/21 1:54 AM, Michael Paquier wrote: > On Fri, Apr 23, 2021 at 08:10:01AM -0400, Andrew Dunstan wrote: >> Yeah, I think it's ok for comparison purposes just to lump them all >> together. Here's a patch that does that and some consequent cleanup. >> Note we now cache the string rather than t

Re: Binary search in ScalarArrayOpExpr for OR'd constant arrays

2021-04-24 Thread David Rowley
On Wed, 14 Apr 2021 at 05:40, James Coleman wrote: > ...and here's a draft patch. I can take this to a new thread if you'd > prefer; the one here already got committed, on the other hand this is > pretty strongly linked to this discussion, so I figured it made sense > to post it here. I only glan