Re: POC: postgres_fdw insert batching

On Sat, 28 Nov 2020, 10:10 Tomas Vondra, wrote: > > > On 11/27/20 7:05 AM, tsunakawa.ta...@fujitsu.com wrote: > > However, the FDW interface as it's implemented today is not designed to > allow that, I believe (we pretty much just invoke the FWD callbacks as > if it was a local AM). It assumes

Re: Improving spin-lock implementation on ARM.

I wrote: > It might be that this hardware is capable of showing a difference with a > better-tuned pgbench test, but with an untuned pgbench run, we just aren't > sufficiently sensitive to the spinlock properties. (Which I guess is good > news, really.) It occurred to me that if we don't insist

Re: POC: postgres_fdw insert batching

On 11/27/20 7:05 AM, tsunakawa.ta...@fujitsu.com wrote: > From: Craig Ringer >> But in the libpq pipelining patch I demonstrated a 300 times >> (3000%) performance improvement on a test workload... > > Wow, impressive number. I've just seen it in the beginning of the > libpq pipelining

Re: A few new options for CHECKPOINT

On 11/27/20, 10:58 AM, "Stephen Frost" wrote: > If you'd like to show that I'm wrong, and it's entirely possible that I > am, then retry the above with actual load on the system, and also > actually look at how much outstanding WAL you end up with given the > different scenarios which has to be

Re: [PoC] Non-volatile WAL buffer

On 11/27/20 1:02 AM, Tomas Vondra wrote: > > Unfortunately, that patch seems to fail for me :-( > > The patches seem to be for PG12, so I applied them on REL_12_STABLE (all > the parts 0001-0005) and then I did this: > > LIBS="-lpmem" ./configure --prefix=/home/tomas/pg-12-pmem --enable-debug >

What to do about the broken btree_gist for inet/cidr?

Hi, As discussed in this thread (https://www.postgresql.org/message-id/201010112055.o9bktzf7011...@wwwmaster.postgresql.org) btree_gist is broken for the inet and cidr types. The reason is that it uses convert_network_to_scalar() to make a lossy GiST index, but convert_network_to_scalar()

Re: Improving spin-lock implementation on ARM.

Peter Eisentraut writes: > I tried this on a M1 MacBook Air. I cannot reproduce these results. > The unpatched numbers are about in the neighborhood of what you showed, > but the patched numbers are only about a few percent better, not the > 1.5x or 2x change that you showed. After redoing

[Doc Patch] Clarify that CREATEROLE roles can GRANT default roles

Hi, https://www.postgresql.org/docs/current/default-roles.html mentions the "Administrator" several times, but does not specify it further. I understand that it is mentioned elsewhere [1], but I think it would be beneficial to remind the reader on that page at least once that "Administrators"

Re: Log message for GSS connection is missing once connection authorization is successful.

Greetings, * vignesh C (vignes...@gmail.com) wrote: > Thanks for testing this, I had missed testing this. The expression > matching was not correct. Attached v6 patch which includes the fix for > this. This generally looks pretty good to me. I did reword the commit message a bit, run pgindent,

Re: bitmaps and correlation

Heikki Linnakangas writes: > Other than that, and a quick pgdindent run, this seems ready to me. I'll > mark it as Ready for Committer. I dunno, this seems largely misguided to me. It's already the case that index correlation is just not the right stat for this purpose, since it doesn't give

Re: Setof RangeType returns

Patrick Handja writes: > This is what I am doing: > static int > get_range_lower(FunctionCallInfo fcinfo, RangeType *r1) > { > /* Return NULL if there's no finite lower bound */ > if (empty || lower.infinite) > PG_RETURN_NULL(); You can't really use PG_RETURN_NULL here, mainly because there is

Re: [PATCH] LWLock self-deadlock detection

On Fri, Nov 27, 2020 at 10:22 AM Heikki Linnakangas wrote: > I've made the mistake of forgetting to release an LWLock many times, > leading to self-deadlock. And I just reviewed a patch that did that this > week [1]. You usually find that mistake very quickly when you start > testing though, I

Re: Setof RangeType returns

Hello Heikki, Thank you for responding to my email. This is what I am doing: //= C file static int get_range_lower(FunctionCallInfo fcinfo, RangeType *r1) { TypeCacheEntry *typcache; RangeBound lower; RangeBound upper; bool empty;

Re: A few new options for CHECKPOINT

Greetings, * Bossart, Nathan (bossa...@amazon.com) wrote: > On 11/27/20, 8:29 AM, "Stephen Frost" wrote: > > Also note that, in all other cases (that is, when there *is* outstanding > > WAL since the last checkpoint), pg_start_backup actually just waits for > > the existing checkpoint to

Re: proposal: possibility to read dumped table's name from file

Greetings, * Pavel Stehule (pavel.steh...@gmail.com) wrote: > > I agree that being able to configure pg_dump via a config file would > > be very useful, but the syntax proposed here feels much more like a > > hacked-up syntax designed to meet this one use case, rather than a > > good

Re: [PATCH] LWLock self-deadlock detection

On 26/11/2020 04:50, Tom Lane wrote: Craig Ringer writes: On Wed, Nov 25, 2020 at 9:23 PM Ashutosh Bapat wrote: I'd prefer to make the lock self deadlock check run for production builds, not just cassert builds. I'd like to register a strong objection to spending any cycles whatsoever on

Re: configure and DocBook XML

Hi Tom, > On 27. Nov, 2020, at 18:54, Tom Lane wrote: > > See the other thread I pointed to. ok, thanks. Cheers, Paul

Re: configure and DocBook XML

=?utf-8?Q?Paul_F=C3=B6rster?= writes: > ok, last question: is there an option for configure to not check for DocBook? > It's not installed and PostgreSQL doesn't need it to compile and run just > fine. See the other thread I pointed to. regards, tom lane

Re: A few new options for CHECKPOINT

On 11/27/20, 8:29 AM, "Stephen Frost" wrote: > Also note that, in all other cases (that is, when there *is* outstanding > WAL since the last checkpoint), pg_start_backup actually just waits for > the existing checkpoint to complete- and while it's waiting for that to > happen, there'll be

Re: configure and DocBook XML

Hi Tom, > On 27. Nov, 2020, at 16:29, Tom Lane wrote: > > To me, it makes sense to have an option to do that, but I do find it > surprising that it's the default. ok, last question: is there an option for configure to not check for DocBook? It's not installed and PostgreSQL doesn't need it to

Re: bitmaps and correlation

On 06/11/2020 19:57, Justin Pryzby wrote: On Fri, Nov 06, 2020 at 01:51:26PM +, Anastasia Lubennikova wrote: The first patch is simply a refactoring and don't see any possible objections against it. The second patch also looks fine to me. The logic is understandable and the code is neat.

Re: configure and DocBook XML

On 2020-11-26 17:48, Tom Lane wrote: =?utf-8?Q?Paul_F=C3=B6rster?= writes: On 26. Nov, 2020, at 17:21, Tom Lane wrote: There's a nearby thread in which I was suggesting that we should just not bother with this configure test [1]. [1]

Re: proposal: possibility to read dumped table's name from file

Greetings, * Tom Lane (t...@sss.pgh.pa.us) wrote: > Dean Rasheed writes: > > Actually, that raises a different possible benefit of passing options > > in an options file -- if the user wants to pass in a table name > > pattern, it can be a nuisance if the shell's argument processing does > >

Re: remove spurious CREATE INDEX CONCURRENTLY wait

Actually, I noticed two things. The first of them, addressed in this new version of the patch, is that REINDEX CONCURRENTLY is doing a lot of repetitive work by reopening each index and table in the build/validate loops, so that they can report progress. This is easy to remedy by adding a couple

Re: Removing unneeded self joins

On 31/10/2020 11:26, Andrey V. Lepikhov wrote: + /* +* Process restrictlist to seperate out the self join quals from +* the other quals. e.g x = x goes to selfjoinquals and a = b to +* otherjoinquals.

Re: A few new options for CHECKPOINT

Greetings, * Bossart, Nathan (bossa...@amazon.com) wrote: > Thanks to all for the feedback. I've attached v2 of the patch. I've > removed the WAIT and FORCE options and renamed IMMEDIATE to FAST. > > On 11/25/20, 7:52 AM, "Stephen Frost" wrote: > > I'm a bit confused by the idea here.. The

Re: Printing LSN made easy

Hi, Here, we cannot use sizeof(but) to get the buf size, because it is a pointer, so it always 8 bytes on 64-bit or 4 bytes on 32-bit machine. +char * +pg_lsn_out_buffer(XLogRecPtr lsn, char *buf) +{ + snprintf(buf, sizeof(buf), LSN_FORMAT, LSN_FORMAT_ARG(lsn)); + + return buf; +}

Re: Online verification of checksums

Greetings, * Magnus Hagander (mag...@hagander.net) wrote: > On Thu, Nov 26, 2020 at 8:42 AM Michael Paquier wrote: > > On Tue, Nov 24, 2020 at 12:38:30PM -0500, David Steele wrote: > > > We are not just looking at one LSN value. Here are the steps we are > > > proposing (I'll skip checks for

Re: parallel distinct union and aggregate support patch

I also had a quick look at the patch and the comments made so far. Summary: 1. The performance results are promising. 2. The code needs comments. Regarding the design: Thomas Munro mentioned the idea of a "Parallel Repartition" node that would redistribute tuples like this. As I understand

Re: configure and DocBook XML

=?utf-8?Q?Paul_F=C3=B6rster?= writes: > So why would xmllint try to download DocBook XML stylesheets if DocBook is > not installed? I'm not a developer but such a thing doesn't make sense to me. You'd have to ask the authors of those programs. To me, it makes sense to have an option to do

Re: proposal: unescape_text function

On 2020-10-07 11:00, Pavel Stehule wrote: Since the idea originated from unescaping unicode string literals i.e.        select unescape('Odpov\u011Bdn\u00E1 osoba'); Shouldn't the built-in function support the above syntax as well? good idea. The prefixes

Re: Printing LSN made easy

Hi, On 2020-11-27 13:40, Ashutosh Bapat wrote: Off list Peter Eisentraut pointed out that we can not use these macros in elog/ereport since it creates problems for translations. He suggested adding functions which return strings and use %s when doing so. The patch has two functions

Re: Improper use about DatumGetInt32

On Thu, Nov 26, 2020 at 9:57 PM Peter Eisentraut < peter.eisentr...@enterprisedb.com> wrote: > On 2020-11-26 14:27, Alvaro Herrera wrote: > > On 2020-Nov-26, Peter Eisentraut wrote: > > > >> The point of the patch is to have the range check somewhere. If you > just > >> cast it, then you won't

Re: Improper use about DatumGetInt32

On Wed, Nov 25, 2020 at 8:13 PM Anastasia Lubennikova < a.lubennik...@postgrespro.ru> wrote: > On 02.11.2020 18:59, Peter Eisentraut wrote: > > I have committed 0003. > > > > For 0001, normal_rand(), I think you should reject negative arguments > > with an error. > > I've updated 0001. The change

Re: Improving spin-lock implementation on ARM.

On 2020-11-26 23:55, Tom Lane wrote: ... and, after retrieving my jaw from the floor, I present the attached. Apple's chips evidently like this style of spinlock a LOT better. The difference is so remarkable that I wonder if I made a mistake somewhere. Can anyone else replicate these results?

Re: Add Information during standby recovery conflicts

On 2020-Nov-27, Drouvot, Bertrand wrote: > + if (nprocs > 0) > + { > + ereport(LOG, > + (errmsg("%s after %ld.%03d ms", > + > get_recovery_conflict_desc(reason),

Re: Feature improvement for pg_stat_statements

On 2020/11/27 21:39, Seino Yuki wrote: 2020-11-27 21:37 に Seino Yuki さんは書きました: 2020-11-16 12:28 に Seino Yuki さんは書きました: Due to similar fixed, we have also merged the patches discussed in the following thread. https://commitfest.postgresql.org/30/2736/ The patch is posted on the 2736 side of

RE: Disable WAL logging to speed up data loading

Thank you, Horiguchi-San > I haven't seen a criteria of whether a record is emitted or not for > wal_leve=none. > > We're emitting only redo logs. So I think theoretically we don't need > anything > other than the shutdown checkpoint record because we don't perform > recovery and checkpoint

Re: [HACKERS] logical decoding of two-phase transactions

On Thu, Nov 26, 2020 at 10:43 PM Amit Kapila wrote: > I think what you need to do to reproduce this is to follow the > snapshot machinery in SnapBuildFindSnapshot. Basically, first, start a > transaction (say transaction-id is 500) and do some operations but > don't commit. Here, if you create

Re: autovac issue with large number of tables

On 2020/11/27 18:38, Kasahara Tatsuhito wrote: Hi, On Fri, Nov 27, 2020 at 1:43 AM Fujii Masao wrote: On 2020/11/26 10:41, Kasahara Tatsuhito wrote: On Wed, Nov 25, 2020 at 8:46 PM Masahiko Sawada wrote: On Wed, Nov 25, 2020 at 4:18 PM Kasahara Tatsuhito wrote: Hi, On Wed, Nov

Re: Feature improvement for pg_stat_statements

2020-11-27 21:37 に Seino Yuki さんは書きました: 2020-11-16 12:28 に Seino Yuki さんは書きました: Due to similar fixed, we have also merged the patches discussed in the following thread. https://commitfest.postgresql.org/30/2736/ The patch is posted on the 2736 side of the thread. Regards. I forgot the

Re: Feature improvement for pg_stat_statements

2020-11-16 12:28 に Seino Yuki さんは書きました: Due to similar fixed, we have also merged the patches discussed in the following thread. https://commitfest.postgresql.org/30/2736/ The patch is posted on the 2736 side of the thread. Regards. The following patches have been committed and we have

Re: A problem about partitionwise join

On Tue, Nov 10, 2020 at 2:43 PM Richard Guo wrote: > > > On Fri, Nov 6, 2020 at 11:26 PM Anastasia Lubennikova > wrote: >> >> Status update for a commitfest entry. >> >> According to CFbot this patch fails to apply. Richard, can you send an >> update, please? >> >> Also, I see that the thread

Re: [patch] CLUSTER blocks scanned progress reporting

On 2020/11/27 1:51, Fujii Masao wrote: On 2020/11/27 1:45, Matthias van de Meent wrote: On Thu, 26 Nov 2020 at 00:55, Fujii Masao wrote: +    * A heap scan need not return tuples for the last page it has +    * scanned. To ensure

Printing LSN made easy

Hi All, An LSN is printed using format "%X/%X" and passing (uint32) (lsn >> 32), (uint32) lsn as arguments to that format specifier. This pattern is repeated at 180 places according to Cscope. I find it hard to remember this pattern every time I have to print LSN. Usually I end up copying from

Re: Extending range type operators to cope with elements

On 31.10.2020 01:08, Tomas Vondra wrote: Hi, On Fri, Oct 30, 2020 at 04:01:27PM +, Georgios Kokolatos wrote: Hi, thank you for your contribution. I did notice that the cfbot [1] is failing for this patch. Please try to address the issues if you can for the upcoming commitfest. I took

Re: autovac issue with large number of tables

On Fri, Nov 27, 2020 at 5:22 PM Masahiko Sawada wrote: > > On Fri, Nov 27, 2020 at 1:43 AM Fujii Masao > wrote: > > > > > > > > On 2020/11/26 10:41, Kasahara Tatsuhito wrote: > > > On Wed, Nov 25, 2020 at 8:46 PM Masahiko Sawada > > > wrote: > > >> > > >> On Wed, Nov 25, 2020 at 4:18 PM

Re: autovac issue with large number of tables

Hi, On Fri, Nov 27, 2020 at 1:43 AM Fujii Masao wrote: > > > > On 2020/11/26 10:41, Kasahara Tatsuhito wrote: > > On Wed, Nov 25, 2020 at 8:46 PM Masahiko Sawada > > wrote: > >> > >> On Wed, Nov 25, 2020 at 4:18 PM Kasahara Tatsuhito > >> wrote: > >>> > >>> Hi, > >>> > >>> On Wed, Nov 25,

RE: Disable WAL logging to speed up data loading

Hi, Tsunakawa-San > I'm afraid "none" doesn't represent the behavior because RM_XLOG_ID and > RM_XACT_ID WAL records, except for XLOG_FPI_*, are emitted. What's the > good name? IIUC, "minimal" is named after the fact that the minimal > amount of WAL necessary for crash recovery is generated.

RE: Disable WAL logging to speed up data loading

Hello, Sawada-San On Friday, November 27, 2020 3:08 PM Masahiko Sawada wrote: > - (errmsg("WAL was generated with wal_level=minimal, > data may be missing"), > + (errmsg("WAL was generated with wal_level<=minimal, > data may be missing"), >

Re: Removing unneeded self joins

On 31.10.2020 12:26, Andrey V. Lepikhov wrote: Thank you for this partial review, I included your changes: On 9/23/20 9:23 AM, David Rowley wrote: On Fri, 3 Apr 2020 at 17:43, Andrey Lepikhov wrote: Doing thing the way I describe will allow you to get rid of all the UniqueRelInfo stuff.

Re: [HACKERS] [PATCH] Generic type subscripting

Hi! I've started to review this patch. My first question is whether we're able to handle different subscript types differently. For instance, one day we could handle jsonpath subscripts for jsonb. And for sure, jsonpath subscripts are expected to be handled differently from text subscripts. I

Re: Improving spin-lock implementation on ARM.

On Fri, Nov 27, 2020 at 11:55 AM Michael Paquier wrote: > Not planning to buy one here, anything I have read on that tells that > it is worth a performance study. Another interesting area for experiments is AWS graviton2 instances. Specification says it supports arm v8.2, so it should have

Re: Add table access method as an option to pgbench

Hello David, Some feedback about v3: In the doc I find TABLEACCESSMETHOD quite hard to read. Use TABLEAM instead? Also, the next entry uses lowercase (tablespace), why change the style? Remove space after comma in help string. I'd use the more readable TABLEAM in the help string rather

Re: Setof RangeType returns

On 26/11/2020 23:28, Patrick Handja wrote: Hello, I am currently working on Library with some specific operators to manipulate RangeType in PostGreSQL. I would like to know if it is possible to return a setof rangetype elements in Postresql in C-Language function using the suggestion like

Re: Improving spin-lock implementation on ARM.

On Fri, Nov 27, 2020 at 02:50:30AM -0500, Tom Lane wrote: > Yeah, that wasn't making sense to me either. The most likely explanation > seems to be that I messed up the test somehow ... but I don't see where. > So, again, I'm wondering if anyone else can replicate or refute this. I do find your

Re: configure and DocBook XML

Hi Tom, > On 26. Nov, 2020, at 17:48, Tom Lane wrote: > > If you don't have the docbook stylesheets, but you do have xmllint, > configure's probe will cause xmllint to try to download those > stylesheets off the net. For me, that always succeeds, but it > takes two or three seconds. I find it

Re: range_agg

Hi! On Thu, Sep 24, 2020 at 3:05 AM Paul A Jungwirth wrote: > On Sun, Aug 16, 2020 at 12:55 PM Paul A Jungwirth > wrote: > > This is rebased on the current master, including some changes to doc > > tables and pg_upgrade handling of type oids. > > Here is a rebased version of this patch,

Re: Disable WAL logging to speed up data loading

At Fri, 27 Nov 2020 07:01:16 +, "tsunakawa.ta...@fujitsu.com" wrote in > From: Masahiko Sawada > > While testing the patch on some workload, I realized that > > XLOG_FPI_FOR_HINT record could still be emitted even when wal_level = > > none. IIUC that WAL record is not necessary during

Re: [proposal] de-TOAST'ing using a iterator

On 02.11.2020 22:08, John Naylor wrote: On Mon, Nov 2, 2020 at 1:30 PM Alvaro Herrera > wrote: On 2020-Nov-02, Anastasia Lubennikova wrote: > Status update for a commitfest entry. > > This entry was inactive for a very long time. > John,

Re: [Patch] Optimize dropping of relation buffers using dlist

Hi! I've found this patch is RFC on commitfest application. I've quickly checked if it's really ready for commit. It seems there are still unaddressed review notes. I'm going to switch it to WFA. -- Regards, Alexander Korotkov

Setof RangeType returns

Hello, I am currently working on Library with some specific operators to manipulate RangeType in PostGreSQL. I would like to know if it is possible to return a setof rangetype elements in Postresql in C-Language function using the suggestion like specified here:

Re: autovac issue with large number of tables

On Fri, Nov 27, 2020 at 1:43 AM Fujii Masao wrote: > > > > On 2020/11/26 10:41, Kasahara Tatsuhito wrote: > > On Wed, Nov 25, 2020 at 8:46 PM Masahiko Sawada > > wrote: > >> > >> On Wed, Nov 25, 2020 at 4:18 PM Kasahara Tatsuhito > >> wrote: > >>> > >>> Hi, > >>> > >>> On Wed, Nov 25, 2020 at