RE: [Patch] Optimize dropping of relation buffers using dlist

2020-09-30 Thread k.jami...@fujitsu.com
On Thursday, October 1, 2020 11:49 AM, Amit Kapila wrote: > On Thu, Oct 1, 2020 at 8:11 AM tsunakawa.ta...@fujitsu.com > wrote: > > > > From: Jamison, Kirk/ジャミソン カーク > > > Recovery performance measurement results below. > > > But it seems there are overhead even with large shared buffers. > > >

Re: Why does PostgresNode.pm set such a low value of max_wal_senders?

2020-09-30 Thread Noah Misch
On Tue, Sep 29, 2020 at 06:13:46PM -0400, Tom Lane wrote: > So I wonder why PostgresNode.pm is doing > > print $conf "max_wal_senders = 5\n"; > > Considering that our default these days is 10 senders, and that a > walsender slot doesn't really cost much, this seems unduly

Re: proposal: schema variables

2020-09-30 Thread Pavel Stehule
čt 1. 10. 2020 v 5:38 odesílatel Michael Paquier napsal: > On Thu, Sep 24, 2020 at 08:49:50PM +0200, Pavel Stehule wrote: > > fixed patch attached > > It looks like there are again conflicts within setrefs.c. > fresh patch Regards Pavel -- > Michael > schema-variables-20201001.patch.gz

Re: [Patch] Optimize dropping of relation buffers using dlist

2020-09-30 Thread Kyotaro Horiguchi
At Thu, 1 Oct 2020 04:20:27 +, "tsunakawa.ta...@fujitsu.com" wrote in > From: Kyotaro Horiguchi > > In more detail, if smgrcachednblocks() returned InvalidBlockNumber for > > any of the forks, we should give up the optimization at all since we > > need to run a full scan anyway. On the

Re: Implementing Incremental View Maintenance

2020-09-30 Thread Yugo NAGATA
On Thu, 1 Oct 2020 13:43:49 +0900 Fujii Masao wrote: > When I glanced the doc patch (i.e., 0012), I found some typos. Thank you for your pointing out typos! I'll fix it. > > +CRATE INCREMENTAL MATERIALIZED VIEW, for example: > > Typo: CRATE should be CREATE ? > > +with __ivm_ and

Re: Online checksums verification in the backend

2020-09-30 Thread Michael Paquier
On Fri, Sep 25, 2020 at 06:11:47PM +0800, Julien Rouhaud wrote: > Thanks a lot for the tests! I'm not surprised that forcing the lock > will slow down the pg_check_relation() execution, but I'm a bit > surprised that holding the buffer mapping lock longer in a workload > that has a lot of

Re: Manager for commit fest 2020-09

2020-09-30 Thread Michael Paquier
On Tue, Sep 01, 2020 at 10:43:47AM +0900, Michael Paquier wrote: > As of the moment this message is written, 10 hours remain until we are > the 1st of September AoE, where I'll switch the commit fest as > officially in progress. It will not be possible to register new > patches to 2020-09 after

Re: Implementing Incremental View Maintenance

2020-09-30 Thread Fujii Masao
On 2020/10/01 13:03, Tatsuo Ishii wrote: On Thu, Sep 17, 2020 at 09:42:45AM +0900, Tatsuo Ishii wrote: I am asking because these patch sets are now getting closer to committable state in my opinion, and if there's someting wrong, it should be fixed soon so that these patches are getting into

Re: Asynchronous Append on postgres_fdw nodes.

2020-09-30 Thread Kyotaro Horiguchi
At Thu, 1 Oct 2020 12:56:02 +0900, Michael Paquier wrote in > On Thu, Oct 01, 2020 at 11:16:53AM +0900, Kyotaro Horiguchi wrote: > > Thanks. Since it starts all remote nodes before local ones, the > > startup gain would be the shorter of the startup time of the fastest > > remote and the time

Re: Logical replication CPU-bound with TRUNCATE/DROP/CREATE many tables

2020-09-30 Thread Dilip Kumar
On Wed, Sep 30, 2020 at 3:00 PM Amit Kapila wrote: > > On Wed, Sep 30, 2020 at 12:16 PM Dilip Kumar wrote: > > > > On Mon, Sep 28, 2020 at 2:22 PM Amit Kapila wrote: > > > > > > > > > I don't have a patch for this idea but you can refer [1] > > >

Re: Disable WAL logging to speed up data loading

2020-09-30 Thread Kyotaro Horiguchi
At Thu, 1 Oct 2020 12:58:19 +0900, Fujii Masao wrote in > the table needs to be rewriitten. One idea for that is to improve that > command > so that it skips the table rewrite if wal_level=minimal. > Of course, also you can change wal_level after marking the table as > unlogged. tablecmd.c: >

Re: Protect syscache from bloating with negative cache entries

2020-09-30 Thread Michael Paquier
On Wed, Jan 22, 2020 at 02:38:19PM +0900, Kyotaro Horiguchi wrote: > I changed my mind to attach the benchmark patch as .txt file, > expecting the checkers not picking up it as a part of the patchset. > > I have in the precise performance measurement mode for a long time, > but I think it's

Re: New statistics for tuning WAL buffer size

2020-09-30 Thread Fujii Masao
On 2020/10/01 12:56, Masahiro Ikeda wrote: On 2020-10-01 11:33, Amit Kapila wrote: On Thu, Oct 1, 2020 at 6:53 AM Kyotaro Horiguchi wrote: At Thu, 1 Oct 2020 09:05:19 +0900, Fujii Masao wrote in > > > On 2020/09/30 20:21, Amit Kapila wrote: > > On Tue, Sep 29, 2020 at 9:23 PM Fujii

Re: PATCH: Batch/pipelining support for libpq

2020-09-30 Thread Michael Paquier
On Mon, Sep 21, 2020 at 07:55:03PM +0200, Matthieu Garrigues wrote: > I merged PQbatchProcessQueue into PQgetResult. > One first init call to PQbatchProcessQueue was also required in > PQsendQueue to have > PQgetResult ready to read the first batch query. > > Tests and documentation are updated

Re: making update/delete of inheritance trees scale better

2020-09-30 Thread Michael Paquier
On Fri, Sep 11, 2020 at 07:20:56PM +0900, Amit Langote wrote: > I have been working away at this and have updated the patches for many > cosmetic and some functional improvements. Please note that this patch set fails to apply. Could you provide a rebase please? -- Michael signature.asc

RE: [Patch] Optimize dropping of relation buffers using dlist

2020-09-30 Thread tsunakawa.ta...@fujitsu.com
From: Kyotaro Horiguchi > In more detail, if smgrcachednblocks() returned InvalidBlockNumber for > any of the forks, we should give up the optimization at all since we > need to run a full scan anyway. On the other hand, if any of the > forks is smaller than the threshold, we still can use the

Re: Logical replication CPU-bound with TRUNCATE/DROP/CREATE many tables

2020-09-30 Thread Dilip Kumar
On Thu, Oct 1, 2020 at 9:19 AM Amit Kapila wrote: > > On Thu, Oct 1, 2020 at 8:22 AM Keisuke Kuroda > wrote: > > > > Hi Dilip, Amit, > > > > Thank you for the patch! > > > > I test the patch on the master HEAD(9796f455) and it works fine. > > * make installcheck-world: passed > > * walsender

Re: POC: postgres_fdw insert batching

2020-09-30 Thread Michael Paquier
On Sun, Jul 12, 2020 at 02:11:01AM +0200, Tomas Vondra wrote: > Because I was focused on speeding-up inserts, and that is not using > CopyMultiInsertBuffer I think. I agree the way the tuples are stored > may be improved, of course. The CF bot is telling that the regression tests of postgres_fdw

Re: Implementing Incremental View Maintenance

2020-09-30 Thread Tatsuo Ishii
> On Thu, Sep 17, 2020 at 09:42:45AM +0900, Tatsuo Ishii wrote: >> I am asking because these patch sets are now getting closer to >> committable state in my opinion, and if there's someting wrong, it >> should be fixed soon so that these patches are getting into the master >> branch. >> >> I

Re: Disable WAL logging to speed up data loading

2020-09-30 Thread Fujii Masao
On 2020/10/01 12:04, osumi.takami...@fujitsu.com wrote: Hello. Can they use a database with all unlogged tables? Unfortunately, no. They want to switch a cluster condition to "without WAL logging" only when they execute night bulk loading for their data warehouse. In other words, they

Re: New statistics for tuning WAL buffer size

2020-09-30 Thread Masahiro Ikeda
On 2020-10-01 11:33, Amit Kapila wrote: On Thu, Oct 1, 2020 at 6:53 AM Kyotaro Horiguchi wrote: At Thu, 1 Oct 2020 09:05:19 +0900, Fujii Masao wrote in > > > On 2020/09/30 20:21, Amit Kapila wrote: > > On Tue, Sep 29, 2020 at 9:23 PM Fujii Masao > > wrote: > >> > >> On 2020/09/29 11:51,

Re: Asynchronous Append on postgres_fdw nodes.

2020-09-30 Thread Michael Paquier
On Thu, Oct 01, 2020 at 11:16:53AM +0900, Kyotaro Horiguchi wrote: > Thanks. Since it starts all remote nodes before local ones, the > startup gain would be the shorter of the startup time of the fastest > remote and the time required for all local nodes. Plus remote > transfer gets asynchronous

Re: ALTER TABLE .. DETACH PARTITION CONCURRENTLY

2020-09-30 Thread Michael Paquier
On Thu, Sep 24, 2020 at 12:51:52PM +0900, Amit Langote wrote: > Also, I noticed that looking up a parent's partitions via > RelationBuildPartitionDesc or directly will inspect inhdetached to > include or exclude partitions, but checking if a child table is a > partition of a given parent table via

Re: Logical replication CPU-bound with TRUNCATE/DROP/CREATE many tables

2020-09-30 Thread Amit Kapila
On Thu, Oct 1, 2020 at 8:22 AM Keisuke Kuroda wrote: > > Hi Dilip, Amit, > > Thank you for the patch! > > I test the patch on the master HEAD(9796f455) and it works fine. > * make installcheck-world: passed > * walsender process continues to use 100% of the CPU 1core when > TRUNCATE 1000

Re: proposal: schema variables

2020-09-30 Thread Michael Paquier
On Thu, Sep 24, 2020 at 08:49:50PM +0200, Pavel Stehule wrote: > fixed patch attached It looks like there are again conflicts within setrefs.c. -- Michael signature.asc Description: PGP signature

Re: Implementing Incremental View Maintenance

2020-09-30 Thread Michael Paquier
On Thu, Sep 17, 2020 at 09:42:45AM +0900, Tatsuo Ishii wrote: > I am asking because these patch sets are now getting closer to > committable state in my opinion, and if there's someting wrong, it > should be fixed soon so that these patches are getting into the master > branch. > > I think this

Re: VACUUM (INTERRUPTIBLE)?

2020-09-30 Thread Andres Freund
Hi, On 2020-09-08 21:11:47 -0700, Andres Freund wrote: > > Not yet sure about what a suitable way to test this for analyze would > > be, unless we'd also accept VacuumDelay as a wait condition :(. I guess > > one fairly easy way would be to include an expression index, and have > > the expression

Re: [Patch] Optimize dropping of relation buffers using dlist

2020-09-30 Thread Kyotaro Horiguchi
At Thu, 1 Oct 2020 02:40:52 +, "tsunakawa.ta...@fujitsu.com" wrote in > With the following code, when the main fork does not meet the > optimization criteria, other forks are not optimized as well. You > want to determine each fork's optimization separately, don't you? In more detail, if

Re: Why does PostgresNode.pm set such a low value of max_wal_senders?

2020-09-30 Thread Michael Paquier
On Tue, Sep 29, 2020 at 07:04:22PM -0400, Tom Lane wrote: > Hm. We could do so back to v10 where that came in, and there are no > src/test/subscription tests before v10, so that should be sufficient. > Sold. Since this stuff has been committed, thorntail has showed a very interesting failure

RE: Disable WAL logging to speed up data loading

2020-09-30 Thread osumi.takami...@fujitsu.com
Hello. > >> Can they use a database with all unlogged tables? > > Unfortunately, no. They want to switch a cluster condition to "without WAL > logging" > > only when they execute night bulk loading for their data warehouse. > > In other words, they would like to keep their other usual operations

RE: [Patch] Optimize dropping of relation buffers using dlist

2020-09-30 Thread tsunakawa.ta...@fujitsu.com
From: Amit Kapila > I have one idea for performance testing. We can even test this for > non-recovery paths by removing the recovery-related check like only > use it when there are cached blocks. You can do this if testing via > recovery path is difficult because at the end performance should be

Re: Disable WAL logging to speed up data loading

2020-09-30 Thread Kyotaro Horiguchi
At Thu, 1 Oct 2020 10:01:56 +0900, Fujii Masao wrote in > > > On 2020/09/30 12:10, osumi.takami...@fujitsu.com wrote: > > Hello, Ashutosh > > > >> Can they use a database with all unlogged tables? > > Unfortunately, no. They want to switch a cluster condition to "without > > WAL logging" > >

Re: Logical replication CPU-bound with TRUNCATE/DROP/CREATE many tables

2020-09-30 Thread Keisuke Kuroda
Hi Dilip, Amit, Thank you for the patch! I test the patch on the master HEAD(9796f455) and it works fine. * make installcheck-world: passed * walsender process continues to use 100% of the CPU 1core when TRUNCATE 1000 partition: 1s or less ** before patch : 230s There is

Re: [Patch] Optimize dropping of relation buffers using dlist

2020-09-30 Thread Amit Kapila
On Thu, Oct 1, 2020 at 8:11 AM tsunakawa.ta...@fujitsu.com wrote: > > From: Jamison, Kirk/ジャミソン カーク > > Recovery performance measurement results below. > > But it seems there are overhead even with large shared buffers. > > > > | s_b | master | patched | %reg | > >

RE: [Patch] Optimize dropping of relation buffers using dlist

2020-09-30 Thread tsunakawa.ta...@fujitsu.com
From: Jamison, Kirk/ジャミソン カーク > Recovery performance measurement results below. > But it seems there are overhead even with large shared buffers. > > | s_b | master | patched | %reg | > |---||-|---| > | 128MB | 36.052 | 39.451 | 8.62% | > | 1GB | 21.731 | 21.73 |

Re: New statistics for tuning WAL buffer size

2020-09-30 Thread Amit Kapila
On Thu, Oct 1, 2020 at 6:53 AM Kyotaro Horiguchi wrote: > > At Thu, 1 Oct 2020 09:05:19 +0900, Fujii Masao > wrote in > > > > > > On 2020/09/30 20:21, Amit Kapila wrote: > > > On Tue, Sep 29, 2020 at 9:23 PM Fujii Masao > > > wrote: > > >> > > >> On 2020/09/29 11:51, Masahiro Ikeda wrote: > >

Re: terminate called after throwing an instance of 'std::bad_alloc'

2020-09-30 Thread Andres Freund
Hi, On 2020-09-30 21:16:10 -0500, Justin Pryzby wrote: > Sep 30 19:40:08 database7 abrt-hook-ccpp: Process 17905 (postgres) of user 26 > killed by SIGABRT - dumping core > Core was generated by `postgres: telsasoft ts 192.168.122.11(34608) SELECT > '. > > Unfortunately, the filesystem

Re: enable_incremental_sort changes query behavior

2020-09-30 Thread James Coleman
On Sat, Sep 26, 2020 at 2:49 PM Jaime Casanova wrote: > > Hi, > > With sqlsmith I found a query that gives this error: > ERROR: ORDER/GROUP BY expression not found in targetlist > > I noted the query (sql query below, sorry it uses custom tables i > couldn't replicate with regression tables)

Re: terminate called after throwing an instance of 'std::bad_alloc'

2020-09-30 Thread Tom Lane
Justin Pryzby writes: > A VM crashed which is now running PG13.0 on centos7: > Sep 30 19:40:08 database7 abrt-hook-ccpp: Process 17905 (postgres) of user 26 > killed by SIGABRT - dumping core > Core was generated by `postgres: telsasoft ts 192.168.122.11(34608) SELECT > '. > Unfortunately,

Re: Asynchronous Append on postgres_fdw nodes.

2020-09-30 Thread Kyotaro Horiguchi
At Wed, 30 Sep 2020 16:30:41 +0900, Etsuro Fujita wrote in > On Mon, Sep 28, 2020 at 10:35 AM Kyotaro Horiguchi > wrote: > > At Sat, 26 Sep 2020 19:45:39 +0900, Etsuro Fujita > > wrote in > > > Your patch (and the original patch by Robert [1]) modified > > > ExecAppend() so that it can

terminate called after throwing an instance of 'std::bad_alloc'

2020-09-30 Thread Justin Pryzby
A VM crashed which is now running PG13.0 on centos7: Sep 30 19:40:08 database7 abrt-hook-ccpp: Process 17905 (postgres) of user 26 killed by SIGABRT - dumping core Core was generated by `postgres: telsasoft ts 192.168.122.11(34608) SELECT'. Unfortunately, the filesystem wasn't large enough

RE: New statistics for tuning WAL buffer size

2020-09-30 Thread tsunakawa.ta...@fujitsu.com
From: Kyotaro Horiguchi > Another reason that I mildly want to object to subdivided functions is > I was annoyed that a stats view makes many individual calls to > functions that internally share the same statistics entry. That > behavior required me to provide an entry-caching feature to my >

Re: __pg_log_level in anonynous enum should be initialized? (Was: pgsql: Change SHA2 implementation based on OpenSSL to use EVP digest ro)

2020-09-30 Thread Michael Paquier
On Wed, Sep 30, 2020 at 03:47:26PM -0400, Tom Lane wrote: > LGTM. Thanks, I have applied this version then. I am curious to see what the buildfarm thinks about it. -- Michael signature.asc Description: PGP signature

Re: New statistics for tuning WAL buffer size

2020-09-30 Thread Kyotaro Horiguchi
At Thu, 1 Oct 2020 09:05:19 +0900, Fujii Masao wrote in > > > On 2020/09/30 20:21, Amit Kapila wrote: > > On Tue, Sep 29, 2020 at 9:23 PM Fujii Masao > > wrote: > >> > >> On 2020/09/29 11:51, Masahiro Ikeda wrote: > >>> On 2020-09-29 11:43, Amit Kapila wrote: > On Tue, Sep 29, 2020 at

Re: Disable WAL logging to speed up data loading

2020-09-30 Thread Fujii Masao
On 2020/09/30 12:10, osumi.takami...@fujitsu.com wrote: Hello, Ashutosh Can they use a database with all unlogged tables? Unfortunately, no. They want to switch a cluster condition to "without WAL logging" only when they execute night bulk loading for their data warehouse. In other words,

Re: BUG #16419: wrong parsing BC year in to_date() function

2020-09-30 Thread Tom Lane
Robert Haas writes: > Right. Ultimately, this comes down to a judgement call about what you > think people are likely to rely on, and what you think they are > unlikely to rely on. Good, so at least we agree on that principle. > But the present case does not seem to me to be comparable. If

Re: BUG #16419: wrong parsing BC year in to_date() function

2020-09-30 Thread David G. Johnston
On Wed, Sep 30, 2020 at 5:24 PM Robert Haas wrote: > The > fact that we've suddenly discovered that this is not what Oracle does > doesn't mean that no users have discovered that it is what PostgreSQL > does. > Presently I cannot seem to make up my mind so I'm going to go with my original

Re: BUG #16419: wrong parsing BC year in to_date() function

2020-09-30 Thread Robert Haas
On Wed, Sep 30, 2020 at 7:26 PM Tom Lane wrote: > We do not have, and never have had, a project policy against > back-patching non-crash-related behavioral changes. If we did, > we would not for example put timezone database updates into the > back branches. It's not terribly hard to imagine

Re: New statistics for tuning WAL buffer size

2020-09-30 Thread Fujii Masao
On 2020/09/30 20:21, Amit Kapila wrote: On Tue, Sep 29, 2020 at 9:23 PM Fujii Masao wrote: On 2020/09/29 11:51, Masahiro Ikeda wrote: On 2020-09-29 11:43, Amit Kapila wrote: On Tue, Sep 29, 2020 at 7:39 AM Masahiro Ikeda wrote: On 2020-09-28 12:43, Amit Kapila wrote: On Mon, Sep 28,

Re: avoid bitmapOR-ing indexes with scan condition inconsistent with partition constraint

2020-09-30 Thread Soumyadeep Chakraborty
Hi Justin, Attached is a minimal patch that is rebased and removes the dependency on Konstantin's earlier patch[1], making it enough to remove the extraneous index scans as Justin brought up. Is this the direction we want to head in? Tagging Konstantin in the email to hear his input on his old

Re: BUG #16419: wrong parsing BC year in to_date() function

2020-09-30 Thread Bruce Momjian
On Wed, Sep 30, 2020 at 07:26:55PM -0400, Tom Lane wrote: > Bruce Momjian writes: > > On Wed, Sep 30, 2020 at 06:10:38PM -0400, Robert Haas wrote: > >> On Wed, Sep 30, 2020 at 5:35 PM Tom Lane wrote: > >>> By that logic, we should never fix any bug in a back branch. > > >> No, by that logic, we

Re: BUG #16419: wrong parsing BC year in to_date() function

2020-09-30 Thread Tom Lane
Bruce Momjian writes: > On Wed, Sep 30, 2020 at 06:10:38PM -0400, Robert Haas wrote: >> On Wed, Sep 30, 2020 at 5:35 PM Tom Lane wrote: >>> By that logic, we should never fix any bug in a back branch. >> No, by that logic, we should not change any behavior in a back-branch >> upon which a

Re: Improving connection scalability: GetSnapshotData()

2020-09-30 Thread Andres Freund
Hi, On 2020-09-14 16:17:18 -0700, Andres Freund wrote: > I've also included a quite heavily revised version of your test. Instead > of using dblink I went for having a long-running psql that I feed over > stdin. The main reason for not liking the previous version is that it > seems fragile, with

Re: BUG #16419: wrong parsing BC year in to_date() function

2020-09-30 Thread Bruce Momjian
On Wed, Sep 30, 2020 at 06:10:38PM -0400, Robert Haas wrote: > On Wed, Sep 30, 2020 at 5:35 PM Tom Lane wrote: > > By that logic, we should never fix any bug in a back branch. > > No, by that logic, we should not change any behavior in a back-branch > upon which a customer is plausibly relying.

Re: BUG #16419: wrong parsing BC year in to_date() function

2020-09-30 Thread Tom Lane
Robert Haas writes: > On Wed, Sep 30, 2020 at 5:35 PM Tom Lane wrote: >> By that logic, we should never fix any bug in a back branch. > No, by that logic, we should not change any behavior in a back-branch > upon which a customer is plausibly relying. I guess where we differ here is on the

CREATE TABLE .. PARTITION OF fails to preserve tgenabled for inherited row triggers

2020-09-30 Thread Justin Pryzby
CREATE TABLE t(i int) PARTITION BY RANGE(i); CREATE TABLE t1 PARTITION OF t FOR VALUES FROM (1) TO (10); CREATE OR REPLACE FUNCTION tgf() RETURNS trigger LANGUAGE plpgsql AS $$ begin raise exception 'except'; end $$; CREATE TRIGGER tg AFTER INSERT ON t FOR EACH ROW EXECUTE FUNCTION tgf(); ALTER

Re: Error on failed COMMIT

2020-09-30 Thread Andrew Dunstan
On 8/4/20 12:19 PM, Dave Cramer wrote: > Attached is the rebased patch for consideration. > > It's a bit sad this has been hanging around so long without attention. The previous discussion seems to give the patch a clean bill of health for most/all of the native drivers. Are there any

Re: BUG #16419: wrong parsing BC year in to_date() function

2020-09-30 Thread Robert Haas
On Wed, Sep 30, 2020 at 5:35 PM Tom Lane wrote: > By that logic, we should never fix any bug in a back branch. No, by that logic, we should not change any behavior in a back-branch upon which a customer is plausibly relying. No one relies on a certain query causing a server crash, for example,

Re: BUG #16419: wrong parsing BC year in to_date() function

2020-09-30 Thread Tom Lane
Robert Haas writes: > On Tue, Sep 29, 2020 at 1:26 PM Tom Lane wrote: >> I think this is nuts. The current behavior is obviously broken; >> we should just treat it as a bug and fix it, including back-patching. >> I do not think there is a compatibility problem of any significance. >> Who out

Re: BUG #16419: wrong parsing BC year in to_date() function

2020-09-30 Thread Robert Haas
On Tue, Sep 29, 2020 at 1:26 PM Tom Lane wrote: > I think this is nuts. The current behavior is obviously broken; > we should just treat it as a bug and fix it, including back-patching. > I do not think there is a compatibility problem of any significance. > Who out there is going to have an

Re: BUG #16419: wrong parsing BC year in to_date() function

2020-09-30 Thread Bruce Momjian
On Wed, Sep 30, 2020 at 03:11:06PM -0400, Tom Lane wrote: > Bruce Momjian writes: > > On Wed, Sep 30, 2020 at 02:50:31PM -0400, Tom Lane wrote: > >> Actually, I was just finishing up back-patching the patch I posted > >> yesterday. I think we should just fix it, not document that it's > >>

Re: [PATCH] Automatic HASH and LIST partition creation

2020-09-30 Thread Rahila Syed
Hi Anastasia, I tested the syntax with some basic commands and it works fine, regression tests also pass. Couple of comments: 1. The syntax used omits the { IMMEDIATE | DEFERRED} keywords suggested in the earlier discussions. I think it is intuitive to include IMMEDIATE with the current

Re: __pg_log_level in anonynous enum should be initialized? (Was: pgsql: Change SHA2 implementation based on OpenSSL to use EVP digest ro)

2020-09-30 Thread Tom Lane
Michael Paquier writes: > However I would prefer the style of the attached. LGTM. regards, tom lane

Re: BUG #16419: wrong parsing BC year in to_date() function

2020-09-30 Thread Tom Lane
Bruce Momjian writes: > On Wed, Sep 30, 2020 at 02:50:31PM -0400, Tom Lane wrote: >> Actually, I was just finishing up back-patching the patch I posted >> yesterday. I think we should just fix it, not document that it's >> broken. > Agreed, that's what I wanted. You stated in a later email you

Re: BUG #16419: wrong parsing BC year in to_date() function

2020-09-30 Thread Bruce Momjian
On Wed, Sep 30, 2020 at 02:50:31PM -0400, Tom Lane wrote: > Bruce Momjian writes: > > On Wed, Sep 30, 2020 at 02:11:54PM -0400, Tom Lane wrote: > >> Hm, I read your reference to "the release notes" as suggesting that > >> we should change it only in a major release, ie HEAD only (and it > >>

Re: BUG #16419: wrong parsing BC year in to_date() function

2020-09-30 Thread Tom Lane
Bruce Momjian writes: > On Wed, Sep 30, 2020 at 02:11:54PM -0400, Tom Lane wrote: >> Hm, I read your reference to "the release notes" as suggesting that >> we should change it only in a major release, ie HEAD only (and it >> looks like David read it the same). If you meant minor release notes,

Re: BUG #16419: wrong parsing BC year in to_date() function

2020-09-30 Thread Bruce Momjian
On Wed, Sep 30, 2020 at 02:11:54PM -0400, Tom Lane wrote: > Bruce Momjian writes: > > On Tue, Sep 29, 2020 at 01:26:29PM -0400, Tom Lane wrote: > >> Bruce Momjian writes: > >>> On Fri, Sep 4, 2020 at 12:45:36PM -0700, David G. Johnston wrote: > Because we already have the to_date/make_date

Re: BUG #16419: wrong parsing BC year in to_date() function

2020-09-30 Thread Tom Lane
Bruce Momjian writes: > On Tue, Sep 29, 2020 at 01:26:29PM -0400, Tom Lane wrote: >> Bruce Momjian writes: >>> On Fri, Sep 4, 2020 at 12:45:36PM -0700, David G. Johnston wrote: Because we already have the to_date/make_date inconsistency, and the -1 to -2 BC mapping is confusing, and

Re: [PATCH] audo-detect and use -moutline-atomics compilation flag for aarch64

2020-09-30 Thread Tom Lane
"Zidenberg, Tsahi" writes: > On 29/09/2020, 10:21, "Heikki Linnakangas" wrote: >> If it's a good idea to use -moutline-atomics, I would expect the >> compiler or distribution to enable it by default. And as you pointed >> out, many have. > -moutline-atomics is only enabled by

Re: Retry Cached Remote Connections for postgres_fdw in case remote backend gets killed/goes away

2020-09-30 Thread Fujii Masao
On 2020/09/30 21:02, Bharath Rupireddy wrote: On Tue, Sep 29, 2020 at 10:01 PM Fujii Masao wrote: I think this is okay, because pg_terminate_backend() sends SIGTERM to the backend, and upon receiving SIGTERM the signal handler die() will be called and since there is no query being

Re: [PATCH] audo-detect and use -moutline-atomics compilation flag for aarch64

2020-09-30 Thread Heikki Linnakangas
On 30/09/2020 19:04, Zidenberg, Tsahi wrote: Ubuntu 20.04 even turned it off by default for gcc-10, which seems like a compatibility step with the main gcc-9 compiler. Ok, I definitely don't want to override that decision. I'm marking this as Rejected in the commitfest. But thanks for the

Re: BUG #16419: wrong parsing BC year in to_date() function

2020-09-30 Thread Bruce Momjian
On Tue, Sep 29, 2020 at 01:26:29PM -0400, Tom Lane wrote: > Bruce Momjian writes: > > On Fri, Sep 4, 2020 at 12:45:36PM -0700, David G. Johnston wrote: > >>> Because we already have the to_date/make_date inconsistency, and the -1 > >>> to -2 BC mapping is confusing, and doesn't match Oracle, I

Re: [PATCH] audo-detect and use -moutline-atomics compilation flag for aarch64

2020-09-30 Thread Zidenberg, Tsahi
On 29/09/2020, 10:21, "Heikki Linnakangas" wrote: > If it's a good idea to use -moutline-atomics, I would expect the > compiler or distribution to enable it by default. And as you pointed > out, many have. -moutline-atomics is only enabled by default on the gcc-10 branch where it

Re: VACUUM PARALLEL option vs. max_parallel_maintenance_workers

2020-09-30 Thread Robert Haas
On Tue, Sep 22, 2020 at 3:20 AM David Rowley wrote: > It would be good if we were consistent with these parallel options. > Right now max_parallel_workers_per_gather will restrict the > parallel_workers reloption. I'd say this > max_parallel_workers_per_gather is similar to >

Re: Residual cpluspluscheck issues

2020-09-30 Thread Tom Lane
Jesse Zhang writes: > So it's been 17 months since you sent this email, so I'm not sure that > nothing has happened (off list or in the code base), but... Well, we fixed the case that was discussed at the time [1]. I'm not exactly convinced about removing the register keywords in s_lock.h.

Re: Residual cpluspluscheck issues

2020-09-30 Thread Jesse Zhang
Hi Tom, So it's been 17 months since you sent this email, so I'm not sure that nothing has happened (off list or in the code base), but... On Sun, Jun 2, 2019 at 9:53 AM Tom Lane wrote: > * On FreeBSD 12, I get > > /home/tgl/pgsql/src/include/utils/hashutils.h:23:23: warning: 'register' >

Re: VACUUM PARALLEL option vs. max_parallel_maintenance_workers

2020-09-30 Thread Amit Kapila
On Tue, Sep 29, 2020 at 3:13 PM Peter Eisentraut wrote: > > On 2020-09-26 07:32, Amit Kapila wrote: > > This is exactly my feeling too. But how about changing documentation a > > bit as proposed above [1] to make it precise. > > > > [1] - > >

Re: [PATCH] Add section headings to index types doc

2020-09-30 Thread Heikki Linnakangas
On 30/09/2020 14:25, Dagfinn Ilmari Mannsåker wrote: Michael Paquier writes: On Mon, Aug 10, 2020 at 12:52:17PM +, Jürgen Purtz wrote: The new status of this patch is: Waiting on Author This has not been answered yet, so I have marked the patch as returned with feedback. Updated

Re: Retry Cached Remote Connections for postgres_fdw in case remote backend gets killed/goes away

2020-09-30 Thread Bharath Rupireddy
On Tue, Sep 29, 2020 at 10:01 PM Fujii Masao wrote: > > > I think this is okay, because pg_terminate_backend() sends SIGTERM to > > the backend, and upon receiving SIGTERM the signal handler die() will > > be called and since there is no query being executed on the backend by > > the time SIGTERM

Re: WIP: BRIN multi-range indexes

2020-09-30 Thread John Naylor
On Mon, Sep 28, 2020 at 10:12 PM Tomas Vondra wrote: > Is it actually all that different from the existing BRIN indexes? > Consider this example: > > create table x (a text, b text, c text); > > create index on x using brin (a,b,c); > > create or replace function random_str(p_len int) returns

Re: Change a constraint's index - ALTER TABLE ... ALTER CONSTRAINT ... USING INDEX ...

2020-09-30 Thread Laurenz Albe
On Wed, 2020-09-30 at 16:27 +0900, Michael Paquier wrote: > On Fri, Sep 04, 2020 at 01:59:47PM +0200, Laurenz Albe wrote: > > I'll set the commitfest entry to "waiting for author". > > This review, as well as any of the follow-up emails, have not been > answered by the author, so I have marked

Re: [PATCH] Add section headings to index types doc

2020-09-30 Thread Dagfinn Ilmari Mannsåker
Michael Paquier writes: > On Mon, Aug 10, 2020 at 12:52:17PM +, Jürgen Purtz wrote: >> The new status of this patch is: Waiting on Author > > This has not been answered yet, so I have marked the patch as returned > with feedback. Updated patch attached, wich reformats the operator lists as

Re: New statistics for tuning WAL buffer size

2020-09-30 Thread Amit Kapila
On Tue, Sep 29, 2020 at 9:23 PM Fujii Masao wrote: > > On 2020/09/29 11:51, Masahiro Ikeda wrote: > > On 2020-09-29 11:43, Amit Kapila wrote: > >> On Tue, Sep 29, 2020 at 7:39 AM Masahiro Ikeda > >> wrote: > >>> > >>> On 2020-09-28 12:43, Amit Kapila wrote: > >>> > On Mon, Sep 28, 2020 at 8:24

Re: Resetting spilled txn statistics in pg_stat_replication

2020-09-30 Thread Dilip Kumar
On Wed, Sep 30, 2020 at 2:40 PM Amit Kapila wrote: > > On Wed, Sep 30, 2020 at 1:12 PM Dilip Kumar wrote: > > > > On Fri, Sep 25, 2020 at 4:33 PM Amit Kapila wrote: > > > > > > On Thu, Sep 24, 2020 at 5:44 PM Amit Kapila > > > wrote: > > > > > > I have done some more testing of this patch

Re: Use PG_FINALLY to simplify code

2020-09-30 Thread Kyotaro Horiguchi
At Tue, 29 Sep 2020 23:10:52 -0400, Tom Lane wrote in tgl> Kyotaro Horiguchi writes: tgl> > At Tue, 29 Sep 2020 01:03:13 +, "Hou, Zhijie" wrote in tgl> >> Since PG_FINALLY can be used now, I think we can use PG_FINALLY to simplify code here. tgl> tgl> > The patch removes PG_RETHROW(),

Re: [HACKERS] logical decoding of two-phase transactions

2020-09-30 Thread Dilip Kumar
On Wed, Sep 30, 2020 at 3:27 PM Amit Kapila wrote: > > On Wed, Sep 30, 2020 at 3:12 PM Dilip Kumar wrote: > > > > On Wed, Sep 30, 2020 at 3:08 PM Amit Kapila wrote: > > > > > > On Wed, Sep 30, 2020 at 2:46 PM Dilip Kumar wrote: > > > > > > > > On Wed, Sep 30, 2020 at 2:36 PM Amit Kapila > >

Re: [HACKERS] logical decoding of two-phase transactions

2020-09-30 Thread Amit Kapila
On Wed, Sep 30, 2020 at 3:12 PM Dilip Kumar wrote: > > On Wed, Sep 30, 2020 at 3:08 PM Amit Kapila wrote: > > > > On Wed, Sep 30, 2020 at 2:46 PM Dilip Kumar wrote: > > > > > > On Wed, Sep 30, 2020 at 2:36 PM Amit Kapila > > > wrote: > > > > > > > > On Tue, Sep 29, 2020 at 8:04 PM Dilip Kumar

Re: [HACKERS] logical decoding of two-phase transactions

2020-09-30 Thread Dilip Kumar
On Wed, Sep 30, 2020 at 3:08 PM Amit Kapila wrote: > > On Wed, Sep 30, 2020 at 2:46 PM Dilip Kumar wrote: > > > > On Wed, Sep 30, 2020 at 2:36 PM Amit Kapila wrote: > > > > > > On Tue, Sep 29, 2020 at 8:04 PM Dilip Kumar wrote: > > > > > > > > I have started looking into you latest patches,

Re: [HACKERS] logical decoding of two-phase transactions

2020-09-30 Thread Amit Kapila
On Wed, Sep 30, 2020 at 2:46 PM Dilip Kumar wrote: > > On Wed, Sep 30, 2020 at 2:36 PM Amit Kapila wrote: > > > > On Tue, Sep 29, 2020 at 8:04 PM Dilip Kumar wrote: > > > > > > I have started looking into you latest patches, as of now I have a > > > few comments. > > > > > > v6-0001 > > > > >

Re: Logical replication CPU-bound with TRUNCATE/DROP/CREATE many tables

2020-09-30 Thread Amit Kapila
On Wed, Sep 30, 2020 at 12:16 PM Dilip Kumar wrote: > > On Mon, Sep 28, 2020 at 2:22 PM Amit Kapila wrote: > > > > > > I don't have a patch for this idea but you can refer [1] > > (v25-0002-Issue-individual-invalidations-with-wal_level-lo) to see > > what I was trying to say about registering

Re: [HACKERS] logical decoding of two-phase transactions

2020-09-30 Thread Dilip Kumar
On Wed, Sep 30, 2020 at 2:36 PM Amit Kapila wrote: > > On Tue, Sep 29, 2020 at 8:04 PM Dilip Kumar wrote: > > > > I have started looking into you latest patches, as of now I have a > > few comments. > > > > v6-0001 > > > > @@ -1987,7 +2072,7 @@ ReorderBufferProcessTXN(ReorderBuffer *rb, > >

Re: Resetting spilled txn statistics in pg_stat_replication

2020-09-30 Thread Amit Kapila
On Wed, Sep 30, 2020 at 1:12 PM Dilip Kumar wrote: > > On Fri, Sep 25, 2020 at 4:33 PM Amit Kapila wrote: > > > > On Thu, Sep 24, 2020 at 5:44 PM Amit Kapila wrote: > > > > I have done some more testing of this patch especially for the case > > where we spill before streaming the transaction

Re: [DOC] Document concurrent index builds waiting on each other

2020-09-30 Thread Michael Paquier
On Tue, Sep 08, 2020 at 01:25:21PM -0400, James Coleman wrote: > Álvaro's patch confused the current state of this thread, so I'm > reattaching (rebased) v2 as v3. + + CREATE INDEX (including the CONCURRENTLY + option) commands are included when VACUUM calculates what + dead tuples are

Re: [HACKERS] logical decoding of two-phase transactions

2020-09-30 Thread Amit Kapila
On Tue, Sep 29, 2020 at 8:04 PM Dilip Kumar wrote: > > I have started looking into you latest patches, as of now I have a > few comments. > > v6-0001 > > @@ -1987,7 +2072,7 @@ ReorderBufferProcessTXN(ReorderBuffer *rb, > ReorderBufferTXN *txn, > prev_lsn = change->lsn; > > /* Set the current

Re: Adding Support for Copy callback functionality on COPY TO api

2020-09-30 Thread Andrey V. Lepikhov
On 7/2/20 2:41 AM, Sanaba, Bilva wrote: Hi hackers, Currently, the COPY TO api does not support callback functions, while the COPY FROM api does. The COPY TO code does, however, include placeholders for supporting callbacks in the future. Rounding out the support of callback functions to

PoC patch: expose TCP socket stats for walsenders

2020-09-30 Thread Craig Ringer
Hi all I've attached a PoC patch demonstrating how to use the linux ioctls SIOCINQ and SIOCOUTQ and its getsockopt option TCP_INFO to expose a lot of useful network socket info directly in system views. Sample output from pg_stat_replication, \x format, trimmed, from a test run where I

Re: pgbench - refactor init functions with buffers

2020-09-30 Thread Heikki Linnakangas
On 09/07/2020 10:05, Fabien COELHO wrote: in favor of *PQExpBuffer(). Attached v7 is rebased v5 which uses PQExpBuffer, per cfbot. Thanks! I pushed this with small changes: - I left out the changes to executeStatement(). I'm not quite convinced it's a good idea or worth it, and it's

Re: Libpq support to connect to standby server as priority

2020-09-30 Thread Greg Nancarrow
On Sun, Sep 27, 2020 at 4:34 AM Tom Lane wrote: > > > Thoughts? > Thanks for your thoughts, patches and all the pointers. I'll be looking at all of them. (And yes, the comma instead of bitwise OR is of course an error, somehow made and gone unnoticed; the next field in the struct is an enum, so

Re: Ltree syntax improvement

2020-09-30 Thread Michael Paquier
On Wed, Jul 01, 2020 at 03:23:30PM +0200, Daniel Gustafsson wrote: > Please submit a rebased version of the patch. Which has not happened after two months, so I have marked the patch as returned with feedback. -- Michael signature.asc Description: PGP signature

Re: Implement for window functions

2020-09-30 Thread Michael Paquier
On Wed, Jul 01, 2020 at 02:27:45PM +0200, Daniel Gustafsson wrote: > This was with GCC in the Travis build, the Windows build passed and so does > clang locally for me. This was two months ago, so this patch has been marked as returned with feedback. Please feel free to resubmit once you have a

  1   2   >