Re: Add parallelism and glibc dependent only options to reindexdb

2019-07-17 Thread Julien Rouhaud
On Wed, Jul 17, 2019 at 9:59 AM Michael Paquier wrote: > > On Tue, Jul 16, 2019 at 02:03:16PM +0200, Julien Rouhaud wrote: > > After more thinking about schema and multiple jobs, I think that > > erroring out is quite user unfriendly, as it's entirely ok to ask > > for > > multiple indexes and

Re: refactoring - share str2*int64 functions

2019-07-17 Thread Andres Freund
Hi, On 2019-07-17 12:18:19 +0900, Michael Paquier wrote: > On Tue, Jul 16, 2019 at 01:04:38PM -0700, Andres Freund wrote: > > There is the issue that there already is pg_strtoint16 and > > pg_strtoint32, which do not have the option to not raise an error. I'd > > probably name the non-error

Re: [HACKERS] WIP: Aggregation push-down

2019-07-17 Thread Antonin Houska
Richard Guo wrote: > Another core dump for query below: > > select sum(t1.s1) from t1, t2, t3, t4 where t1.j1 = t2.j2 group by t1.g1, > t2.g2; > > This is due to a small mistake: > > diff --git a/src/backend/optimizer/util/relnode.c > b/src/backend/optimizer/util/relnode.c > index

Re: refactoring - share str2*int64 functions

2019-07-17 Thread Andres Freund
Hi, On 2019-07-17 12:04:32 +0900, Michael Paquier wrote: > On Tue, Jul 16, 2019 at 01:18:38PM -0700, Andres Freund wrote: > > I'd probably also just use the implementation we have for signed > > integers (minus the relevant negation and overflow checks, obviously) - > > it's a lot faster, and I

Re: [PATCH] Make configuration file "include" directive handling more robust

2019-07-17 Thread Ian Barwick
On 7/17/19 5:34 PM, Kyotaro Horiguchi wrote:> Hello. > > At Wed, 17 Jul 2019 12:29:43 +0900, Ian Barwick wrote in <8c8bcbca-3bd9-dc6e-8986-04a5abdef...@2ndquadrant.com> >> Hi >> >> While poking about with [1], I noticed a few potential issues with the >> inclusion handling for configuration

Re: pg_stat_database update stats_reset only by pg_stat_reset

2019-07-17 Thread 张连壮
Yeah, I agree. this is not necessary, i will remove the commitfest at '2019-07-19'. Tomas Vondra 于2019年7月12日周五 下午9:07写道: > On Fri, Jul 12, 2019 at 01:51:50PM +0900, Michael Paquier wrote: > >On Thu, Jul 11, 2019 at 04:34:20PM +0200, Daniel Verite wrote: > >> I can understand why you'd want that

Re: refactoring - share str2*int64 functions

2019-07-17 Thread Andres Freund
Hi, On 2019-07-17 07:55:39 +, Fabien COELHO wrote: > > - The str->integer conversion routines, which actually have very > > similar characteristics to the strtol families as they remove trailing > > whitespaces first, check for a sign, etc, except that they work only > > on base 10. And here

Re: fix for BUG #3720: wrong results at using ltree

2019-07-17 Thread Alexander Korotkov
Hi Nikita, On Tue, Jul 16, 2019 at 6:52 PM Nikita Glukhov wrote: > I looked at "ltree syntax improvement" patch and found two more very > old bugs in ltree/lquery (fixes are attached): Thank you for the fixes. I've couple notes on them. 0001-Fix-max-size-checking-for-ltree-and-lquery.patch

Re: [HACKERS] [WIP] Effective storage of duplicates in B-tree index.

2019-07-17 Thread Anastasia Lubennikova
11.07.2019 21:19, Peter Geoghegan wrote: On Thu, Jul 11, 2019 at 8:34 AM Rafia Sabih wrote: Hi, Peter, Rafia, thanks for the review. New version is attached. + elog(DEBUG4, "insert_itupprev_to_page. compressState->ntuples %d IndexTupleSize %zu free %zu", + compressState->ntuples,

Re: Intermittent pg_ctl failures on Windows

2019-07-17 Thread r . zharkov
On 2019-07-17 20:51, Tom Lane wrote: r.zhar...@postgrespro.ru writes: pg_ctl now opens the postmaster.pid file using pgwin32_open() function to correctly handle share locks. HEAD already does that, no? See f02259fe9. You are right. I tested branch REL_11_STABLE and it is my mistake. On

Re: POC: converting Lists into arrays

2019-07-17 Thread Tom Lane
Daniel Gustafsson writes: > For cases where an Oid list is copied and then head elements immediately > removed, as in fetch_search_path, couldn’t we instead use a counter and > list_copy_tail to avoid repeated list_delete_first calls? Perhaps, but I'm having a hard time getting excited about it.

Re: pg_receivewal documentation

2019-07-17 Thread Jesper Pedersen
Hi, On 7/17/19 4:04 AM, Michael Paquier wrote: How about adding "or priority-based" after "quorum-based"? I would be fine with that for the first part. I am not sure of what a good formulation would be for the second part of the sentence. Now it only refers to quorum, but with priority sets

Re: Parallel Append subplan order instability on aye-aye

2019-07-17 Thread Tom Lane
David Rowley writes: > Surely it can't be that since that just sets what *pages gets set to. > Tom mentioned that following was returning 0 pages and tuples: > -- Temporary hack to investigate whether extra vacuum/analyze is happening > select relname, relpages, reltuples > from pg_class > where

Re: Change ereport level for QueuePartitionConstraintValidation

2019-07-17 Thread Tom Lane
Alvaro Herrera writes: > On 2019-Jul-15, David Rowley wrote: >> I think the only argument against it was around lack of ability to >> test if the constraint was used to verify no row breaks the partition >> bound during the ATTACH PARTITION. > Would it work to set client_min_messages to DEBUG1

Re: POC: converting Lists into arrays

2019-07-17 Thread Tom Lane
David Rowley writes: > I've only looked at 0002. Here are my thoughts: Thanks for looking! > get_tables_to_cluster: > Looks fine. It's a heap scan. Any previous order was accidental, so if > it causes issues then we might need to think of using a more > well-defined order for CLUSTER; Check.

Re: block-level incremental backup

2019-07-17 Thread Jeevan Chalke
On Wed, Jul 17, 2019 at 7:38 PM Ibrar Ahmed wrote: > > > On Wed, Jul 17, 2019 at 6:43 PM Jeevan Chalke < > jeevan.cha...@enterprisedb.com> wrote: > >> On Wed, Jul 17, 2019 at 2:15 PM Ibrar Ahmed >> wrote: >> >>> >>> At what stage you will apply the WAL generated in between the START/STOP >>>

sepgsql seems rather thoroughly broken on Fedora 30

2019-07-17 Thread Tom Lane
I tried to run the contrib/sepgsql tests, following the instructions, on a recently-set-up Fedora 30 machine. I've done that successfully on previous Fedora releases, but it's no go with F30. First off, building the sepgsql-regtest.pp policy file spews a bunch of complaints that I don't recall

Re: sepgsql seems rather thoroughly broken on Fedora 30

2019-07-17 Thread Tom Lane
I wrote: > I tried to run the contrib/sepgsql tests, following the instructions, > on a recently-set-up Fedora 30 machine. I've done that successfully > on previous Fedora releases, but it's no go with F30. > ... > I'm pretty sure the test recipe last worked for me on F28. > Off to try F29. On

Re: buildfarm's typedefs list has gone completely nutso

2019-07-17 Thread Andres Freund
Hi, On 2019-07-16 22:21:34 -0700, Andres Freund wrote: > It turns out that -Wi is actually all we need - so I'll probably patch > my animals to use that for now did that now. > until the bug is fixed. Bug report: https://sourceware.org/bugzilla/show_bug.cgi?id=24818 Greetings, Andres Freund

Re: Change ereport level for QueuePartitionConstraintValidation

2019-07-17 Thread Alvaro Herrera
On 2019-Jul-15, David Rowley wrote: > I think the only argument against it was around lack of ability to > test if the constraint was used to verify no row breaks the partition > bound during the ATTACH PARTITION. Would it work to set client_min_messages to DEBUG1 for the duration of the test,

Re: Change ereport level for QueuePartitionConstraintValidation

2019-07-17 Thread Alvaro Herrera
On 2019-Jul-15, David Rowley wrote: > I think the only argument against it was around lack of ability to > test if the constraint was used to verify no row breaks the partition > bound during the ATTACH PARTITION. Would it work to set client_min_messages to DEBUG1 for the duration of the test,

Re: [Proposal] Table-level Transparent Data Encryption (TDE) and Key Management Service (KMS)

2019-07-17 Thread Alvaro Herrera
On 2019-Jul-15, Bruce Momjian wrote: > My point is that doing encryption of only some data might actually make > the system slower due to the lookups, so I think we need to implement > all-cluster encryption and then see what the overhead is, and if there > are use-cases for not encrypting only

Re: A little report on informal commit tag usage

2019-07-17 Thread Alvaro Herrera
On 2019-Jul-16, Daniel Gustafsson wrote: > The green gamification dot on people’s Github profiles might light up if the > machine readable format with email address was used (and the user has that > specific email connected to their Github account unless it’s a primary email). > Looking at commit

Re: refactoring - share str2*int64 functions

2019-07-17 Thread Alvaro Herrera
On 2019-Jul-16, Thomas Munro wrote: > On Mon, Jul 15, 2019 at 5:08 PM Fabien COELHO wrote: > > The compromise I can offer is to change the name of the first one, say to > > "pg_scanint8" to reflect its former backend name. Attached a v4 which does > > a renaming so as to avoid the name

Re: Ltree syntax improvement

2019-07-17 Thread Nikita Glukhov
Hi! I have looked at the patch and found some problems. 1. I fixed some bugs (fixed patch with additional test cases is attached): -- NULL 'lptr' pointer dereference at lquery_in() =# SELECT '*'::lquery; -- crash -- '|' after '*{n}' is wrongly handled (LQPRS_WAITEND state) =# SELECT

Re: ERROR after writing PREPARE WAL record

2019-07-17 Thread Tom Lane
Asim R P writes: > Cancel/terminate requests are held off during "PREPARE TRANSACTION" > processing in function PrepareTransaction(). However, a subroutine invoked > by PrepareTransaction() may perform elog(ERROR) or elog(FATAL). Doing anything that's likely to fail in the post-commit code path

Re: block-level incremental backup

2019-07-17 Thread Jeevan Chalke
On Wed, Jul 17, 2019 at 2:15 PM Ibrar Ahmed wrote: > > At what stage you will apply the WAL generated in between the START/STOP > backup. > In this design, we are not touching any WAL related code. The WAL files will get copied with each backup either full or incremental. And thus, the last

Re: Intermittent pg_ctl failures on Windows

2019-07-17 Thread Tom Lane
r.zhar...@postgrespro.ru writes: > pg_ctl now opens the postmaster.pid file using pgwin32_open() function > to correctly handle share locks. HEAD already does that, no? See f02259fe9. > On Windows systems we cannot handle ERROR_DELETE_PENDING because > GetLastError() returns

Re: block-level incremental backup

2019-07-17 Thread Ibrar Ahmed
On Wed, Jul 17, 2019 at 6:43 PM Jeevan Chalke < jeevan.cha...@enterprisedb.com> wrote: > On Wed, Jul 17, 2019 at 2:15 PM Ibrar Ahmed wrote: > >> >> At what stage you will apply the WAL generated in between the START/STOP >> backup. >> > > In this design, we are not touching any WAL related code.

Re: POC: converting Lists into arrays

2019-07-17 Thread Daniel Gustafsson
> On 17 Jul 2019, at 01:06, Tom Lane wrote: > There are a bunch of places that are using list_delete_first to remove > the next-to-process entry from a List used as a queue. In principle, > we could invert the order of those queues and then use list_delete_last, > but I thought this would

Re: Allow simplehash to use already-calculated hash values

2019-07-17 Thread Jeff Davis
On Tue, 2019-07-16 at 15:46 -0700, Andres Freund wrote: > ISTM that it'd be best to just have a static inline helper function > employed both the hash-passing and the "traditional" insertion > routines? > Then that problem ought to not exist anymore. Agreed, attached. Regards, Jeff Davis

Re: Change ereport level for QueuePartitionConstraintValidation

2019-07-17 Thread Sergei Kornilov
Hi > It's not awful. I tried inserting "set client_min_messages = debug1" > into alter_table.sql We already did this in March. And this change was reverted in 5655565c077c53b6e9b4b9bfcdf96439cf3af065 because this will not work on buildfarm animals with log_statement = 'all' regards, Sergei

Re: refactoring - share str2*int64 functions

2019-07-17 Thread Andres Freund
Hi, On 2019-07-17 17:29:58 +0900, Michael Paquier wrote: > Actually, one thing which may be a problem is that we lack currently > the equivalents of pg_mul_s16_overflow and such for unsigned > integers. It's much simpler to implement them for unsigned than for signed, because unsigned overflow

Unwanted expression simplification in PG12b2

2019-07-17 Thread Komяpa
Hi, Many thanks for the parallel improvements in Postgres 12. Here is one of cases where a costy function gets moved from a parallel worker into main one, rendering spatial processing single core once again on some queries. Perhaps an assumption "expressions should be mashed together as much as

Re: Allow simplehash to use already-calculated hash values

2019-07-17 Thread Andres Freund
Hi, On 2019-07-17 11:17:46 -0700, Jeff Davis wrote: > From a6aba8e53f7a36a42922add68098682c2c96683e Mon Sep 17 00:00:00 2001 > From: Jeff Davis > Date: Wed, 17 Jul 2019 10:52:15 -0700 > Subject: [PATCH] Allow simplehash to use already-calculated hash values. > > Add _lookup_hash and

Re: Change ereport level for QueuePartitionConstraintValidation

2019-07-17 Thread Tom Lane
Sergei Kornilov writes: >> It's not awful. I tried inserting "set client_min_messages = debug1" >> into alter_table.sql > We already did this in March. And this change was reverted in > 5655565c077c53b6e9b4b9bfcdf96439cf3af065 because this will not work on > buildfarm animals with

Re: Allow simplehash to use already-calculated hash values

2019-07-17 Thread Jeff Davis
On Wed, 2019-07-17 at 11:59 -0700, Andres Freund wrote: > I've not tested it, but this looks reasonable to me. Do you actually > need the lookup variant, or is that more for completeness? Yes. If the hash table is full, I do a lookup. If not, I do an insert. > nitpick^3: I'd s/This is intended

Re: FETCH FIRST clause PERCENT option

2019-07-17 Thread Ryan Lambert
Surafel, On Wed, Jul 17, 2019 at 3:45 AM Surafel Temesgen wrote: > > Hi Ryan, > On Tue, Jul 9, 2019 at 4:13 PM Ryan Lambert > wrote: > >> >> "It is possible for FETCH FIRST N PERCENT to create poorly performing >> query plans when the N supplied exceeds 50 percent. In these cases query >>

Further hacking on SPITupleTable struct

2019-07-17 Thread Tom Lane
Thinking more about the public/private field distinction we just specified --- it's always annoyed me that SPITupleTable doesn't provide a number-of-valid-rows field, so that callers have to look at the entirely separate SPI_processed variable in order to make sense of SPI_tuptable. I looked a

Re: sepgsql seems rather thoroughly broken on Fedora 30

2019-07-17 Thread Joe Conway
On 7/17/19 12:54 PM, Tom Lane wrote: > I wrote: >> I tried to run the contrib/sepgsql tests, following the instructions, >> on a recently-set-up Fedora 30 machine. I've done that successfully >> on previous Fedora releases, but it's no go with F30. >> ... >> I'm pretty sure the test recipe last

Re: Unwanted expression simplification in PG12b2

2019-07-17 Thread Tom Lane
=?UTF-8?Q?Darafei_=22Kom=D1=8Fpa=22_Praliaskouski?= writes: > Many thanks for the parallel improvements in Postgres 12. Here is one of > cases where a costy function gets moved from a parallel worker into main > one, rendering spatial processing single core once again on some queries. > Perhaps

Re: using explicit_bzero

2019-07-17 Thread Alvaro Herrera
On 2019-Jul-11, Thomas Munro wrote: > Following a trail of crumbs beginning at OpenSSH's fallback > implementation of this[1], I learned that C11 has standardised > memset_s[2] for this purpose. Macs have memset_s but no > explicit_bzero. FreeBSD has both. I wonder if it'd be better to make >

Re: Unwanted expression simplification in PG12b2

2019-07-17 Thread Komяpa
Hi, On Wed, Jul 17, 2019 at 11:58 PM Tom Lane wrote: > =?UTF-8?Q?Darafei_=22Kom=D1=8Fpa=22_Praliaskouski?= > writes: > > Many thanks for the parallel improvements in Postgres 12. Here is one of > > cases where a costy function gets moved from a parallel worker into main > > one, rendering

Re: pg_receivewal documentation

2019-07-17 Thread Laurenz Albe
On Wed, 2019-07-17 at 13:59 -0400, Jesper Pedersen wrote: > + > +Note that while WAL will be flushed with this setting, > +pg_receivewal never applies it, so > + must not be set to > +remote_apply or on > +if pg_receivewal is the only synchronous >

Re: getting ERROR "relation 16401 has no triggers" with partition foreign key alter

2019-07-17 Thread Alvaro Herrera
On 2019-Jul-16, Tom Lane wrote: > Rajkumar Raghuwanshi writes: > > I am getting ERROR: relation 16401 has no triggers error while executing > > below query. > > Yeah, I can reproduce that back to v11. If you try the same scenario > with a non-partitioned table you get > > ERROR: 55006:

Re: getting ERROR "relation 16401 has no triggers" with partition foreign key alter

2019-07-17 Thread Alvaro Herrera
On 2019-Jul-17, Alvaro Herrera wrote: > I think we should just run CheckTableNotInUse for each partition in > ATExecDropConstraint. Trying that out now. Actually, that doesn't fix this problem, because the partitioned side is the *referencing* side, and ATExecDropConstraint is obsessed about

Re: PG 11 JIT deform failure

2019-07-17 Thread Andres Freund
Hi, I still haven't heard an explanation why you see a problem here. On 2019-06-27 15:54:28 +0200, didier wrote: > I searched the mailing list but found nothing. Any reason why > TupleDescAttr is a macro and not a static inline? It's present in branches that can't rely on static inlines being

New "-b slim" option in 2019b zic: should we turn that on?

2019-07-17 Thread Tom Lane
I just finished updating our timezone code to match IANA release 2019b. There's an interesting new switch in zic: if you say "-b slim", it generates zone data files that have only 64-bit data (not the 32-bit plus 64-bit data that it's been emitting for years), and it drops other space-wasting

Re: using explicit_bzero

2019-07-17 Thread Tom Lane
Alvaro Herrera writes: > On 2019-Jul-11, Thomas Munro wrote: >> Following a trail of crumbs beginning at OpenSSH's fallback >> implementation of this[1], I learned that C11 has standardised >> memset_s[2] for this purpose. Macs have memset_s but no >> explicit_bzero. FreeBSD has both. I wonder

Re: getting ERROR "relation 16401 has no triggers" with partition foreign key alter

2019-07-17 Thread Alvaro Herrera
On 2019-Jul-17, Alvaro Herrera wrote: > Actually, that doesn't fix this problem, because the partitioned side is > the *referencing* side, and ATExecDropConstraint is obsessed about the > *referenced* side only and assumes that the calling code has already > dealt with the referencing side

Re: refactoring - share str2*int64 functions

2019-07-17 Thread Fabien COELHO
- The str->integer conversion routines, which actually have very similar characteristics to the strtol families as they remove trailing whitespaces first, check for a sign, etc, except that they work only on base 10. And here we get into a state where pg_scanint8 should be actually called

Re: Parallel Append subplan order instability on aye-aye

2019-07-17 Thread Andres Freund
On 2019-07-17 11:53:48 -0400, Tom Lane wrote: > David Rowley writes: > > Surely it can't be that since that just sets what *pages gets set to. > > Tom mentioned that following was returning 0 pages and tuples: > > > -- Temporary hack to investigate whether extra vacuum/analyze is happening > >

Re: refactoring - share str2*int64 functions

2019-07-17 Thread Andres Freund
Hi, On 2019-07-17 22:59:01 +, Fabien COELHO wrote: > > > > with an interface inconsistent with its int32/int16 relatives now only > > > > in the backend. > > > > > > We can, but I'm not at ease with changing the error handling approach. > > > > Why? > > If a function reports an error to

Re: Parallel Append subplan order instability on aye-aye

2019-07-17 Thread Tom Lane
Andres Freund writes: > On 2019-07-17 11:53:48 -0400, Tom Lane wrote: >> A brute-force way to fix this (or at least reduce the odds quite a bit) >> would be to have sanity_check.sql issue a CHECKPOINT before its VACUUM, >> thereby guaranteeing that none of these pages are still in need of being

Re: Custom table AMs need to include heapam.h because of BulkInsertState

2019-07-17 Thread David Rowley
On Wed, 17 Jul 2019 at 06:46, Andres Freund wrote: > 1) Have ExecFindPartition() return via a bool* whether the partition is >being accessed for the first time. In copy.c push the partition onto >a list of to-be-bulk-finished tables. > 2) Add a execPartition.c function that returns all

Re: [PATCH] Implement INSERT SET syntax

2019-07-17 Thread Gareth Palmer
Hi Marko, > On 17/07/2019, at 5:52 PM, Marko Tiikkaja wrote: > > On Wed, Jul 17, 2019 at 7:30 AM Gareth Palmer > wrote: > Attached is a patch that adds the option of using SET clause to specify > the columns and values in an INSERT statement in the same manner as that > of an UPDATE

Re: Custom table AMs need to include heapam.h because of BulkInsertState

2019-07-17 Thread Andres Freund
Hi, On 2019-07-18 11:29:37 +1200, David Rowley wrote: > On Wed, 17 Jul 2019 at 06:46, Andres Freund wrote: > > 1) Have ExecFindPartition() return via a bool* whether the partition is > >being accessed for the first time. In copy.c push the partition onto > >a list of to-be-bulk-finished

Re: Extracting only the columns needed for a query

2019-07-17 Thread Melanie Plageman
On Sat, Jun 15, 2019 at 10:01 AM Tom Lane wrote: > Melanie Plageman writes: > > While hacking on zedstore, we needed to get a list of the columns to > > be projected--basically all of the columns needed to satisfy the > > query. The two use cases we have for this is > > 1) to pass this column

Re: Custom table AMs need to include heapam.h because of BulkInsertState

2019-07-17 Thread David Rowley
On Thu, 18 Jul 2019 at 11:36, Andres Freund wrote: > > Hi, > > On 2019-07-18 11:29:37 +1200, David Rowley wrote: > > On Wed, 17 Jul 2019 at 06:46, Andres Freund wrote: > > > 1) Have ExecFindPartition() return via a bool* whether the partition is > > >being accessed for the first time. In

Re: refactoring - share str2*int64 functions

2019-07-17 Thread Michael Paquier
On Wed, Jul 17, 2019 at 11:14:28AM -0700, Andres Freund wrote: > That'd be considerably slower, so I'm *strongly* against that. These > conversion routines are *really* hot in a number of workloads, > e.g. bulk-loading with COPY. Check e.g. >

Re: Add parallelism and glibc dependent only options to reindexdb

2019-07-17 Thread Michael Paquier
On Wed, Jul 17, 2019 at 07:46:10PM +0200, Julien Rouhaud wrote: > On Wed, Jul 17, 2019 at 9:59 AM Michael Paquier wrote: >> On top of that quick lookup, I have done an in-depth review on 0001 to >> bring it to a committable state, fixing a couple of typos, incorrect >> comments (description of

Re: Tid scan improvements

2019-07-17 Thread Edmund Horner
Thanks for the edits and fixing that pretty glaring copy-paste bug. Regarding enable_tidscan, I couldn't decide whether we really need it, and erred on the side of not adding yet another setting. The current patch only creates a tid range path if there's at least one ctid qual. But during

partition routing layering in nodeModifyTable.c

2019-07-17 Thread Andres Freund
Hi, While discussing partition related code with David in [1], I again was confused by the layering of partition related code in nodeModifyTable.c. 1) How come partition routing is done outside of ExecInsert()? case CMD_INSERT: /* Prepare for tuple routing if

Re: refactoring - share str2*int64 functions

2019-07-17 Thread Andres Freund
Hi, On 2019-07-18 09:28:28 +0900, Michael Paquier wrote: > On Wed, Jul 17, 2019 at 11:14:28AM -0700, Andres Freund wrote: > > That'd be considerably slower, so I'm *strongly* against that. These > > conversion routines are *really* hot in a number of workloads, > > e.g. bulk-loading with COPY.

Re: Custom table AMs need to include heapam.h because of BulkInsertState

2019-07-17 Thread Andres Freund
Hi, On 2019-07-18 11:57:44 +1200, David Rowley wrote: > However, I spent quite a bit of time trying to make that function as > fast as possible in v12, and since #2 seems like a perfectly good > alternative, I'd rather go with that than to add pollution to > ExecFindPartition's signature. Also,

Re: Add client connection check during the execution of the query

2019-07-17 Thread Thomas Munro
On Sat, Jul 6, 2019 at 12:27 AM Stas Kelvich wrote: > Well, indeed in case of cable disconnect only way to detect it with > proposed approach is to have tcp keepalive. However if disconnection > happens due to client application shutdown then client OS should itself > properly close than

Re: d25ea01275 and partitionwise join

2019-07-17 Thread Amit Langote
Fujita-san, Thanks for looking at this. On Tue, Jul 16, 2019 at 8:22 PM Etsuro Fujita wrote: > > On Tue, Jul 2, 2019 at 6:29 PM Amit Langote wrote: > > 0001 - fix partitionwise join to work correctly with n-way joins of > > which some are full joins (+ cosmetic improvements around the code > >

Re: Tid scan improvements

2019-07-17 Thread Andres Freund
Hi, On 2019-07-17 23:10:52 +1200, David Rowley wrote: > When I mentioned up-thread about the optional scan_setlimits table AM > callback, I'd forgotten that you'd not have access to check that > directly during planning. As you mention above, you've added > RelOptInfo has_scan_setlimits so the

Re: buildfarm's typedefs list has gone completely nutso

2019-07-17 Thread Andres Freund
Hi, On 2019-07-17 10:33:02 -0700, Andres Freund wrote: > On 2019-07-16 22:21:34 -0700, Andres Freund wrote: > > It turns out that -Wi is actually all we need - so I'll probably patch > > my animals to use that for now > > did that now. Looks like that made the generated typedef lists sane. Any

Re: Comment typo in tableam.h

2019-07-17 Thread Andres Freund
Hi, On 2019-07-11 20:44:02 -0500, Brad DeJong wrote: > More typos in tableam.h along with a few grammar changes. Thanks! Applied. Greetings, Andres Freund

Re: Speed up transaction completion faster after many relations are accessed in a transaction

2019-07-17 Thread David Rowley
On Thu, 27 Jun 2019 at 12:59, Tsunakawa, Takayuki wrote: > > From: David Rowley [mailto:david.row...@2ndquadrant.com] > Thank you, looks good. I find it ready for committer (I noticed the status > is already set so.) Thanks for looking. I've just been looking at this again and I thought I'd

Re: Built-in connection pooler

2019-07-17 Thread Ryan Lambert
Hi Konstantin, Thanks for your work on this. I'll try to do more testing in the next few days, here's what I have so far. make installcheck-world: passed The v8 patch [1] applies, though I get indent and whitespace errors: :79: tab in indent. "Each proxy launches its own

Re: [Proposal] Table-level Transparent Data Encryption (TDE) and Key Management Service (KMS)

2019-07-17 Thread Masahiko Sawada
On Fri, Jul 12, 2019 at 7:37 AM Bruce Momjian wrote: > > On Wed, Jul 10, 2019 at 12:26:24PM -0400, Bruce Momjian wrote: > > On Wed, Jul 10, 2019 at 08:31:17AM -0400, Joe Conway wrote: > > > Please see my other reply (and > > >

Re: Add client connection check during the execution of the query

2019-07-17 Thread Tatsuo Ishii
> Yeah. > > +1 for this patch, with a few adjustments including making the test > use pg_sleep() as mentioned. It does something useful, namely > cancelling very long running queries sooner if the client has gone > away instead of discovering that potentially much later when sending a >

Re: dropdb --force

2019-07-17 Thread Ryan Lambert
The following review has been posted through the commitfest application: make installcheck-world: tested, passed Implements feature: tested, passed Spec compliant: not tested Documentation:not tested Hi, The latest patch [1] applies cleanly and basic functionality

Re: [proposal] de-TOAST'ing using a iterator

2019-07-17 Thread John Naylor
On Tue, Jul 16, 2019 at 9:14 PM Binguo Bao wrote: > In the compressed beginning case, the test result is different from yours > since the patch is ~1.75x faster > rather than no improvement. The interesting thing is that the patch if 4% > faster than master in the uncompressed end case. > I

Re: ERROR after writing PREPARE WAL record

2019-07-17 Thread Asim R P
On Wed, Jul 17, 2019 at 7:08 PM Tom Lane wrote: > > Asim R P writes: > > Cancel/terminate requests are held off during "PREPARE TRANSACTION" > > processing in function PrepareTransaction(). However, a subroutine invoked > > by PrepareTransaction() may perform elog(ERROR) or elog(FATAL). > > The

Re: Add client connection check during the execution of the query

2019-07-17 Thread Thomas Munro
On Thu, Jul 18, 2019 at 3:19 PM Tatsuo Ishii wrote: > So the performance is about 5% down with the feature enabled in this > case. For me, 5% down is not subtle. Probably we should warn this in > the doc. Yeah, the timer logic is wrong. I didn't have time to look into it but with truss/strace

Re: Add client connection check during the execution of the query

2019-07-17 Thread Tatsuo Ishii
> Yeah, the timer logic is wrong. I didn't have time to look into it > but with truss/strace for some reason I see 3 setitimer() syscalls for > every query, but I think this doesn't even need to set the timer for > every query. Hum. I see 2 settimer(), instead of 3. Best regards, -- Tatsuo

Re: Add client connection check during the execution of the query

2019-07-17 Thread Tom Lane
Tatsuo Ishii writes: >> Yeah, the timer logic is wrong. I didn't have time to look into it >> but with truss/strace for some reason I see 3 setitimer() syscalls for >> every query, but I think this doesn't even need to set the timer for >> every query. > Hum. I see 2 settimer(), instead of 3.

Re: partition routing layering in nodeModifyTable.c

2019-07-17 Thread Amit Langote
Hi Andres, On Thu, Jul 18, 2019 at 10:09 AM Andres Freund wrote: > 1) How come partition routing is done outside of ExecInsert()? > > case CMD_INSERT: > /* Prepare for tuple routing if needed. */ > if (proute) > slot =

Re: pg_receivewal documentation

2019-07-17 Thread Michael Paquier
On Wed, Jul 17, 2019 at 11:21:06PM +0200, Laurenz Albe wrote: > Ok, here's another attempt: > >Note that while WAL will be flushed with this setting, >pg_receivewal never applies it, so > must not be set to >remote_apply if pg_receivewal >is the only synchronous standby. >

Re: Intermittent pg_ctl failures on Windows

2019-07-17 Thread Michael Paquier
On Wed, Jul 17, 2019 at 09:51:48AM -0400, Tom Lane wrote: > r.zhar...@postgrespro.ru writes: >> On Windows systems we cannot handle ERROR_DELETE_PENDING because >> GetLastError() returns ERROR_ACCESS_DENIED instead. >> So we rename the lock files before delete them. > > This seems improbably

Re: POC: Cleaning up orphaned files using undo logs

2019-07-17 Thread Amit Kapila
On Wed, Jul 17, 2019 at 3:37 AM Andres Freund wrote: > > Hi, > > On 2019-07-13 15:55:51 +0530, Amit Kapila wrote: > > On Fri, Jul 12, 2019 at 7:08 PM Robert Haas wrote: > > > > I think even if we currently go with a binary heap, it will be > > > > possible to change it to rbtree later, but I am

Re: partition routing layering in nodeModifyTable.c

2019-07-17 Thread Andres Freund
Hi, On 2019-07-18 14:24:29 +0900, Amit Langote wrote: > On Thu, Jul 18, 2019 at 10:09 AM Andres Freund wrote: > > 1) How come partition routing is done outside of ExecInsert()? > > > > case CMD_INSERT: > > /* Prepare for tuple routing if needed. */ > >

Re: POC: Cleaning up orphaned files using undo logs

2019-07-17 Thread Andres Freund
Hi, On 2019-07-18 11:15:05 +0530, Amit Kapila wrote: > On Wed, Jul 17, 2019 at 3:37 AM Andres Freund wrote: > > I'm not yet sure whether we'd want the rbtree nodes being pointed to > > directly by the hashtable, or whether we'd want one indirection. > > > > e.g. either something like: > > > > >

Fw: Documentation fix for adding a column with a default value

2019-07-17 Thread Daniel Westermann (DWE)
>__ >From: Daniel Westermann (DWE) >Sent: Monday, July 15, 2019 13:01 >To: pgsql-hack...@postgresql.org >Subject: Documentation fix for adding a column with a default value > >Hi, > >the tip in the "Adding a column" section is not true anymore since PostgreSQL

Re: POC: Cleaning up orphaned files using undo logs

2019-07-17 Thread Amit Kapila
On Tue, Jul 16, 2019 at 9:52 PM Robert Haas wrote: > > On Tue, Jul 16, 2019 at 7:13 AM Amit Kapila wrote: > > > I also strongly suspect it is altogether wrong to do > > > this before CommitSubTransaction sets s->state to TRANS_COMMIT; what > > > if a subxact callback throws an error? > > > >

Re: make \d pg_toast.foo show its indices ; and, \d toast show its main table ; and \d relkind=I show its partitions

2019-07-17 Thread Justin Pryzby
Find attached updated patches which also work against old servers. 1) avoid ::regnamespace; 2) don't PQgetvalue() fields which don't exist and then crash. >From 16b31dc1e4142ed6d0f5f7ed6d65c6184f546a3c Mon Sep 17 00:00:00 2001 From: Justin Pryzby Date: Tue, 30 Apr 2019 19:05:53 -0500 Subject:

Re: POC: Cleaning up orphaned files using undo logs

2019-07-17 Thread Amit Kapila
On Wed, Jul 17, 2019 at 3:53 AM Andres Freund wrote: > On 2019-07-15 12:26:21 -0400, Robert Haas wrote: Responding again with some more details. > > > > But, my understanding of the current design being implemented is that > > there is a hard limit on the number of transactions that can be > >

Re: Fw: Documentation fix for adding a column with a default value

2019-07-17 Thread Ian Barwick
On Wed, 17 Jul 2019 at 15:42, Daniel Westermann (DWE) < daniel.westerm...@dbi-services.com> wrote: > >__ > >From: Daniel Westermann (DWE) > >Sent: Monday, July 15, 2019 13:01 > >To: pgsql-hack...@postgresql.org > >Subject: Documentation fix for adding a column

Re: POC: Cleaning up orphaned files using undo logs

2019-07-17 Thread Amit Kapila
On Tue, Jul 16, 2019 at 9:44 PM Robert Haas wrote: > > On Tue, Jul 16, 2019 at 12:32 AM Amit Kapila wrote: > > The idea is that the queues can get full, but not rollback hash table. > > In the case where the error queue gets full, we mark the entry as > > Invalid in the hash table and later when

Re: UCT (Re: pgsql: Update time zone data files to tzdata release 2019a.)

2019-07-17 Thread Thomas Munro
On Thu, Jun 27, 2019 at 10:48 AM Andrew Gierth wrote: > > "Tom" == Tom Lane writes: > Tom> No zone1970.tab. > > zone.tab is an adequate substitute - a fact which I thought was > sufficiently obvious as to not be worth mentioning. > > (also see https://reviews.freebsd.org/D20646 ) FWIW this

Re: [PATCH] Make configuration file "include" directive handling more robust

2019-07-17 Thread Kyotaro Horiguchi
Hello. At Wed, 17 Jul 2019 12:29:43 +0900, Ian Barwick wrote in <8c8bcbca-3bd9-dc6e-8986-04a5abdef...@2ndquadrant.com> > Hi > > While poking about with [1], I noticed a few potential issues with the > inclusion handling for configuration files; another issue is > demonstrated in [2]. > > [1]

Re: pg_receivewal documentation

2019-07-17 Thread Michael Paquier
On Wed, Jul 17, 2019 at 07:40:48AM +0200, Laurenz Albe wrote: > I understand the concern, I'm just worried that too much accuracy may > render the sentence hard to read. > > How about adding "or priority-based" after "quorum-based"? I would be fine with that for the first part. I am not sure of

RE: libpq debug log

2019-07-17 Thread Iwata, Aya
Hi, This is a summary of the whole thread. I am currently improving PQtrace() by adjusting its output to one line per protocol message as per the advice of reviewers. Purpose: If a problem occurs, such as a slow query, you want to know which query takes time. In Current libpq, there is

Re: refactoring - share str2*int64 functions

2019-07-17 Thread Michael Paquier
On Wed, Jul 17, 2019 at 07:55:39AM +, Fabien COELHO wrote: >> numutils.c also has pg_strtoint16 and pg_strtoint32, so the locations >> become rather inconsistent with inconsistent APIs for the manipulation >> of int2 and int4 fields, and scanint8 is just a derivative of the same >> logic. We

Re: FETCH FIRST clause PERCENT option

2019-07-17 Thread Surafel Temesgen
Hi Ryan, On Tue, Jul 9, 2019 at 4:13 PM Ryan Lambert wrote: > > "It is possible for FETCH FIRST N PERCENT to create poorly performing > query plans when the N supplied exceeds 50 percent. In these cases query > execution can take an order of magnitude longer to execute than simply > returning

Re: refactoring - share str2*int64 functions

2019-07-17 Thread Fabien COELHO
Bonjour Michaël, FWIW, I was looking forward to putting my hands on this patch and try to get it merged so as we can get rid of those duplications. Here are some comments. +#ifdef FRONTEND + fprintf(stderr, + "invalid input syntax for type %s: \"%s\"\n", "bigint", str);

Re: Fw: Documentation fix for adding a column with a default value

2019-07-17 Thread Daniel Westermann (DWE)
>> Seems the first mail didn't make it ... >Actually it did, I was about to reply to it :) > >The suggested change pares down the "Tip" to more of a brief "Note", which >IMHO is a bit >terse for that section of the documentation (which has more of a tutorial >character), >and the contents of

  1   2   >