Re: Delay locking partitions during query execution

2019-01-27 Thread Amit Langote
On 2019/01/28 10:26, David Rowley wrote: > On Tue, 4 Dec 2018 at 00:42, David Rowley > wrote: >> Over here and along similar lines to the above, but this time I'd like >> to take this even further and change things so we don't lock *any* >> partitions during AcquireExecutorLocks() and instead

Re: Covering GiST indexes

2019-01-27 Thread Andreas Karlsson
On 11/26/18 5:56 PM, Andrey Borodin wrote: Here's rebased version. Thanks! Here is my review. = General The features seems useful and an obvious extension of covering B-trees, and a potentially useful feature together with exclusion constraints. The patch still applies (with git apply

Re: Emacs vs pg_indent's weird indentation for function declarations

2019-01-27 Thread Michael Paquier
On Mon, Jan 28, 2019 at 12:28:30AM -0500, Tom Lane wrote: > Thomas Munro writes: >> That's ... annoying. I wish indent wouldn't do that, because it means >> that my declarations get moved around every time I write code. > > If you can fix it, I'd vote for accepting the patch. I don't

Re: Prepare Transaction support for ON COMMIT DROP temporary tables

2019-01-27 Thread Michael Paquier
On Sat, Jan 19, 2019 at 10:39:43AM +0900, Michael Paquier wrote: > I have not looked at the patch in details, but we should really be > careful that if we do that the namespace does not remain behind when > performing such transactions so as it cannot be dropped. On my very > recent lookups of

Re: Tab completion for ALTER INDEX|TABLE ALTER COLUMN SET STATISTICS

2019-01-27 Thread Tatsuro Yamada
Hi Michael, On 2019/01/28 15:31, Michael Paquier wrote: On Mon, Jan 28, 2019 at 02:18:25PM +0900, Tatsuro Yamada wrote: I modified the patch to handle the both: - quoted index names - complete partial numbers Committed, which should close the loop for this thread. If you have

Re: Tab completion for ALTER INDEX|TABLE ALTER COLUMN SET STATISTICS

2019-01-27 Thread Michael Paquier
On Mon, Jan 28, 2019 at 02:18:25PM +0900, Tatsuro Yamada wrote: > I modified the patch to handle the both: > - quoted index names > - complete partial numbers Committed, which should close the loop for this thread. If you have suggestions for the documentation, maybe it would be better to

Re: pgsql: Avoid creation of the free space map for small heap relations.

2019-01-27 Thread John Naylor
On Mon, Jan 28, 2019 at 6:58 AM Amit Kapila wrote: > > On Mon, Jan 28, 2019 at 11:10 AM Andrew Gierth > you just need to create free space on a page that didn't have enough > > before? It might be worth tweaking the fillfactor rather than trying to > > delete anything. > > > > No, it also depends

Re: speeding up planning with partitions

2019-01-27 Thread Amit Langote
Imai-san, Thanks for testing. On 2019/01/24 15:09, Imai, Yoshikazu wrote: > [pgbench commands] > pgbench -n -f update.sql -T 30 postgres > > [update.sql(updating partkey case)] > update rt set a = 1; > > [update.sql(updating non-partkey case)] > update rt set b = 1; > > [results] > updating

Re: pgsql: Avoid creation of the free space map for small heap relations.

2019-01-27 Thread Tom Lane
Amit Kapila writes: >> I don't know what the common thread is here, but you don't get to leave >> the buildfarm broken this badly while you figure it out. > Sure, but I am wondering why none of this ever shown in local tests, > as we have done quite some testing related to pgbench as well. Not

Re: pgsql: Avoid creation of the free space map for small heap relations.

2019-01-27 Thread Amit Kapila
On Mon, Jan 28, 2019 at 11:10 AM Andrew Gierth wrote: > > > "Amit" == Amit Kapila writes: > > Amit> Yes, so this could be the cause of the problem. I think we need > Amit> to change the tests added by the patch such that they don't rely > Amit> on vacuum to remove dead-row versions? Do

Re: pgsql: Avoid creation of the free space map for small heap relations.

2019-01-27 Thread Amit Kapila
On Mon, Jan 28, 2019 at 11:18 AM Tom Lane wrote: > > Amit Kapila writes: > > Yes, so this could be the cause of the problem. I think we need to > > change the tests added by the patch such that they don't rely on > > vacuum to remove dead-row versions? Do you or anybody else see any > > better

Re: A few new options for vacuumdb

2019-01-27 Thread Michael Paquier
On Thu, Jan 24, 2019 at 12:49:28AM +, Bossart, Nathan wrote: > Oh, wow. Thanks for pointing this out. I should have caught this. > With 0002, we are basically just throwing out the column lists > entirely as we obtain the qualified identifiers from the catalog > query. To fix this, I've

Re: pgsql: Avoid creation of the free space map for small heap relations.

2019-01-27 Thread Tom Lane
Amit Kapila writes: > Yes, so this could be the cause of the problem. I think we need to > change the tests added by the patch such that they don't rely on > vacuum to remove dead-row versions? Do you or anybody else see any > better way to fix this? To be blunt, this patch needs to be

Re: pgsql: Avoid creation of the free space map for small heap relations.

2019-01-27 Thread Andrew Gierth
> "Amit" == Amit Kapila writes: Amit> Yes, so this could be the cause of the problem. I think we need Amit> to change the tests added by the patch such that they don't rely Amit> on vacuum to remove dead-row versions? Do you or anybody else see Amit> any better way to fix this? Do you

Re: pgsql: Avoid creation of the free space map for small heap relations.

2019-01-27 Thread Amit Kapila
On Mon, Jan 28, 2019 at 10:25 AM Andrew Gierth wrote: > > > "Amit" == Amit Kapila writes: > > Amit> One possibility is that autovacuum has triggered to perform > Amit> truncation of some other relation (remove pages at the end) which > Amit> doesn't allow the FSM test to remove the

Re: Emacs vs pg_indent's weird indentation for function declarations

2019-01-27 Thread Tom Lane
Thomas Munro writes: > For a while I've been baffled by that: the first arguments of later > lines don't line up with that of the first line, but they're also not > in a constant column (it varies from function to function), and it's > also not caused by 8-space vs 4-space confusion. It was only

Re: Tab completion for ALTER INDEX|TABLE ALTER COLUMN SET STATISTICS

2019-01-27 Thread Tatsuro Yamada
Hi Peter, On 2019/01/25 20:09, Peter Eisentraut wrote: On 26/12/2018 07:07, Tatsuro Yamada wrote: +#define Query_for_list_of_attribute_numbers \ +"SELECT attnum "\ +" FROM pg_catalog.pg_attribute a, "\ +" pg_catalog.pg_class c "\ +" WHERE c.oid = a.attrelid "\ +" AND a.attnum > 0 "\

Re: pgsql: Avoid creation of the free space map for small heap relations.

2019-01-27 Thread Andrew Gierth
> "Amit" == Amit Kapila writes: Amit> One possibility is that autovacuum has triggered to perform Amit> truncation of some other relation (remove pages at the end) which Amit> doesn't allow the FSM test to remove the rows/perform truncation Amit> and thus let to the failure. Can there be

Re: WIP: Avoid creation of the free space map for small tables

2019-01-27 Thread Amit Kapila
On Mon, Jan 28, 2019 at 10:03 AM John Naylor wrote: > > On Mon, Jan 28, 2019 at 4:53 AM Amit Kapila wrote: > > There are a few buildfarm failures due to this commit, see my email on > > pgsql-committers. If you have time, you can also once look into > > those. > > I didn't see anything in

RE: Timeout parameters

2019-01-27 Thread Nagaura, Ryohei
Hi, Sorry for my late. On Tue, Dec 25, 2018 at 7:40 PM, Fabien COELHO wrote: > I still do not understand the use-case specifics: for me, aborting the > connection, or a softer cancelling the statement, will result in the server > stopping the statement, so the server does NOT "continue the job",

Re: WIP: Avoid creation of the free space map for small tables

2019-01-27 Thread John Naylor
On Mon, Jan 28, 2019 at 4:53 AM Amit Kapila wrote: > There are a few buildfarm failures due to this commit, see my email on > pgsql-committers. If you have time, you can also once look into > those. I didn't see anything in common with the configs of the failed members. None have a non-default

Re: Protect syscache from bloating with negative cache entries

2019-01-27 Thread Kyotaro HORIGUCHI
At Fri, 25 Jan 2019 07:26:46 +, "Tsunakawa, Takayuki" wrote in <0A3221C70F24FB45833433255569204D1FB70E6B@G01JPEXMBYT05> > From: Robert Haas [mailto:robertmh...@gmail.com] > > On Thu, Jan 24, 2019 at 10:02 AM Tom Lane wrote: > > > I will argue hard that we should not do it at all, ever. > >

Re: speeding up planning with partitions

2019-01-27 Thread David Rowley
On Sat, 12 Jan 2019 at 02:00, Amit Langote wrote: > > On 2019/01/09 9:09, David Rowley wrote: > > postgres=# update parent set c = c where a = 333; > > server closed the connection unexpectedly > > This probably means the server terminated abnormally > > before or while processing

Re: pgsql: Avoid creation of the free space map for small heap relations.

2019-01-27 Thread Amit Kapila
On Mon, Jan 28, 2019 at 8:49 AM Amit Kapila wrote: > > On Mon, Jan 28, 2019 at 8:17 AM Amit Kapila wrote: > > > > Avoid creation of the free space map for small heap relations. > > > > It seems there is some failure due to this on build farm machines. I > will investigate! > The failure is as

Re: [Sender Address Forgery]Re: using expression syntax for partition bounds

2019-01-27 Thread Amit Langote
Hi Peter, On 2019/01/26 17:25, Peter Eisentraut wrote: > On 25/01/2019 16:19, Tom Lane wrote: >> Peter Eisentraut writes: >>> committed >> >> Some of the buildfarm members are having sort-ordering problems >> with this. Looks like you could work around it with different >> partition names

Re: WIP: Avoid creation of the free space map for small tables

2019-01-27 Thread Amit Kapila
On Mon, Jan 28, 2019 at 9:16 AM John Naylor wrote: > > On Mon, Jan 28, 2019 at 3:53 AM Amit Kapila wrote: > > On Thu, Jan 24, 2019 at 9:14 AM Amit Kapila wrote: > > > Sure, apart from this I have run pgindent on the patches and make some > > > changes accordingly. Latest patches attached (only

Re: Protect syscache from bloating with negative cache entries

2019-01-27 Thread Kyotaro HORIGUCHI
At Fri, 25 Jan 2019 08:14:19 +, "Tsunakawa, Takayuki" wrote in <0A3221C70F24FB45833433255569204D1FB70EFB@G01JPEXMBYT05> > Hi Horiguchi-san, Bruce, > > From: Bruce Momjian [mailto:br...@momjian.us] > > I suggest you go with just syscache_prune_min_age, get that into PG 12, > > and we can

Re: WIP: Avoid creation of the free space map for small tables

2019-01-27 Thread John Naylor
On Mon, Jan 28, 2019 at 3:53 AM Amit Kapila wrote: > On Thu, Jan 24, 2019 at 9:14 AM Amit Kapila wrote: > > Sure, apart from this I have run pgindent on the patches and make some > > changes accordingly. Latest patches attached (only second patch has > > some changes). I will take one more

Re: problems with foreign keys on partitioned tables

2019-01-27 Thread Amit Langote
On 2019/01/25 2:18, Alvaro Herrera wrote: > On 2019-Jan-24, Amit Langote wrote: > >> A few hunks of the originally proposed patch are attached here as 0001, >> especially the part which fixes ATAddForeignKeyConstraint to pass the >> correct value of connoninherit to CreateConstraintEntry (which

Re: WIP: Avoid creation of the free space map for small tables

2019-01-27 Thread Amit Kapila
On Thu, Jan 24, 2019 at 9:14 AM Amit Kapila wrote: > > Sure, apart from this I have run pgindent on the patches and make some > changes accordingly. Latest patches attached (only second patch has > some changes). I will take one more pass on Monday morning (28th Jan) > and will commit unless

Re: Use an enum for RELKIND_*?

2019-01-27 Thread Kyotaro HORIGUCHI
At Thu, 24 Jan 2019 09:37:41 -0500, Tom Lane wrote in <15760.1548340...@sss.pgh.pa.us> > Kyotaro HORIGUCHI writes: > > I might misunderstand something, but my compiler (gcc 7.3.1) > > won't be quiet about omitted value even with default:. > > ... > > I would call that a compiler bug, TBH. The

RE: Log a sample of transactions

2019-01-27 Thread Kuroda, Hayato
Dear Adrien, >> * xact_is_sampled is left at the end of a transaction. >> Should the parameter be set to false at the lowest layer of the transaction >> system? >> I understand it is unnecessary for the functionality, but it have more >> symmetry. > > Yes, it is not necessary. I wonder what is

RE: speeding up planning with partitions

2019-01-27 Thread Imai, Yoshikazu
On Thu, Jan 24, 2019 at 6:10 AM, Imai, Yoshikazu wrote: > updating partkey case: > > part-num master 0001 0002 0003 0004 > 18215.34 7924.99 7931.15 8407.40 8475.65 > 27137.49 7026.45 7128.84 7583.08 7593.73 > 45880.54 5896.47 6014.82 6405.33

[ANN] pg2arrow

2019-01-27 Thread Kohei KaiGai
Hello, I made a utility program to dump PostgreSQL database in Apache Arrow format. Apache Arrow is a kind of data format for columnar-based structured data; actively developed by Spark and comprehensive communities. It is suitable data representation for static and read-only but large number of

Re: Delay locking partitions during query execution

2019-01-27 Thread David Rowley
On Tue, 4 Dec 2018 at 00:42, David Rowley wrote: > Over here and along similar lines to the above, but this time I'd like > to take this even further and change things so we don't lock *any* > partitions during AcquireExecutorLocks() and instead just lock them > when we first access them with

RE: pg_upgrade: Pass -j down to vacuumdb

2019-01-27 Thread Jamison, Kirk
Hi Jesper, On Friday, January 25, 2019, Jesper Pedersen > Thanks for your feedback ! > > As per Peter's comments I have changed the patch (v2) to not pass down the -j > option to vacuumdb. > > Only an update to the documentation and console output is made in order to > make it more clear.

Re: log bind parameter values on error

2019-01-27 Thread Alexey Bashtanov
Hi Peter, With your patch, with log_statement=all and log_parameters=on, you get the same, but with log_statement=all and log_parameters=off you get LOG: execute : SELECT abalance FROM pgbench_accounts WHERE aid = $1; DETAIL: parameters: $1 = UNKNOWN TYPE Thanks for spotting this, I've

Re: "SELECT ... FROM DUAL" is not quite as silly as it appears

2019-01-27 Thread Stefan Keller
Dear all, I'm following this list since years - especially PostGIS related - and you and PG are just awesome! Pls. let me chime in as a university teacher, therefore used to explain every year the same things :-). My 2 cents here are: Pls. try to give DUAL a better name, since it's IMHO neither

Re: FETCH FIRST clause PERCENT option

2019-01-27 Thread Tomas Vondra
On 1/24/19 10:57 AM, Surafel Temesgen wrote: > > > On Wed, Jan 9, 2019 at 8:18 PM Tomas Vondra > mailto:tomas.von...@2ndquadrant.com>> wrote: >   > > > See the attached patch, which recomputes the count regularly. I don't > claim the patch is committable or that it has no other

Emacs vs pg_indent's weird indentation for function declarations

2019-01-27 Thread Thomas Munro
Hello, Using either the .dir-locals.el settings or the "more complete" src/tools/editors/emacs.samples, I have never convinced Emacs to produce multi-line function declarations in .h files that satisfy pg_indent. For example, latch.c has the following definition: int

Re: WIP: Avoid creation of the free space map for small tables

2019-01-27 Thread John Naylor
On Sat, Jan 26, 2019 at 2:14 PM Amit Kapila wrote: > > On Sat, Jan 26, 2019 at 5:05 AM John Naylor > wrote: > > > > So, in v19 we check pg_class.relpages and if it's > > a heap and less than or equal the threshold we call stat on the 0th > > segment to verify. > > > > Okay, but the way logic is

Re: "SELECT ... FROM DUAL" is not quite as silly as it appears

2019-01-27 Thread Mark Dilger
> On Jan 27, 2019, at 12:04 PM, Mark Dilger wrote: > > > >> On Jan 25, 2019, at 5:09 PM, Tom Lane wrote: >> >> David Rowley writes: >>> As far as I can see the patch is ready to go, but I'll defer to Mark, >>> who's also listed on the reviewer list for this patch. >> >> Mark, are you

Re: "SELECT ... FROM DUAL" is not quite as silly as it appears

2019-01-27 Thread Mark Dilger
> On Jan 25, 2019, at 5:09 PM, Tom Lane wrote: > > David Rowley writes: >> As far as I can see the patch is ready to go, but I'll defer to Mark, >> who's also listed on the reviewer list for this patch. > > Mark, are you planning to do further review on this patch? > I'd like to move it

Re: "SELECT ... FROM DUAL" is not quite as silly as it appears

2019-01-27 Thread Mark Dilger
> On Jan 25, 2019, at 5:09 PM, Tom Lane wrote: > > David Rowley writes: >> As far as I can see the patch is ready to go, but I'll defer to Mark, >> who's also listed on the reviewer list for this patch. > > Mark, are you planning to do further review on this patch? > I'd like to move it

Re: [Patch] Log10 and hyperbolic functions for SQL:2016 compliance

2019-01-27 Thread Lætitia Avrot
Hi, Thanks for your time and advice, Tom! > > [ adding_log10_and_hyperbolic_functions_v1.patch ] > > No objection to the feature, but > > - Why are you using the float4-width library functions (coshf etc) > rather than the float8-width ones (cosh etc)? > > Well, I guess the only reason is that

Re: libpq environment variables in the server

2019-01-27 Thread Dmitry Igrishin
пн, 21 янв. 2019 г. в 13:42, Peter Eisentraut : > > When libpq is loaded in the server (libpqwalreceiver, dblink, > postgres_fdw), it may use libpq environment variables set in the > postmaster environment for connection parameter defaults. I have > noticed that this has some confusing effects in

Re: Early WIP/PoC for inlining CTEs

2019-01-27 Thread Andreas Karlsson
On 1/26/19 11:55 PM, Tom Lane wrote:> Hearing no immediate pushback on that proposal, I went ahead and made a version of the patch that does it like that, as attached. I also took a stab at documenting it fully. Thanks! This version of the patch looks solid, including the documentation. The

Re: Early WIP/PoC for inlining CTEs

2019-01-27 Thread Andreas Karlsson
On 1/27/19 4:21 PM, Tom Lane wrote: Andrew Gierth writes: I'm not sure we should nail down the rule that the absence of NOT MATERIALIZED will mean a multiply-referenced CTE is evaluated once. One would hope that in the future the planner might be taught to inline or not in that case depending

Re: Index Skip Scan

2019-01-27 Thread Dmitry Dolgov
> On Sat, Jan 26, 2019 at 6:45 PM Dmitry Dolgov <9erthali...@gmail.com> wrote: > > Rebased version after rd_amroutine was renamed. And one more to fix the documentation. Also I've noticed few TODOs in the patch about the missing docs, and replaced them with a required explanation of the feature.

Re: Early WIP/PoC for inlining CTEs

2019-01-27 Thread Tom Lane
Andrew Gierth writes: > I'm not sure we should nail down the rule that the absence of NOT > MATERIALIZED will mean a multiply-referenced CTE is evaluated once. One > would hope that in the future the planner might be taught to inline or > not in that case depending on cost. I think it makes more

Re: Allowing extensions to supply operator-/function-specific info

2019-01-27 Thread Tom Lane
Simon Riggs writes: > On Sun, 20 Jan 2019 at 23:48, Tom Lane wrote: >> What I'm envisioning therefore is that we allow an auxiliary function ... > Does this help with these cases? > * Allow a set returning function to specify number of output rows, in cases > where that is variable and

Re: Opossum vs. float4 NaN

2019-01-27 Thread Tom Lane
Glyn Astill writes: > I guess the main question is; does anybody care about builds on a 20 year old > netbsd/mipsel dinosaur?  I noticed there are now mips64el and mips64eb > build-farm members. I tend to think that variety in the buildfarm is intrinsically a good thing. I'd rather see you

Re: A few new options for vacuumdb

2019-01-27 Thread Michael Paquier
On Thu, Jan 24, 2019 at 12:49:28AM +, Bossart, Nathan wrote: > Oh, wow. Thanks for pointing this out. I should have caught this. > With 0002, we are basically just throwing out the column lists > entirely as we obtain the qualified identifiers from the catalog > query. To fix this, I've

Re: Early WIP/PoC for inlining CTEs

2019-01-27 Thread Andrew Gierth
> "Tom" == Tom Lane writes: Tom> I was interested to find, while writing the docs, that it's a real Tom> struggle to invent plausible reasons to write MATERIALIZED given Tom> the above specification. You pretty much have to have lied to the Tom> planner, eg by making a volatile function

Re: backslash-dot quoting in COPY CSV

2019-01-27 Thread Michael Paquier
On Thu, Jan 24, 2019 at 10:09:30PM -0500, Bruce Momjian wrote: > This seems like a bug to me. Looking at the code, psql issues the > prompts for STDIN, but when it sees \. alone on a line, it has no idea > you are in a quoted CSV string, so it thinks the copy is done and sends > the result to the

Re: pg_basebackup, walreceiver and wal_sender_timeout

2019-01-27 Thread Michael Paquier
On Sat, Jan 26, 2019 at 01:45:46PM +0100, Magnus Hagander wrote: > One workaround you could perhaps look at here is to run pg_basebackup > with --no-sync. That way there will be no fsyncs issued while running. You > will then of course have to take care of syncing all the files to disk > after

Re: pg_upgrade: Pass -j down to vacuumdb

2019-01-27 Thread Michael Paquier
On Fri, Jan 25, 2019 at 12:16:49PM -0500, Tom Lane wrote: > Alvaro Herrera writes: >> So let's have it write with a $VACUUMDB_OPTS variable, which is by >> default defined as empty but with a comment suggesting that maybe the >> user wants to add the -j option. This way, if they have to edit it,

Re: relcache reference leak with pglogical replication to insert-only partitioned table?

2019-01-27 Thread Michael Paquier
On Sat, Jan 26, 2019 at 09:19:49PM -0600, Jeremy Finzel wrote: > I understand it's not fully supported to replicate to a differently > partitioned setup on a subscriber with either pglogical or the native > logical replication, however I also know that INSERT triggers can be fired > in replication

Re: Opossum vs. float4 NaN

2019-01-27 Thread Glyn Astill
>> On Saturday, 26 January 2019, 16:00:24 GMT, Tom Lane wrote: >> >> I'm thinking we should regretfully retire opossum, unless there's >> a software update available for it that fixes this bug. I'm happy to update opossum to see if the issue goes away; likewise I'm just as happy to retire it

Re: Allowing extensions to supply operator-/function-specific info

2019-01-27 Thread Simon Riggs
On Sun, 20 Jan 2019 at 23:48, Tom Lane wrote: > What I'm envisioning therefore is that we allow an auxiliary function to > be attached to any operator or function that can provide functionality > like this, and that we set things up so that the set of tasks that > such functions can perform can