Re: ALTER INDEX ... ALTER COLUMN not present in dump

2018-12-16 Thread amul sul
On Mon, Dec 17, 2018 at 1:04 PM Michael Paquier wrote: > > On Mon, Dec 17, 2018 at 12:49:03PM +0530, amul sul wrote: > > On Mon, Dec 17, 2018 at 11:54 AM Michael Paquier wrote: > >> So this settles the argument that we had better not do anything before > >> v11. Switching the dump code to use

Re: Fixing typos in tests of partition_info.sql

2018-12-16 Thread Amit Langote
On 2018/12/17 16:38, Michael Paquier wrote: > On Mon, Dec 17, 2018 at 04:14:07PM +0900, Amit Langote wrote: >> --- A table not part of a partition tree works is the only member listed. >> +-- A table not part of a partition tree is the only member listed. >> >> How about: >> >> -- Table that is

Re: Fixing typos in tests of partition_info.sql

2018-12-16 Thread Michael Paquier
On Mon, Dec 17, 2018 at 04:14:07PM +0900, Amit Langote wrote: > --- A table not part of a partition tree works is the only member listed. > +-- A table not part of a partition tree is the only member listed. > > How about: > > -- Table that is not part of any partition tree is the only member

Re: ALTER INDEX ... ALTER COLUMN not present in dump

2018-12-16 Thread Michael Paquier
On Mon, Dec 17, 2018 at 12:49:03PM +0530, amul sul wrote: > On Mon, Dec 17, 2018 at 11:54 AM Michael Paquier wrote: >> So this settles the argument that we had better not do anything before >> v11. Switching the dump code to use column numbers has not proved to be >> complicated as only the

Re: Remove double trailing semicolons

2018-12-16 Thread Amit Kapila
On Mon, Dec 17, 2018 at 1:53 AM David Rowley wrote: > > While looking over some recent commits, I noticed there are some code > lines with a double trailing semicolon instead of a single one. The > attached fixes these. > LGTM. I will commit it. -- With Regards, Amit Kapila. EnterpriseDB:

Re: Fixing typos in tests of partition_info.sql

2018-12-16 Thread Amit Langote
On 2018/12/17 15:52, Michael Paquier wrote: > On Mon, Dec 17, 2018 at 03:40:28PM +0900, Michael Paquier wrote: >> I was just going through some of the tests, when I noticed that the >> tests of partition_info.sql have two typos and that the last set of >> tests is imprecise about the expected

Re: Fixing typos in tests of partition_info.sql

2018-12-16 Thread Amit Langote
Hi, On 2018/12/17 15:40, Michael Paquier wrote: > Hi Amit, > (CC: -hackers) > > I was just going through some of the tests, when I noticed that the > tests of partition_info.sql have two typos and that the last set of > tests is imprecise about the expected behavior of the functions. > > Do you

Re: Fixing typos in tests of partition_info.sql

2018-12-16 Thread Michael Paquier
On Mon, Dec 17, 2018 at 03:40:28PM +0900, Michael Paquier wrote: > I was just going through some of the tests, when I noticed that the > tests of partition_info.sql have two typos and that the last set of > tests is imprecise about the expected behavior of the functions. > > Do you think that

Re: ALTER INDEX ... ALTER COLUMN not present in dump

2018-12-16 Thread amul sul
On Mon, Dec 17, 2018 at 11:54 AM Michael Paquier wrote: > > On Mon, Dec 17, 2018 at 12:24:15AM -0500, Tom Lane wrote: > > If we were to rename the "foo.expr" column at this point, > > and then dump and reload, the expression column in the > > second index would presumably acquire the name "expr"

Fixing typos in tests of partition_info.sql

2018-12-16 Thread Michael Paquier
Hi Amit, (CC: -hackers) I was just going through some of the tests, when I noticed that the tests of partition_info.sql have two typos and that the last set of tests is imprecise about the expected behavior of the functions. Do you think that something like the attached is an improvement?

Re: ALTER INDEX ... ALTER COLUMN not present in dump

2018-12-16 Thread Michael Paquier
On Mon, Dec 17, 2018 at 12:24:15AM -0500, Tom Lane wrote: > If we were to rename the "foo.expr" column at this point, > and then dump and reload, the expression column in the > second index would presumably acquire the name "expr" > not "expr1", because "expr" would no longer be taken. > So if

Re: ALTER INDEX ... ALTER COLUMN not present in dump

2018-12-16 Thread Michael Paquier
On Mon, Dec 17, 2018 at 10:59:08AM +0530, amul sul wrote: > Patch is missing? Here you go. The patch is still using atttribute names, which is a bad idea ;) -- Michael diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c index 637c79af48..09e90ea62c 100644 ---

Re: ALTER INDEX ... ALTER COLUMN not present in dump

2018-12-16 Thread amul sul
On Mon, Dec 17, 2018 at 10:44 AM Michael Paquier wrote: > > On Fri, Dec 14, 2018 at 08:08:45AM +, Amul Sul wrote: > > dump-alter-index-stats-v2.patch looks pretty much reasonable to me, passing > > on committer. > > > > The new status of this patch is: Ready for Committer > > Thanks Amul for

Re: ALTER INDEX ... ALTER COLUMN not present in dump

2018-12-16 Thread Tom Lane
Michael Paquier writes: > As Alexander and others state on this thread, it looks a bit weird to > use internally-produced attribute names in those SQL queries, which is > why the new grammar has been added. At the same time, it looks more > solid to me to represent the dumps with those column

Re: ALTER INDEX ... ALTER COLUMN not present in dump

2018-12-16 Thread Michael Paquier
On Fri, Dec 14, 2018 at 08:08:45AM +, Amul Sul wrote: > dump-alter-index-stats-v2.patch looks pretty much reasonable to me, passing > on committer. > > The new status of this patch is: Ready for Committer Thanks Amul for the review. I got the occasion to look again at this patch, and I

Re: 'infinity'::Interval should be added

2018-12-16 Thread Isaac Morland
On Sun, 16 Dec 2018 at 22:27, Robert Haas wrote: > Simon's argument for adding this is that we support 'infinity' for > timestamp, but is that a good argument for making 'interval' do it, > given that there are many other types like date for which we don't > support it? > postgres=> select

Re: 'infinity'::Interval should be added

2018-12-16 Thread Tom Lane
Robert Haas writes: > Simon's argument for adding this is that we support 'infinity' for > timestamp, but is that a good argument for making 'interval' do it, > given that there are many other types like date for which we don't > support it? My feeling is that date is to timestamp as integer is

Re: Catalog views failed to show partitioned table information.

2018-12-16 Thread Michael Paquier
On Mon, Dec 17, 2018 at 10:22:28AM +0900, Amit Langote wrote: > On 2018/12/15 8:00, Michael Paquier wrote: >> I tend to agree with your comment here. pg_tables lists partitioned >> tables, but pg_indexes is forgotting about partitioned indexes. So this >> is a good thing to add. > > +1 I'll go

Re: Should new partitions inherit their tablespace from their parent?

2018-12-16 Thread David Rowley
On Mon, 17 Dec 2018 at 12:59, Michael Paquier wrote: > Okay, I think that you should add an assertion on > CheckRelationLockedByMe() as MergeAttributes()'s only caller is > DefineRelation(). Better safe than sorry later. Would that not just double up the work that's already done in

Re: Catalog views failed to show partitioned table information.

2018-12-16 Thread Amit Langote
Hi, On 2018/12/15 8:00, Michael Paquier wrote: > On Fri, Dec 14, 2018 at 05:21:49PM +0530, Suraj Kharage wrote: >> There are some catalog views which do not show the partitioned table and >> its index entry. >> One of them is "pg_indexes" which failed to show the partitioned index. >> Attached

Re: gist microvacuum doesn't appear to care about hot standby?

2018-12-16 Thread Alexander Korotkov
On Mon, Dec 17, 2018 at 1:25 AM Andres Freund wrote: > On 2018-12-17 01:03:52 +0300, Alexander Korotkov wrote: > > On Thu, Dec 13, 2018 at 7:28 AM Alexander Korotkov > > wrote: > > > On Thu, Dec 13, 2018 at 1:45 AM Andres Freund wrote: > > > > Is there any reason something like that isn't

Re: Should new partitions inherit their tablespace from their parent?

2018-12-16 Thread Michael Paquier
On Sun, Dec 16, 2018 at 07:07:35PM -0300, Alvaro Herrera wrote: > I'll self-review this again tomorrow, 'cause I now have to run. > - if (!is_partition) > - relation = heap_openrv(parent, > ShareUpdateExclusiveLock); > - else > -

Re: select limit error in file_fdw

2018-12-16 Thread Tom Lane
Erik Rijkers writes: > Thank you very much. I've now also tested with the original, much larger > file, which gives no problem anymore. I am really glad this works again, > we use this stuff a lot. We appreciate you noticing the problem before 11.2 got out ... regards,

Re: slight tweaks to documentation about runtime pruning

2018-12-16 Thread Alvaro Herrera
On 2018-Dec-10, David Rowley wrote: > On Wed, 5 Dec 2018 at 20:24, Amit Langote > wrote: > > However, for pruned partitions' subplans, what's actually shown is the > > string "(never executed)", not loops. So, wouldn't it be better to tell > > the readers to look for that instead of "loops"?

Re: gist microvacuum doesn't appear to care about hot standby?

2018-12-16 Thread Andres Freund
Hi, On 2018-12-17 01:03:52 +0300, Alexander Korotkov wrote: > On Thu, Dec 13, 2018 at 7:28 AM Alexander Korotkov > wrote: > > On Thu, Dec 13, 2018 at 1:45 AM Andres Freund wrote: > > > Is there any reason something like that isn't necessary for gist? If so, > > > where's that documented? If

Re: Should new partitions inherit their tablespace from their parent?

2018-12-16 Thread Alvaro Herrera
I didn't like this, so I rewrote it a little bit. First, I changed the Assert() to use the macro for relations with storage that I just posted in the other thread that Michael mentioned. I then noticed that we're doing a heap_openrv() in the parent relation and closing it before MergeAttribute()

Re: gist microvacuum doesn't appear to care about hot standby?

2018-12-16 Thread Alexander Korotkov
On Thu, Dec 13, 2018 at 7:28 AM Alexander Korotkov wrote: > On Thu, Dec 13, 2018 at 1:45 AM Andres Freund wrote: > > Is there any reason something like that isn't necessary for gist? If so, > > where's that documented? If not, uh, isn't that a somewhat serious bug > > in gist? > > Thank you for

Re: Valgrind failures in Apply Launcher's bgworker_quickdie() exit

2018-12-16 Thread Andres Freund
Hi, On 2018-12-17 08:25:38 +1100, Thomas Munro wrote: > On Mon, Dec 17, 2018 at 7:57 AM Andres Freund wrote: > > The interesting bit is that if I replace the _exit(2) in > > bgworker_quickdie() with an exit(2) (i.e. processing atexit handlers), > > or manully add an OPENSSL_cleanup() before the

Re: Valgrind failures in Apply Launcher's bgworker_quickdie() exit

2018-12-16 Thread Thomas Munro
On Mon, Dec 17, 2018 at 7:57 AM Andres Freund wrote: > The interesting bit is that if I replace the _exit(2) in > bgworker_quickdie() with an exit(2) (i.e. processing atexit handlers), > or manully add an OPENSSL_cleanup() before the _exit(2), valgrind > doesn't find errors. Weird. Well I can

Re: select limit error in file_fdw

2018-12-16 Thread Erik Rijkers
On 2018-12-16 19:10, Tom Lane wrote: Anyway, we know what to do, so I'll go do it. Thank you very much. I've now also tested with the original, much larger file, which gives no problem anymore. I am really glad this works again, we use this stuff a lot. Erik Rijkers

Re: Improving collation-dependent indexes in system catalogs

2018-12-16 Thread Tom Lane
Alvaro Herrera writes: > I notice that some information_schema view columns end up with C > collation after this patch, and others remain with default collation. > Is that sensible? (I think the only two cases where this might matter > at all are information_schema.parameters.parameter_name, >

Re: Valgrind failures in Apply Launcher's bgworker_quickdie() exit

2018-12-16 Thread Andres Freund
Hi, On 2018-12-16 22:33:00 +1100, Thomas Munro wrote: > On Fri, Dec 14, 2018 at 4:14 PM Tom Lane wrote: > > Andres Freund writes: > > > On December 13, 2018 6:01:04 PM PST, Tom Lane wrote: > > >> Has anyone tried to reproduce this on other platforms? > > > > > I recently also hit this locally,

Re: don't create storage when unnecessary

2018-12-16 Thread Alvaro Herrera
On 2018-Dec-07, Michael Paquier wrote: > A macro makes sense to control that. I added Ashutosh's RELKIND_HAS_STORAGE, but renamed it to RELKIND_CAN_HAVE_STORAGE, because some of the relkinds can be mapped and thus would have relfilenode set to 0. I think this is a bit misleading either way. >

Re: reorderbuffer: memory overconsumption with medium-size subxacts

2018-12-16 Thread Andres Freund
Hi, On 2018-12-16 17:30:30 -0300, Alvaro Herrera wrote: > On 2018-Dec-16, Andres Freund wrote: > > > > I think there's a one-line fix, attached: just add the number of changes > > > in a subxact to nentries_mem when the transaction is assigned to the > > > parent. > > > > Isn't this going to

Re: reorderbuffer: memory overconsumption with medium-size subxacts

2018-12-16 Thread Alvaro Herrera
On 2018-Dec-16, Andres Freund wrote: > > I think there's a one-line fix, attached: just add the number of changes > > in a subxact to nentries_mem when the transaction is assigned to the > > parent. > > Isn't this going to cause significant breakage, because we rely on > nentries_mem to be

Remove double trailing semicolons

2018-12-16 Thread David Rowley
While looking over some recent commits, I noticed there are some code lines with a double trailing semicolon instead of a single one. The attached fixes these. -- David Rowley http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services

Re: reorderbuffer: memory overconsumption with medium-size subxacts

2018-12-16 Thread Andres Freund
Hi, On 2018-12-16 12:06:16 -0300, Alvaro Herrera wrote: > Found this on Postgres 9.6, but I think it affects back to 9.4. > > I've seen a case where reorderbuffer keeps very large amounts of memory > in use, without spilling to disk, if the main transaction does little or > no changes and many

Re: Why aren't we using strsignal(3) ?

2018-12-16 Thread Alvaro Herrera
On 2018-Dec-16, Tom Lane wrote: > I propose to replace all these places with code like > > snprintf(str, sizeof(str), > _("child process was terminated by signal %d: %s"), > WTERMSIG(exitstatus), pg_strsignal(WTERMSIG(exitstatus))); > > where

Re: Should new partitions inherit their tablespace from their parent?

2018-12-16 Thread David Rowley
On Tue, 11 Dec 2018 at 15:43, Michael Paquier wrote: > + parentrel = heap_openrv(parent, AccessExclusiveLock); > So, in order to determine which tablespace should be used here, an > exclusive lock is taken on the parent because its partition descriptor > gets updated by the addition of the

Re: reorderbuffer: memory overconsumption with medium-size subxacts

2018-12-16 Thread Tomas Vondra
On 12/16/18 4:06 PM, Alvaro Herrera wrote: > Hello > > Found this on Postgres 9.6, but I think it affects back to 9.4. > > I've seen a case where reorderbuffer keeps very large amounts of memory > in use, without spilling to disk, if the main transaction does little or > no changes and many

Re: select limit error in file_fdw

2018-12-16 Thread Tom Lane
Erik Rijkers writes: > On 2018-12-16 16:52, Tom Lane wrote: >> However, something else occurred to me this morning, and a bit >> later I can reproduce the problem! I did it by changing the >> table's definition to use a shell pipeline: > /bin/sh seems to be dash, here. Hm. That must be the

Why aren't we using strsignal(3) ?

2018-12-16 Thread Tom Lane
While poking at the signal-reporting bug just pointed out by Erik Rijkers, I couldn't help noticing how many places we have that are doing some equivalent of this ugly dance: #if defined(HAVE_DECL_SYS_SIGLIST) && HAVE_DECL_SYS_SIGLIST { charstr2[256]; snprintf(str2,

Re: PATCH: logical_work_mem and logical streaming of large in-progress transactions

2018-12-16 Thread Tomas Vondra
FWIW the original CF entry in 2018-07 [1] was marked as RWF. I'm not sure what's the right way to resubmit such patches, so I've created a new entry in 2019-01 [2] referencing the same hackers thread (and with the same authors/reviewers metadata). [1] https://commitfest.postgresql.org/19/1429/

reducing the footprint of ScanKeyword (was Re: Large writable variables)

2018-12-16 Thread John Naylor
On 10/15/18, Tom Lane wrote: > Andres Freund writes: >> On 2018-10-15 16:36:26 -0400, Tom Lane wrote: >>> We could possibly fix these by changing the data structure so that >>> what's in a ScanKeywords entry is an offset into some giant string >>> constant somewhere. No idea how that would

Re: select limit error in file_fdw

2018-12-16 Thread Erik Rijkers
On 2018-12-16 16:52, Tom Lane wrote: Erik Rijkers writes: On 2018-12-16 07:03, Tom Lane wrote: Um ... this example works for me, in both HEAD and v11 branch tip. Moreover, the behavior you describe is exactly what ffa4cbd623 was intended to fix. Is there any chance that you got 11.1 and v11

Re: Alternative to \copy in psql modelled after \g

2018-12-16 Thread Tom Lane
"Daniel Verite" writes: > So as a replacement for the \copyto I was proposing earlier, PFA a patch > for COPY TO STDOUT to make use of the argument to \g. Sounds plausible, please add to next commitfest so we don't forget it. regards, tom lane

Re: select limit error in file_fdw

2018-12-16 Thread Tom Lane
I wrote: > It remains unclear why you had an intervening shell process when > I didn't, but perhaps that can be chalked up to use of a different > shell? To provide some data on that: popen() is presumably invoking /bin/sh, which on my box is $ /bin/sh --version GNU bash, version

Re: select limit error in file_fdw

2018-12-16 Thread Tom Lane
Erik Rijkers writes: > On 2018-12-16 07:03, Tom Lane wrote: >> Um ... this example works for me, in both HEAD and v11 branch tip. >> Moreover, the behavior you describe is exactly what ffa4cbd623 was >> intended to fix. Is there any chance that you got 11.1 and v11 >> branch tip mixed up? > [

Re: Improving collation-dependent indexes in system catalogs

2018-12-16 Thread Alvaro Herrera
On 2018-Dec-15, Tom Lane wrote: > I wrote: > > While fooling with the idea of making type "name" collation > > aware, it occurred to me that there's a better, more general > > answer, which is to insist that collation-aware system catalog > > columns must be marked with C collation. >

Grant documentation about "all tables"

2018-12-16 Thread Lætitia Avrot
Hi all, When you look at Postgres' SQL reference documentation for `GRANT`, the `ALL TABLES` clause is explained as : > ALL TABLES also affects views and foreign tables, just like the specific-object GRANT command. A colleague of mine was asking himself if it included materialized views or not

reorderbuffer: memory overconsumption with medium-size subxacts

2018-12-16 Thread Alvaro Herrera
Hello Found this on Postgres 9.6, but I think it affects back to 9.4. I've seen a case where reorderbuffer keeps very large amounts of memory in use, without spilling to disk, if the main transaction does little or no changes and many subtransactions execute changes just below the threshold to

Re: PATCH: logical_work_mem and logical streaming of large in-progress transactions

2018-12-16 Thread Tomas Vondra
imit-ReorderBuffer-20181216.patch.gz Description: application/gzip 0002-Immediately-WAL-log-assignments-20181216.patch.gz Description: application/gzip 0003-Issue-individual-invalidations-with-wal_lev-20181216.patch.gz Description: application/gzip 0004-Extend-the-output-plugin-API-with-stream-me-2018

Re: New function pg_stat_statements_reset_query() to reset statistics of a specific query

2018-12-16 Thread Alvaro Herrera
On 2018-Dec-15, Peter Eisentraut wrote: > An example: > > select pg_stat_statements_reset( > 10, > (select min(oid) from pg_database where datname like 'test%'), > 1234); > > (This is obviously a weird example, but it illustrates the > language-theoretic point.) This is not at all weird. Lack

Re: Alternative to \copy in psql modelled after \g

2018-12-16 Thread Daniel Verite
I wrote: > I admit that if we could improve \g to handle COPY, it would be more > elegant than the current proposal adding two meta-commands. > > But the copy-workflow and non-copy-workflow are different, and in > order to know which one to start, \g would need to analyze the query It

Re: [HACKERS] logical decoding of two-phase transactions

2018-12-16 Thread Tomas Vondra
Hi Nikhil, Thanks for the updated patch - I've started working on a review, with the hope of getting it committed sometime in 2019-01. But the patch bit-rotted again a bit (probably due to d3c09b9b), which broke the last part. Can you post a fixed version? regards -- Tomas Vondra

Re: select limit error in file_fdw

2018-12-16 Thread Erik Rijkers
On 2018-12-16 11:19, Erik Rijkers wrote: On 2018-12-16 07:03, Tom Lane wrote: Erik Rijkers writes: I have noticed that since ffa4cbd623, a foreign table that pulls data from a PROGRAM (in this case an unzip call) will fail if there is a LIMIT on the SELECT (while succeeding without LIMIT).

Re: Valgrind failures in Apply Launcher's bgworker_quickdie() exit

2018-12-16 Thread Thomas Munro
On Fri, Dec 14, 2018 at 4:14 PM Tom Lane wrote: > Andres Freund writes: > > On December 13, 2018 6:01:04 PM PST, Tom Lane wrote: > >> Has anyone tried to reproduce this on other platforms? > > > I recently also hit this locally, but since that's also Debian unstable... > > Note that removing

Re: select limit error in file_fdw

2018-12-16 Thread Erik Rijkers
On 2018-12-16 07:03, Tom Lane wrote: Erik Rijkers writes: I have noticed that since ffa4cbd623, a foreign table that pulls data from a PROGRAM (in this case an unzip call) will fail if there is a LIMIT on the SELECT (while succeeding without LIMIT). Below is an example. Um ... this example

Re: plpgsql plugin - stmt_beg/end is not called for top level block of statements

2018-12-16 Thread Pavel Stehule
po 19. 11. 2018 v 19:37 odesílatel Pavel Stehule napsal: > Hi > > I am playing with plpgsql profiling and and plpgsql plugin API. I found so > callback stmt_beg and stmt_end was not called for top statement due direct > call exec_stmt_block function. > > <-->estate.err_text = NULL; >

Re: proposal: plpgsql pragma statement

2018-12-16 Thread Pavel Stehule
Hi st 12. 12. 2018 v 9:03 odesílatel Pavel Stehule napsal: > > > čt 6. 12. 2018 v 18:27 odesílatel Pavel Stehule > napsal: > >> >> >> čt 6. 12. 2018 v 18:17 odesílatel Robert Haas >> napsal: >> >>> On Thu, Dec 6, 2018 at 12:13 PM Pavel Stehule >>> wrote: >>> > My idea about plpgsql PRAGMA is