Re: Proposal: Make use of C99 designated initialisers for nulls/values arrays

2019-10-16 Thread Thomas Munro
On Tue, Oct 8, 2019 at 11:09 PM Amit Kapila wrote: > On Sat, Oct 5, 2019 at 3:10 AM Andres Freund wrote: > > On 2019-10-04 17:08:29 -0400, Tom Lane wrote: > > > Andres Freund writes: > > > > On 2019-10-04 16:31:29 -0400, Bruce Momjian wrote: > > > >> Yeah, it is certainly weird that you have to

Re: recovery_min_apply_delay in archive recovery causes assertion failure in latch

2019-10-16 Thread Michael Paquier
On Tue, Oct 08, 2019 at 02:18:00AM +0900, Fujii Masao wrote: > On Fri, Oct 4, 2019 at 9:03 PM Fujii Masao wrote: >> So archive recovery with recovery_min_apply_delay enabled would be >> intended configuration. My patch that changes archive recovery so that >> it always ignores thesetting might be

Re: [HACKERS] Block level parallel vacuum

2019-10-16 Thread Masahiko Sawada
On Wed, Oct 16, 2019 at 3:02 PM Amit Kapila wrote: > > On Wed, Oct 16, 2019 at 6:50 AM Masahiko Sawada wrote: > > > > On Tue, Oct 15, 2019 at 6:33 PM Amit Kapila wrote: > > > > > > > Attached updated patch set. 0001 patch introduces new index AM field > > amcanparallelvacuum. All index AMs

Re: SegFault on 9.6.14

2019-10-16 Thread Thomas Munro
On Fri, Sep 13, 2019 at 1:35 AM Robert Haas wrote: > On Thu, Sep 12, 2019 at 8:55 AM Amit Kapila wrote: > > Robert, Thomas, do you have any more suggestions related to this. I > > am planning to commit the above-discussed patch (Forbid Limit node to > > shutdown resources.) coming Monday, so

Re: maintenance_work_mem used by Vacuum

2019-10-16 Thread Amit Kapila
On Thu, Oct 17, 2019 at 6:07 AM Greg Stark wrote: > > It's a bit unfortunate that we're doing the pending list flush while the > vacuum memory is allocated at all. Is there any reason other than the way the > callbacks are defined that gin doesn't do the pending list flush before > vacuum does

Re: Proposal: Make use of C99 designated initialisers for nulls/values arrays

2019-10-16 Thread vignesh C
On Tue, Oct 8, 2019 at 4:43 AM Smith, Peter wrote: > > From: Amit Kapila Sent: Friday, 4 October 2019 4:50 > PM > > >>How about I just define them both the same? > >>#define INIT_ALL_ELEMS_ZERO {0} > >>#define INIT_ALL_ELEMS_FALSE{0} > > > >I think using one define would be preferred,

Re: Questions/Observations related to Gist vacuum

2019-10-16 Thread Dilip Kumar
On Thu, Oct 17, 2019 at 9:15 AM Amit Kapila wrote: > > On Wed, Oct 16, 2019 at 7:21 PM Heikki Linnakangas wrote: > > > > On 16 October 2019 12:57:03 CEST, Amit Kapila > > wrote: > > >On Tue, Oct 15, 2019 at 7:13 PM Heikki Linnakangas > > >wrote: > > >> All things > > >> considered, I'm not

Re: Questions/Observations related to Gist vacuum

2019-10-16 Thread Amit Kapila
On Wed, Oct 16, 2019 at 7:21 PM Heikki Linnakangas wrote: > > On 16 October 2019 12:57:03 CEST, Amit Kapila wrote: > >On Tue, Oct 15, 2019 at 7:13 PM Heikki Linnakangas > >wrote: > >> All things > >> considered, I'm not sure which is better. > > > >Yeah, this is a tough call to make, but if we

Re: configure fails for perl check on CentOS8

2019-10-16 Thread Kyotaro Horiguchi
Hello. # I'm almost comming back.. At Wed, 16 Oct 2019 10:41:19 -0400, Andrew Dunstan wrote in > On Wed, Oct 16, 2019 at 8:32 AM Tom Lane wrote: > > >> $ ./configure --with-perl CFLAGS=-O0 > > >> It is bash-aliased and survived without being found for a long time on > > >> my Cent7

Re: Questions/Observations related to Gist vacuum

2019-10-16 Thread Amit Kapila
On Wed, Oct 16, 2019 at 11:20 AM Dilip Kumar wrote: > > On Tue, Oct 15, 2019 at 7:13 PM Heikki Linnakangas wrote: > > > > On 15/10/2019 09:37, Amit Kapila wrote: > > > While reviewing a parallel vacuum patch [1], we noticed a few things > > > about $SUBJECT implemented in commit - > > >

Re: ICU for global collation

2019-10-16 Thread Thomas Munro
On Wed, Oct 9, 2019 at 12:16 AM Marius Timmer wrote: > like the others before me we (the university of Münster) are happy to > see this feature as well. Thank you this. > > When I applied the patch two weeks ago I run into the issue that initdb > did not recognize the new parameters

RE: New SQL counter statistics view (pg_stat_sql)

2019-10-16 Thread Smith, Peter
Dear Hackers, We have resurrected this 2 year old "SQL statements statistics counter" proposal from Hari. The attached patch addresses the previous review issues. The commit-fest entry has been moved forward to the next commit-fest https://commitfest.postgresql.org/25/790/ Please review

Remaining calls of heap_close/heap_open in the tree

2019-10-16 Thread Michael Paquier
Hi all, I have just bumped into $subject, and we now use the table_* equivalents in the code. Any objections to the simple patch attached to clean up that? Thanks, -- Michael diff --git a/src/backend/commands/statscmds.c b/src/backend/commands/statscmds.c index f51eb7bb64..0ab43deb71 100644 ---

Re: Cache lookup errors with functions manipulation object addresses

2019-10-16 Thread Michael Paquier
On Thu, Sep 26, 2019 at 03:52:03PM +0900, Michael Paquier wrote: > On Wed, Sep 25, 2019 at 09:21:03AM -0300, Alvaro Herrera wrote: >> On 2019-Sep-24, Michael Paquier wrote: >>> + * - FORMAT_TYPE_FORCE_NULL >>> + * if the type OID is invalid or unknown, return NULL instead of ??? >>> + * or

Re: Cache lookup errors with functions manipulation object addresses

2019-10-16 Thread Michael Paquier
On Wed, Oct 16, 2019 at 01:04:57PM +0200, Dmitry Dolgov wrote: > Thanks for the update. Looking at v17 0003 I have one more question. In all > the > places where we have to do systable_endscan, it followed by heap_close, > although in the rest of the file table_close is used. I guess this logic

Re: v12.0: segfault in reindex CONCURRENTLY

2019-10-16 Thread Michael Paquier
On Wed, Oct 16, 2019 at 04:11:46PM -0500, Justin Pryzby wrote: > On Sun, Oct 13, 2019 at 04:18:34PM -0300, Alvaro Herrera wrote: >> (FWIW I expect the crash is possible not just in reindex but also in >> CREATE INDEX CONCURRENTLY.) > > FWIW, for sake of list archives, and for anyone running v12

Re: v12.0: segfault in reindex CONCURRENTLY

2019-10-16 Thread Michael Paquier
On Wed, Oct 16, 2019 at 09:53:56AM -0300, Alvaro Herrera wrote: > Thanks, pushed. Thanks, Alvaro. -- Michael signature.asc Description: PGP signature

Re: maintenance_work_mem used by Vacuum

2019-10-16 Thread Greg Stark
It's a bit unfortunate that we're doing the pending list flush while the vacuum memory is allocated at all. Is there any reason other than the way the callbacks are defined that gin doesn't do the pending list flush before vacuum does the heap scan and before it allocates any memory using

Re: ERROR: multixact X from before cutoff Y found to be still running

2019-10-16 Thread Thomas Munro
On Thu, Oct 17, 2019 at 6:11 AM Jeremy Schneider wrote: > On 10/16/19 10:09, Bossart, Nathan wrote: > > On 10/15/19, 11:11 PM, "Thomas Munro" wrote: > >> Here's a version with a proposed commit message and a comment. Please > >> let me know if I credited things to the right people! > > > >

Re: Connect as multiple users using single client certificate

2019-10-16 Thread Stephen Frost
Greetings, * Kyle Bateman (k...@batemans.org) wrote: > What I hope to accomplish is: Establish a secure, encrypted connection to > Postgresql from a trusted process, possibly running on another machine, whom > I trust to tell me which user (within a limited set, defined by a role) it > would like

Re: v12 and pg_restore -f-

2019-10-16 Thread Stephen Frost
Greetings, * Justin Pryzby (pry...@telsasoft.com) wrote: > On Wed, Oct 16, 2019 at 03:04:52PM -0400, Stephen Frost wrote: > > > > On Wed, Oct 16, 2019 at 01:21:48PM -0400, Stephen Frost wrote: > > > > [...] if they actually need to work with both concurrently (which > > > > strikes me > > > >

Re: v12.0: segfault in reindex CONCURRENTLY

2019-10-16 Thread Justin Pryzby
On Sun, Oct 13, 2019 at 04:18:34PM -0300, Alvaro Herrera wrote: > (FWIW I expect the crash is possible not just in reindex but also in > CREATE INDEX CONCURRENTLY.) FWIW, for sake of list archives, and for anyone running v12 hoping to avoid crashing, I believe we hit this for DROP INDEX

Re: WIP/PoC for parallel backup

2019-10-16 Thread Jeevan Ladhe
I quickly tried to have a look at your 0001-refactor patch. Here are some comments: 1. The patch fails to compile. Sorry if I am missing something, but am not able to understand why in new function collectTablespaces() you have added an extra parameter NULL while calling sendTablespace(), it

Re: v12 and pg_restore -f-

2019-10-16 Thread Justin Pryzby
On Wed, Oct 16, 2019 at 03:04:52PM -0400, Stephen Frost wrote: > > On Wed, Oct 16, 2019 at 01:21:48PM -0400, Stephen Frost wrote: > > > [...] if they actually need to work with both concurrently (which strikes > > > me > > > as already at least relatively uncommon...). > > > > I doubt it's

[Patch proposal] libpq portal support

2019-10-16 Thread Sergei Fedorov
Hello everybody, Our company was in desperate need of portals in async interface of libpq, so we patched it. We would be happy to upstream the changes. The description of changes: Two functions in libpq-fe.h: PQsendPortalBindParams for sending a command to bind a portal to a previously

Re: v12 and pg_restore -f-

2019-10-16 Thread Stephen Frost
Greetings, * Justin Pryzby (pry...@telsasoft.com) wrote: > On Sun, Oct 06, 2019 at 04:43:13PM -0400, Tom Lane wrote: > > Nobody is going to wish that to mean "write to a file named '-'", so > > Probably right, but it occurs to me that someone could make a named pipe > called > that, possibly to

Re: [Proposal] Global temporary tables

2019-10-16 Thread 曾文旌(义从)
> 2019年10月12日 下午1:16,Pavel Stehule 写道: > > > > pá 11. 10. 2019 v 15:50 odesílatel Konstantin Knizhnik > mailto:k.knizh...@postgrespro.ru>> napsal: > > > On 11.10.2019 15:15, 曾文旌(义从) wrote: >> Dear Hackers, >> >> This propose a way to develop global temporary tables in PostgreSQL. >> >>

Re: v12 and pg_restore -f-

2019-10-16 Thread Justin Pryzby
On Sun, Oct 06, 2019 at 04:43:13PM -0400, Tom Lane wrote: > Nobody is going to wish that to mean "write to a file named '-'", so Probably right, but it occurs to me that someone could make a named pipe called that, possibly to make "dump to stdout" work with scripts that don't support dumping to

Re: v12 and pg_restore -f-

2019-10-16 Thread Stephen Frost
Greetings, * imai.yoshik...@fujitsu.com (imai.yoshik...@fujitsu.com) wrote: > On Sun, Oct 6, 2019 at 7:09 PM, Justin Pryzby wrote: > > I saw this and updated our scripts with pg_restore -f- > > https://www.postgresql.org/docs/12/release-12.html > > |In pg_restore, require specification of -f - to

Re: ERROR: multixact X from before cutoff Y found to be still running

2019-10-16 Thread Bossart, Nathan
On 10/15/19, 11:11 PM, "Thomas Munro" wrote: > Here's a version with a proposed commit message and a comment. Please > let me know if I credited things to the right people! Looks good to me. Thanks! Nathan

Re: configure fails for perl check on CentOS8

2019-10-16 Thread Andrew Dunstan
On Wed, Oct 16, 2019 at 8:32 AM Tom Lane wrote: > > Andrew Dunstan writes: > > On Tue, Oct 15, 2019 at 6:45 AM Kyotaro Horiguchi > > wrote: > >> The problematic command line boils down to: > >> $ ./configure --with-perl CFLAGS=-O0 > >> It is bash-aliased and survived without being found for a

Re: Questions/Observations related to Gist vacuum

2019-10-16 Thread Heikki Linnakangas
On 16 October 2019 12:57:03 CEST, Amit Kapila wrote: >On Tue, Oct 15, 2019 at 7:13 PM Heikki Linnakangas >wrote: >> All things >> considered, I'm not sure which is better. > >Yeah, this is a tough call to make, but if we can allow it to delete >the pages in bulkdelete conditionally for parallel

Re: WIP/PoC for parallel backup

2019-10-16 Thread Asif Rehman
On Mon, Oct 7, 2019 at 6:35 PM Asif Rehman wrote: > > > On Mon, Oct 7, 2019 at 6:05 PM Robert Haas wrote: > >> On Mon, Oct 7, 2019 at 8:48 AM Asif Rehman >> wrote: >> > Sure. Though the backup manifest patch calculates and includes the >> checksum of backup files and is done >> > while the

Re: [HACKERS] advanced partition matching algorithm for partition-wise join

2019-10-16 Thread Etsuro Fujita
On Wed, Sep 25, 2019 at 12:59 AM Etsuro Fujita wrote: > I will continue to review the rest of the patch. I've been reviewing the rest of the patch. Here are my review comments: * map_and_merge_partitions() checks whether the two partitions from the outer and inner sides can be merged in two

Re: fairywren failures

2019-10-16 Thread Andrew Dunstan
On Sat, Oct 12, 2019 at 3:56 PM Peter Eisentraut wrote: > > On 2019-10-03 16:21, Andrew Dunstan wrote: > > My new msys2 animal fairywren > > Could you please check how this animal is labeled? AFAICT, this is not > an msys2 build but a mingw build (x86_64-w64-mingw32). It is indeed an msys2

Re: Cache lookup errors with functions manipulation object addresses

2019-10-16 Thread Dmitry Dolgov
> On Tue, Sep 24, 2019 at 1:58 AM Michael Paquier wrote: > > Please feel free to use the updated versions attached. These can > apply on top of HEAD at 30d1379. Thanks for the update. Looking at v17 0003 I have one more question. In all the places where we have to do systable_endscan, it

Re: Questions/Observations related to Gist vacuum

2019-10-16 Thread Amit Kapila
On Tue, Oct 15, 2019 at 7:13 PM Heikki Linnakangas wrote: > > On 15/10/2019 09:37, Amit Kapila wrote: > > 2. Right now, in gistbulkdelete we make a note of empty leaf pages and > > internals pages and then in the second pass during gistvacuumcleanup, > > we delete all the empty leaf pages. I was

Re: strong memory leak in plpgsql from handled rollback and lazy cast

2019-10-16 Thread Andres Freund
Hi, On 2019-09-22 20:16:15 -0400, Tom Lane wrote: > I think you're considering a much smaller slice of the system than > what seems to me to be at risk here. As an example, an ExprState > tree would also include any fn_extra-linked state that individual > functions might've set up. > > Hm. I

Re: Understanding TupleQueue impact and overheads?

2019-10-16 Thread Andres Freund
Hi, On 2019-10-16 01:24:04 +, Tom Mercha wrote: > What I am having difficulty understanding is what happens to the > location of the HeapTuple as it moves from one TupleTableSlot to the > other as described above. Since there most likely is a reference to a > physical tuple involved, am I

Bug about drop index concurrently

2019-10-16 Thread 李杰(慎追)
Hi hackers, In recently, I discovered a postgres bug, and I hope I can ask you for the best solution. The problem is as follows: postgres=# explain analyze select * from xxx where a=500; ERROR: could not open relation with OID 25989 The structure of my table is as follows: postgres=# \d xxx

Re: maintenance_work_mem used by Vacuum

2019-10-16 Thread Amit Kapila
On Wed, Oct 16, 2019 at 7:20 AM Masahiko Sawada wrote: > > On Sat, Oct 12, 2019 at 8:45 PM Amit Kapila wrote: > > > > On Sat, Oct 12, 2019 at 10:49 AM Masahiko Sawada > > wrote: > > > > > > On Fri, Oct 11, 2019 at 5:13 PM Amit Kapila > > > wrote: > > > > > > > > That's right, but OTOH, if

RE: v12 and pg_restore -f-

2019-10-16 Thread imai.yoshik...@fujitsu.com
Hi, On Sun, Oct 6, 2019 at 7:09 PM, Justin Pryzby wrote: > I saw this and updated our scripts with pg_restore -f- > https://www.postgresql.org/docs/12/release-12.html > |In pg_restore, require specification of -f - to send the dump contents to > standard output (Euler Taveira) > |Previously,

Re: Non-Active links being referred in our source code

2019-10-16 Thread Michael Paquier
On Mon, Oct 14, 2019 at 09:48:12PM +0530, vignesh C wrote: > About pg_crc.h, I have made the changes with the correct links. > The patch for the same is attached. Confirmed, so applied. Thanks, Vignesh. -- Michael signature.asc Description: PGP signature

Re: ERROR: multixact X from before cutoff Y found to be still running

2019-10-16 Thread Thomas Munro
On Wed, Sep 18, 2019 at 8:11 AM Bossart, Nathan wrote: > Thanks for the detailed background information. FWIW I am now in > favor of the v2 patch. Here's a version with a proposed commit message and a comment. Please let me know if I credited things to the right people!

RE: Copy data to DSA area

2019-10-16 Thread ideriha.take...@fujitsu.com
Hi, >ShmZoneContext for SharedPlan and SharedRelCache is not implemented but I'm >going to do it following your points. After looking into existing code, I'm thinking Generation Memory Context seems to have the similar purpose. So I'll implement ShmZoneContext by reference it. Generation

Re: [HACKERS] Block level parallel vacuum

2019-10-16 Thread Amit Kapila
On Wed, Oct 16, 2019 at 6:50 AM Masahiko Sawada wrote: > > On Tue, Oct 15, 2019 at 6:33 PM Amit Kapila wrote: > > > > Attached updated patch set. 0001 patch introduces new index AM field > amcanparallelvacuum. All index AMs except for gist sets true for now. > 0002 patch incorporated the all