Re: Commitfest 2018-11

2018-11-01 Thread Dmitry Dolgov
> On Thu, 1 Nov 2018 at 14:11, Stephen Frost wrote: > I've added you to the 'cf admins' group Thanks. > so please give it a shot now and let me know if you run into any troubles. Hm...I don't see any difference in CF app, what should be changed?

Re: zheap: a new storage format for PostgreSQL

2018-11-01 Thread Tomas Vondra
On 11/01/2018 07:43 AM, Amit Kapila wrote: You can find the latest code at https://github.com/EnterpriseDB/zheap Seems valgrind complains about a couple of places in the code - nothing major, might be noise, but probably worth a look. regards -- Tomas Vondra

Re: [PATCH][PROPOSAL] Add enum releation option type

2018-11-01 Thread Robert Haas
On Wed, Mar 7, 2018 at 10:23 AM Nikolay Shaplov wrote: > > I see you lost the Oxford comma: > > > > -DETAIL: Valid values are "on", "off", and "auto". > > +DETAIL: Valid values are "auto", "on" and "off". > > > > Please put these back. > Actually that's me who have lost it. The code with

Re: PG vs macOS Mojave

2018-11-01 Thread Daniel Gustafsson
> On 1 Nov 2018, at 15:53, Tom Lane wrote: > > Daniel Gustafsson writes: >>> On 1 Nov 2018, at 15:14, Tom Lane wrote: >>> Wow ... could it be that it actually varies depending on the combination >>> of compiler and OS versions? That would be weird. > >> Or the version of XCode and the set of

Re: [PATCH][PROPOSAL] Add enum releation option type

2018-11-01 Thread Tom Lane
Robert Haas writes: > On Wed, Mar 7, 2018 at 10:23 AM Nikolay Shaplov wrote: >>> I see you lost the Oxford comma: >>> >>> -DETAIL: Valid values are "on", "off", and "auto". >>> +DETAIL: Valid values are "auto", "on" and "off". >>> >>> Please put these back. >> Actually that's me who have

Re: [PATCH][PROPOSAL] Add enum releation option type

2018-11-01 Thread Nikolay Shaplov
В письме от 1 ноября 2018 11:10:14 пользователь Tom Lane написал: > >>> I see you lost the Oxford comma: > >>> > >>> -DETAIL: Valid values are "on", "off", and "auto". > >>> +DETAIL: Valid values are "auto", "on" and "off". > >>> > >>> Please put these back. > >> > >> Actually that's me who

Re: pgsql: Avoid duplicate XIDs at recovery when building initial snapshot

2018-11-01 Thread Michael Paquier
On Tue, Oct 23, 2018 at 10:43:38AM +0900, Michael Paquier wrote: > Well, following the same kind of thoughts, txid_current_snapshot() uses > sort_snapshot() to remove all the duplicates after fetching its data > from GetSnapshotData(), so wouldn't we want to do something about > removal of

Hooks to Modify Execution Flow and Query Planner

2018-11-01 Thread Vincent Mirian
Hi all, I would like to create a library with UDFs written in C that implements different Query Planner tasks (e.g. scan, hash, join, etc...). I am looking for a document that provides an overview of execution flow within postgres and the query planner. I am also looking for a description of the

Re: PostgreSQL Limits and lack of documentation about them.

2018-11-01 Thread Andrew Gierth
> "Nasby," == Nasby, Jim writes: >> I did try a table with 1600 text columns then inserted values of >> several kB each. Trying with BIGINT columns the row was too large >> for the page. I've never really gotten a chance to explore these >> limits before, so I guess this is about the

Re: Parallel threads in query

2018-11-01 Thread Konstantin Knizhnik
On 31.10.2018 22:07, Darafei "Komяpa" Praliaskouski wrote: Hi, I've tried porting some of PostGIS algorithms to utilize multiple cores via OpenMP to return faster. Question is, what's the best policy to allocate cores so we can play nice with rest of postgres? What I'd like to see is

Re: row filtering for logical replication

2018-11-01 Thread Erik Rijkers
eaking of PATHs, PG-variables (PGPASSFILE) etc). This is the same program I used in march when you first posted a version of this patch alhough the error is different. thanks, Erik Rijkers #!/bin/bash # postgres binary compiled with # # 20181101 # 0001-Remove-unused-atttypmod-colum

Re: Hooks to Modify Execution Flow and Query Planner

2018-11-01 Thread Amit Langote
Hi, On 2018/11/01 16:58, Vincent Mirian wrote: > Hi all, > > I would like to create a library with UDFs written in C that implements > different Query Planner tasks (e.g. scan, hash, join, etc...). I am looking > for a document that provides an overview of execution flow within postgres > and

Re: Ordered Partitioned Table Scans

2018-11-01 Thread Antonin Houska
David Rowley wrote: > On 1 November 2018 at 04:01, Antonin Houska wrote: > > * As for the logic, I found generate_mergeappend_paths() to be the most > > interesting part: > > > > Imagine table partitioned by "i", so "partition_pathkeys" is {i}. > > > > partition 1: > > > > i | j > > --+-- > >

Re: INSTALL file

2018-11-01 Thread Michael Paquier
On Thu, Nov 01, 2018 at 01:32:09AM +0100, Andreas 'ads' Scherbaum wrote: > Picking up on this idea, attached is a first draft for changing the > README. Why don't you add it to the upcoming commit fest? It would be good to get some traction with a formal review. > It includes links to the

Re: zheap: a new storage format for PostgreSQL

2018-11-01 Thread Amit Kapila
On Sat, May 26, 2018 at 6:33 PM Amit Kapila wrote: > On Fri, Mar 2, 2018 at 4:05 PM, Alexander Korotkov > wrote: > > It's been a while since we have updated the progress on this project, > so here is an update. > Yet, another update. > This is based on the features that were not > working (as

RE: [Proposal] Add accumulated statistics for wait event

2018-11-01 Thread Yotsunaga, Naoki
On Mon, Oct 29, 2018 at 1:52 AM, Phil Florent wrote: Hi, thank you for comments. >Yes you will be able to solve bottlenecks with sampling. In interactive mode, >a 1s interval is probably too large. I use 0s1 - 0s01 with my tool and it is >normally OK. With the tool you are using, can you

Re: row filtering for logical replication

2018-11-01 Thread Erik Rijkers
On 2018-11-01 08:56, Erik Rijkers wrote: On 2018-11-01 01:29, Euler Taveira wrote: Em qua, 28 de fev de 2018 às 20:03, Euler Taveira escreveu: The attached patches add support for filtering rows in the publisher. I ran pgbench-over-logical-replication with a WHERE-clause and could not get

Re: Speeding up INSERTs and UPDATEs to partitioned tables

2018-11-01 Thread Amit Langote
On 2018/11/01 10:30, David Rowley wrote: > It's great to know the patch is now so perfect that we've only the > macro naming left to debate ;-) I looked over v12 again and noticed a couple minor issues. + * table then we store the index into parenting + *

Re: [HACKERS] generated columns

2018-11-01 Thread John Naylor
On 10/30/18, Peter Eisentraut wrote: > 3. Radical alternative: Collapse everything into one new column. We > could combine atthasdef and attgenerated and even attidentity into a new > column. (Only one of the three can be the case.) This would give > client code a clean break, which may or may

Re: CVE-2017-7484-induced bugs, or, btree cmp functions are not leakproof?

2018-11-01 Thread Dilip Kumar
On Mon, Oct 29, 2018 at 2:53 PM Amit Langote wrote: > > Thank you for creating the patch. > > On 2018/10/28 20:35, Dilip Kumar wrote: > > On Sat, Oct 27, 2018 at 10:07 AM Dilip Kumar wrote: > >> On Fri, Oct 26, 2018 at 1:12 PM Amit Langote wrote: > >>> On 2018/10/25 19:54, Dilip Kumar wrote: >

Doubts about pushing LIMIT to MergeAppendPath

2018-11-01 Thread Antonin Houska
Review of [1] made me think of this optimization, currently used only in create_merge_append_path(): /* * Apply query-wide LIMIT if known and path is for sole base relation. * (Handling this at this low level is a bit klugy.) */ if

Re: Is there way to detect uncommitted 'new table' in pg_class?

2018-11-01 Thread Hubert Zhang
Thanks On Thu, Nov 1, 2018 at 8:38 AM Michael Paquier wrote: > On Wed, Oct 31, 2018 at 01:30:52PM -0400, Robert Haas wrote: > > In theory, at least, you could write C code to scan the catalog tables > > with SnapshotDirty to find the catalog entries, but I don't think that > > helps a whole

Re: Super PathKeys (Allowing sort order through precision loss functions)

2018-11-01 Thread Tomas Vondra
On 11/01/2018 02:40 AM, David Rowley wrote: On 1 November 2018 at 12:24, Andres Freund wrote: FWIW, I kind of wonder if we built proper infrastructure to allow to make such inferrences from function calls, whether it could also be made to support the transformation of LIKEs into indexable <=

Re: pg_promote not marked as parallel-restricted in pg_proc.dat

2018-11-01 Thread Amit Kapila
On Thu, Nov 1, 2018 at 6:14 AM Michael Paquier wrote: > > On Wed, Oct 31, 2018 at 01:09:53PM -0400, Robert Haas wrote: > > There's no rule whatsoever that a parallel worker can't write to the > > disk. pg_start_backup and pg_stop_backup have to be > > parallel-restricted because, when used in

Re: FETCH FIRST clause WITH TIES option

2018-11-01 Thread Tomas Vondra
On 10/31/2018 06:17 PM, Robert Haas wrote: On Mon, Oct 29, 2018 at 12:48 PM Andrew Gierth wrote: Then FETCH FIRST N WITH TIES becomes "stop when the expression rank() over (order by ...) <= N is no longer true" (where the ... is the existing top level order by) Wouldn't that be wicked

Re: Connection slots reserved for replication

2018-11-01 Thread Alexander Kukushkin
Hi, Attached rebased version patch to the current HEAD and created commit fest entry On Fri, 21 Sep 2018 at 13:43, Alexander Kukushkin wrote: > > Hi, > > On 20 September 2018 at 08:18, Kyotaro HORIGUCHI > wrote: > > > > > Instaed, we can iterally "reserve" connection slots for the > > specific

Re: More issues with pg_verify_checksums and checksum verification in base backups

2018-11-01 Thread Amit Kapila
On Sun, Oct 21, 2018 at 7:12 PM Michael Paquier wrote: > > Hi all, > > This is a follow-up of the following thread: > https://www.postgresql.org/message-id/20181012010411.re53cwcistcpi...@alap3.anarazel.de > > In a nutshell, b34e84f1 has added TAP tests for pg_verify_checksums, and > the

Re: FETCH FIRST clause WITH TIES option

2018-11-01 Thread Surafel Temesgen
hi, The attached patch include all the comment given by Tomas and i check sql standard about LIMIT and this feature it did not say anything about it but I think its good idea to include it to LIMIT too and I will add it if we have consensus on it. regards surafel diff --git

Re: PG vs macOS Mojave

2018-11-01 Thread Daniel Gustafsson
> On 1 Nov 2018, at 04:17, Tom Lane wrote: > and on Xcode 10.0 I get Odd. I don’t actually get -isysroot on XCode 10.0 on my 10.13.6 installation, on 10.12 with XCode 8.3.3 I do however get -isysroot. > Right now I think the only plausible > fix is to go back to adding "-isysroot $PG_SYSROOT"

Commitfest 2018-11

2018-11-01 Thread Daniel Gustafsson
We are now in November and the 2018-11 commitfest has started (or rather, will one someone pushes the button) but AFAICT there hasn’t been anyone volunteering to do be CFM this time apart from Dmitry Dolgov who showed an interest (in

Re: Commitfest 2018-11

2018-11-01 Thread Dmitry Dolgov
> On Thu, 1 Nov 2018 at 13:05, Daniel Gustafsson wrote: > > Dmitry: Are you still interested in running this commitfest? Yes, I'm still interested. Do I need to have any special permissions in CF app for that (e.g. now I can't "push the button" to start the current one)?

Re: bugfix: BUG #15477: Procedure call with named inout refcursor parameter - "invalid input syntax for type boolean"

2018-11-01 Thread Pavel Stehule
Cleaned patch with regress tests diff --git a/src/pl/plpgsql/src/expected/plpgsql_call.out b/src/pl/plpgsql/src/expected/plpgsql_call.out index 547ca22a55..8762e1335c 100644 --- a/src/pl/plpgsql/src/expected/plpgsql_call.out +++ b/src/pl/plpgsql/src/expected/plpgsql_call.out @@ -276,3 +276,43 @@

CF app feature request

2018-11-01 Thread Andrew Dunstan
Yesterday Fabien and I submitted the same item to the Commitfest (1859 and 1860). Unfortunately there doesn't seem to be any way for one of these to be withdrawn. "Rejected" and "Returned with Feedback" seem wrong. Ideally, there would be a way for someone who submits an item in error to

bugfix: BUG #15477: Procedure call with named inout refcursor parameter - "invalid input syntax for type boolean"

2018-11-01 Thread Pavel Stehule
Hi The processing of named parameters inside CALL statement is not correct. It is my code :-/. I am sorry Attached patch fix it. This still doesn't fix INOUT variables with default value - so is not fully correct, but in this moment, it can show, where is a core of this issue. Regards Pavel

Re: bugfix: BUG #15477: Procedure call with named inout refcursor parameter - "invalid input syntax for type boolean"

2018-11-01 Thread Pavel Stehule
čt 1. 11. 2018 v 13:00 odesílatel Pavel Stehule napsal: > Hi > > The processing of named parameters inside CALL statement is not correct. > > It is my code :-/. I am sorry > > Attached patch fix it. > > This still doesn't fix INOUT variables with default value - so is not > fully correct, but in

Re: Commitfest 2018-11

2018-11-01 Thread Daniel Gustafsson
> On 1 Nov 2018, at 13:19, Dmitry Dolgov <9erthali...@gmail.com> wrote: > >> On Thu, 1 Nov 2018 at 13:05, Daniel Gustafsson wrote: >> >> Dmitry: Are you still interested in running this commitfest? > > Yes, I'm still interested. Great! > Do I need to have any special permissions in CF app >

Re: bugfix: BUG #15477: Procedure call with named inout refcursor parameter - "invalid input syntax for type boolean"

2018-11-01 Thread Pavel Stehule
čt 1. 11. 2018 v 13:10 odesílatel Pavel Stehule napsal: > > > čt 1. 11. 2018 v 13:00 odesílatel Pavel Stehule > napsal: > >> Hi >> >> The processing of named parameters inside CALL statement is not correct. >> >> It is my code :-/. I am sorry >> >> Attached patch fix it. >> >> This still

Re: Commitfest 2018-11

2018-11-01 Thread Stephen Frost
Greetings, * Daniel Gustafsson (dan...@yesql.se) wrote: > > On 1 Nov 2018, at 13:19, Dmitry Dolgov <9erthali...@gmail.com> wrote: > > Do I need to have any special permissions in CF app > > for that (e.g. now I can't "push the button" to start the current one)? > > Yes, I believe either Magnus

Re: INSTALL file

2018-11-01 Thread Andreas 'ads' Scherbaum
On 01.11.18 07:26, Michael Paquier wrote: On Thu, Nov 01, 2018 at 01:32:09AM +0100, Andreas 'ads' Scherbaum wrote: Picking up on this idea, attached is a first draft for changing the README. Why don't you add it to the upcoming commit fest? It would be good to get some traction with a formal

Re: New vacuum option to do only freezing

2018-11-01 Thread Robert Haas
On Mon, Oct 1, 2018 at 6:23 AM Masahiko Sawada wrote: > Attached patch adds a new option FREEZE_ONLY to VACUUM command. This > option is same as FREEZE option except for it disables reclaiming dead > tuples. That is, with this option vacuum does pruning HOT chain, > freezing live tuples and

Re: pg_promote not marked as parallel-restricted in pg_proc.dat

2018-11-01 Thread Robert Haas
On Wed, Oct 31, 2018 at 8:43 PM Michael Paquier wrote: > Okay, but likely we would not want to signal the postmaster > unnecessarily, no? FALLBACK_PROMOTE_SIGNAL_FILE gets discarded if > promotion is triggered more than once, but that does not like a sane > thing to do if not necessary. Uh,

Re: New vacuum option to do only freezing

2018-11-01 Thread Bossart, Nathan
Hi, On 10/1/18, 5:23 AM, "Masahiko Sawada" wrote: > Attached patch adds a new option FREEZE_ONLY to VACUUM command. This > option is same as FREEZE option except for it disables reclaiming dead > tuples. That is, with this option vacuum does pruning HOT chain, > freezing live tuples and

Re: Parallel threads in query

2018-11-01 Thread Paul Ramsey
On Wed, Oct 31, 2018 at 2:11 PM Tom Lane wrote: > =?UTF-8?Q?Darafei_=22Kom=D1=8Fpa=22_Praliaskouski?= > writes: > > Question is, what's the best policy to allocate cores so we can play nice > > with rest of postgres? > > There is not, because we do not use or support multiple threads inside >

Re: Parallel threads in query

2018-11-01 Thread Andres Freund
On 2018-11-01 10:10:33 -0700, Paul Ramsey wrote: > On Wed, Oct 31, 2018 at 2:11 PM Tom Lane wrote: > > > =?UTF-8?Q?Darafei_=22Kom=D1=8Fpa=22_Praliaskouski?= > > writes: > > > Question is, what's the best policy to allocate cores so we can play nice > > > with rest of postgres? > > > > > >

Re: Commitfest 2018-11

2018-11-01 Thread Stephen Frost
Greetings, * Dmitry Dolgov (9erthali...@gmail.com) wrote: > > On Thu, 1 Nov 2018 at 14:11, Stephen Frost wrote: > > I've added you to the 'cf admins' group > > Thanks. > > > so please give it a shot now and let me know if you run into any troubles. > > Hm...I don't see any difference in CF

Re: Commitfest 2018-11

2018-11-01 Thread Dmitry Dolgov
> On Thu, 1 Nov 2018 at 15:11, Stephen Frost wrote: > Hmm... Can you try it again? Yep, now I see the administration menu, thanks.

Re: PG vs macOS Mojave

2018-11-01 Thread Tom Lane
Daniel Gustafsson writes: > Odd. I don’t actually get -isysroot on XCode 10.0 on my 10.13.6 installation, > on 10.12 with XCode 8.3.3 I do however get -isysroot. Wow ... could it be that it actually varies depending on the combination of compiler and OS versions? That would be weird.

Re: PG vs macOS Mojave

2018-11-01 Thread Daniel Gustafsson
> On 1 Nov 2018, at 15:14, Tom Lane wrote: > > Daniel Gustafsson writes: >> Odd. I don’t actually get -isysroot on XCode 10.0 on my 10.13.6 >> installation, >> on 10.12 with XCode 8.3.3 I do however get -isysroot. > > Wow ... could it be that it actually varies depending on the combination >

Re: POC: Cleaning up orphaned files using undo logs

2018-11-01 Thread Robert Haas
While I've been involved in the design discussions for this patch set, I haven't looked at any of the code personally in a very long time. I certainly don't claim to be an independent reviewer, and I encourage others to review this work also. That said, here are some review comments. I decided

Re: PG vs macOS Mojave

2018-11-01 Thread Tom Lane
Daniel Gustafsson writes: >> On 1 Nov 2018, at 15:14, Tom Lane wrote: >> Wow ... could it be that it actually varies depending on the combination >> of compiler and OS versions? That would be weird. > Or the version of XCode and the set of installed SDKs? I only have a single > SDK installed

Re: Parallel threads in query

2018-11-01 Thread Tomas Vondra
On 11/01/2018 07:50 PM, Andres Freund wrote: > Hi, > > On 2018-11-01 19:44:54 +0100, Tomas Vondra wrote: >> On 11/01/2018 07:40 PM, Andres Freund wrote: >>> On 2018-11-01 19:33:39 +0100, Tomas Vondra wrote: In theory, simulating such global limit should be possible using a bit of shared

Re: Doubts about pushing LIMIT to MergeAppendPath

2018-11-01 Thread Tomas Vondra
On 11/01/2018 08:51 PM, Antonin Houska wrote: > Tomas Vondra wrote: > >> On 11/01/2018 12:48 PM, Antonin Houska wrote: >>> Review of [1] made me think of this optimization, currently used only in >>> create_merge_append_path(): >>> >>> /* >>> * Apply query-wide LIMIT if known and

Re: CF app feature request

2018-11-01 Thread Andrew Dunstan
On 11/01/2018 05:50 PM, Magnus Hagander wrote: On Thu, Nov 1, 2018 at 2:44 PM Andrew Dunstan > wrote: Yesterday Fabien and I submitted the same item to the Commitfest (1859 and 1860). Unfortunately there doesn't seem to be any way for

Re: PG vs macOS Mojave

2018-11-01 Thread Tom Lane
So it seems like there are two ways we could go about this. One is to go back to the scheme of adding an -isysroot switch to CPPFLAGS, where it'd have global effects. We could make this slightly less painful for scenarios like Jakob's if we set things up in Makefile.global this way: CPPFLAGS =

Re: PG vs macOS Mojave

2018-11-01 Thread Tom Lane
I wrote: > Then somebody who wants to build on a different SDK version still needs > to do "make PG_SYSROOT=/proper/path", but only if they're trying to > build PL/Perl or related extensions. So this second way seems uglier > in some sense but less likely to cause problems for most people. >

Re: Parallel threads in query

2018-11-01 Thread Tomas Vondra
On 11/01/2018 08:03 PM, Andres Freund wrote: > On 2018-11-01 19:57:17 +0100, Tomas Vondra wrote: I think that very much depends on how expensive the tasks handled by the threads are. It may still be cheaper than a reasonable IPC, and if you don't create/destroy threads, that also

Re: Buildfarm failures for hash indexes: buffer leaks

2018-11-01 Thread Fabien COELHO
Their commit r265375 fixed the ability to compile itself, but built PostgreSQL binaries remain broken there and thereafter. |...] Thanks a lot for this investigation! I can fill in a gcc bug report. There would be a enormous work to narrow it down to a small test case, it is unclear how

Re: PG vs macOS Mojave

2018-11-01 Thread Peter Eisentraut
On 01/11/2018 22:17, Tom Lane wrote: > The other idea that's occurred to me is to go back to the scheme of > commit 68fc227dd, where we inject the sysroot path into just the -I > switches used for PL/Perl and PL/Tcl. We could improve on that > commit by injecting it symbolically similar to what I

Re: Doubts about pushing LIMIT to MergeAppendPath

2018-11-01 Thread Tomas Vondra
On 11/01/2018 08:11 PM, Tomas Vondra wrote: > On 11/01/2018 12:48 PM, Antonin Houska wrote: >> Review of [1] made me think of this optimization, currently used only in >> create_merge_append_path(): >> >> /* >> * Apply query-wide LIMIT if known and path is for sole base relation. >>

Re: Remove obsolete pg_attrdef.adsrc column

2018-11-01 Thread Peter Eisentraut
On 27/10/2018 23:19, Daniel Gustafsson wrote: >> On 27 Oct 2018, at 12:57, Peter Eisentraut >> wrote: >> >> On 23/10/2018 19:48, Daniel Gustafsson wrote: On 23 Oct 2018, at 15:17, Peter Eisentraut wrote: I propose the attached patch to remove the long-unused catalog column

Re: PG vs macOS Mojave

2018-11-01 Thread Tom Lane
I wrote: > Peter Eisentraut writes: >> On 01/11/2018 22:17, Tom Lane wrote: >>> The other idea that's occurred to me is to go back to the scheme of >>> commit 68fc227dd, where we inject the sysroot path into just the -I >>> switches used for PL/Perl and PL/Tcl. We could improve on that >>>

Re: Doubts about pushing LIMIT to MergeAppendPath

2018-11-01 Thread Antonin Houska
Tomas Vondra wrote: > On 11/01/2018 12:48 PM, Antonin Houska wrote: > > Review of [1] made me think of this optimization, currently used only in > > create_merge_append_path(): > > > > /* > > * Apply query-wide LIMIT if known and path is for sole base relation. > > * (Handling

Re: CF app feature request

2018-11-01 Thread Tom Lane
Andrew Dunstan writes: > On 11/01/2018 05:50 PM, Magnus Hagander wrote: >> Are you thinking basically another status that's "Withdrawn", but >> keeping it, or actually removing the records completely? > I don't know enough about the app internals to comment. But it probably > shouldn't appear

Re: Parallel threads in query

2018-11-01 Thread Komяpa
> > > Because you said "faster than reasonable IPC" - which to me implies that > you don't do full blown IPC. Which using threads in a bgworker is very > strongly implying. What you're proposing strongly implies multiple > context switches just to process a few results. Even before, but >

Re: Hash Joins vs. Bloom Filters / take 2

2018-11-01 Thread Thomas Munro
On Fri, Nov 2, 2018 at 9:23 AM Jim Finnerty wrote: > I'm very interested in this patch, and particularly in possible > extensions to push the Bloom filter down on the probe side of the join. I > made a few small edits to the patch to enable it to compile on PG11, and can > send it to you if

Re: Compressed TOAST Slicing

2018-11-01 Thread Stephen Frost
Greetings, * Paul Ramsey (pram...@cleverelephant.ca) wrote: > The attached patch adds in a code path to do a partial decompression of the > TOAST entry, when the requested slice is at the start of the object. Neat! > As usual, doing less work is faster. Definitely. > Interesting note to

Re: Compressed TOAST Slicing

2018-11-01 Thread Paul Ramsey
On Thu, Nov 1, 2018 at 2:29 PM Stephen Frost wrote: > Greetings, > > * Paul Ramsey (pram...@cleverelephant.ca) wrote: > > The attached patch adds in a code path to do a partial decompression of > the > > TOAST entry, when the requested slice is at the start of the object. > > There two things

Re: Compressed TOAST Slicing

2018-11-01 Thread Tom Lane
Paul Ramsey writes: > On Thu, Nov 1, 2018 at 2:29 PM Stephen Frost wrote: >> and secondly, why we wouldn't consider >> handling a non-zero offset. A non-zero offset would, of course, still >> require decompressing from the start and then just throwing away what we >> skip over, but we're going

Compressed TOAST Slicing

2018-11-01 Thread Paul Ramsey
Currently, PG_DETOAST_DATUM_SLICE when run on a compressed TOAST entry will first decompress the whole object, then extract the relevant slice. When the desired slice is at or near the front of the object, this is obviously non-optimal. The attached patch adds in a code path to do a partial

Re: Hash Joins vs. Bloom Filters / take 2

2018-11-01 Thread Tomas Vondra
On 11/01/2018 10:06 PM, Thomas Munro wrote: > On Fri, Nov 2, 2018 at 9:23 AM Jim Finnerty wrote: >> I'm very interested in this patch, and particularly in possible >> extensions to push the Bloom filter down on the probe side of the join. I >> made a few small edits to the patch to enable it

Re: PG vs macOS Mojave

2018-11-01 Thread Tom Lane
Peter Eisentraut writes: > On 01/11/2018 22:17, Tom Lane wrote: >> The other idea that's occurred to me is to go back to the scheme of >> commit 68fc227dd, where we inject the sysroot path into just the -I >> switches used for PL/Perl and PL/Tcl. We could improve on that >> commit by injecting

Re: Parallel threads in query

2018-11-01 Thread Andres Freund
On 2018-11-01 19:57:17 +0100, Tomas Vondra wrote: > >> I think that very much depends on how expensive the tasks handled by the > >> threads are. It may still be cheaper than a reasonable IPC, and if you > >> don't create/destroy threads, that also saves quite a bit of time. > > > > I'm not

Re: Doubts about pushing LIMIT to MergeAppendPath

2018-11-01 Thread Tomas Vondra
On 11/01/2018 12:48 PM, Antonin Houska wrote: > Review of [1] made me think of this optimization, currently used only in > create_merge_append_path(): > > /* >* Apply query-wide LIMIT if known and path is for sole base relation. >* (Handling this at this low level is a bit

Re: Parallel threads in query

2018-11-01 Thread Andres Freund
Hi, On 2018-11-01 09:17:56 -1000, Darafei "Komяpa" Praliaskouski wrote: > So, do I understand correctly that I need to start a parallel worker that > does nothing for each thread I launch to consume the parallel worker limit? No, I don't think that'd be reasonable. I think what we're saying is

Re: replication_slots usability issue

2018-11-01 Thread Michael Paquier
On Thu, Nov 01, 2018 at 10:54:23AM -0700, Andres Freund wrote: > On 2018-11-01 09:34:05 +0900, Michael Paquier wrote: > That has absolutely nothing to do with the issue at hand though, so I > don't think it'd have made much sense to do it at the same time. Nor do > I think it's particularly

Re: ToDo: show size of partitioned table

2018-11-01 Thread Amit Langote
Hi, On 2018/11/01 2:19, Pavel Stehule wrote: > st 31. 10. 2018 v 7:34 odesílatel Amit Langote < > langote_amit...@lab.ntt.co.jp> napsal: >> On 2018/10/31 15:30, Pavel Stehule wrote: >>> st 31. 10. 2018 v 3:27 odesílatel Amit Langote < >>> langote_amit...@lab.ntt.co.jp> napsal: +

Re: CF app feature request

2018-11-01 Thread Magnus Hagander
On Thu, Nov 1, 2018 at 2:44 PM Andrew Dunstan < andrew.duns...@2ndquadrant.com> wrote: > > Yesterday Fabien and I submitted the same item to the Commitfest (1859 > and 1860). Unfortunately there doesn't seem to be any way for one of > these to be withdrawn. "Rejected" and "Returned with Feedback"

Re: COPY FROM WHEN condition

2018-11-01 Thread Pavel Stehule
pá 2. 11. 2018 v 3:57 odesílatel Corey Huinker napsal: > > Are you thinking something like having a COPY command that provides >> > results in such a way that they could be referenced in a FROM clause >> > (perhaps a COPY that defines a cursor…)? >> >> That would also be nice, but what I was

Re: Getting ERROR: could not open file "base/13164/t3_16388" with partition table with ON COMMIT

2018-11-01 Thread Michael Paquier
On Fri, Nov 02, 2018 at 01:36:05PM +0900, Amit Langote wrote: > When writing the test, I noticed something to be pointed out. As of > 1c7c317cd9d, partitions of a temporary partition table themselves must be > temporary, but the ON COMMIT action has to be specified for each table > separately.

Re: Getting ERROR: could not open file "base/13164/t3_16388" with partition table with ON COMMIT

2018-11-01 Thread Michael Paquier
On Fri, Nov 02, 2018 at 02:18:04PM +0900, Michael Paquier wrote: > This case is funky. The parent gets dropped at commit time, but it does > not know that it should drop the child as well per their dependencies. > This actually goes into the internals of performDeletion(), which is > scary to

Re: Pluggable Storage - Andres's take

2018-11-01 Thread Haribabu Kommi
On Wed, Oct 31, 2018 at 9:34 PM Dmitry Dolgov <9erthali...@gmail.com> wrote: > > On Mon, 29 Oct 2018 at 05:56, Haribabu Kommi > wrote: > > > >> This problem couldn't be reproduced on the master branch, so I've tried > to > >> investigate it. It comes from nodeModifyTable.c:1267, when we've got >

Re: INSTALL file

2018-11-01 Thread Stephen Frost
Greetings, * Andreas 'ads' Scherbaum (a...@pgug.de) wrote: > How about the attached one? Picked up your draft, and cleaned it up a bit. (unsurprisingly) this is looking pretty good to me. A few additional notes: > PostgreSQL Database Management System > = >

Re: partitioned indexes and tablespaces

2018-11-01 Thread Michael Paquier
On Thu, Nov 01, 2018 at 09:31:38PM -0300, Alvaro Herrera wrote: > A customer reported to us that partitioned indexes are not working > consistently with tablespaces: Let's see... > 1. When a CREATE INDEX specifies a tablespace, existing partitions get > the index in the correct tablespace;

Re: CF app feature request

2018-11-01 Thread Andrew Dunstan
On 11/01/2018 08:40 PM, Michael Paquier wrote: On Thu, Nov 01, 2018 at 05:56:24PM -0400, Andrew Dunstan wrote: On 11/01/2018 05:50 PM, Magnus Hagander wrote: Are you thinking basically another status that's "Withdrawn", but keeping it, or actually removing the records completely? I don't

Re: WIP Patch: Add a function that returns binary JSONB as a bytea

2018-11-01 Thread Christian Ohler
On Wed, Oct 31, 2018 at 7:22 AM Tom Lane wrote: > If we're going to expose the > internal format, let's just change the definition of the type's binary > I/O format, thereby getting a win for purposes like COPY BINARY as well. > We'd need to ensure that jsonb_recv could tell whether it was

Re: INSTALL file

2018-11-01 Thread Michael Paquier
On Thu, Nov 01, 2018 at 01:41:33PM -0400, Stephen Frost wrote: > I'm not sure that I'm really following this, because we aren't pointing > to the development documentation, just the 'current' documentation, and > that seems fine, and there's links on that page to the other versions of > the page

partitioned indexes and tablespaces

2018-11-01 Thread Alvaro Herrera
Hello A customer reported to us that partitioned indexes are not working consistently with tablespaces: 1. When a CREATE INDEX specifies a tablespace, existing partitions get the index in the correct tablespace; however, the parent index itself does not record the tablespace. So when new

Re: WIP Patch: Add a function that returns binary JSONB as a bytea

2018-11-01 Thread Tom Lane
Christian Ohler writes: > On Wed, Oct 31, 2018 at 7:22 AM Tom Lane wrote: >> If we're going to expose the >> internal format, let's just change the definition of the type's binary >> I/O format, thereby getting a win for purposes like COPY BINARY as well. > How would this work from the driver's

Re: bugfix: BUG #15477: Procedure call with named inout refcursor parameter - "invalid input syntax for type boolean"

2018-11-01 Thread Pavel Stehule
čt 1. 11. 2018 v 14:31 odesílatel Pavel Stehule napsal: > Cleaned patch with regress tests > > minor cleaning Regards Pavel diff --git a/src/pl/plpgsql/src/expected/plpgsql_call.out b/src/pl/plpgsql/src/expected/plpgsql_call.out index 547ca22a55..8762e1335c 100644 ---

Re: More issues with pg_verify_checksums and checksum verification in base backups

2018-11-01 Thread Michael Paquier
On Thu, Nov 01, 2018 at 04:44:40PM +0530, Amit Kapila wrote: > This sounds like a good argument for having a whitelist approach, but > is it really a big problem if a user gets warning for files that the > utility is not able to verify checksums for? I think in some sense > this message can be

Re: row filtering for logical replication

2018-11-01 Thread Euler Taveira
Em qui, 1 de nov de 2018 às 05:30, Erik Rijkers escreveu: > > I ran pgbench-over-logical-replication with a WHERE-clause and could > > not get this to do a correct replication. Below is the output of the > > attached test program. > > > > > > $ ./logrep_rowfilter.sh > Erik, thanks for testing.

Re: partitioned indexes and tablespaces

2018-11-01 Thread Amit Langote
Hi, On 2018/11/02 10:27, Michael Paquier wrote: > On Thu, Nov 01, 2018 at 09:31:38PM -0300, Alvaro Herrera wrote: >> A customer reported to us that partitioned indexes are not working >> consistently with tablespaces: > > Let's see... > >> 1. When a CREATE INDEX specifies a tablespace, existing

Re: pg_promote not marked as parallel-restricted in pg_proc.dat

2018-11-01 Thread Michael Paquier
On Thu, Nov 01, 2018 at 12:59:47PM -0400, Robert Haas wrote: > If you do the same analysis for pg_start_backup(), you'll immediately > notice that it calls get_backup_status(), and if you look at that > function, you'll see that it returns the value of a global variable. > If you check parallel.c,

Re: INSTALL file

2018-11-01 Thread Stephen Frost
Greetings, * Michael Paquier (mich...@paquier.xyz) wrote: > On Thu, Nov 01, 2018 at 01:41:33PM -0400, Stephen Frost wrote: > > I'm not sure that I'm really following this, because we aren't pointing > > to the development documentation, just the 'current' documentation, and > > that seems fine,

Re: COPY FROM WHEN condition

2018-11-01 Thread Corey Huinker
> > > Are you thinking something like having a COPY command that provides > > results in such a way that they could be referenced in a FROM clause > > (perhaps a COPY that defines a cursor…)? > > That would also be nice, but what I was thinking of was that some > highly restricted subset of cases

Re: CF app feature request

2018-11-01 Thread Michael Paquier
On Thu, Nov 01, 2018 at 05:56:24PM -0400, Andrew Dunstan wrote: > On 11/01/2018 05:50 PM, Magnus Hagander wrote: >> Are you thinking basically another status that's "Withdrawn", but >> keeping it, or actually removing the records completely? > > I don't know enough about the app internals to

Re: INSTALL file

2018-11-01 Thread Michael Paquier
On Thu, Nov 01, 2018 at 08:42:34PM -0400, Stephen Frost wrote: > If we go down this route, the master branch should probably link to the > regularly built devel documentation, so that if/when we do make such a > change, we'll point people at the updated documentation too. I don't know how others

Re: Getting ERROR: could not open file "base/13164/t3_16388" with partition table with ON COMMIT

2018-11-01 Thread Michael Paquier
On Thu, Nov 01, 2018 at 01:04:43PM +0900, Michael Paquier wrote: > On Thu, Nov 01, 2018 at 12:39:16PM +0900, Amit Langote wrote: >> Rajkumar pointed out off-list that the patch still remains to be applied. >> Considering that there is a planned point release on Nov 8, maybe we >> should do

Re: heap_sync seems rather oblivious to partitioned tables (wal_level=minimal)

2018-11-01 Thread Steve Singer
On Thu, 1 Nov 2018, David Rowley wrote: I ended up going with: However, this consideration only applies when is minimal for non-partitioned tables as all commands must write WAL otherwise. I've changed this to: Rows will be frozen only if the table being loaded has been

Re: INSTALL file

2018-11-01 Thread Andreas 'ads' Scherbaum
On 01.11.18 18:41, Stephen Frost wrote: Greetings, * Andreas 'ads' Scherbaum (a...@pgug.de) wrote: On 01.11.18 07:26, Michael Paquier wrote: It includes links to the website, as well as the short version of the installation instructions. +The installation instructions are listed on the

  1   2   >