Re: libpq compression

2018-06-05 Thread Thomas Munro
On Thu, May 17, 2018 at 3:54 AM, Konstantin Knizhnik wrote: > Concerning specification of compression level: I have made many experiments > with different data sets and both zlib/zstd and in both cases using > compression level higher than default doesn't cause some noticeable increase > of

Re: commitfest 2018-07

2018-06-05 Thread Michael Paquier
On Mon, Jun 04, 2018 at 11:32:18PM -0400, Tom Lane wrote: > +1 for just renaming 2018-09 to 2018-07, if we can do that. We'll end > up postponing some entries back to -09, but that seems like less churn > than the other way. Okay. If we tend toward this direction, I propose to do this switch in

Re: Spilling hashed SetOps and aggregates to disk

2018-06-05 Thread Tomas Vondra
On 06/05/2018 09:22 AM, David Rowley wrote: On 5 June 2018 at 17:04, Tomas Vondra wrote: On 06/05/2018 04:56 AM, David Rowley wrote: Isn't there still a problem determining when the memory exhaustion actually happens though? As far as I know, we've still little knowledge how much memory

Re: Remove mention in docs that foreign keys on partitioned tables are not supported

2018-06-05 Thread Ashutosh Bapat
On Tue, Jun 5, 2018 at 1:07 PM, Amit Langote wrote: > On 2018/06/05 1:25, Alvaro Herrera wrote: >> In the meantime, my inclination is to fix the documentation to point out >> that AFTER triggers are allowed but BEFORE triggers are not. > > Wasn't that already fixed by bcded2609ade6? > > We don't

Re: Remove mention in docs that foreign keys on partitioned tables are not supported

2018-06-05 Thread Amit Langote
On 2018/06/05 1:25, Alvaro Herrera wrote: > In the meantime, my inclination is to fix the documentation to point out > that AFTER triggers are allowed but BEFORE triggers are not. Wasn't that already fixed by bcded2609ade6? We don't say anything about AFTER triggers per se, but the following

Re: SCRAM with channel binding downgrade attack

2018-06-05 Thread Michael Paquier
On Sat, Jun 02, 2018 at 01:08:56PM -0400, Heikki Linnakangas wrote: > On 28/05/18 15:08, Michael Paquier wrote: >> On Mon, May 28, 2018 at 12:26:37PM +0300, Heikki Linnakangas wrote: >> > Sounds good. >> >> Okay. Done this way as attached. If the backend forces anything else >> than SCRAM then

Re: libpq compression

2018-06-05 Thread Michael Paquier
On Tue, Jun 05, 2018 at 06:04:21PM +1200, Thomas Munro wrote: > On Thu, May 17, 2018 at 3:54 AM, Konstantin Knizhnik > Speaking of configuration, are you planning to support multiple > compression libraries at the same time? It looks like the current > patch implicitly requires client and server

Re: Spilling hashed SetOps and aggregates to disk

2018-06-05 Thread David Rowley
On 5 June 2018 at 17:04, Tomas Vondra wrote: > On 06/05/2018 04:56 AM, David Rowley wrote: >> Isn't there still a problem determining when the memory exhaustion >> actually happens though? As far as I know, we've still little >> knowledge how much memory each aggregate state occupies. >> >>

Re: Remove mention in docs that foreign keys on partitioned tables are not supported

2018-06-05 Thread Amit Langote
On 2018/06/05 16:41, Ashutosh Bapat wrote: > On Tue, Jun 5, 2018 at 1:07 PM, Amit Langote > wrote: >> On 2018/06/05 1:25, Alvaro Herrera wrote: >>> In the meantime, my inclination is to fix the documentation to point out >>> that AFTER triggers are allowed but BEFORE triggers are not. >> >>

Re: Spilling hashed SetOps and aggregates to disk

2018-06-05 Thread Tomas Vondra
On 06/05/2018 07:46 AM, Jeff Davis wrote: On Tue, 2018-06-05 at 07:04 +0200, Tomas Vondra wrote: I expect the eviction strategy to be the primary design challenge of this patch. The other bits will be mostly determined by this one piece. Not sure I agree that this is the primary challenge.

Re: Spilling hashed SetOps and aggregates to disk

2018-06-05 Thread Andres Freund
Hi, On 2018-06-06 01:06:39 +1200, David Rowley wrote: > On 6 June 2018 at 00:57, Andres Freund wrote: > > I think it's ok to only handle this gracefully if serialization is > > supported. > > > > But I think my proposal to continue use a hashtable for the already > > known groups, and sorting

Re: [HACKERS] Moving relation extension locks out of heavyweight lock manager

2018-06-05 Thread Konstantin Knizhnik
On 04.06.2018 21:42, Andres Freund wrote: Hi, On 2018-06-04 16:47:29 +0300, Konstantin Knizhnik wrote: We in PostgresProc were faced with lock extension contention problem at two more customers and tried to use this patch (v13) to address this issue. Unfortunately replacing heavy lock with

Re: [HACKERS] Moving relation extension locks out of heavyweight lock manager

2018-06-05 Thread Konstantin Knizhnik
On 05.06.2018 07:22, Masahiko Sawada wrote: On Mon, Jun 4, 2018 at 10:47 PM, Konstantin Knizhnik wrote: On 26.04.2018 09:10, Masahiko Sawada wrote: On Thu, Apr 26, 2018 at 3:30 AM, Robert Haas wrote: On Tue, Apr 10, 2018 at 9:08 PM, Masahiko Sawada wrote: Never mind. There was a lot

AtEOXact_ApplyLauncher() and subtransactions

2018-06-05 Thread Amit Khandekar
Hi, When a SUBSCRIPTION is altered, then the currently running table-synchronization workers that are no longer needed for the altered subscription, are terminated. This is done by the function AtEOXact_ApplyLauncher() inside CommitTransaction(). So during each ALTER-SUBSCRIPTION command, the

Re: Spilling hashed SetOps and aggregates to disk

2018-06-05 Thread Andres Freund
Hi, On 2018-06-05 10:05:35 +0200, Tomas Vondra wrote: > My concern is more about what happens when the input tuple ordering is > inherently incompatible with the eviction strategy, greatly increasing the > amount of data written to disk during evictions. > > Say for example that we can fit 1000

Re: [HACKERS] Moving relation extension locks out of heavyweight lock manager

2018-06-05 Thread Andres Freund
On 2018-06-05 13:09:08 +0300, Alexander Korotkov wrote: > On Tue, Jun 5, 2018 at 12:48 PM Konstantin Knizhnik > wrote: > > Workload is combination of inserts and selects. > > Looks like shared locks obtained by select cause starvation of inserts, > > trying to get exclusive relation extension

Re: [HACKERS] Transactions involving multiple postgres foreign servers

2018-06-05 Thread Masahiko Sawada
On Sat, May 26, 2018 at 12:25 AM, Robert Haas wrote: > On Fri, May 18, 2018 at 11:21 AM, Masahiko Sawada > wrote: >> Regarding to API design, should we use 2PC for a distributed >> transaction if both two or more 2PC-capable foreign servers and >> 2PC-non-capable foreign server are involved

Re: [HACKERS] Moving relation extension locks out of heavyweight lock manager

2018-06-05 Thread Masahiko Sawada
On Tue, Jun 5, 2018 at 6:47 PM, Konstantin Knizhnik wrote: > > > On 05.06.2018 07:22, Masahiko Sawada wrote: >> >> On Mon, Jun 4, 2018 at 10:47 PM, Konstantin Knizhnik >> wrote: >>> >>> >>> On 26.04.2018 09:10, Masahiko Sawada wrote: On Thu, Apr 26, 2018 at 3:30 AM, Robert Haas

Re: [HACKERS] Moving relation extension locks out of heavyweight lock manager

2018-06-05 Thread Konstantin Knizhnik
On 05.06.2018 13:29, Masahiko Sawada wrote: On Tue, Jun 5, 2018 at 6:47 PM, Konstantin Knizhnik wrote: On 05.06.2018 07:22, Masahiko Sawada wrote: On Mon, Jun 4, 2018 at 10:47 PM, Konstantin Knizhnik wrote: On 26.04.2018 09:10, Masahiko Sawada wrote: On Thu, Apr 26, 2018 at 3:30 AM,

Re: Spilling hashed SetOps and aggregates to disk

2018-06-05 Thread Andres Freund
On 2018-06-06 00:53:42 +1200, David Rowley wrote: > On 6 June 2018 at 00:45, Andres Freund wrote: > > On 2018-06-05 09:35:13 +0200, Tomas Vondra wrote: > >> I wonder if an aggregate might use a custom context > >> internally (I don't recall anything like that). The accounting capability > >>

Re: Problem while updating a foreign table pointing to a partitioned table on foreign server

2018-06-05 Thread Kyotaro HORIGUCHI
Hello. At Mon, 04 Jun 2018 20:58:28 +0900 (Tokyo Standard Time), Kyotaro HORIGUCHI wrote in <20180604.205828.208262556.horiguchi.kyot...@lab.ntt.co.jp> > It fails on some join-pushdown cases since it doesn't add tid > columns to join tlist. I suppose that build_tlist_to_deparse > needs

Re: pg_replication_slot_advance to return NULL instead of 0/0 if slot not advanced

2018-06-05 Thread Petr Jelinek
On 05/06/18 06:28, Michael Paquier wrote: > On Mon, Jun 04, 2018 at 11:51:35AM +0200, Petr Jelinek wrote: >> On 01/06/18 21:13, Michael Paquier wrote: >>> -startlsn =3D MyReplicationSlot->data.confirmed_flush; >>> +if (OidIsValid(MyReplicationSlot->data.database)) >>> +startlsn

Re: Spilling hashed SetOps and aggregates to disk

2018-06-05 Thread David Rowley
On 6 June 2018 at 00:45, Andres Freund wrote: > On 2018-06-05 09:35:13 +0200, Tomas Vondra wrote: >> I wonder if an aggregate might use a custom context >> internally (I don't recall anything like that). The accounting capability >> seems potentially useful for other places, and those might not

Re: commitfest 2018-07

2018-06-05 Thread Stephen Frost
Greetings, * Ashutosh Bapat (ashutosh.ba...@enterprisedb.com) wrote: > On Tue, Jun 5, 2018 at 9:02 AM, Tom Lane wrote: > > Michael Paquier writes: > >> On Mon, Jun 04, 2018 at 07:16:33PM -0400, Peter Eisentraut wrote: > >>> There were some discussions about renaming the existing 2018-09 entry >

Re: Spilling hashed SetOps and aggregates to disk

2018-06-05 Thread David Rowley
On 6 June 2018 at 00:57, Andres Freund wrote: > On 2018-06-06 00:53:42 +1200, David Rowley wrote: >> On 6 June 2018 at 00:45, Andres Freund wrote: >> > On 2018-06-05 09:35:13 +0200, Tomas Vondra wrote: >> >> I wonder if an aggregate might use a custom context >> >> internally (I don't recall

Re: Spilling hashed SetOps and aggregates to disk

2018-06-05 Thread David Rowley
On 6 June 2018 at 01:09, Andres Freund wrote: > On 2018-06-06 01:06:39 +1200, David Rowley wrote: >> My concern is that only accounting memory for the group and not the >> state is only solving half the problem. It might be fine for >> aggregates that don't stray far from their aggtransspace, but

Re: Spilling hashed SetOps and aggregates to disk

2018-06-05 Thread Tomas Vondra
On 06/05/2018 02:49 PM, Andres Freund wrote: Hi, On 2018-06-05 10:05:35 +0200, Tomas Vondra wrote: My concern is more about what happens when the input tuple ordering is inherently incompatible with the eviction strategy, greatly increasing the amount of data written to disk during evictions.

Re: why partition pruning doesn't work?

2018-06-05 Thread Amit Langote
Hi Dmitry, Thanks for creating the patch. I looked at it and have some comments. On 2018/06/04 22:30, Dmitry Dolgov wrote: >> On 3 June 2018 at 19:11, Tom Lane wrote: >> Dmitry Dolgov <9erthali...@gmail.com> writes: >>> Just to clarify for myself, for evaluating any stable function here would

Re: \d t: ERROR: XX000: cache lookup failed for relation

2018-06-05 Thread Teodor Sigaev
Ah, I think this is the missing, essential component: CREATE INDEX ON t(right(i::text,1)) WHERE i::text LIKE '%1'; Finally, I reproduce it with attached script. INSERT 0 99 <- first insertion ERROR: cache lookup failed for relation 1032219 ALTER TABLE ERROR: cache lookup failed for

Re: commitfest 2018-07

2018-06-05 Thread Andres Freund
On 2018-06-04 23:32:18 -0400, Tom Lane wrote: > Michael Paquier writes: > > On Mon, Jun 04, 2018 at 07:16:33PM -0400, Peter Eisentraut wrote: > >> There were some discussions about renaming the existing 2018-09 entry > >> versus inserting a new one at -07 and requiring patches to be moved back >

Re: [HACKERS] Moving relation extension locks out of heavyweight lock manager

2018-06-05 Thread Alexander Korotkov
On Tue, Jun 5, 2018 at 12:48 PM Konstantin Knizhnik wrote: > Workload is combination of inserts and selects. > Looks like shared locks obtained by select cause starvation of inserts, > trying to get exclusive relation extension lock. > The problem is fixed by fair lwlock patch, implemented by

Re: commitfest 2018-07

2018-06-05 Thread Ashutosh Bapat
On Tue, Jun 5, 2018 at 9:02 AM, Tom Lane wrote: > Michael Paquier writes: >> On Mon, Jun 04, 2018 at 07:16:33PM -0400, Peter Eisentraut wrote: >>> There were some discussions about renaming the existing 2018-09 entry >>> versus inserting a new one at -07 and requiring patches to be moved back

Re: plans for PostgreSQL 12

2018-06-05 Thread Andres Freund
Hi, On 2018-06-05 06:32:31 +0200, Pavel Stehule wrote: > ./configure --with-libxml --enable-tap-tests --enable-debug --with-perl > CFLAGS="-ggdb -Og -g3 -fno-omit-frame-pointer" > > [pavel@nemesis postgresql]$ gcc --version > gcc (GCC) 8.1.1 20180502 (Red Hat 8.1.1-1) > > I executed simple

Re: why partition pruning doesn't work?

2018-06-05 Thread Ashutosh Bapat
On Tue, Jun 5, 2018 at 6:24 PM, Dmitry Dolgov <9erthali...@gmail.com> wrote: >> On 5 June 2018 at 12:31, Amit Langote wrote: >> >> doesn't look quite right. What says expr is really a Param? The patch >> appears to work because, by setting pinfo->execparams to *something*, it >> triggers

Re: Spilling hashed SetOps and aggregates to disk

2018-06-05 Thread Tomas Vondra
On 06/05/2018 03:41 PM, se...@rielau.com wrote: In our code base we have added a WithStats-Flavor for creating memory contexts. This api accepts a pointer to metric for accounting and it is inherited by all subcontexts unless overridden. So we only needed to change context creation API where we

Re: libpq compression

2018-06-05 Thread Konstantin Knizhnik
On 05.06.2018 08:26, Thomas Munro wrote: On Thu, May 17, 2018 at 3:54 AM, Konstantin Knizhnik wrote: Thank you for this notice. Updated and rebased patch is attached. Hi Konstantin, Seems very useful. +1. + rc = inflate(>rx, Z_SYNC_FLUSH); + if (rc != Z_OK) + { + return

automating perl compile time checking

2018-06-05 Thread Andrew Dunstan
Here is a follow up patch to last weeks commit allowing all perl files to be checked clean for compile time errors and warnings. The patch contains a simple script to run the checks. The code that finds perl files is put in a function in a single file that is sourced by the three locations

Re: Code of Conduct plan

2018-06-05 Thread Chris Travers
On Mon, Jun 4, 2018 at 6:59 PM, Joshua D. Drake wrote: > On 06/03/2018 04:08 PM, Gavin Flower wrote: > > My comments: >>> >>> 1) Reiterate my contention that this is a solution is search of problem. >>> Still it looks like it is going forward, so see below. >>> >>> 2) "... engaging in behavior

Re: commitfest 2018-07

2018-06-05 Thread Joe Conway
On 06/05/2018 10:43 AM, Andres Freund wrote: > On 2018-06-05 10:20:55 -0400, Tom Lane wrote: >> Andres Freund writes: >>> I'd rather create a new 2018-07, and just manually move old patches to >>> it. Otherwise we'll not really focus on the glut of old things, but >>> everyone just restarts

Re: libpq compression

2018-06-05 Thread Konstantin Knizhnik
On 05.06.2018 10:09, Michael Paquier wrote: On Tue, Jun 05, 2018 at 06:04:21PM +1200, Thomas Munro wrote: On Thu, May 17, 2018 at 3:54 AM, Konstantin Knizhnik Speaking of configuration, are you planning to support multiple compression libraries at the same time? It looks like the current

Re: Loaded footgun open_datasync on Windows

2018-06-05 Thread Laurenz Albe
Amit Kapila wrote: > On Fri, Jun 1, 2018 at 8:18 PM, Laurenz Albe wrote: > > Amit Kapila wrote: > > > On Fri, Jun 1, 2018 at 3:13 PM, Laurenz Albe > > > wrote: > > > > I recently read our documentation about reliability on Windows: > > > > > > > > > On Windows, if wal_sync_method is

Re: commitfest 2018-07

2018-06-05 Thread Tom Lane
Michael Paquier writes: > Okay. If we tend toward this direction, I propose to do this switch in > two days my time (Thursday afternoon in Tokyo) if there are no > objections, so as anybody has hopefully time to argue back. I think we probably have to wait longer. It's not clear to me when the

RE: Re: Spilling hashed SetOps and aggregates to disk

2018-06-05 Thread serge
In our code base we have added a WithStats-Flavor for creating memory contexts. This api accepts a pointer to metric for accounting and it is inherited by all subcontexts unless overridden. So we only needed to change context creation API where we wanted (such as TopTansactionContext, Message

Re: Concurrency bug in UPDATE of partition-key

2018-06-05 Thread Amit Khandekar
Attached is a rebased patch version. Also included it in the upcoming commitfest : https://commitfest.postgresql.org/18/1660/ In the rebased version, the new test cases are added in the existing isolation/specs/partition-key-update-1.spec test. -- Thanks, -Amit Khandekar EnterpriseDB

Re: commitfest 2018-07

2018-06-05 Thread Peter Eisentraut
On 6/5/18 09:12, Andres Freund wrote: > I'd rather create a new 2018-07, and just manually move old patches to > it. My concern is whether the commitfest app will handle that well. There is no "move to previous commit fest" button. So you'd have to do it in some evil way, possibly confusing the

location reporting in TAP test failures

2018-06-05 Thread Peter Eisentraut
Right now, when a TAP test reports a failure, it looks something like this: # Failed test 'creating a replication slot' # at /../postgresql/src/bin/pg_basebackup/../../../src/test/perl/TestLib.pm line 371. That file location is where we call out to the test function provided by

Re: Spilling hashed SetOps and aggregates to disk

2018-06-05 Thread Tomas Vondra
On 06/05/2018 03:17 PM, David Rowley wrote: On 6 June 2018 at 01:09, Andres Freund wrote: On 2018-06-06 01:06:39 +1200, David Rowley wrote: My concern is that only accounting memory for the group and not the state is only solving half the problem. It might be fine for aggregates that don't

install doesn't work on HEAD

2018-06-05 Thread Amit Kapila
Hi, On my win-7, I am facing $SUBJECT. I am consistently getting below error: Copying build output files...Could not copy postgres.exe at install.pl line 26. On digging, I found that it started failing with commit 3a7cc727 (Don't fall off the end of perl functions). It seems that we have

Re: libpq compression

2018-06-05 Thread Konstantin Knizhnik
On 05.06.2018 09:04, Thomas Munro wrote: On Thu, May 17, 2018 at 3:54 AM, Konstantin Knizhnik wrote: Concerning specification of compression level: I have made many experiments with different data sets and both zlib/zstd and in both cases using compression level higher than default doesn't

Re: commitfest 2018-07

2018-06-05 Thread David Rowley
On 6 June 2018 at 03:17, Andres Freund wrote: > On 2018-06-06 03:14:58 +1200, David Rowley wrote: >> On 6 June 2018 at 02:20, Tom Lane wrote: >> > I thought the idea was to clear out the underbrush of small, ready-to-go >> > patches. How old they are doesn't enter into that. >> >> I don't

Re: why partition pruning doesn't work?

2018-06-05 Thread Pavel Stehule
2018-06-05 17:07 GMT+02:00 David Rowley : > On 5 June 2018 at 22:31, Amit Langote > wrote: > > Maybe, David (added to cc) has something to say about all this, > especially > > whether he considers this a feature and not a bug fix. > > Thanks, Amit. I had missed this thread. > > Yeah. I admit if

Re: Code of Conduct plan

2018-06-05 Thread Joshua D. Drake
On 06/05/2018 07:45 AM, Chris Travers wrote: It is my hope that PostgreSQL.Org -Core chooses members for that committee that are exceedingly diverse otherwise it is just an echo chamber for a single ideology and that will destroy this community. If I may suggest:  The committee

Re: commitfest 2018-07

2018-06-05 Thread Tom Lane
Joe Conway writes: > On 06/05/2018 10:43 AM, Andres Freund wrote: >> I think we've not fully agreed on that. I'd argue we should manually >> filter things into the next CF. And both small patches and older things >> should qualify. > Would it work to rename 2018-09 to 2018-07 and then make a

Re: commitfest 2018-07

2018-06-05 Thread Andres Freund
On 2018-06-06 03:14:58 +1200, David Rowley wrote: > On 6 June 2018 at 02:20, Tom Lane wrote: > > I thought the idea was to clear out the underbrush of small, ready-to-go > > patches. How old they are doesn't enter into that. > > I don't recall a 'fest where small ready to go patches getting >

buildfarm vs code

2018-06-05 Thread Andrew Dunstan
At my talk at pgcon last Friday [1] I presented some ideas for how people could run a full buildfarm run against their code, including a 4 line recipe using some Docker recipes I had created. Thomas Munro suggested it would be even nicer if you could just point something like Appveypr at the

Re: I'd like to discuss scaleout at PGCon

2018-06-05 Thread MauMau
From: Robert Haas On Thu, May 31, 2018 at 8:12 AM, MauMau wrote: >> Oh, I didn't know you support FDW approach mainly for analytics. I >> guessed the first target was OLTP read-write scalability. > > That seems like a harder target to me, because you will have an extra > hop involved -- SQL from

Re: buildfarm vs code

2018-06-05 Thread Andrew Dunstan
On 06/05/2018 12:09 PM, Andrew Dunstan wrote: notice at lines 36 and 34 or the Appveyor output, (from lines 16 and 19 of the recipe appveyor.yml) that this recipe does make a couple of adjustments. Of course that's lines 36 and 43. I seem to be getting more dyslectic as I get older ...

Re: commitfest 2018-07

2018-06-05 Thread Tom Lane
Andres Freund writes: > I'd rather create a new 2018-07, and just manually move old patches to > it. Otherwise we'll not really focus on the glut of old things, but > everyone just restarts working on their own new thing. I thought the idea was to clear out the underbrush of small, ready-to-go

Re: adding tab completions

2018-06-05 Thread Arthur Zakirov
On Sun, Jun 03, 2018 at 10:39:22PM -0500, Justin Pryzby wrote: > Find attached an update which also supports column completion using the legacy > non-parenthesized syntax. Thank you! > BTW..should that be toast.tuple_target ?? I think shouldn't. From the documentation "This parameter cannot be

Re: commitfest 2018-07

2018-06-05 Thread Andres Freund
On 2018-06-05 10:20:55 -0400, Tom Lane wrote: > Andres Freund writes: > > I'd rather create a new 2018-07, and just manually move old patches to > > it. Otherwise we'll not really focus on the glut of old things, but > > everyone just restarts working on their own new thing. > > I thought the

Re: [MASSMAIL]Re: Code of Conduct plan

2018-06-05 Thread gilberto . castillo
El 2018-06-05 10:54, gilberto.casti...@etecsa.cu escribió: Hello, Maybe must include policy of money support from several at member from country less earnings. El 2018-06-05 10:45, Chris Travers escribió: On Mon, Jun 4, 2018 at 6:59 PM, Joshua D. Drake wrote: On 06/03/2018 04:08 PM, Gavin

Re: commitfest 2018-07

2018-06-05 Thread Stephen Frost
Greetings, * Tom Lane (t...@sss.pgh.pa.us) wrote: > Joe Conway writes: > > On 06/05/2018 10:43 AM, Andres Freund wrote: > >> I think we've not fully agreed on that. I'd argue we should manually > >> filter things into the next CF. And both small patches and older things > >> should qualify. >

Re: commitfest 2018-07

2018-06-05 Thread Andrew Dunstan
On 06/05/2018 10:43 AM, Andres Freund wrote: On 2018-06-05 10:20:55 -0400, Tom Lane wrote: Andres Freund writes: I'd rather create a new 2018-07, and just manually move old patches to it. Otherwise we'll not really focus on the glut of old things, but everyone just restarts working on

Re: commitfest 2018-07

2018-06-05 Thread David Rowley
On 6 June 2018 at 02:20, Tom Lane wrote: > I thought the idea was to clear out the underbrush of small, ready-to-go > patches. How old they are doesn't enter into that. I don't recall a 'fest where small ready to go patches getting anything but priority. -- David Rowley

Re: Spilling hashed SetOps and aggregates to disk

2018-06-05 Thread David Fetter
On Tue, Jun 05, 2018 at 02:56:23PM +1200, David Rowley wrote: > On 5 June 2018 at 06:52, Andres Freund wrote: > > That part has gotten a bit easier since, because we have serialize / > > deserialize operations for aggregates these days. > > True. Although not all built in aggregates have those

Re: Code of Conduct plan

2018-06-05 Thread Magnus Hagander
On Tue, Jun 5, 2018 at 4:45 PM, Chris Travers wrote: > > > On Mon, Jun 4, 2018 at 6:59 PM, Joshua D. Drake > wrote: > >> On 06/03/2018 04:08 PM, Gavin Flower wrote: >> >> My comments: 1) Reiterate my contention that this is a solution is search of problem. Still it looks like it

Re: Spilling hashed SetOps and aggregates to disk

2018-06-05 Thread Andres Freund
On 2018-06-05 19:04:11 +0200, David Fetter wrote: > On Tue, Jun 05, 2018 at 02:56:23PM +1200, David Rowley wrote: > > On 5 June 2018 at 06:52, Andres Freund wrote: > > > That part has gotten a bit easier since, because we have serialize > > > / deserialize operations for aggregates these days. >

Re: [PATCH] Trim trailing whitespace in vim and emacs

2018-06-05 Thread Tom Lane
Alvaro Herrera writes: > On 2018-Jun-05, David Fetter wrote: >> Is there any interest in such a feature? > I'd rather have the editor warn me (highlight) such things rather than > fix them silently (I wonder if it'd cause a mess with regression .out > files for example, which I do edit on

Re: [PATCH] Trim trailing whitespace in vim and emacs

2018-06-05 Thread David Fetter
On Tue, Jun 05, 2018 at 01:28:54PM -0400, Alvaro Herrera wrote: > On 2018-Jun-05, David Fetter wrote: > Hi David > > > Here's my attempt to $subject. I've tested with vim, but I'm much less > > certain I got the EMACS code right. > > > > Is there any interest in such a feature? > > I'd rather

Re: I'd like to discuss scaleout at PGCon

2018-06-05 Thread MauMau
From: Ashutosh Bapat > In order to normalize parse trees, we need to at > least replace various OIDs in parse-tree with something that the > foreign server will understand correctly like table name on the > foreign table pointed to by local foreign table OR (schema qualified) > function names and

Re: Spilling hashed SetOps and aggregates to disk

2018-06-05 Thread David Fetter
On Tue, Jun 05, 2018 at 02:56:23PM +1200, David Rowley wrote: > On 5 June 2018 at 06:52, Andres Freund wrote: > > That part has gotten a bit easier since, because we have serialize > > / deserialize operations for aggregates these days. > > True. Although not all built in aggregates have those

Re: Remove mention in docs that foreign keys on partitioned tables are not supported

2018-06-05 Thread Alvaro Herrera
On 2018-Jun-05, Amit Langote wrote: > On 2018/06/05 16:41, Ashutosh Bapat wrote: > > On Tue, Jun 5, 2018 at 1:07 PM, Amit Langote > > wrote: > >> On 2018/06/05 1:25, Alvaro Herrera wrote: > >>> In the meantime, my inclination is to fix the documentation to point out > >>> that AFTER triggers are

Re: I'd like to discuss scaleout at PGCon

2018-06-05 Thread MauMau
From: Merlin Moncure > FWIW, Distributed analytical queries is the right market to be in. > This is the field in which I work, and this is where the action is at. > I am very, very, sure about this. My view is that many of the > existing solutions to this problem (in particular hadoop class >

Re: Variable-length FunctionCallInfoData

2018-06-05 Thread Andres Freund
Hi, On 2018-06-05 10:40:22 -0700, se...@rielau.com wrote: > Big +1 on this one. Cool. > Here is what we did. It's very crude, but minimized the amount of pain: I think I'd rather go for my approach in core though. Needlessly carrying around a bunch of pointers, and adding the necessary

Re: [PATCH] Trim trailing whitespace in vim and emacs

2018-06-05 Thread Teodor Sigaev
I once tried to have vim highlight trailing whitespace, spaces before tabs, and overlength lines (>80 chars), and while I could do each thing in isolation, I wasn't able to get it to highlight the three of them at the same time. If you have a recipe for that, I welcome it. I use FileStly

[PATCH] Trim trailing whitespace in vim and emacs

2018-06-05 Thread David Fetter
Folks, Here's my attempt to $subject. I've tested with vim, but I'm much less certain I got the EMACS code right. Is there any interest in such a feature? Best, David. --- .dir-locals.el| 3 ++- src/tools/editors/vim.samples | 1 + 2 files changed, 3 insertions(+), 1

Variable-length FunctionCallInfoData

2018-06-05 Thread Andres Freund
Hi, While prototyping codegen improvements for JITed expression evaluation, I once more hit the issue that the FunctionCallInfoData structs are really large (936 bytes), despite arguments beyond the fourth barely every being used. I think we should fix that. What I think we should do is convert

Re: Spilling hashed SetOps and aggregates to disk

2018-06-05 Thread Jeff Davis
On Tue, 2018-06-05 at 05:42 -0700, Andres Freund wrote: > That's an interesting idea, but it seems simpler to stick to > > hashing > > rather than using a combination strategy. It also seems like it > > would > > take less CPU effort. > Isn't the locality of access going to considerably better

Re: I'd like to discuss scaleout at PGCon

2018-06-05 Thread MauMau
From: Simon Riggs On 1 June 2018 at 16:56, Ashutosh Bapat wrote: >> I think partitioning + FDW provide basic infrastructure for >> distributing data, planning queries working with such data. We need >> more glue to support node management, cluster configuration. So, I >> agree with your

Re: [PATCH] Trim trailing whitespace in vim and emacs

2018-06-05 Thread Teodor Sigaev
I use FileStly plugin to vim [1]. But I slightly modify it,  see in attachment. FileStyle, sorry. -- Teodor Sigaev E-mail: teo...@sigaev.ru WWW: http://www.sigaev.ru/

Re: [PATCH] Improve geometric types

2018-06-05 Thread Emre Hasegeli
> Those underscore-prefixed names are defined in Microsoft's > [3][4]. So now I'm wondering if win32_port.h needs to > #include if (_MSC_VER < 1800). I don't have the C experience to decide the correct way. There are currently many .c files that are including float.h conditionally or

Re: I'd like to discuss scaleout at PGCon

2018-06-05 Thread MauMau
From: Ashutosh Bapat > Each node need to be confiugred and maintained. That requires efforts. > So we need to keep the number of nodes to a minimum. With a > coordinator and worker node segregation, we require at least two nodes > in a cluster and just that configuration doesn't provide much >

Re: libpq compression

2018-06-05 Thread Peter Eisentraut
On 6/5/18 03:09, Michael Paquier wrote: > I just had a quick look at this patch, lured by the smell of your latest > messages... And it seems to me that this patch needs a heavy amount of > work as presented. There are a couple of things which are not really > nice, like forcing the presentation

Re: commitfest 2018-07

2018-06-05 Thread Jonathan S. Katz
> On Jun 5, 2018, at 10:14 AM, Tom Lane wrote: > > Michael Paquier writes: >> Okay. If we tend toward this direction, I propose to do this switch in >> two days my time (Thursday afternoon in Tokyo) if there are no >> objections, so as anybody has hopefully time to argue back. > > I think

Re: [PATCH v17] GSSAPI encryption support

2018-06-05 Thread Robbie Harwood
Thomas Munro writes: > On Sat, May 26, 2018 at 6:58 AM, Robbie Harwood wrote: >> Me and the bot are having an argument. This should green Linux but I >> dunno about Windows. > > BTW if you're looking for a way to try stuff out on Windows exactly > the way cfbot does it without posting a new

Re: I'd like to discuss scaleout at PGCon

2018-06-05 Thread MauMau
From: Ashutosh Bapat > In order to normalize parse trees, we need to at > least replace various OIDs in parse-tree with something that the > foreign server will understand correctly like table name on the > foreign table pointed to by local foreign table OR (schema qualified) > function names and

Re: [PATCH] Trim trailing whitespace in vim and emacs

2018-06-05 Thread Alvaro Herrera
On 2018-Jun-05, David Fetter wrote: Hi David > Here's my attempt to $subject. I've tested with vim, but I'm much less > certain I got the EMACS code right. > > Is there any interest in such a feature? I'd rather have the editor warn me (highlight) such things rather than fix them silently (I

Re: Re: Spilling hashed SetOps and aggregates to disk

2018-06-05 Thread Jeff Davis
On Tue, 2018-06-05 at 08:39 -0700, se...@rielau.com wrote: > Strange. We don't see this overeahd and measure a lot more than just > a single metric: Let's investigate again. I wasn't able to repro the overhead on x86; Robert saw it on a POWER machine, and it was somewhat noisy. I don't think we

Re: Spilling hashed SetOps and aggregates to disk

2018-06-05 Thread Andres Freund
Hi, On 2018-06-05 10:47:49 -0700, Jeff Davis wrote: > The thing I don't like about it is that it requires running two memory- > hungry operations at once. How much of work_mem do we use for sorted > runs, and how much do we use for the hash table? Is that necessarily true? I'd assume that we'd

Re: POC: GROUP BY optimization

2018-06-05 Thread Teodor Sigaev
Thanks for the patch. This (missing) optimization popped-up repeatedly recently, and I was planning to look into it for PG12. So now I don't have to, because you've done all the hard work ;-) You are welcome. Actually one of out customers faced the problem with GROUP BY column order and

Re: Spilling hashed SetOps and aggregates to disk

2018-06-05 Thread Tom Lane
David Fetter writes: > On Tue, Jun 05, 2018 at 02:56:23PM +1200, David Rowley wrote: >> True. Although not all built in aggregates have those defined. > Just out of curiosity, which ones don't? As of > 3f85c62d9e825eedd1315d249ef1ad793ca78ed4, pg_aggregate has both of > those as NOT NULL. NOT

RE: Variable-length FunctionCallInfoData

2018-06-05 Thread serge
Big +1 on this one. Here is what we did. It's very crude, but minimized the amount of pain: It helps that the C compiler treats arrays and pointers the same. I can dig for the complete patch if you want... Cheers Serge /* * This struct is the data actually passed to an fmgr-called

Re: Code of Conduct plan

2018-06-05 Thread Stephen Frost
Greetings, * Magnus Hagander (mag...@hagander.net) wrote: > On Tue, Jun 5, 2018 at 4:45 PM, Chris Travers > wrote: > > If I may suggest: The committee should be international as well and > > include people from around the world. The last thing we want is for it to > > be dominated by people

Re: Variable-length FunctionCallInfoData

2018-06-05 Thread Peter Eisentraut
On 6/5/18 13:29, Andres Freund wrote: > Besides the change here, I think we should also go much further with the > conversion to NullableDatum. There's two main areas of change: I want > to move the execExpr.c related code so steps return data into > NullableDatums - that removes a good chunk of

Re: [PATCH] Trim trailing whitespace in vim and emacs

2018-06-05 Thread Peter Eisentraut
On 6/5/18 13:37, Tom Lane wrote: > I note that Peter E. seems to have a recipe for finding such issues, > which I suspect is grounded in some obscure git feature or other. > That might be easier to work with, since you'd only need one fix > not one per editor. I have a git alias:

Why is fncollation in FunctionCallInfoData rather than fmgr_info?

2018-06-05 Thread Andres Freund
Hi, In my understanding FunctionCallInfoData is basically per-call data, whereas FmgrInfo is information about the function. It makes some sense that ->context is in FunctionCallInfoData, after all it's used for per-row data like the trigger context. But we don't really change the collation of

Re: automating perl compile time checking

2018-06-05 Thread Andrew Dunstan
On 06/05/2018 05:14 PM, Daniel Gustafsson wrote: This comment should say “perlcheck/..” and not “pgperlcritic/.." I assume: Thanks. will fix. cheers andrew -- Andrew Dunstanhttps://www.2ndQuadrant.com PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Re: [PATCH] Trim trailing whitespace in vim and emacs

2018-06-05 Thread Andrew Gierth
> "Matthew" == Matthew Woodcraft writes: Matthew> A few days ago there was a suggestion on this list to add a Matthew> .editorconfig file specifying tab width: Matthew> https://www.postgresql.org/message-id/87efhuz9be.fsf%40news-spur.riddles.org.uk Matthew> The .editorconfig format

Re: Variable-length FunctionCallInfoData

2018-06-05 Thread Andres Freund
Hi, On 2018-06-05 15:08:33 -0400, Peter Eisentraut wrote: > On 6/5/18 13:29, Andres Freund wrote: > > Besides the change here, I think we should also go much further with the > > conversion to NullableDatum. There's two main areas of change: I want > > to move the execExpr.c related code so

Re: Periods

2018-06-05 Thread Paul A Jungwirth
On Tue, Jun 5, 2018 at 12:47 PM, Paul A Jungwirth wrote: > Also, this may not be very helpful, but I started an extension to > support temporal foreign keys here: > > https://github.com/pjungwir/time_for_keys > > It uses intervals, not periods, but maybe you can steal some ideas. Sorry for two

  1   2   >