Re: [HACKERS] Transactions involving multiple postgres foreign servers

2015-01-11 Thread Michael Paquier
On Sun, Jan 11, 2015 at 10:37 AM, Jim Nasby wrote: > On 1/10/15, 7:11 AM, Michael Paquier wrote: >>> >>> If we had an independent transaction coordinator then I agree with you >>> >Kevin. I think Robert is proposing that if we are controlling one of the >>> >nodes that's participating as well as c

Re: [HACKERS] Parallel Seq Scan

2015-01-11 Thread Stephen Frost
* Robert Haas (robertmh...@gmail.com) wrote: > On Thu, Jan 8, 2015 at 2:46 PM, Stephen Frost wrote: > > Yeah, if we come up with a plan for X workers and end up not being able > > to spawn that many then I could see that being worth a warning or notice > > or something. Not sure what EXPLAIN has

Re: [HACKERS] Parallel Seq Scan

2015-01-11 Thread Stephen Frost
* Robert Haas (robertmh...@gmail.com) wrote: > On Fri, Jan 9, 2015 at 12:24 PM, Stephen Frost wrote: > > Yeah, we also need to consider the i/o side of this, which will > > definitely be tricky. There are i/o systems out there which are faster > > than a single CPU and ones where a single CPU can

Re: [HACKERS] Parallel Seq Scan

2015-01-11 Thread Stephen Frost
Amit, * Amit Kapila (amit.kapil...@gmail.com) wrote: > On Sun, Jan 11, 2015 at 9:09 AM, Robert Haas wrote: > > I don't think you should be "switching" queues. The tuples should be > > sent to the tuple queue, and errors and notices to the error queue. Agreed. > To achieve what you said (The tu

[HACKERS] Hash Function

2015-01-11 Thread Ravi Kiran
Hi, I want to know what kind of hash function postgres is using currently, can someone please explain the algorithm postgres is using for the hash function in the hash join algorithm. Thanks, K.Ravikiran

Re: [HACKERS] Parallel Seq Scan

2015-01-11 Thread Stefan Kaltenbrunner
On 01/11/2015 11:27 AM, Stephen Frost wrote: > * Robert Haas (robertmh...@gmail.com) wrote: >> On Thu, Jan 8, 2015 at 2:46 PM, Stephen Frost wrote: >>> Yeah, if we come up with a plan for X workers and end up not being able >>> to spawn that many then I could see that being worth a warning or noti

Re: [HACKERS] Using 128-bit integers for sum, avg and statistics aggregates

2015-01-11 Thread Tom Lane
Andreas Karlsson writes: > On 01/11/2015 02:36 AM, Andres Freund wrote: >> b) I'm doubtful that AC_CHECK_TYPES is a sufficiently good test on all >> platforms. > Should I fix it to actually compile some code which uses the 128-bit types? We used to have code in configure to test that int64 works

Re: [HACKERS] libpq 9.4 requires /etc/passwd?

2015-01-11 Thread Tom Lane
Christoph Berg writes: > Re: Tom Lane 2015-01-10 <22432.1420915...@sss.pgh.pa.us> >> So what I propose we do with this is patch HEAD and 9.4 only. >> We need to do *something* in 9.4 to address Christoph's complaint, and >> that branch is new enough that we can probably get away with changing >> o

Re: [HACKERS] libpq 9.4 requires /etc/passwd?

2015-01-11 Thread Christoph Berg
Re: Tom Lane 2015-01-11 <13609.1420998...@sss.pgh.pa.us> > > The problem isn't present in 9.3 and earlier (at least with > > postfix-pgsql), so there's no need to go back further. > > I've committed a fix for this in HEAD and 9.4. I've just tested with the HEAD libpq and the issue is fixed. Thank

Re: [HACKERS] Using 128-bit integers for sum, avg and statistics aggregates

2015-01-11 Thread Petr Jelinek
On 11/01/15 05:07, Andreas Karlsson wrote: On 01/11/2015 02:36 AM, Andres Freund wrote: @@ -3030,6 +3139,18 @@ int8_avg_accum(PG_FUNCTION_ARGS) Datum int2_accum_inv(PG_FUNCTION_ARGS) { +#ifdef HAVE_INT128 +Int16AggState *state; + +state = PG_ARGISNULL(0) ? NULL : (Int16AggState *)

Re: Custom/Foreign-Join-APIs (Re: [HACKERS] [v9.5] Custom Plan API)

2015-01-11 Thread Petr Jelinek
On 11/01/15 08:56, Kohei KaiGai wrote: 2015-01-11 10:40 GMT+09:00 Jim Nasby : Yeah there are actually several places in the code where "relid" means index in range table and not oid of relation, it still manages to confuse me. Nothing this patch can do about that. Well, since it's confused 3

Re: Custom/Foreign-Join-APIs (Re: [HACKERS] [v9.5] Custom Plan API)

2015-01-11 Thread Tom Lane
Petr Jelinek writes: > On 11/01/15 08:56, Kohei KaiGai wrote: >> 2015-01-11 10:40 GMT+09:00 Jim Nasby : Yeah there are actually several places in the code where "relid" means index in range table and not oid of relation, it still manages to confuse me. Nothing this patch can do abou

Re: [HACKERS] s_lock.h default definitions are rather confused

2015-01-11 Thread Tom Lane
Andres Freund writes: > On 2015-01-10 18:40:58 -0500, Tom Lane wrote: >> Andres Freund writes: >>> Actually. It looks like I only translated the logic from barrier.h 1:1 >>> and it already was broken there. Hm, it looks like the current code >>> essentially is from 89779bf2c8f9aa480e0ceb8883f93e9

Re: [HACKERS] Escaping from blocked send() reprised.

2015-01-11 Thread Noah Misch
On Sat, Jan 10, 2015 at 03:25:42AM +0100, Andres Freund wrote: > 0001-Allow-latches-to-wait-for-socket-writability-without.patch > Imo pretty close to commit and can be committed independently. The key open question is whether all platforms of interest can reliably detect end-of-file when pol

Re: [HACKERS] Escaping from blocked send() reprised.

2015-01-11 Thread Robert Haas
On Sat, Jan 10, 2015 at 11:35 AM, Andres Freund wrote: > Interesting. I dimly remembered you mentioning this, that's how I > rediscovered this message. > > Do you remember any details? No, not really. > My guess that's not so much the overhead of the latch itself, but the > lack of the directed

Re: [HACKERS] Parallel Seq Scan

2015-01-11 Thread Robert Haas
On Sat, Jan 10, 2015 at 11:14 PM, Amit Kapila wrote: >> I don't think you should be "switching" queues. The tuples should be >> sent to the tuple queue, and errors and notices to the error queue. > To achieve what you said (The tuples should be sent to the tuple > queue, and errors and notices to

Re: [HACKERS] Parallel Seq Scan

2015-01-11 Thread Robert Haas
On Sun, Jan 11, 2015 at 5:27 AM, Stephen Frost wrote: > * Robert Haas (robertmh...@gmail.com) wrote: >> On Thu, Jan 8, 2015 at 2:46 PM, Stephen Frost wrote: >> > Yeah, if we come up with a plan for X workers and end up not being able >> > to spawn that many then I could see that being worth a war

Re: [HACKERS] Parallel Seq Scan

2015-01-11 Thread Robert Haas
On Sun, Jan 11, 2015 at 6:01 AM, Stephen Frost wrote: > So, for my 2c, I've long expected us to parallelize at the relation-file > level for these kinds of operations. This goes back to my other > thoughts on how we should be thinking about parallelizing inbound data > for bulk data loads but it

Re: [HACKERS] Parallel Seq Scan

2015-01-11 Thread Robert Haas
On Sun, Jan 11, 2015 at 6:09 AM, Stephen Frost wrote: > I think what Robert's getting at here is that pq_redirect_to_shm_mq() > might be fine for the normal data heading back, but we need something > separate for errors and notices. Switching everything back and forth > between the normal and err

Re: [HACKERS] Escaping from blocked send() reprised.

2015-01-11 Thread Andres Freund
On 2015-01-11 16:36:07 -0500, Noah Misch wrote: > On Sat, Jan 10, 2015 at 03:25:42AM +0100, Andres Freund wrote: > > 0001-Allow-latches-to-wait-for-socket-writability-without.patch > > Imo pretty close to commit and can be committed independently. > > The key open question is whether all plat

Re: [HACKERS] Escaping from blocked send() reprised.

2015-01-11 Thread Andres Freund
On 2015-01-11 16:47:53 -0500, Robert Haas wrote: > > My guess that's not so much the overhead of the latch itself, but the > > lack of the directed wakeup stuff the OS provides for semaphores. > > That's possible. > On Sat, Jan 10, 2015 at 11:35 AM, Andres Freund > wrote: > > Interesting. I dim

Re: [HACKERS] PATCH: decreasing memory needlessly consumed by array_agg

2015-01-11 Thread Ali Akbar
> > Or else we implement what you suggest below (more comments below): > > > > Thinking about the 'release' flag a bit more - maybe we could do > > this > > instead: > > > > if (release && astate->private_cxt) > > MemoryContextDelete(astate->m

Re: [HACKERS] Using 128-bit integers for sum, avg and statistics aggregates

2015-01-11 Thread Andres Freund
On 2015-01-11 05:07:13 +0100, Andreas Karlsson wrote: > On 01/11/2015 02:36 AM, Andres Freund wrote: > >a) Afaics only __int128/unsigned __int128 is defined. See > >https://gcc.gnu.org/onlinedocs/gcc/_005f_005fint128.html > > Both GCC and Clang defines both of them. Which you use seems to just

Re: [HACKERS] Escaping from blocked send() reprised.

2015-01-11 Thread Noah Misch
On Mon, Jan 12, 2015 at 12:40:50AM +0100, Andres Freund wrote: > On 2015-01-11 16:36:07 -0500, Noah Misch wrote: > > On Sat, Jan 10, 2015 at 03:25:42AM +0100, Andres Freund wrote: > > > 0001-Allow-latches-to-wait-for-socket-writability-without.patch > > > Imo pretty close to commit and can be

Re: [HACKERS] Escaping from blocked send() reprised.

2015-01-11 Thread Andres Freund
On 2015-01-11 19:37:53 -0500, Noah Misch wrote: > I recommend either (a) taking no action or (b) adding a regression test > verifying WaitLatchOrSocket() conformance in this scenario. Do you have a good idea how to test b) save a C function in regress.c that does what your test does using latches?

Re: [HACKERS] Escaping from blocked send() reprised.

2015-01-11 Thread Noah Misch
On Mon, Jan 12, 2015 at 01:45:41AM +0100, Andres Freund wrote: > On 2015-01-11 19:37:53 -0500, Noah Misch wrote: > > I recommend either (a) taking no action or (b) adding a regression test > > verifying WaitLatchOrSocket() conformance in this scenario. > > Do you have a good idea how to test b) sa

Re: [HACKERS] Removing INNER JOINs

2015-01-11 Thread Robert Haas
On Thu, Jan 8, 2015 at 6:31 AM, David Rowley wrote: > I'd be keen to know what people's thoughts are about the nodeAlternativePlan > only surviving until the plan is initialised. I find it scary, although sometimes I am easily frightened. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com

Re: [HACKERS] Parallel Seq Scan

2015-01-11 Thread Amit Kapila
On Mon, Jan 12, 2015 at 3:30 AM, Robert Haas wrote: > > On Sun, Jan 11, 2015 at 6:01 AM, Stephen Frost wrote: > >> So, if the workers have been started but aren't keeping up, the master > >> should do nothing until they produce tuples rather than participating? > >> That doesn't seem right. > >

Re: [HACKERS] Parallel Seq Scan

2015-01-11 Thread Amit Kapila
On Mon, Jan 12, 2015 at 3:27 AM, Robert Haas wrote: > > On Sun, Jan 11, 2015 at 5:27 AM, Stephen Frost wrote: > > * Robert Haas (robertmh...@gmail.com) wrote: > >> On Thu, Jan 8, 2015 at 2:46 PM, Stephen Frost wrote: > >> > Yeah, if we come up with a plan for X workers and end up not being able