Re: [HACKERS] Explanation for bug #13908: hash joins are badly broken

Tomas Vondra writes: > I believe the attached patch should fix this by actually copying the > tuples into the densely allocated chunks. Haven't tested it though, will > do in a few hours. BTW, I confirmed that this patch fixes the wrong-number-of-output-tuples

Re: [HACKERS] Explanation for bug #13908: hash joins are badly broken

On 02/06/2016 08:39 PM, Andres Freund wrote: On 2016-02-06 20:34:07 +0100, Tomas Vondra wrote: On 02/06/2016 06:47 PM, Tom Lane wrote: * It incorporates a bespoke reimplementation of palloc into hash joins. This is not a maintainable/scalable way to go about reducing memory consumption. It

[HACKERS] Recently added typedef "string" is a horrid idea

I see that commit b47b4dbf6 added this to varlena.c: typedef struct varlena string; This is a remarkably bad idea. It will cause pgindent to do strange things anywhere it sees a variable or field named "string", of which we have quite a few. Remember that the effects of typedef names

Re: [HACKERS] Explanation for bug #13908: hash joins are badly broken

Tomas Vondra writes: > On 02/06/2016 08:39 PM, Andres Freund wrote: >> FWIW, I've done that at some point. Noticeable speedups (that's what >> I cared about), but a bit annoying to use. There's many random >> pfree()s around, and then there's MemoryContextContains(),

Re: [HACKERS] Explanation for bug #13908: hash joins are badly broken

Tomas Vondra writes: > On 02/06/2016 06:47 PM, Tom Lane wrote: >> I note also that while the idea of ExecHashRemoveNextSkewBucket is >> to reduce memory consumed by the skew table to make it available to >> the main hash table, in point of fact it's unlikely that the

Re: [HACKERS] Explanation for bug #13908: hash joins are badly broken

On 02/06/2016 09:55 PM, Tom Lane wrote: Tomas Vondra writes: On 02/06/2016 06:47 PM, Tom Lane wrote: I note also that while the idea of ExecHashRemoveNextSkewBucket is to reduce memory consumed by the skew table to make it available to the main hash table, in

Re: [HACKERS] Optimization- Check the set of conditionals on a WHERE clause against CHECK constraints.

Shubham Barai writes: > I was searching for project ideas and found this > 1.Optimization- Check the set of conditionals on a WHERE clause against > CHECK constraints on the table being queried and remove any conditionals > which *must* be true due to the CHECK

Re: [HACKERS] Explanation for bug #13908: hash joins are badly broken

Tomas Vondra writes: > What about using the dense allocation even for the skew buckets, but not > one context for all skew buckets but one per bucket? Then when we delete > a bucket, we simply destroy the context (and free the chunks, just like > we do with the

Re: [HACKERS] Re: BUG #13685: Archiving while idle every archive_timeout with wal_level hot_standby

On Sun, Feb 7, 2016 at 2:49 AM, Andres Freund wrote: > On 2016-02-06 22:03:15 +0900, Michael Paquier wrote: >> + /* >> + * Update the progress LSN positions. At least one WAL insertion lock >> + * is already taken appropriately before doing that, and it is just

Re: [HACKERS] [Reveiw] Idle In Transaction Session Timeout, revived

On 31/01/2016 14:33, Vik Fearing wrote: > Attached is a rebased and revised version of my > idle_in_transaction_session_timeout patch from last year. > > This version does not suffer the problems the old one did where it would > jump out of SSL code thanks to Andres' patch in commit >

Re: [HACKERS] proposal: make NOTIFY list de-duplication optional

Brendan Jurd writes: > On Sat, 6 Feb 2016 at 12:50 Tom Lane wrote: >> Yeah, I agree that a GUC for this is quite unappetizing. > How would you feel about a variant for calling NOTIFY? If we decide that this ought to be user-visible, then an extra NOTIFY

Re: [HACKERS] silent data loss with ext4 / all current versions

On 2016-02-06 17:43:48 +0100, Tomas Vondra wrote: > >Still the data is here... But well. I won't insist. > > Huh? This thread started by an example how to cause loss of committed > transactions. That fits my definition of "data loss" quite well. Agreed, that view doesn't seem to make much sense.

Re: [HACKERS] Re: BUG #13685: Archiving while idle every archive_timeout with wal_level hot_standby

On 2016-02-06 22:03:15 +0900, Michael Paquier wrote: > The patch attached will apply on master, on 9.5 there is one minor > conflict. For older versions we will need another reworked patch. FWIW, I don't think we should backpatch this. It'd look noticeably different in the back branches, and this

Re: [HACKERS] Explanation for bug #13908: hash joins are badly broken

Tomas Vondra writes: > On 02/06/2016 02:34 AM, Tom Lane wrote: >> I have sussed what's happening in bug #13908. Basically, commit >> 45f6240a8fa9d355 ("Pack tuples in a hash join batch densely, to save >> memory") broke things for the case where a hash join is using

Re: [HACKERS] Explanation for bug #13908: hash joins are badly broken

On 02/06/2016 06:47 PM, Tom Lane wrote: Tomas Vondra writes: On 02/06/2016 02:34 AM, Tom Lane wrote: I have sussed what's happening in bug #13908. Basically, commit 45f6240a8fa9d355 ("Pack tuples in a hash join batch densely, to save memory") broke things for the

Re: [HACKERS] silent data loss with ext4 / all current versions

Hi, On 02/06/2016 01:16 PM, Michael Paquier wrote: On Sat, Feb 6, 2016 at 2:11 AM, Tomas Vondra wrote: On 02/04/2016 09:59 AM, Michael Paquier wrote: On Tue, Feb 2, 2016 at 9:59 AM, Andres Freund wrote: On 2016-02-02 09:56:40 +0900,

Re: [HACKERS] proposal: make NOTIFY list de-duplication optional

On 02/05/2016 08:49 PM, Tom Lane wrote: Yeah, I agree that a GUC for this is quite unappetizing. Agreed. One idea would be to build a hashtable to aid with duplicate detection (perhaps only once the pending-notify list gets long). Another thought is that it's already the case that

[HACKERS] Optimization- Check the set of conditionals on a WHERE clause against CHECK constraints.

Hello hackers, I was searching for project ideas and found this 1.Optimization- Check the set of conditionals on a WHERE clause against CHECK constraints on the table being queried and remove any conditionals which *must* be true due to the CHECK constraints. Is it expensive for simple

Re: [HACKERS] Explanation for bug #13908: hash joins are badly broken

On 2016-02-06 20:34:07 +0100, Tomas Vondra wrote: > On 02/06/2016 06:47 PM, Tom Lane wrote: > >* It incorporates a bespoke reimplementation of palloc into hash > >joins. This is not a maintainable/scalable way to go about reducing > >memory consumption. It should have been done with an

Re: [HACKERS] proposal: make NOTIFY list de-duplication optional

+1 ... and a patch (only adding ALL keyword, no hash table implemented yet). On Sat, Feb 6, 2016 at 2:35 PM, Brendan Jurd wrote: > On Sat, 6 Feb 2016 at 12:50 Tom Lane wrote: >> >> Robert Haas writes: >> > I agree with what

Re: [HACKERS] Recently added typedef "string" is a horrid idea

On Sat, Feb 6, 2016 at 5:11 PM, Tom Lane wrote: > I see that commit b47b4dbf6 added this to varlena.c: > > typedef struct varlena string; > > This is a remarkably bad idea. It will cause pgindent to do strange > things anywhere it sees a variable or field named

Re: [HACKERS] Patch: fix lock contention for HASHHDR.mutex

On Thu, Jan 21, 2016 at 9:03 AM, Anastasia Lubennikova wrote: > First of all, why not merge both patches into one? They aren't too big > anyway. So looking over this patch, it institutes a new coding rule that all shared hash tables must have the same number of

Re: [HACKERS] "using previous checkpoint record at" maybe not the greatest idea?

On Tue, Feb 2, 2016 at 5:28 AM, Andres Freund wrote: > > Hi, > > currently if, when not in standby mode, we can't read a checkpoint > record, we automatically fall back to the previous checkpoint, and start > replay from there. > > Doing so without user intervention doesn't

Re: [HACKERS] postgres_fdw join pushdown (was Re: Custom/Foreign-Join-APIs)

On Sat, Feb 6, 2016 at 12:46 AM, Ashutosh Bapat wrote: > Here it is rebased. Thanks for the pgindent run and committing core changes. > I have to manage only one patch now :) > > pgindent is giving trouble with following two comments > > 2213 /* Run

Re: [HACKERS] "using previous checkpoint record at" maybe not the greatest idea?

On Sun, Feb 7, 2016 at 10:54 AM, Amit Kapila wrote: > > On Tue, Feb 2, 2016 at 5:28 AM, Andres Freund wrote: > > > > Hi, > > > > currently if, when not in standby mode, we can't read a checkpoint > > record, we automatically fall back to the previous

Re: [HACKERS] pgbench small bug fix

On Wed, Jan 27, 2016 at 2:31 PM, Fabien COELHO wrote: > > While testing for something else I encountered two small bugs under very low > rate (--rate=0.1). The attached patches fixes these. > > - when a duration (-T) is specified, ensure that pgbench ends at that >time

Re: [HACKERS] WIP: Make timestamptz_out less slow.

David Rowley writes: [ timestamp_out_speedup_2015-11-05.patch ] Pushed with a bunch of cosmetic tweaks. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription:

Re: CustomScan in a larger structure (RE: [HACKERS] CustomScan support on readfuncs.c)

On Wed, Feb 3, 2016 at 11:57 PM, Kouhei Kaigai wrote: > At this moment, I tried to write up description at nodes/nodes.h. > The amount of description is about 100lines. It is on a borderline > whether we split off this chunk into another header file, in my sense. > > > On

Re: [HACKERS] Explanation for bug #13908: hash joins are badly broken

On 02/06/2016 02:34 AM, Tom Lane wrote: I have sussed what's happening in bug #13908. Basically, commit 45f6240a8fa9d355 ("Pack tuples in a hash join batch densely, to save memory") broke things for the case where a hash join is using a skew table. The reason is that that commit only changed the

Re: [HACKERS] silent data loss with ext4 / all current versions

On Sat, Feb 6, 2016 at 2:11 AM, Tomas Vondra wrote: > On 02/04/2016 09:59 AM, Michael Paquier wrote: >> >> On Tue, Feb 2, 2016 at 9:59 AM, Andres Freund wrote: >>> >>> On 2016-02-02 09:56:40 +0900, Michael Paquier wrote: And there is no

Re: [HACKERS] Re: BUG #13685: Archiving while idle every archive_timeout with wal_level hot_standby

On Thu, Feb 4, 2016 at 6:38 PM, Michael Paquier wrote: > On Thu, Feb 4, 2016 at 4:10 PM, Andres Freund wrote: >> On 2016-02-04 18:21:41 +0530, Amit Kapila wrote: >>> I think generally it is good idea, but one thing worth a thought is that >>> by

Re: [HACKERS] proposal: make NOTIFY list de-duplication optional

On Sat, 6 Feb 2016 at 12:50 Tom Lane wrote: > Robert Haas writes: > > I agree with what Merlin said about this: > > > http://www.postgresql.org/message-id/CAHyXU0yoHe8Qc=yc10ahu1nfia1tbhsg+35ds-oeueuapo7...@mail.gmail.com > > Yeah, I agree that a GUC