[HACKERS] [PATCH] parallel & isolated makefile for plpython

2016-10-01 Thread Pavel Raiskup
Hi, we observed issues with parallel make during RPM build in plpython, seems like the attached patch 0002 should help. Feel free to reject 0001, but comment like that would save some time to me as a "newcomer" into that Makefile. Pavel >From b8722c8fb1e3d5f752d75e4d0740d04793577185 Mon Sep 17

Re: [HACKERS] VACUUM's ancillary tasks

2016-10-01 Thread Thomas Munro
On Mon, Aug 29, 2016 at 1:26 PM, Vik Fearing wrote: > The attached two patches scratch two itches I've been having for a > while. I'm attaching them together because the second depends on the first. > > Both deal with the fact that [auto]vacuum has taken on more roles than >

Re: [HACKERS] Show hash / bitmap sizes in EXPLAIN ANALYZE?

2016-10-01 Thread Tomas Vondra
On 10/01/2016 01:37 AM, Andres Freund wrote: Hi, At the moment in-memory sort and hash nodes show their memory usage in explain: │ -> Sort (cost=59.83..62.33 rows=1000 width=4) (actual time=0.512..0.632 rows=1000 loops=1)│ │ Sort Key: a.a

Re: [HACKERS] [GENERAL] pg_upgrade from 9.5 to 9.6 fails with "invalid argument"

2016-10-01 Thread Tom Lane
Andrew Dunstan writes: > On 09/30/2016 12:24 PM, Tom Lane wrote: >> Seems to be some additional prep work needed somewhere ... >> No upgrade_install_root at >> /home/bfarm/bf-scripts/build-farm-4.17/PGBuild/Modules/TestUpgradeXversion.pm >> line 51. > Oh sorry, you also

Re: [HACKERS] Macro customizable hashtable / bitmapscan & aggregation perf

2016-10-01 Thread Greg Stark
On Sat, Oct 1, 2016 at 1:44 AM, Andres Freund wrote: > > Unfortunately I'm running out battery right now, so I don't want to > re-run the benchmarks posted upthread (loading takes a while). But the > last time I ran them all the results after the patches were better than >

Re: [HACKERS] On conflict update & hint bits

2016-10-01 Thread Peter Geoghegan
On Fri, Sep 30, 2016 at 5:33 PM, Konstantin Knizhnik wrote: > So the question is whether it is correct that ExecOnConflictUpdate tries to > access and update tuple without holding lock on the buffer? You're right -- this is a bug in Postgres. I'm travelling from

Re: [HACKERS] PoC: Make it possible to disallow WHERE-less UPDATE and DELETE

2016-10-01 Thread Michael Paquier
On Sat, Oct 1, 2016 at 5:08 AM, Thomas Munro wrote: > Right. These cases work because they show up as CMD_DELETE/CMD_UPDATE: > > postgres=# set require_where.delete = on; > SET > postgres=# with answer as (select 42) delete from foo; > ERROR: DELETE requires a

Re: [HACKERS] COPY command with RLS bug

2016-10-01 Thread Michael Paquier
On Sat, Oct 1, 2016 at 3:11 AM, Stephen Frost wrote: > Comments and testing welcome, of course, though it's looking pretty good > to me at this point and I'll likely commit it in another day or two > unless issues are found. +* nodes/value.h) that

Re: [HACKERS] PoC: Make it possible to disallow WHERE-less UPDATE and DELETE

2016-10-01 Thread Michael Paquier
On Sat, Oct 1, 2016 at 5:08 AM, Thomas Munro wrote: > I guess you need something involving query_tree_walker or some other > kind of recursive traversal if you want to find DELETE/UPDATE lurking > in there. > > One option would be to document it as working for top

Re: [HACKERS] Macro customizable hashtable / bitmapscan & aggregation perf

2016-10-01 Thread Andres Freund
Hi, On 2016-10-02 01:37:35 +0200, Tomas Vondra wrote: > On 10/01/2016 09:59 PM, Andres Freund wrote: > >>What about running a bigger benchmark - say, TPC-DS - and evaluating > >>the impact? > > > >Worthwhile, although obviously the impact will be a lot smaller, > >since they're not entirely

Re: [HACKERS] pgbench more operators & functions

2016-10-01 Thread Fabien COELHO
Hello Stephen, bitwise: <<, >>, &, |, ^/#, ~ comparisons: =/==, <>/!=, <, <=, >, >= logical: and/&&, or/||, xor/^^, not, ! I'm not sure that we want to introduce operators '&&', '||' as logical 'and' and 'or' when those have specific meaning in PG which is different (array overlaps and

Re: [HACKERS] [PATCH] parallel & isolated makefile for plpython

2016-10-01 Thread Tom Lane
Pavel Raiskup writes: > Hi, we observed issues with parallel make during RPM build in plpython, > seems like the attached patch 0002 should help. Feel free to reject 0001, > but comment like that would save some time to me as a "newcomer" into that > Makefile. Hi Pavel! >

Re: [HACKERS] PATCH: two slab-like memory allocators

2016-10-01 Thread Tomas Vondra
On 10/02/2016 01:53 AM, Jim Nasby wrote: On 9/26/16 9:10 PM, Tomas Vondra wrote: Attached is v2 of the patch, updated based on the review. That means: +/* make sure the block can store at least one chunk (with 1B for a bitmap)? */ (and the comment below it) I find the question to be

Re: [HACKERS] PATCH: two slab-like memory allocators

2016-10-01 Thread Tomas Vondra
On 10/02/2016 12:23 AM, John Gorman wrote: I reproduced the quadradic pfree performance problem and verified that these patches solved it. The slab.c data structures and functions contain no quadradic components. I noticed the sizing loop in SlabContextCreate() and came up with a similar

Re: [HACKERS] PATCH: two slab-like memory allocators

2016-10-01 Thread Tomas Vondra
On 10/02/2016 12:23 AM, John Gorman wrote: I reproduced the quadradic pfree performance problem and verified that these patches solved it. The slab.c data structures and functions contain no quadradic components. I noticed the sizing loop in SlabContextCreate() and came up with a similar

Re: [HACKERS] Macro customizable hashtable / bitmapscan & aggregation perf

2016-10-01 Thread Tomas Vondra
On 10/02/2016 02:17 AM, Andres Freund wrote: Hi, > ... I think a crucial part of the benchmarking will be identifying and measuring corner cases, e.g. a lot of rows with the same key, etc. Although that probably is not a major issue for the two places switched to the new implementation (e.g.

Re: [HACKERS] [PATCH] parallel & isolated makefile for plpython

2016-10-01 Thread Pavel Raiskup
Hi Tom, On Saturday, October 1, 2016 12:23:03 PM CEST Tom Lane wrote: > Hm, actually that's unnecessary because Makefile.global already > established 'all' as the default target. I'm inclined to think > that the comment in Makefile.shlib is wrong and should be removed > or at least rewritten,

Re: [HACKERS] Macro customizable hashtable / bitmapscan & aggregation perf

2016-10-01 Thread Andres Freund
Hi, On 2016-10-01 20:19:21 +0200, Tomas Vondra wrote: > On 10/01/2016 02:44 AM, Andres Freund wrote: > > Hi, > > > > On 2016-07-26 17:43:33 -0700, Andres Freund wrote: > > > In the attached patch I've attached simplehash.h, which can be > > > customized by a bunch of macros, before being

Re: [HACKERS] Macro customizable hashtable / bitmapscan & aggregation perf

2016-10-01 Thread Andres Freund
On 2016-10-01 20:04:18 +0100, Greg Stark wrote: > On Sat, Oct 1, 2016 at 1:44 AM, Andres Freund wrote: > > > > Unfortunately I'm running out battery right now, so I don't want to > > re-run the benchmarks posted upthread (loading takes a while). But the > > last time I ran

Re: [HACKERS] VACUUM's ancillary tasks

2016-10-01 Thread Vik Fearing
On 10/01/2016 09:28 AM, Thomas Munro wrote: > On Mon, Aug 29, 2016 at 1:26 PM, Vik Fearing wrote: >> The attached two patches scratch two itches I've been having for a >> while. I'm attaching them together because the second depends on the first. >> >> Both deal with the

[HACKERS] Contains and is contained by operators of inet datatypes

2016-10-01 Thread Emre Hasegeli
Attached patch adds <@, @>, <<@, and @>> operator symbols for inet datatype to replace <<=, >>=, <<, and >>. <@ and @> symbols are used for containment for all datatypes except inet, particularly on the geometric types, arrays; cube, hstore, intaray, ltree extensions. <@ and @> symbols are

Re: [HACKERS] VACUUM's ancillary tasks

2016-10-01 Thread Thomas Munro
On Sun, Oct 2, 2016 at 9:34 AM, Vik Fearing wrote: > On 10/01/2016 09:28 AM, Thomas Munro wrote: >> On Mon, Aug 29, 2016 at 1:26 PM, Vik Fearing wrote: >>> The attached two patches scratch two itches I've been having for a >>> while. I'm attaching them

Re: [HACKERS] Hash Indexes

2016-10-01 Thread ktm
Andres Freund : On 2016-09-30 17:39:04 +0100, Peter Geoghegan wrote: On Fri, Sep 30, 2016 at 4:46 PM, Robert Haas wrote: > I would just be very disappointed if, after the amount of work that > Amit and others have put into this project, the code gets

Re: [HACKERS] Macro customizable hashtable / bitmapscan & aggregation perf

2016-10-01 Thread Tomas Vondra
On 10/01/2016 02:44 AM, Andres Freund wrote: Hi, On 2016-07-26 17:43:33 -0700, Andres Freund wrote: In the attached patch I've attached simplehash.h, which can be customized by a bunch of macros, before being inlined. There's also a patch using this for tidbitmap.c and nodeAgg/nodeSubplan/...

Re: [HACKERS] Hash Indexes

2016-10-01 Thread Greg Stark
On Fri, Sep 30, 2016 at 2:11 AM, Robert Haas wrote: > For one thing, we can stop shipping a totally broken feature in release after > release For what it's worth I'm for any patch that can accomplish that. In retrospect I think we should have done the hash-over-btree

Re: [HACKERS] PATCH: two slab-like memory allocators

2016-10-01 Thread John Gorman
I reproduced the quadradic pfree performance problem and verified that these patches solved it. The slab.c data structures and functions contain no quadradic components. I noticed the sizing loop in SlabContextCreate() and came up with a similar formula to determine chunksPerBlock that you

Re: [HACKERS] Macro customizable hashtable / bitmapscan & aggregation perf

2016-10-01 Thread Tomas Vondra
On 10/01/2016 09:59 PM, Andres Freund wrote: Hi, On 2016-10-01 20:19:21 +0200, Tomas Vondra wrote: On 10/01/2016 02:44 AM, Andres Freund wrote: Hi, On 2016-07-26 17:43:33 -0700, Andres Freund wrote: In the attached patch I've attached simplehash.h, which can be customized by a bunch of

Re: [HACKERS] PATCH: two slab-like memory allocators

2016-10-01 Thread Jim Nasby
On 9/26/16 9:10 PM, Tomas Vondra wrote: Attached is v2 of the patch, updated based on the review. That means: + /* make sure the block can store at least one chunk (with 1B for a bitmap)? */ (and the comment below it) I find the question to be confusing... I think these would be better as