Re: [HACKERS] Question about lazy_space_alloc() / linux over-commit

2015-03-06 Thread Noah Misch
On Sat, Mar 07, 2015 at 12:46:42AM -0500, Tom Lane wrote: > Noah Misch writes: > > On Thu, Mar 05, 2015 at 03:28:12PM -0600, Jim Nasby wrote: > >> I was thinking the simpler route of just repalloc'ing... the memcpy would > >> suck, but much less so than the extra index pass. 64M gets us 11M tuples

Re: [HACKERS] Question about lazy_space_alloc() / linux over-commit

2015-03-06 Thread Tom Lane
Noah Misch writes: > On Thu, Mar 05, 2015 at 03:28:12PM -0600, Jim Nasby wrote: >> I was thinking the simpler route of just repalloc'ing... the memcpy would >> suck, but much less so than the extra index pass. 64M gets us 11M tuples, >> which probably isn't very common. > +1. Start far below 64

Re: [HACKERS] Question about lazy_space_alloc() / linux over-commit

2015-03-06 Thread Noah Misch
On Thu, Mar 05, 2015 at 03:28:12PM -0600, Jim Nasby wrote: > On 3/4/15 9:10 AM, Robert Haas wrote: > >On Wed, Feb 25, 2015 at 5:06 PM, Jim Nasby wrote: > >>Could the large allocation[2] for the dead tuple array in lazy_space_alloc > >>cause problems with linux OOM? [1] and some other things I've r

Re: [HACKERS] CATUPDATE confusion?

2015-03-06 Thread Tom Lane
Peter Eisentraut writes: > On 12/29/14 7:16 PM, Adam Brightwell wrote: >> Given this discussion, I have attached a patch that removes CATUPDATE >> for review/discussion. > committed this version Hmm .. I'm not sure that summarily removing usecatupd from those three system views was well thought

Re: [HACKERS] CATUPDATE confusion?

2015-03-06 Thread Peter Eisentraut
On 12/29/14 7:16 PM, Adam Brightwell wrote: > Given this discussion, I have attached a patch that removes CATUPDATE > for review/discussion. committed this version -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.

Re: [HACKERS] MD5 authentication needs help

2015-03-06 Thread Stephen Frost
* Bruce Momjian (br...@momjian.us) wrote: > On Fri, Mar 6, 2015 at 12:50:14PM -0800, Josh Berkus wrote: > > On 03/06/2015 08:19 AM, Stephen Frost wrote: > > > Well, server-side, we already have that- have pgbouncer run on the > > > database server (something which I'm typically in favor of anyway)

Re: [HACKERS] MD5 authentication needs help

2015-03-06 Thread Stephen Frost
* Bruce Momjian (br...@momjian.us) wrote: > On Thu, Mar 5, 2015 at 11:15:55AM -0500, Stephen Frost wrote: > > * Bruce Momjian (br...@momjian.us) wrote: > > > On Wed, Mar 4, 2015 at 05:56:25PM -0800, Josh Berkus wrote: > > > > So, are we more worried about attackers getting a copy of pg_authid, or

Re: [HACKERS] Clamping reulst row number of joins.

2015-03-06 Thread Tom Lane
I wrote: > Hm, the problem evidently is that we get a default selectivity estimate > for the ON condition. I think a proper fix for this would involve > teaching eqjoinsel (and ideally other join selectivity functions) how > to drill down into appendrels and combine estimates for the child > relat

Re: [HACKERS] MD5 authentication needs help

2015-03-06 Thread Bruce Momjian
On Fri, Mar 6, 2015 at 12:50:14PM -0800, Josh Berkus wrote: > On 03/06/2015 08:19 AM, Stephen Frost wrote: > > * Alvaro Herrera (alvhe...@2ndquadrant.com) wrote: > >> Stephen Frost wrote: > >> Sure. I was thinking we would have some mechanism to authenticate the > >> connection as coming from a p

Re: [HACKERS] MD5 authentication needs help

2015-03-06 Thread Bruce Momjian
On Thu, Mar 5, 2015 at 11:15:55AM -0500, Stephen Frost wrote: > * Bruce Momjian (br...@momjian.us) wrote: > > On Wed, Mar 4, 2015 at 05:56:25PM -0800, Josh Berkus wrote: > > > So, are we more worried about attackers getting a copy of pg_authid, or > > > sniffing the hash on the wire? > > > > Bot

Re: [HACKERS] alter user/role CURRENT_USER

2015-03-06 Thread Alvaro Herrera
There is something odd here going on: alvherre=# alter group test add user current_user; ERROR: role "test" is a member of role "(null)" Surely (null) is not the right name to be reporting there ... Attached is a rebased patch, which also has some incomplete doc changes. With this patch applie

Re: [HACKERS] Bootstrap DATA is a pita

2015-03-06 Thread Alvaro Herrera
Robert Haas wrote: > On Wed, Mar 4, 2015 at 2:27 PM, Alvaro Herrera > wrote: > > BTW one solution to the merge problem is to have unique separators for > > each entry. For instance, instead of > Speaking from entirely too much experience, that's not nearly enough. > git only needs 3 lines of c

Re: [HACKERS] Add more tests on event triggers

2015-03-06 Thread Alvaro Herrera
Robert Haas wrote: > On Wed, Feb 25, 2015 at 10:03 AM, Fabien COELHO wrote: > >> You can add tests in src/test/modules/dummy_seclabel. > > > > Patch attached to test sec label there, in addition to the other more > > standard checks in event_trigger. > > These tests seem worthwhile to me. Pushed

[HACKERS] get_object_address support for additional object types

2015-03-06 Thread Alvaro Herrera
This is extracted from the DDL deparse series. These patches add get_object_address support for the following object types: - user mappings - default ACLs - operators and functions of operator families In each case I had to create a new value in ObjectType. These object types can not be created

Re: [HACKERS] Clamping reulst row number of joins.

2015-03-06 Thread Tom Lane
Tom Lane writes: > Stephen Frost writes: >> * Tom Lane (t...@sss.pgh.pa.us) wrote: >>> BTW, is that JOIN (VALUES(...)) thing common in applications, or did you >>> just use it to make a compact example? If it were something worth >>> optimizing, it seems like we could teach the planner to "pull

Re: [HACKERS] MD5 authentication needs help

2015-03-06 Thread Josh Berkus
On 03/06/2015 08:19 AM, Stephen Frost wrote: > * Alvaro Herrera (alvhe...@2ndquadrant.com) wrote: >> Stephen Frost wrote: >> Sure. I was thinking we would have some mechanism to authenticate the >> connection as coming from a pooler that has been previously authorized; >> something simple as a new

Re: [HACKERS] extend pgbench expressions with functions

2015-03-06 Thread Fabien COELHO
This patch extends pgbench expression with functions. Currently only one "abs" function is added. The point is rather to bootstrap the infrastructure for other functions (such as hash, random variants...) to be added later. Here is a small v2 update: - with better error messages on non exist

Re: [HACKERS] pg_upgrade and rsync

2015-03-06 Thread Bruce Momjian
On Fri, Mar 6, 2015 at 10:50:27AM +0300, Vladimir Borodin wrote: > What I have done is to update the pg_upgrade instructions to add this > required step. Updated doc patch attached. (I also added the --delete > flag to rsync.) Thanks so much for your detailed report. > > > It seem

Re: [HACKERS] Weirdly pesimistic estimates in optimizer

2015-03-06 Thread Tom Lane
I wrote: > I chewed on this for awhile and decided that there'd be no real harm in > taking identification of the unique expressions out of > create_unique_path() and doing it earlier, in initsplan.c; we'd need a > couple more fields in SpecialJoinInfo but that doesn't seem like a > problem. Howe

Re: [HACKERS] Clamping reulst row number of joins.

2015-03-06 Thread Tom Lane
Stephen Frost writes: > * Tom Lane (t...@sss.pgh.pa.us) wrote: >> BTW, is that JOIN (VALUES(...)) thing common in applications, or did you >> just use it to make a compact example? If it were something worth >> optimizing, it seems like we could teach the planner to "pull up VALUES" >> in the sam

Re: [HACKERS] MD5 authentication needs help

2015-03-06 Thread Stephen Frost
* Alvaro Herrera (alvhe...@2ndquadrant.com) wrote: > Stephen Frost wrote: > > * Alvaro Herrera (alvhe...@2ndquadrant.com) wrote: > > > Perhaps one of the requirements of a new auth method should be to allow > > > middlemen such as connection poolers. It's been over two years since I > > > had a lo

Re: [HACKERS] File based Incremental backup v8

2015-03-06 Thread Robert Haas
On Fri, Mar 6, 2015 at 9:38 AM, Gabriele Bartolini wrote: > I believe the main point is to look at a user interface point of view. > If/When we switch to a block level incremental support, this will be > completely transparent to the end user, even if we start with a file-level > approach with LSN

Re: [HACKERS] MD5 authentication needs help

2015-03-06 Thread Alvaro Herrera
Stephen Frost wrote: > Alvaro, > > * Alvaro Herrera (alvhe...@2ndquadrant.com) wrote: > > Perhaps one of the requirements of a new auth method should be to allow > > middlemen such as connection poolers. It's been over two years since I > > had a look, but IIRC pgbouncer had the very ugly requir

Re: [HACKERS] CATUPDATE confusion?

2015-03-06 Thread Adam Brightwell
All, Thanks for all the feedback and review. > I think I vote for (1). I do not like (2) because of the argument I made > > upthread that write access on system catalogs is far more dangerous than > > a naive DBA might think. (0) has some attraction but really CATUPDATE > > is one more concept

Re: [HACKERS] MD5 authentication needs help

2015-03-06 Thread Stephen Frost
Alvaro, * Alvaro Herrera (alvhe...@2ndquadrant.com) wrote: > Stephen Frost wrote: > > * Josh Berkus (j...@agliodbs.com) wrote: > > > > > 3) Using the user name for the MD5 storage salt allows the MD5 stored > > > > hash to be used on a different cluster if the user used the same > > > > password

Re: [HACKERS] Clamping reulst row number of joins.

2015-03-06 Thread Stephen Frost
* Tom Lane (t...@sss.pgh.pa.us) wrote: > BTW, is that JOIN (VALUES(...)) thing common in applications, or did you > just use it to make a compact example? If it were something worth > optimizing, it seems like we could teach the planner to "pull up VALUES" > in the same way that it flattens sub-se

Re: [HACKERS] Clamping reulst row number of joins.

2015-03-06 Thread Tom Lane
Kyotaro HORIGUCHI writes: > Hello, I had a report that a query gets wired estimated row > numbers and it makes subsequent plans wrong. > Although the detailed explanation is shown later in this mail, > the problem here was that a query like following makes the path > with apparently wrong row num

Re: [HACKERS] MD5 authentication needs help

2015-03-06 Thread Alvaro Herrera
Stephen Frost wrote: > * Josh Berkus (j...@agliodbs.com) wrote: > > > 3) Using the user name for the MD5 storage salt allows the MD5 stored > > > hash to be used on a different cluster if the user used the same > > > password. > > > > This is a feature as well as a bug. For example, pgBouncer r

Re: [HACKERS] [PATCH] Add transforms feature

2015-03-06 Thread Pavel Stehule
Hi I am checking this patch, but it is broken still Regards Pavel 2015-02-13 8:14 GMT+01:00 Michael Paquier : > > > On Mon, Dec 22, 2014 at 12:19 PM, Michael Paquier < > michael.paqu...@gmail.com> wrote: > >> On Mon, Dec 15, 2014 at 3:10 PM, Peter Eisentraut >> wrote: >> > fixed >> This pa

Re: [HACKERS] File based Incremental backup v8

2015-03-06 Thread Gabriele Bartolini
Hi Robert, 2015-03-06 3:10 GMT+11:00 Robert Haas : > But I agree with Fujii to the extent that I see little value in > committing this patch in the form proposed. Being smart enough to use > the LSN to identify changed blocks, but then sending the entirety of > every file anyway because you don'

Re: [HACKERS] Rethinking pg_dump's function sorting code

2015-03-06 Thread Tom Lane
Noah Misch writes: > Comparing argument type names sounds fine. Comparing argument type OID does > not offer enough to justify the loss of cross-cluster sort equivalence. Fair enough. > So as to stably compare f(nsp0.t) to f(nsp1.t), this should also compare the > dobj.namespace name. Ah, good

Re: [HACKERS] INSERT ... ON CONFLICT UPDATE and RLS

2015-03-06 Thread Stephen Frost
* Dean Rasheed (dean.a.rash...@gmail.com) wrote: > On 26 February 2015 at 09:50, Dean Rasheed wrote: > > On 26 February 2015 at 05:43, Stephen Frost wrote: > >> I wonder if there are some documentation updates which need to be done > >> for this also? I'm planning to look as I vauguely recall me

Re: [HACKERS] MD5 authentication needs help

2015-03-06 Thread Stephen Frost
Greg, * Greg Stark (st...@mit.edu) wrote: > Locked accounts are a terrible terrible idea. All they do is hand attackers > an easy DOS vulnerability. They're pure security theatre if your > authentication isn't vulnerable to brute force attacks and an unreliable > band-aid if they are. For starter

Re: [HACKERS] MD5 authentication needs help

2015-03-06 Thread Stephen Frost
* Albe Laurenz (laurenz.a...@wien.gv.at) wrote: > Stephen Frost wrote: > > Yes, it certainly was. I think Bruce was thinking that we could simply > > hash what goes on to disk with an additional salt that's stored, but > > that wouldn't actually work without requiring a change to the wireline > >

Re: [HACKERS] parallel mode and parallel contexts

2015-03-06 Thread Robert Haas
On Fri, Mar 6, 2015 at 7:01 AM, Amit Kapila wrote: > Today, while testing parallel_seqscan patch, I encountered one > intermittent issue (it hangs in below function) and I have question > related to below function. > > +void > +WaitForParallelWorkersToFinish(ParallelContext *pcxt) > +{ > .. > + fo

Re: [HACKERS] parallel mode and parallel contexts

2015-03-06 Thread Amit Kapila
On Sun, Feb 15, 2015 at 11:48 AM, Robert Haas wrote: > > On Fri, Feb 13, 2015 at 2:22 AM, Michael Paquier > wrote: > > On Thu, Feb 12, 2015 at 3:59 AM, Robert Haas wrote: > >> > >> We're not seeing eye to eye here yet, since I don't accept your > >> example scenario and you don't accept mine. L

Re: [HACKERS] MD5 authentication needs help

2015-03-06 Thread Greg Stark
Locked accounts are a terrible terrible idea. All they do is hand attackers an easy DOS vulnerability. They're pure security theatre if your authentication isn't vulnerable to brute force attacks and an unreliable band-aid if they are. Having dealt with mechanisms for locking accounts in other dat

Re: [HACKERS] Join push-down support for foreign tables

2015-03-06 Thread Ashutosh Bapat
Hi Kaigai-san, Hanada-san, Attached please find a patch to print the column names prefixed by the relation names. I haven't tested the patch fully. The same changes will be needed for CustomPlan node specific code. Now I am able to make sense out of the Output information postgres=# explain verbo

[HACKERS] extend pgbench expressions with functions

2015-03-06 Thread Fabien COELHO
This patch extends pgbench expression with functions. Currently only one "abs" function is added. The point is rather to bootstrap the infrastructure for other functions (such as hash, random variants...) to be added later. -- Fabien.diff --git a/contrib/pgbench/exprparse.y b/contrib/pgbench

Re: [HACKERS] MD5 authentication needs help

2015-03-06 Thread Albe Laurenz
Stephen Frost wrote: > * Tom Lane (t...@sss.pgh.pa.us) wrote: >> Bruce Momjian writes: >>> Let me update my list of possible improvements: >> >>> 1) MD5 makes users feel uneasy (though our usage is mostly safe) >> >>> 2) The per-session salt sent to the client is only 32-bits, meaning >>> that i

Re: [HACKERS] INSERT ... ON CONFLICT UPDATE and RLS

2015-03-06 Thread Dean Rasheed
On 26 February 2015 at 09:50, Dean Rasheed wrote: > On 26 February 2015 at 05:43, Stephen Frost wrote: >> I wonder if there are some documentation updates which need to be done >> for this also? I'm planning to look as I vauguely recall mentioning the >> ordering of operations somewhere along th

[HACKERS] Clamping reulst row number of joins.

2015-03-06 Thread Kyotaro HORIGUCHI
Hello, I had a report that a query gets wired estimated row numbers and it makes subsequent plans wrong. Although the detailed explanation is shown later in this mail, the problem here was that a query like following makes the path with apparently wrong row number. EXPLAIN SELECT a FROM (SELECT a

Re: [HACKERS] Rethinking pg_dump's function sorting code

2015-03-06 Thread Noah Misch
On Thu, Mar 05, 2015 at 07:28:33PM -0500, Tom Lane wrote: > In bug #12832 Marko Tiikkaja points out that commit > 7b583b20b1c95acb621c71251150beef958bb603 created a rather unnecessary > dump failure hazard, since it applies pg_get_function_identity_arguments() > to every function in the database, e

Re: [HACKERS] Rethinking pg_dump's function sorting code

2015-03-06 Thread Marko Tiikkaja
On 2015-03-06 01:28, Tom Lane wrote: In bug #12832 Marko Tiikkaja points out that commit 7b583b20b1c95acb621c71251150beef958bb603 created a rather unnecessary dump failure hazard, since it applies pg_get_function_identity_arguments() to every function in the database, even those that won't get du