Re: [HACKERS] Reduce pinning in btree indexes

2015-03-04 Thread Kyotaro HORIGUCHI
Hello, > It looks like the remaining performance regression was indeed a > result of code alignment. I found two "paranoia" assignments I had > accidentally failed to put back with the rest of the mark/restore > optimization; after that trivial change the patched version is > ever-so-slightly fas

Re: [HACKERS] Join push-down support for foreign tables

2015-03-04 Thread Etsuro Fujita
On 2015/03/03 21:34, Shigeru Hanada wrote: I rebased "join push-down" patch onto Kaigai-san's Custom/Foreign Join v6 patch. Thanks for the work, Hanada-san and KaiGai-san! Maybe I'm missing something, but did we agree to take this approach, ie, "join push-down" on top of custom join? There i

Re: [HACKERS] Optimization for updating foreign tables in Postgres FDW

2015-03-04 Thread Etsuro Fujita
On 2015/03/04 16:58, Albe Laurenz wrote: Etsuro Fujita wrote: While updating the patch, I noticed that in the previous patch, there is a bug in pushing down parameterized UPDATE/DELETE queries; generic plans for such queries fail with a can't-happen error. I fixed the bug and tried to add the r

Re: [HACKERS] add modulo (%) operator to pgbench

2015-03-04 Thread Fabien COELHO
On Mon, Mar 2, 2015 at 5:41 PM, Fabien COELHO wrote: but I'd like to have a more robust discussion about what we want the error reporting to look like rather than just sliding it into this patch. As an input, I suggest that the error reporting feature should provide a clue about where the is

Re: [HACKERS] Join push-down support for foreign tables

2015-03-04 Thread Kouhei Kaigai
> On 2015/03/03 21:34, Shigeru Hanada wrote: > > I rebased "join push-down" patch onto Kaigai-san's Custom/Foreign Join > > v6 patch. > > Thanks for the work, Hanada-san and KaiGai-san! > > Maybe I'm missing something, but did we agree to take this approach, ie, > "join push-down" on top of custo

Re: [HACKERS] Join push-down support for foreign tables

2015-03-04 Thread Etsuro Fujita
On 2015/03/04 17:31, Kouhei Kaigai wrote: On 2015/03/03 21:34, Shigeru Hanada wrote: I rebased "join push-down" patch onto Kaigai-san's Custom/Foreign Join v6 patch. Maybe I'm missing something, but did we agree to take this approach, ie, "join push-down" on top of custom join? There is a co

Re: [HACKERS] Bootstrap DATA is a pita

2015-03-04 Thread Andres Freund
On 2015-03-03 21:49:21 -0500, Robert Haas wrote: > On Sat, Feb 21, 2015 at 11:34 AM, Tom Lane wrote: > > Andres Freund writes: > >> On 2015-02-20 22:19:54 -0500, Peter Eisentraut wrote: > >>> On 2/20/15 8:46 PM, Josh Berkus wrote: > Or what about just doing CSV? > > > >>> I don't think that

Re: [HACKERS] Join push-down support for foreign tables

2015-03-04 Thread Shigeru Hanada
2015-03-04 17:00 GMT+09:00 Etsuro Fujita : > On 2015/03/03 21:34, Shigeru Hanada wrote: >> >> I rebased "join push-down" patch onto Kaigai-san's Custom/Foreign Join >> v6 patch. > > > Thanks for the work, Hanada-san and KaiGai-san! > > Maybe I'm missing something, but did we agree to take this appr

Re: [HACKERS] Performance improvement for joins where outer side is unique

2015-03-04 Thread David Rowley
On 27 February 2015 at 06:48, Tomas Vondra wrote: > On 26.2.2015 18:34, Tom Lane wrote: > > Tomas Vondra writes: > >> FWIW this apparently happens because the code only expect that > >> EquivalenceMembers only contain Var, but in this particular case that's > >> not the case - it contains Relabe

Re: [HACKERS] Performance improvement for joins where outer side is unique

2015-03-04 Thread David Rowley
On 26 February 2015 at 13:15, Tomas Vondra wrote: > > BTW, I find this coding (first cast, then check) rather strange: > > Var *var = (Var *) ecm->em_expr; > > if (!IsA(var, Var)) > continue; /* Ignore Consts */ > > It's probably harmless, but I find it confusing and I can't remem

Re: [HACKERS] Combining Aggregates

2015-03-04 Thread David Rowley
On 25 February 2015 at 08:15, Peter Eisentraut wrote: > On 2/20/15 3:32 PM, Tomas Vondra wrote: > > > Also, there are aggregate functions like array_agg() or string_agg() > > that make this impossible, just like for many custom aggregates (like > > hyperloglog for example). Again, I might not und

Re: [HACKERS] Join push-down support for foreign tables

2015-03-04 Thread Etsuro Fujita
On 2015/03/04 17:57, Shigeru Hanada wrote: 2015-03-04 17:00 GMT+09:00 Etsuro Fujita : On 2015/03/03 21:34, Shigeru Hanada wrote: I rebased "join push-down" patch onto Kaigai-san's Custom/Foreign Join v6 patch. but still the patch has an issue about joins underlying UPDATE or DELETE. Now I'm

Re: [HACKERS] Join push-down support for foreign tables

2015-03-04 Thread Shigeru Hanada
Here is v4 patch of Join push-down support for foreign tables. This patch requires Custom/Foreign join patch v7 posted by Kaigai-san. In this version I added check about query type which gives up pushing down joins when the join is a part of an underlying query of UPDATE/DELETE. As of now post

Re: [HACKERS] Combining Aggregates

2015-03-04 Thread David Rowley
On 18 February 2015 at 21:13, Kouhei Kaigai wrote: > > This patch itself looks good as an infrastructure towards > the big picture, however, we still don't reach the consensus > how combined functions are used instead of usual translation > functions. Thank you for taking the time to look at the

Re: [HACKERS] Combining Aggregates

2015-03-04 Thread David Rowley
On 21 February 2015 at 07:16, Tomas Vondra wrote: > > 2) serialize/deserialize functions > -- > > This thread mentions "parallel queries" as a use case, but that means > passing data between processes, and that requires being able to > serialize and deserialize the

Re: [HACKERS] How about to have relnamespace and relrole?

2015-03-04 Thread Jeevan Chalke
The following review has been posted through the commitfest application: make installcheck-world: tested, passed Implements feature: tested, passed Spec compliant: tested, passed Documentation:tested, passed > 1. > +#include "utils/acl.h" > > Can you pl

Re: [HACKERS] Join push-down support for foreign tables

2015-03-04 Thread Ashutosh Bapat
Hi Hanada-san, I am looking at the patch. Here are my comments In create_foreignscan_path() we have lines like - 1587 pathnode->path.param_info = get_baserel_parampathinfo(root, rel, 1588 required_outer); Now, that the same function is being used for creating foreign scan paths for joins, we s

Re: [HACKERS] pg_upgrade and rsync

2015-03-04 Thread Vladimir Borodin
> 3 марта 2015 г., в 18:01, Bruce Momjian написал(а): > > On Tue, Mar 3, 2015 at 04:55:56PM +0300, Vladimir Borodin wrote: >>OK, hmmm. Thanks for testing. It feels like you didn't have your new >>master set up for streaming replication when you ran pg_upgrade. Is >>that correct?

Re: [HACKERS] [REVIEW] Re: Compression of full-page-writes

2015-03-04 Thread Syed, Rahila
Hello, >Are there any other flag bits that we should or are planning to add into WAL >header newly, except the above two? If yes and they are required by even a >block which doesn't have an image, I will change my mind and agree to add >something like chunk ID to a block header. >But I guess t

Re: [HACKERS] NULL-pointer check and incorrect comment for pstate in addRangeTableEntry

2015-03-04 Thread Greg Stark
This sounded familiar... I pointed out the same thing a while back and Tom had some feedback on what to do about it: http://www.postgresql.org/message-id/23294.1384142...@sss.pgh.pa.us On 4 Mar 2015 00:37, "Michael Paquier" wrote: > On Wed, Mar 4, 2015 at 6:43 AM, Robert Haas wrote: > > That see

Re: [HACKERS] Install shared libs in lib/ and bin/ with MSVC (Was: install libpq.dll in bin directory on Windows / Cygwin)

2015-03-04 Thread Michael Paquier
On Wed, Mar 4, 2015 at 4:43 PM, Michael Paquier wrote: > On Tue, Mar 3, 2015 at 8:36 PM, Asif Naeem wrote: >> Thank you Michael. I have looked the patch. > > Thanks for the review! > >> Overall logic looks good to me, >> I have checked it with MSVC{2013,2008}. It works for MSVC 2013 but fail for >>

Re: [HACKERS] Providing catalog view to pg_hba.conf file - Patch submission

2015-03-04 Thread Greg Stark
On Wed, Mar 4, 2015 at 1:35 AM, Haribabu Kommi wrote: > I feel there is no problem of current pg_hba reloads, because the > check_for_interrupts > doesn't reload the conf files. It will be done in the "postgresMain" > function once the > query finishes. Am I missing something? Ah, no I guess that

Re: [HACKERS] NULL-pointer check and incorrect comment for pstate in addRangeTableEntry

2015-03-04 Thread Michael Paquier
On Wed, Mar 4, 2015 at 9:12 PM, Greg Stark wrote: > This sounded familiar... I pointed out the same thing a while back and Tom > had some feedback on what to do about it: > > http://www.postgresql.org/message-id/23294.1384142...@sss.pgh.pa.us Translated into code I guess that this gives the attac

Re: [HACKERS] POLA violation with \c service=

2015-03-04 Thread David Fetter
On Tue, Mar 03, 2015 at 09:52:55PM -0500, Robert Haas wrote: > On Mon, Mar 2, 2015 at 5:05 PM, David Fetter wrote: > > So just to clarify, are you against back-patching the behavior > > change, or the addition to src/common? > > Mostly the latter. So you're saying the former isn't a problem? To

Re: [HACKERS] Bootstrap DATA is a pita

2015-03-04 Thread Robert Haas
>> Even this promises to vastly increase the number of lines in the file, >> and make it harder to compare entries by grepping out some common >> substring. I agree that the current format is a pain in the tail, but >> pg_proc.h is >5k lines already. I don't want it to be 100k lines >> instead. >

Re: [HACKERS] Bootstrap DATA is a pita

2015-03-04 Thread Andres Freund
On 2015-03-04 08:47:44 -0500, Robert Haas wrote: > >> Even this promises to vastly increase the number of lines in the file, > >> and make it harder to compare entries by grepping out some common > >> substring. I agree that the current format is a pain in the tail, but > >> pg_proc.h is >5k lines

Re: [HACKERS] POLA violation with \c service=

2015-03-04 Thread Robert Haas
On Wed, Mar 4, 2015 at 8:33 AM, David Fetter wrote: > On Tue, Mar 03, 2015 at 09:52:55PM -0500, Robert Haas wrote: >> On Mon, Mar 2, 2015 at 5:05 PM, David Fetter wrote: >> > So just to clarify, are you against back-patching the behavior >> > change, or the addition to src/common? >> >> Mostly th

Re: [HACKERS] Bootstrap DATA is a pita

2015-03-04 Thread Peter Eisentraut
On 3/3/15 9:49 PM, Robert Haas wrote: >> Yeah. One thought though is that I don't think we need the "data" layer >> in your proposal; that is, I'd flatten the representation to something >> more like >> >> { >> oid => 2249, >> oiddefine => 'CSTRINGOID', >> typname =

Re: [HACKERS] NULL-pointer check and incorrect comment for pstate in addRangeTableEntry

2015-03-04 Thread Greg Stark
On Wed, Mar 4, 2015 at 12:38 PM, Michael Paquier wrote: > Translated into code I guess that this gives the attached patch. Probably want to update a comment somewhere but yes, those are the same three call sites I had found back then. I don't see any patch lying around so I don't think I got any

Re: [HACKERS] Bootstrap DATA is a pita

2015-03-04 Thread Tom Lane
Peter Eisentraut writes: > On 3/3/15 9:49 PM, Robert Haas wrote: >> Even this promises to vastly increase the number of lines in the file, > I think lines are cheap. Columns are much harder to deal with. Yeah. pg_proc.h is already impossible to work with in a standard 80-column window. I don'

Re: [HACKERS] Bootstrap DATA is a pita

2015-03-04 Thread Robert Haas
On Wed, Mar 4, 2015 at 9:06 AM, Peter Eisentraut wrote: >> and make it harder to compare entries by grepping out some common >> substring. > > Could you give an example of the sort of thing you wish to do? e.g. grep for a function name and check that all the matches have the same volatility. --

Re: [HACKERS] Bootstrap DATA is a pita

2015-03-04 Thread Robert Haas
On Wed, Mar 4, 2015 at 9:42 AM, Tom Lane wrote: >>> and make it harder to compare entries by grepping out some common >>> substring. > >> Could you give an example of the sort of thing you wish to do? > > On that angle, I'm dubious that a format that allows omission of fields is > going to be easy

Re: [HACKERS] improve pgbench syntax error messages

2015-03-04 Thread Robert Haas
On Tue, Mar 3, 2015 at 3:48 AM, Fabien COELHO wrote: > Report the origin of syntax errors from pgbench. > > Currently only the column number (for expressions) and command are > essentially reported: > > sh> ./pgbench -f bad.sql > syntax error at column 14 > set: parse error > > The patch hel

Re: [HACKERS] Bootstrap DATA is a pita

2015-03-04 Thread Tom Lane
Robert Haas writes: > On Wed, Mar 4, 2015 at 9:06 AM, Peter Eisentraut wrote: >> Could you give an example of the sort of thing you wish to do? > e.g. grep for a function name and check that all the matches have the > same volatility. Well, grep is not going to work too well anymore, but extrac

Re: [HACKERS] Bootstrap DATA is a pita

2015-03-04 Thread Andrew Dunstan
On 03/04/2015 09:42 AM, Tom Lane wrote: Peter Eisentraut writes: On 3/3/15 9:49 PM, Robert Haas wrote: Even this promises to vastly increase the number of lines in the file, I think lines are cheap. Columns are much harder to deal with. Yeah. pg_proc.h is already impossible to work with i

Re: [HACKERS] Bootstrap DATA is a pita

2015-03-04 Thread Stephen Frost
* Robert Haas (robertmh...@gmail.com) wrote: > On Wed, Mar 4, 2015 at 9:42 AM, Tom Lane wrote: > >>> and make it harder to compare entries by grepping out some common > >>> substring. > > > >> Could you give an example of the sort of thing you wish to do? > > > > On that angle, I'm dubious that a

Re: [HACKERS] Bootstrap DATA is a pita

2015-03-04 Thread Andrew Dunstan
On 03/04/2015 09:51 AM, Robert Haas wrote: On Wed, Mar 4, 2015 at 9:06 AM, Peter Eisentraut wrote: and make it harder to compare entries by grepping out some common substring. Could you give an example of the sort of thing you wish to do? e.g. grep for a function name and check that all the

Re: [HACKERS] Question about lazy_space_alloc() / linux over-commit

2015-03-04 Thread Robert Haas
On Wed, Feb 25, 2015 at 5:06 PM, Jim Nasby wrote: > Could the large allocation[2] for the dead tuple array in lazy_space_alloc > cause problems with linux OOM? [1] and some other things I've read indicate > that a large mmap will count towards total system memory, including > producing a failure i

Re: [HACKERS] collations in shared catalogs?

2015-03-04 Thread Robert Haas
On Wed, Feb 25, 2015 at 7:54 PM, David Steele wrote: > +1 on 128/256 character names. > >> /me runs and hides. > > /stands brazenly in the open and volunteers to try it if I don't get > clobbered within seconds. I think the question is whether making lots of rows in system catalogs better is goin

Re: [HACKERS] Bootstrap DATA is a pita

2015-03-04 Thread Andres Freund
On 2015-03-04 09:55:01 -0500, Robert Haas wrote: > On Wed, Mar 4, 2015 at 9:42 AM, Tom Lane wrote: > I wonder if we should have a tool in our repository to help people > edit the file. So instead of going in there yourself and changing > things by hand, or writing your own script, you can do: >

Re: [HACKERS] Bootstrap DATA is a pita

2015-03-04 Thread Robert Haas
On Wed, Mar 4, 2015 at 10:04 AM, Andrew Dunstan wrote: > Is it necessarily an all or nothing deal? > > Taking a previous example, we could have something like: > > { > oid => 2249, oiddefine => 'CSTRINGOID', typname => 'cstring', > typlen => -2, typbyval => 1, > ..

Re: [HACKERS] Weirdly pesimistic estimates in optimizer

2015-03-04 Thread Robert Haas
On Mon, Mar 2, 2015 at 2:19 PM, David Kubečka wrote: > The question is why optimizer, or rather the cost estimator, produced so > much different estimates upon very small change in input. Moreover it seems > that the main culprit of bad estimates isn't actually directly related to > outer table, b

Re: [HACKERS] Bootstrap DATA is a pita

2015-03-04 Thread Tom Lane
Robert Haas writes: > Another advantage of this is that it would probably make git less > likely to fumble a rebase. If there are lots of places in the file > where we have the same 10 lines in a row with occasional variations, > rebasing a patch could easily pick the the wrong place to reapply t

Re: [HACKERS] collations in shared catalogs?

2015-03-04 Thread David Steele
Hi Robert, On 3/4/15 10:14 AM, Robert Haas wrote: > On Wed, Feb 25, 2015 at 7:54 PM, David Steele wrote: >> +1 on 128/256 character names. >> >>> /me runs and hides. >> >> /stands brazenly in the open and volunteers to try it if I don't get >> clobbered within seconds. > > I think the question i

Re: [HACKERS] Strange assertion using VACOPT_FREEZE in vacuum.c

2015-03-04 Thread Robert Haas
On Sun, Mar 1, 2015 at 6:58 AM, Michael Paquier wrote: > Hm, why not. That would remove all inconsistencies between the parser > and the autovacuum code path. Perhaps something like the attached > makes sense then? I really don't see this patch, or any of the previous ones, as solving any actual

Re: [HACKERS] MD5 authentication needs help

2015-03-04 Thread Magnus Hagander
On Wed, Mar 4, 2015 at 4:52 PM, Stephen Frost wrote: > > A lot of discussion has been going on with SCRAM and SASL, which is all > great, but that means we end up with a dependency on SASL or we have to > reimplement SCRAM (which I've been thinking might not be a bad idea- > it's actually not tha

Re: [HACKERS] MD5 authentication needs help

2015-03-04 Thread Stephen Frost
Bruce, all, I've been discussing this with a few folks outside of the PG community (Debian and Openwall people specifically) and a few interesting ideas have come out of that which might be useful to discuss. The first is a "don't break anything" approach which would move the needle between "netw

Re: [HACKERS] Abbreviated keys for Numeric

2015-03-04 Thread Robert Haas
On Mon, Feb 23, 2015 at 5:59 AM, Andrew Gierth wrote: >> "Tomas" == Tomas Vondra writes: > Tomas> Interesting, but I think Gavin was asking about how much > Tomas> variation was there for each tested case (e.g. query executed on > Tomas> the same code / dataset). And in those cases the pad

Re: [HACKERS] Strange assertion using VACOPT_FREEZE in vacuum.c

2015-03-04 Thread Stephen Frost
* Robert Haas (robertmh...@gmail.com) wrote: > On Sun, Mar 1, 2015 at 6:58 AM, Michael Paquier > wrote: > > Hm, why not. That would remove all inconsistencies between the parser > > and the autovacuum code path. Perhaps something like the attached > > makes sense then? > > I really don't see this

Re: [HACKERS] MD5 authentication needs help

2015-03-04 Thread Andres Freund
Hi, On 2015-03-04 10:52:30 -0500, Stephen Frost wrote: > I've been discussing this with a few folks outside of the PG community > (Debian and Openwall people specifically) and a few interesting ideas > have come out of that which might be useful to discuss. > > The first is a "don't break anythin

Re: [HACKERS] MD5 authentication needs help

2015-03-04 Thread Stephen Frost
Magnus, * Magnus Hagander (mag...@hagander.net) wrote: > On Wed, Mar 4, 2015 at 4:52 PM, Stephen Frost wrote: > > A lot of discussion has been going on with SCRAM and SASL, which is all > > great, but that means we end up with a dependency on SASL or we have to > > reimplement SCRAM (which I've b

Re: [HACKERS] MD5 authentication needs help

2015-03-04 Thread Magnus Hagander
On Wed, Mar 4, 2015 at 5:03 PM, Stephen Frost wrote: > Magnus, > > * Magnus Hagander (mag...@hagander.net) wrote: > > On Wed, Mar 4, 2015 at 4:52 PM, Stephen Frost > wrote: > > > A lot of discussion has been going on with SCRAM and SASL, which is all > > > great, but that means we end up with a

Re: [HACKERS] MD5 authentication needs help

2015-03-04 Thread Stephen Frost
* Andres Freund (and...@2ndquadrant.com) wrote: > Hi, > > On 2015-03-04 10:52:30 -0500, Stephen Frost wrote: > > I've been discussing this with a few folks outside of the PG community > > (Debian and Openwall people specifically) and a few interesting ideas > > have come out of that which might be

Re: [HACKERS] Combining Aggregates

2015-03-04 Thread Robert Haas
On Wed, Mar 4, 2015 at 4:41 AM, David Rowley wrote: >> This thread mentions "parallel queries" as a use case, but that means >> passing data between processes, and that requires being able to >> serialize and deserialize the aggregate state somehow. For actual data >> types that's not overly diffi

Re: [HACKERS] MD5 authentication needs help

2015-03-04 Thread Andres Freund
On 2015-03-04 11:06:33 -0500, Stephen Frost wrote: > * Andres Freund (and...@2ndquadrant.com) wrote: > > On 2015-03-04 10:52:30 -0500, Stephen Frost wrote: > > > The first is a "don't break anything" approach which would move the > > > needle between "network data sensitivity" and "on-disk data sen

Re: [HACKERS] xpath changes in the recent back branches

2015-03-04 Thread Robert Haas
On Thu, Feb 19, 2015 at 5:53 AM, Marko Tiikkaja wrote: > Commit 79af9a1d2668c9edc8171f03c39e7fed571eeb98 changed xpath handling with > regard to namespaces, and it seems to be fixing an actual issue. However, it > was also backpatched to all branches despite it breaking for example code > like thi

Re: [HACKERS] xpath changes in the recent back branches

2015-03-04 Thread Tom Lane
Robert Haas writes: > On Thu, Feb 19, 2015 at 5:53 AM, Marko Tiikkaja wrote: >> I'm not sure how changing behavior like this in a minor release was >> considered acceptable. > I'm guessing that the fact that it changes behavior in cases like this > wasn't recognized, but I suppose Peter will hav

Re: [HACKERS] MD5 authentication needs help

2015-03-04 Thread Stephen Frost
* Andres Freund (and...@2ndquadrant.com) wrote: > On 2015-03-04 11:06:33 -0500, Stephen Frost wrote: > > * Andres Freund (and...@2ndquadrant.com) wrote: > > > On 2015-03-04 10:52:30 -0500, Stephen Frost wrote: > > > > The first is a "don't break anything" approach which would move the > > > > needl

Re: [HACKERS] Comparing primary/HS standby in tests

2015-03-04 Thread Jeff Janes
On Tue, Mar 3, 2015 at 7:49 AM, Andres Freund wrote: > Hi, > > I've regularly wished we had automated tests that setup HS and then > compare primary/standby at the end to verify replay worked > correctly. > > Heikki's page comparison tools deals with some of that verification, but > it's really q

Re: [HACKERS] improve pgbench syntax error messages

2015-03-04 Thread Fabien COELHO
As I mentioned on the other thread, I'd really like to get this into a better format, where each error message is on one line. Looking at that, you can't tell whether you've got one mistake, two mistakes, or three mistakes. Indeed. Here is a v2. sh> ./pgbench -f bad.sql bad.sql:3: syntax

Re: [HACKERS] xpath changes in the recent back branches

2015-03-04 Thread Mike Rylander
On Thu, Feb 19, 2015 at 5:53 AM, Marko Tiikkaja wrote: > Hi, > > Commit 79af9a1d2668c9edc8171f03c39e7fed571eeb98 changed xpath handling > with regard to namespaces, and it seems to be fixing an actual issue. > However, it was also backpatched to all branches despite it breaking for > example code

Re: [HACKERS] compress method for spgist - 2

2015-03-04 Thread Paul Ramsey
On Wed, Feb 25, 2015 at 6:13 AM, Heikki Linnakangas wrote: > In the original post on this, you mentioned that the PostGIS guys planned to > use this to store polygons, as bounding boxes > (http://www.postgresql.org/message-id/5447b3ff.2080...@sigaev.ru). Any idea > how would that work? Poorly, by

Re: [HACKERS] File based Incremental backup v8

2015-03-04 Thread Marco Nenciarini
Hi Fujii, Il 03/03/15 11:48, Fujii Masao ha scritto: > On Tue, Mar 3, 2015 at 12:36 AM, Marco Nenciarini > wrote: >> Il 02/03/15 14:21, Fujii Masao ha scritto: >>> On Thu, Feb 12, 2015 at 10:50 PM, Marco Nenciarini >>> wrote: Hi, I've attached an updated version of the patch. >>>

Re: [HACKERS] Comparing primary/HS standby in tests

2015-03-04 Thread Andres Freund
On 2015-03-04 08:41:23 -0800, Jeff Janes wrote: > Couldn't we just arbitrarily exclude sequence internal states from the > comparison? Not sure what you mean? You mean just not dump them? I guess we could by editing the contents of a custom format dump? A bit annoying to have a script doing that..

Re: [HACKERS] pg_upgrade and rsync

2015-03-04 Thread Bruce Momjian
On Wed, Mar 4, 2015 at 01:53:47PM +0300, Vladimir Borodin wrote: > After running initdb to create the new master, but before running > pg_upgrade, modify the new master's postgresql.conf and change wal_level > = hot_standby. (Don't modify pg_hba.conf at this stage.) > > > > That do

Re: [HACKERS] xpath changes in the recent back branches

2015-03-04 Thread Marko Tiikkaja
On 3/4/15 5:26 PM, Tom Lane wrote: Robert Haas writes: On Thu, Feb 19, 2015 at 5:53 AM, Marko Tiikkaja wrote: I'm not sure how changing behavior like this in a minor release was considered acceptable. I'm guessing that the fact that it changes behavior in cases like this wasn't recognized,

Re: [HACKERS] xpath changes in the recent back branches

2015-03-04 Thread Marko Tiikkaja
On 3/4/15 6:19 PM, I wrote: On 3/4/15 5:26 PM, Tom Lane wrote: It was considered to be a bug fix; more, given the few complaints about the clearly-broken old behavior, we thought it was a fix that would affect few people, and them positively. Yeah, but these things usually go the other way. "

Re: [HACKERS] MD5 authentication needs help

2015-03-04 Thread Bruce Momjian
On Wed, Mar 4, 2015 at 10:52:30AM -0500, Stephen Frost wrote: > The first is a "don't break anything" approach which would move the > needle between "network data sensitivity" and "on-disk data sensitivity" > a bit back in the direction of making the network data more sensitive. > > this approach

Re: [HACKERS] MD5 authentication needs help

2015-03-04 Thread Bruce Momjian
On Wed, Mar 4, 2015 at 11:36:23AM -0500, Stephen Frost wrote: > * Andres Freund (and...@2ndquadrant.com) wrote: > > On 2015-03-04 11:06:33 -0500, Stephen Frost wrote: > > > * Andres Freund (and...@2ndquadrant.com) wrote: > > > > On 2015-03-04 10:52:30 -0500, Stephen Frost wrote: > > > > > The firs

Re: [HACKERS] MD5 authentication needs help

2015-03-04 Thread Stephen Frost
* Bruce Momjian (br...@momjian.us) wrote: > On Wed, Mar 4, 2015 at 10:52:30AM -0500, Stephen Frost wrote: > > The first is a "don't break anything" approach which would move the > > needle between "network data sensitivity" and "on-disk data sensitivity" > > a bit back in the direction of making t

Re: [HACKERS] MD5 authentication needs help

2015-03-04 Thread Stephen Frost
* Bruce Momjian (br...@momjian.us) wrote: > On Wed, Mar 4, 2015 at 11:36:23AM -0500, Stephen Frost wrote: > > * Andres Freund (and...@2ndquadrant.com) wrote: > > > On 2015-03-04 11:06:33 -0500, Stephen Frost wrote: > > > > * Andres Freund (and...@2ndquadrant.com) wrote: > > > > > On 2015-03-04 10:

Re: [HACKERS] MD5 authentication needs help

2015-03-04 Thread Stephen Frost
* Stephen Frost (sfr...@snowman.net) wrote: > * Bruce Momjian (br...@momjian.us) wrote: > > On Wed, Mar 4, 2015 at 11:36:23AM -0500, Stephen Frost wrote: > > > * Andres Freund (and...@2ndquadrant.com) wrote: > > > > On 2015-03-04 11:06:33 -0500, Stephen Frost wrote: > > > > > * Andres Freund (and.

Re: [HACKERS] MD5 authentication needs help

2015-03-04 Thread Bruce Momjian
On Wed, Mar 4, 2015 at 12:43:54PM -0500, Stephen Frost wrote: > > What does storing multiple hash(password || stoarage_salt) values do for > > us that session_salt doesn't already do? > > By storing a hash of the result of the challenge/response, we wouldn't > be susceptible to attacks where the

Re: [HACKERS] Patch: raise default for max_wal_segments to 1GB

2015-03-04 Thread Greg Stark
On Tue, Mar 3, 2015 at 7:57 PM, Tom Lane wrote: > Now, if we were to change the server so that it *refused* settings that > didn't have a unit, that argument would become moot. But I'm not going > to defend the castle against the villagers who will show up if you do > that. That might be somethi

Re: [HACKERS] Normalizing units for postgresql.conf WAS: Patch: raise default for max_wal_segments to 1GB

2015-03-04 Thread Robert Haas
On Tue, Mar 3, 2015 at 1:42 PM, Josh Berkus wrote: >>> Sure. Although, do we take (s) for tcp_keepalives_idle? Or only an INT? >> >> It's a "time unit", so you can say "10s" or "1ms". If you don't >> specify a unit, it implies seconds. > > So if we're going to make this consistent, let's mak

Re: [HACKERS] MD5 authentication needs help

2015-03-04 Thread Stephen Frost
* Bruce Momjian (br...@momjian.us) wrote: > On Wed, Mar 4, 2015 at 12:43:54PM -0500, Stephen Frost wrote: > > > What does storing multiple hash(password || stoarage_salt) values do for > > > us that session_salt doesn't already do? > > > > By storing a hash of the result of the challenge/response

Re: [HACKERS] MD5 authentication needs help

2015-03-04 Thread Stefan Kaltenbrunner
On 03/04/2015 04:52 PM, Stephen Frost wrote: > Bruce, all, > > I've been discussing this with a few folks outside of the PG community > (Debian and Openwall people specifically) and a few interesting ideas > have come out of that which might be useful to discuss. > > The first is a "don't break a

Re: [HACKERS] MD5 authentication needs help

2015-03-04 Thread Heikki Linnakangas
On 03/04/2015 06:11 PM, Stephen Frost wrote: * Magnus Hagander (mag...@hagander.net) wrote: On Wed, Mar 4, 2015 at 5:03 PM, Stephen Frost wrote: No, I'm not suggesting that OpenSSL or TLS become mandatory but was thinking it might be good alternative as a middle-ground between full client-and-

Re: [HACKERS] failures with tuplesort and ordered set aggregates (due to 5cefbf5a6c44)

2015-03-04 Thread Peter Geoghegan
On Wed, Mar 4, 2015 at 8:26 AM, Robert Haas wrote: > I think we should commit my patch, and if a future patch needs > sortKeys set in more places, it can make that change itself. There's > no reason why it's needed with the code as it is today, and no reason > to let bits of future changes leak i

Re: [HACKERS] Reduce pinning in btree indexes

2015-03-04 Thread Kevin Grittner
Kyotaro HORIGUCHI wrote: > The performance which your test shows looks great. The gain might > comes from removing of buffer locking on _bt_next. Yeah, I had been hoping that removing some buffer pins and locks from the common path of scanning forward from one page to the next might have some di

Re: [HACKERS] MD5 authentication needs help

2015-03-04 Thread Stephen Frost
* Heikki Linnakangas (hlinn...@iki.fi) wrote: > The big difference between SRP and SCRAM is that if you eavesdrop > the SCRAM handshake, you can use that information to launch a > brute-force or dictionary attack. With SRP, you cannot do that. That > makes it relatively safe to use weak passwords w

Re: [HACKERS] MD5 authentication needs help

2015-03-04 Thread Bruce Momjian
On Wed, Mar 4, 2015 at 01:27:32PM -0500, Stephen Frost wrote: > This further makes what is sent over the network not directly > susceptible to a replay attack because the server has multiple values > available to pick for the salt to use and sends one at random to the > client, exactly how our cur

Re: [HACKERS] MD5 authentication needs help

2015-03-04 Thread Heikki Linnakangas
On 03/04/2015 08:59 PM, Stephen Frost wrote: * Heikki Linnakangas (hlinn...@iki.fi) wrote: The big difference between SRP and SCRAM is that if you eavesdrop the SCRAM handshake, you can use that information to launch a brute-force or dictionary attack. With SRP, you cannot do that. That makes it

Re: [HACKERS] MD5 authentication needs help

2015-03-04 Thread Stephen Frost
* Heikki Linnakangas (hlinn...@iki.fi) wrote: > I'm not sure how expensive a brute force attack on SRP would be, > using a stolen backup tape. There doesn't seem to be an iterations > count similar to SCRAM. But note that SRP's resistance to > brute-forcing the authentication handshake is of a diff

Re: [HACKERS] MD5 authentication needs help

2015-03-04 Thread Stephen Frost
* Bruce Momjian (br...@momjian.us) wrote: > On Wed, Mar 4, 2015 at 01:27:32PM -0500, Stephen Frost wrote: > > This further makes what is sent over the network not directly > > susceptible to a replay attack because the server has multiple values > > available to pick for the salt to use and sends

Re: [HACKERS] Bootstrap DATA is a pita

2015-03-04 Thread Alvaro Herrera
Andrew Dunstan wrote: > > On 03/04/2015 09:51 AM, Robert Haas wrote: > >On Wed, Mar 4, 2015 at 9:06 AM, Peter Eisentraut wrote: > >>>and make it harder to compare entries by grepping out some common > >>>substring. > >>Could you give an example of the sort of thing you wish to do? > >e.g. grep fo

Re: [HACKERS] MD5 authentication needs help

2015-03-04 Thread Stephen Frost
* Bruce Momjian (br...@momjian.us) wrote: > On Wed, Mar 4, 2015 at 02:21:51PM -0500, Stephen Frost wrote: > > * Bruce Momjian (br...@momjian.us) wrote: > > > On Wed, Mar 4, 2015 at 01:27:32PM -0500, Stephen Frost wrote: > > > > This further makes what is sent over the network not directly > > > >

Re: [HACKERS] MD5 authentication needs help

2015-03-04 Thread Bruce Momjian
On Wed, Mar 4, 2015 at 02:21:51PM -0500, Stephen Frost wrote: > * Bruce Momjian (br...@momjian.us) wrote: > > On Wed, Mar 4, 2015 at 01:27:32PM -0500, Stephen Frost wrote: > > > This further makes what is sent over the network not directly > > > susceptible to a replay attack because the server h

Re: [HACKERS] Bootstrap DATA is a pita

2015-03-04 Thread Peter Eisentraut
On 3/4/15 9:51 AM, Robert Haas wrote: > On Wed, Mar 4, 2015 at 9:06 AM, Peter Eisentraut wrote: >>> and make it harder to compare entries by grepping out some common >>> substring. >> >> Could you give an example of the sort of thing you wish to do? > > e.g. grep for a function name and check tha

Re: [HACKERS] MD5 authentication needs help

2015-03-04 Thread Stephen Frost
* Magnus Hagander (mag...@hagander.net) wrote: > On Wed, Mar 4, 2015 at 5:03 PM, Stephen Frost wrote: > > No, I'm not suggesting that OpenSSL or TLS become mandatory but was > > thinking it might be good alternative as a middle-ground between full > > client-and-server side certificates and straig

Re: [HACKERS] MD5 authentication needs help

2015-03-04 Thread Bruce Momjian
On Wed, Mar 4, 2015 at 02:46:54PM -0500, Stephen Frost wrote: > > Well, passwords are already addressed by certificate authentication, so > > what's your point? I think we decided we wanted a way to do passwords > > without requiring network encryption. > > It's completely unclear to me what you

Re: [HACKERS] MD5 authentication needs help

2015-03-04 Thread Tom Lane
Bruce Momjian writes: > Let me update my list of possible improvements: > 1) MD5 makes users feel uneasy (though our usage is mostly safe) > 2) The per-session salt sent to the client is only 32-bits, meaning > that it is possible to reply an observed MD5 hash in ~16k connection > attempts. >

Re: [HACKERS] MD5 authentication needs help

2015-03-04 Thread Stephen Frost
* Bruce Momjian (br...@momjian.us) wrote: > On Wed, Mar 4, 2015 at 02:46:54PM -0500, Stephen Frost wrote: > > > Well, passwords are already addressed by certificate authentication, so > > > what's your point? I think we decided we wanted a way to do passwords > > > without requiring network encry

Re: [HACKERS] MD5 authentication needs help

2015-03-04 Thread Stephen Frost
* Tom Lane (t...@sss.pgh.pa.us) wrote: > Bruce Momjian writes: > > Let me update my list of possible improvements: > > > 1) MD5 makes users feel uneasy (though our usage is mostly safe) > > > 2) The per-session salt sent to the client is only 32-bits, meaning > > that it is possible to reply a

Re: [HACKERS] MD5 authentication needs help

2015-03-04 Thread Tom Lane
Stephen Frost writes: > * Tom Lane (t...@sss.pgh.pa.us) wrote: >> What happened to "possession of the contents of pg_authid is sufficient >> to log in"? I thought fixing that was one of the objectives here. > Yes, it certainly was. I think Bruce was thinking that we could simply > hash what goe

Re: [HACKERS] xpath changes in the recent back branches

2015-03-04 Thread Peter Eisentraut
On 3/4/15 12:20 PM, Marko Tiikkaja wrote: > On 3/4/15 6:19 PM, I wrote: >> On 3/4/15 5:26 PM, Tom Lane wrote: >>> It was considered to be a bug fix; more, given the few complaints about >>> the clearly-broken old behavior, we thought it was a fix that would >>> affect >>> few people, and them posit

Re: [HACKERS] improve pgbench syntax error messages

2015-03-04 Thread Fabien COELHO
Indeed. Here is a v2. Here is a v3, which (1) activates better error messages from bison and (2) improves the error reporting from the scanner as well. sh> ./pgbench -f bad.sql bad.sql:3: syntax error at column 23 in command "set" \set aid (1021 * :id) % ^ error fou

Re: [HACKERS] MD5 authentication needs help

2015-03-04 Thread Stephen Frost
* Tom Lane (t...@sss.pgh.pa.us) wrote: > Stephen Frost writes: > > * Tom Lane (t...@sss.pgh.pa.us) wrote: > >> What happened to "possession of the contents of pg_authid is sufficient > >> to log in"? I thought fixing that was one of the objectives here. > > > Yes, it certainly was. I think Bruc

Re: [HACKERS] Bootstrap DATA is a pita

2015-03-04 Thread Robert Haas
On Wed, Mar 4, 2015 at 2:27 PM, Alvaro Herrera wrote: > Andrew Dunstan wrote: >> On 03/04/2015 09:51 AM, Robert Haas wrote: >> >On Wed, Mar 4, 2015 at 9:06 AM, Peter Eisentraut wrote: >> >>>and make it harder to compare entries by grepping out some common >> >>>substring. >> >>Could you give an e

Re: [HACKERS] Providing catalog view to pg_hba.conf file - Patch submission

2015-03-04 Thread Peter Eisentraut
On 3/3/15 7:17 PM, Jim Nasby wrote: > I think we're screwed in that regard anyway, because of the special > constructs. You'd need different logic to handle things like +role and > sameuser. We might even end up painted in a corner where we can't change > it in the future because it'll break everyo

  1   2   >