Re: [HACKERS] Commitfest problems

2014-12-19 Thread Mark Kirkwood
On 19/12/14 20:48, Andres Freund wrote: On 2014-12-18 10:02:25 -0800, Joshua D. Drake wrote: I think a lot of hackers forget exactly how tender their egos are. Now I say this knowing that a lot of them will go, Oh give me a break but as someone who employs hackers, deals with open source AND

Re: [HACKERS] Postgres TR for missing chunk

2014-12-19 Thread M Tarkeshwar Rao
Hello friends, Thanks for your useful inputs. We are facing this issue and want to analyse this through logging. can you please share a sample Postgres config file to enable max logging with syslog support? What should be the debug level so that I can capture the failure information? Regards

Re: [HACKERS] [COMMITTERS] pgsql: Keep track of transaction commit timestamps

2014-12-19 Thread Petr Jelinek
On 10/12/14 16:03, Petr Jelinek wrote: On 10/12/14 04:26, Michael Paquier wrote: On Thu, Dec 4, 2014 at 9:26 PM, Heikki Linnakangas hlinnakan...@vmware.com wrote: Yeah, it was raised. I don't think it was really addressed. There was lengthy discussion on whether to include LSN, node id, and/or

[HACKERS] Re: [pgsql-pkg-debian] Updated libpq5 packages cause connection errors on postgresql 9.2

2014-12-19 Thread Christoph Berg
Re: Chris Butler 2014-12-19 1155204201.65430.1418975376728.javamail.zim...@zedcore.com One of our servers is currently running on postgres 9.2 using the 9.2.9-1.pgdg70+1 packages from pgdg. After an apt update this morning which brought in the libpq5 package version 9.4.0-1.pgdg70+1,

Re: [HACKERS] TODO : Allow parallel cores to be used by vacuumdb [ WIP ]

2014-12-19 Thread Amit Kapila
On Mon, Dec 15, 2014 at 4:18 PM, Dilip kumar dilip.ku...@huawei.com wrote: On December 2014 17:31 Amit Kapila Wrote, Hmm, theoretically I think new behaviour could lead to more I/O in certain cases as compare to existing behaviour. The reason for more I/O is that in the new behaviour,

Re: [HACKERS] Combining Aggregates

2014-12-19 Thread David Rowley
On 18 December 2014 at 02:48, Simon Riggs si...@2ndquadrant.com wrote: David, if you can update your patch with some docs to explain the behaviour, it looks complete enough to think about committing it in early January, to allow other patches that depend upon it to stand a chance of getting

Re: [HACKERS] Role Attribute Bitmask Catalog Representation

2014-12-19 Thread Alvaro Herrera
Stephen Frost wrote: Alvaro, * Alvaro Herrera (alvhe...@2ndquadrant.com) wrote: FWIW I've been giving this patch a look and and adjusting some coding details here and there. Do you intend to commit it yourself? You're not listed as reviewer or committer for it in the commitfest app,

[HACKERS] Re: [pgsql-pkg-debian] Updated libpq5 packages cause connection errors on postgresql 9.2

2014-12-19 Thread Magnus Hagander
On Fri, Dec 19, 2014 at 11:52 AM, Christoph Berg c...@df7cb.de wrote: Re: Chris Butler 2014-12-19 1155204201.65430.1418975376728.javamail.zim...@zedcore.com One of our servers is currently running on postgres 9.2 using the 9.2.9-1.pgdg70+1 packages from pgdg. After an apt update this

Re: [HACKERS] [COMMITTERS] pgsql: Keep track of transaction commit timestamps

2014-12-19 Thread Michael Paquier
On Fri, Dec 19, 2014 at 6:30 PM, Petr Jelinek p...@2ndquadrant.com wrote: On 10/12/14 16:03, Petr Jelinek wrote: On 10/12/14 04:26, Michael Paquier wrote: On Thu, Dec 4, 2014 at 9:26 PM, Heikki Linnakangas hlinnakan...@vmware.com wrote: Yeah, it was raised. I don't think it was really

Re: [HACKERS] [COMMITTERS] pgsql: Keep track of transaction commit timestamps

2014-12-19 Thread Petr Jelinek
On 19/12/14 13:17, Michael Paquier wrote: On Fri, Dec 19, 2014 at 6:30 PM, Petr Jelinek p...@2ndquadrant.com wrote: On 10/12/14 16:03, Petr Jelinek wrote: On 10/12/14 04:26, Michael Paquier wrote: On Thu, Dec 4, 2014 at 9:26 PM, Heikki Linnakangas hlinnakan...@vmware.com wrote: Yeah, it

Re: [HACKERS] Parallel Seq Scan

2014-12-19 Thread Stephen Frost
Amit, * Amit Kapila (amit.kapil...@gmail.com) wrote: 1. Parallel workers help a lot when there is an expensive qualification to evaluated, the more expensive the qualification the more better are results. I'd certainly hope so. ;) 2. It works well for low selectivity quals and as the

[HACKERS] Bogus WAL segments archived after promotion

2014-12-19 Thread Heikki Linnakangas
When streaming replication was introduced in 9.0, we started to recycle old WAL segments in archive recovery, like we do during normal operation. The WAL segments are recycled on the current timeline. There is no guarantee that they are useful, if the current timeline changes, because we step

Re: [HACKERS] Parallel Seq Scan

2014-12-19 Thread Robert Haas
On Fri, Dec 19, 2014 at 7:51 AM, Stephen Frost sfr...@snowman.net wrote: 3. After certain point, increasing having more number of workers won't help and rather have negative impact, refer Test-4. Yes, I see that too and it's also interesting- have you been able to identify why? What is the

Re: [HACKERS] Logical Replication Helpers WIP for discussion

2014-12-19 Thread Petr Jelinek
On 15/12/14 19:42, Robert Haas wrote: On Mon, Dec 15, 2014 at 12:57 AM, Petr Jelinek p...@2ndquadrant.com wrote: we've made few helper functions for making logical replication easier, I bundled it into contrib module as this is mainly for discussion at this time (I don't expect this to get

Re: [HACKERS] NUMERIC private methods?

2014-12-19 Thread Robert Haas
On Thu, Dec 18, 2014 at 11:51 PM, Tom Lane t...@sss.pgh.pa.us wrote: What it boils down to is that numeric is great for storing given decimal inputs exactly, and it can do exact addition/subtraction/multiplication on those too, but as soon as you get into territory where the result is

Re: [HACKERS] Role Attribute Bitmask Catalog Representation

2014-12-19 Thread Alvaro Herrera
The fact that the ROLE_ATTR_* definitions are in pg_authid.h means that there are now a lot of files that need to include that one. I think the includes relative to ACLs and roles is rather messy now, and this patch makes it a bit worse. I think we should create a new header file (maybe

Re: [HACKERS] parallel mode and parallel contexts

2014-12-19 Thread Robert Haas
On Wed, Dec 17, 2014 at 2:53 PM, Robert Haas robertmh...@gmail.com wrote: In the meantime, I had a good chat with Heikki on IM yesterday which gave me some new ideas on how to fix up the transaction handling in here, which I am working on implementing. So hopefully I will have that by then.

Re: [HACKERS] Parallel Seq Scan

2014-12-19 Thread Stephen Frost
Robert, * Robert Haas (robertmh...@gmail.com) wrote: On Fri, Dec 19, 2014 at 7:51 AM, Stephen Frost sfr...@snowman.net wrote: 3. After certain point, increasing having more number of workers won't help and rather have negative impact, refer Test-4. Yes, I see that too and it's also

Re: [HACKERS] Parallel Seq Scan

2014-12-19 Thread Marko Tiikkaja
On 12/19/14 3:27 PM, Stephen Frost wrote: We'd have to coach our users to constantly be tweaking the enable_parallel_query (or whatever) option for the queries where it helps and turning it off for others. I'm not so excited about that. I'd be perfectly (that means 100%) happy if it just

Re: [HACKERS] Parallel Seq Scan

2014-12-19 Thread Stephen Frost
* Marko Tiikkaja (ma...@joh.to) wrote: On 12/19/14 3:27 PM, Stephen Frost wrote: We'd have to coach our users to constantly be tweaking the enable_parallel_query (or whatever) option for the queries where it helps and turning it off for others. I'm not so excited about that. I'd be

Re: [HACKERS] Parallel Seq Scan

2014-12-19 Thread Robert Haas
On Fri, Dec 19, 2014 at 9:39 AM, Stephen Frost sfr...@snowman.net wrote: Perhaps we should reconsider our general position on hints then and add them so users can define the plan to be used.. For my part, I don't see this as all that much different. Consider if we were just adding HashJoin

Re: [HACKERS] Parallel Seq Scan

2014-12-19 Thread Heikki Linnakangas
On 12/19/2014 04:39 PM, Stephen Frost wrote: * Marko Tiikkaja (ma...@joh.to) wrote: On 12/19/14 3:27 PM, Stephen Frost wrote: We'd have to coach our users to constantly be tweaking the enable_parallel_query (or whatever) option for the queries where it helps and turning it off for others. I'm

Re: [HACKERS] Re: [pgsql-pkg-debian] Updated libpq5 packages cause connection errors on postgresql 9.2

2014-12-19 Thread Tom Lane
Magnus Hagander mag...@hagander.net writes: On Fri, Dec 19, 2014 at 11:52 AM, Christoph Berg c...@df7cb.de wrote: Googling for digest too big for rsa key seems to indicate that this problem occurs when you are using (client?) certificates with short RSA keys. 512 bits is most often cited in

Re: [HACKERS] [PATCH] HINT: pg_hba.conf changed since last config reload

2014-12-19 Thread Steve Singer
On 12/15/2014 11:38 AM, Alex Shulgin wrote: These are all valid concerns IMHO. Attached is the modified version of the original patch by Craig, addressing the handling of the new hint_log error data field and removing the client-side HINT. I'm also moving this to the current CF. -- Alex

[HACKERS] Re: [pgsql-pkg-debian] Updated libpq5 packages cause connection errors on postgresql 9.2

2014-12-19 Thread Chris Butler
Hi Christoph, - Original Message - From: Christoph Berg c...@df7cb.de To: Chris Butler cbut...@zedcore.com Googling for digest too big for rsa key seems to indicate that this problem occurs when you are using (client?) certificates with short RSA keys. 512 bits is most often cited

Re: [HACKERS] [PATCH] HINT: pg_hba.conf changed since last config reload

2014-12-19 Thread Alex Shulgin
Steve Singer st...@ssinger.info writes: On 12/15/2014 11:38 AM, Alex Shulgin wrote: These are all valid concerns IMHO. Attached is the modified version of the original patch by Craig, addressing the handling of the new hint_log error data field and removing the client-side HINT. I'm also

Re: [HACKERS] [PATCH] HINT: pg_hba.conf changed since last config reload

2014-12-19 Thread Craig Ringer
On 12/19/2014 11:41 PM, Alex Shulgin wrote: I don't think so. The scenario this patch relies on assumes that the DBA will remember to look in the log if something goes wrong Well, actually, the whole point was that the user who's connecting (likely also the DBA) will see a HINT telling them

Re: [HACKERS] NUMERIC private methods?

2014-12-19 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: I think what it boils down to is that several people here (and I'll add my voice to the chorus) are saying, hey, numeric is really useful, and we'd like to be able to manipulate numerics without all the palloc and fmgr overhead, and your response

Suppressing elog.c context messages (was Re: [HACKERS] Wait free LW_SHARED acquisition)

2014-12-19 Thread Andres Freund
Hi, When debugging lwlock issues I found PRINT_LWDEBUG/LOG_LWDEBUG rather painful to use because of the amount of elog contexts/statements emitted. Given the number of lwlock acquirations that's just not doable. To solve that during development I've solved that by basically replacing: if

Re: [HACKERS] NUMERIC private methods?

2014-12-19 Thread Robert Haas
On Fri, Dec 19, 2014 at 10:56 AM, Tom Lane t...@sss.pgh.pa.us wrote: Well, there are two components to what I'm saying. One is that the example David started with looks like it could use some better-informed consideration about which datatype to use. The other is that numeric leaves quite a

Re: [HACKERS] [PATCH] HINT: pg_hba.conf changed since last config reload

2014-12-19 Thread Alex Shulgin
Craig Ringer cr...@2ndquadrant.com writes: On 12/19/2014 11:41 PM, Alex Shulgin wrote: I don't think so. The scenario this patch relies on assumes that the DBA will remember to look in the log if something goes wrong Well, actually, the whole point was that the user who's connecting

Re: [HACKERS] Commitfest problems

2014-12-19 Thread Arthur Silva
On Fri, Dec 19, 2014 at 6:28 AM, Mark Kirkwood mark.kirkw...@catalyst.net.nz wrote: On 19/12/14 20:48, Andres Freund wrote: On 2014-12-18 10:02:25 -0800, Joshua D. Drake wrote: I think a lot of hackers forget exactly how tender their egos are. Now I say this knowing that a lot of them

Re: [HACKERS] Role Attribute Bitmask Catalog Representation

2014-12-19 Thread Alvaro Herrera
Alvaro Herrera wrote: The fact that the ROLE_ATTR_* definitions are in pg_authid.h means that there are now a lot of files that need to include that one. I think the includes relative to ACLs and roles is rather messy now, and this patch makes it a bit worse. I think we should create a new

Re: [HACKERS] NUMERIC private methods?

2014-12-19 Thread David Fetter
On Thu, Dec 18, 2014 at 11:51:37PM -0300, Alvaro Herrera wrote: Robert Haas wrote: I think that's ridiculous. You're basically arguing that numeric doesn't offer meaningful advantages over float8, which flies in the face of the fact that essentially every database application I've ever

Re: [HACKERS] replicating DROP commands across servers

2014-12-19 Thread Alvaro Herrera
Alvaro Herrera wrote: Alvaro Herrera wrote: Andres Freund wrote: Having reread the patch just now I basically see two things to criticize: a) why isn't this accessible at SQL level? That seems easy to address. b) Arguably some of this could well be done in separate commits.

Re: [HACKERS] Commitfest problems

2014-12-19 Thread Bruce Momjian
On Wed, Dec 17, 2014 at 02:00:18PM -0500, Stephen Frost wrote: Another thought I had was to suggest we consider *everyone* to be a contributor and implement a way to tie together the mailing list archives with the commit history and perhaps the commitfest app and make it searchable and indexed

Re: [HACKERS] Parallel Seq Scan

2014-12-19 Thread Gavin Flower
On 20/12/14 03:54, Heikki Linnakangas wrote: On 12/19/2014 04:39 PM, Stephen Frost wrote: * Marko Tiikkaja (ma...@joh.to) wrote: On 12/19/14 3:27 PM, Stephen Frost wrote: We'd have to coach our users to constantly be tweaking the enable_parallel_query (or whatever) option for the queries

Re: [HACKERS] Parallel Seq Scan

2014-12-19 Thread Stephen Frost
Robert, * Robert Haas (robertmh...@gmail.com) wrote: On Fri, Dec 19, 2014 at 9:39 AM, Stephen Frost sfr...@snowman.net wrote: Perhaps we should reconsider our general position on hints then and add them so users can define the plan to be used.. For my part, I don't see this as all that

Re: [HACKERS] Parallel Seq Scan

2014-12-19 Thread Stephen Frost
* Heikki Linnakangas (hlinnakan...@vmware.com) wrote: On 12/19/2014 04:39 PM, Stephen Frost wrote: * Marko Tiikkaja (ma...@joh.to) wrote: I'd be perfectly (that means 100%) happy if it just defaulted to off, but I could turn it up to 11 whenever I needed it. I don't believe to be the only

Re: [HACKERS] Role Attribute Bitmask Catalog Representation

2014-12-19 Thread Stephen Frost
* Alvaro Herrera (alvhe...@2ndquadrant.com) wrote: Alvaro Herrera wrote: I think we should create a new header file (maybe acltypes.h or acldefs.h), which only contains the AclMode and RoleAttr typedefs and their associated defines; that one would be included from parsenodes.h, acl.h and

Re: [HACKERS] Bogus WAL segments archived after promotion

2014-12-19 Thread Heikki Linnakangas
On 12/19/2014 02:55 PM, Heikki Linnakangas wrote: I'm thinking that we should add a step to promotion, where we scan pg_xlog for any segments higher than the timeline switch point, and remove them, or mark them with .done so that they are not archived. There might be some real WAL that was

Re: [HACKERS] Streaming replication and WAL archive interactions

2014-12-19 Thread Heikki Linnakangas
On 12/18/2014 12:32 PM, Fujii Masao wrote: On Wed, Dec 17, 2014 at 4:11 AM, Heikki Linnakangas hlinnakan...@vmware.com wrote: On 12/16/2014 10:24 AM, Borodin Vladimir wrote: 12 дек. 2014 г., в 16:46, Heikki Linnakangas hlinnakan...@vmware.com написал(а): There have been a few threads on the

Re: [HACKERS] Commitfest problems

2014-12-19 Thread Joshua D. Drake
On 12/19/2014 12:28 AM, Mark Kirkwood wrote: To me that's a bit over the top stereotyping. +1 Having been mentioned one or two times myself - it was an unexpected wow - cool rather than a grumpy/fragile I must be noticed thing. I think some folk have forgotten the underlying principle of

Re: hash_create API changes (was Re: [HACKERS] speedup tidbitmap patch: hash BlockNumber)

2014-12-19 Thread Jim Nasby
On 12/18/14, 5:00 PM, Jim Nasby wrote: 2201582 20 -- Mostly LOCALLOCK and Shared Buffer Started looking into this; perhaps https://code.google.com/p/fast-hash/ would be worth looking at, though it requires uint64. It also occurs to me that we're needlessly shoving a lot of 0's into the hash

Re: hash_create API changes (was Re: [HACKERS] speedup tidbitmap patch: hash BlockNumber)

2014-12-19 Thread Tom Lane
Jim Nasby jim.na...@bluetreble.com writes: On 12/18/14, 5:00 PM, Jim Nasby wrote: 2201582 20 -- Mostly LOCALLOCK and Shared Buffer Started looking into this; perhaps https://code.google.com/p/fast-hash/ would be worth looking at, though it requires uint64. It also occurs to me that we're

Re: hash_create API changes (was Re: [HACKERS] speedup tidbitmap patch: hash BlockNumber)

2014-12-19 Thread k...@rice.edu
On Fri, Dec 19, 2014 at 04:41:51PM -0600, Jim Nasby wrote: On 12/18/14, 5:00 PM, Jim Nasby wrote: 2201582 20 -- Mostly LOCALLOCK and Shared Buffer Started looking into this; perhaps https://code.google.com/p/fast-hash/ would be worth looking at, though it requires uint64. It also occurs

Re: [HACKERS] Commitfest problems

2014-12-19 Thread Josh Berkus
On 12/18/2014 05:36 PM, Stephen Frost wrote: I tend to agree that we want to avoid complicated rules. The corollary to that is the concern Andrew raised about my earlier off-the-cuff proposal- how do we avoid debasing the value of being recognized as a PG contributor? I find that less of a

Re: hash_create API changes (was Re: [HACKERS] speedup tidbitmap patch: hash BlockNumber)

2014-12-19 Thread Tom Lane
k...@rice.edu k...@rice.edu writes: If we are going to consider changing the hash function, we should consider something like xxhash which runs at 13.8GB/s on a 2.7GHz x86_64 for the XXH64 variant and 6.8GB/s for the XXH32 variant which is double the speed of fast-hash according to the page

Re: [HACKERS] Commitfest problems

2014-12-19 Thread Tom Lane
Josh Berkus j...@agliodbs.com writes: On 12/18/2014 05:36 PM, Stephen Frost wrote: I do agree that we need to give credit in some form, though. I'm just saying can we please not put the responsibility on committers. Ugh, yeah, I certainly wouldn't want to have to work out some complex set

Re: [HACKERS] INSERT ... ON CONFLICT {UPDATE | IGNORE}

2014-12-19 Thread Peter Geoghegan
On Thu, Dec 18, 2014 at 6:59 AM, Heikki Linnakangas hlinnakan...@vmware.com wrote: Good point. For the IGNORE case: I guess the syntax just isn't that flexible. I agree that that isn't ideal. It should be simple to allow multiple key specifications: INSERT INTO persons (username,

Re: [HACKERS] POLA violation with \c service=

2014-12-19 Thread David G Johnston
On Wed, Dec 17, 2014 at 8:25 AM, David Fetter [via PostgreSQL] ml-node+s1045698n5831124...@n5.nabble.com wrote: On Wed, Dec 17, 2014 at 08:14:04AM -0500, Andrew Dunstan wrote: On 12/17/2014 04:11 AM, Heikki Linnakangas wrote: On 12/17/2014 10:03 AM, Albe Laurenz wrote: David Fetter

[HACKERS] table vs domain constraints in ObjType

2014-12-19 Thread Alvaro Herrera
Right now, both table and domain constraints share the same classification in the ObjType enum, OBJECT_CONSTRAINT. For my patch to implement improved object drop reporting in event triggers, it is necessary to separate them, as per in the attach patch. One emerging side-effect is that it is now

Re: [HACKERS] Reducing lock strength of adding foreign keys

2014-12-19 Thread Noah Misch
On Wed, Dec 17, 2014 at 02:41:39PM +, Simon Riggs wrote: Is there anything different here from work in my original patch series? Not to my knowledge. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription:

Re: hash_create API changes (was Re: [HACKERS] speedup tidbitmap patch: hash BlockNumber)

2014-12-19 Thread Jim Nasby
On 12/19/14, 5:13 PM, Tom Lane wrote: Jim Nasby jim.na...@bluetreble.com writes: On 12/18/14, 5:00 PM, Jim Nasby wrote: 2201582 20 -- Mostly LOCALLOCK and Shared Buffer Started looking into this; perhaps https://code.google.com/p/fast-hash/ would be worth looking at, though it requires

Re: [HACKERS] Commitfest problems

2014-12-19 Thread Jim Nasby
On 12/19/14, 6:16 PM, Tom Lane wrote: Could we establish an expectation that whoever sets a CF entry to ready for committer is responsible for reviewing the authors/reviewers lists and making sure that those fairly represent who should get credit? That would divide the labor a bit, and there

Re: [HACKERS] Commitfest problems

2014-12-19 Thread Andres Freund
On 2014-12-19 22:17:54 -0600, Jim Nasby wrote: git does allow you to revise a commit message; it just makes downstream pulls uglier if the commit was already pushed (see https://help.github.com/articles/changing-a-commit-message/). It might be possible to minimize or even eliminate that pain

Re: hash_create API changes (was Re: [HACKERS] speedup tidbitmap patch: hash BlockNumber)

2014-12-19 Thread Andres Freund
On 2014-12-19 22:03:55 -0600, Jim Nasby wrote: I'm not suggesting we change BufferTag or BufferLookupEnt; clearly we can't simply throw away any of the fields I was talking about (well, except possibly tablespace ID. AFAICT that's completely redundant for searching because relid is UNIQUE).

[HACKERS] Re: pgsql: Allow pushdown of WHERE quals into subqueries with window functi

2014-12-19 Thread Noah Misch
On Sat, Jun 28, 2014 at 06:08:05AM +, Tom Lane wrote: Allow pushdown of WHERE quals into subqueries with window functions. We can allow this even without any specific knowledge of the semantics of the window function, so long as pushed-down quals will either accept every row in a given

Re: [HACKERS] Commitfest problems

2014-12-19 Thread Mark Kirkwood
On 20/12/14 11:22, Joshua D. Drake wrote: On 12/19/2014 12:28 AM, Mark Kirkwood wrote: To me that's a bit over the top stereotyping. +1 Having been mentioned one or two times myself - it was an unexpected wow - cool rather than a grumpy/fragile I must be noticed thing. I think some folk

[HACKERS] Initdb-cs_CZ.WIN-1250 buildfarm failures

2014-12-19 Thread Noah Misch
Buildfarm members magpie, treepie and fulmar went absent on 2014-10-29. Since returning on 2014-11-16, they have consistently failed with 'initdb: invalid locale name cs_CZ.WIN-1250'. No commits in that period readily explain a regression in this area. Did the underlying system configurations

[HACKERS] New Python vs. old PG on raccoon and jaguarundi

2014-12-19 Thread Noah Misch
PostgreSQL 9.2 and later support Python 3.3 and Python 3.4; PostgreSQL 9.1 and 9.0 do not. Buildfarm member raccoon upgraded from Python 2.7 to Python 3.3 for its 2013-08-18 run, and it has failed for REL9_1_STABLE and REL9_0_STABLE ever since. Please omit --with-python for those branches.