Re: [HACKERS] CREATE INDEX CONCURRENTLY?

2014-11-01 Thread Simon Riggs
On 31 October 2014 17:46, Michael Banck michael.ba...@credativ.de wrote: I wonder whether that is pilot error (fair enough), or whether something could be done about this? When originally written the constraints were tighter, but have since been relaxed. Even so a CIC waits until all

Re: [HACKERS] Let's drop two obsolete features which are bear-traps for novices

2014-11-01 Thread Amit Langote
Hi, On Sat, Nov 1, 2014 at 1:21 PM, Eric Ridge eeb...@gmail.com wrote: On Fri, Oct 31, 2014 at 6:07 PM, Tom Lane t...@sss.pgh.pa.us wrote: I don't know if/when that will happen as such, but Simon was making noises about writing code to treat hash indexes as unlogged automatically, which

Re: [HACKERS] Patch: Add launchd Support

2014-11-01 Thread Peter Eisentraut
On 10/20/14 8:53 PM, Wim Lewis wrote: Apple has published their changes to Postgres (since they ship it in recent versions of OSX) here, fwiw, including the launchd plist they use: http://www.opensource.apple.com/source/PostgreSQL/ One thing I noticed is that Apple also used the label

Re: [HACKERS] tracking commit timestamps

2014-11-01 Thread Petr Jelinek
Hi, thanks for review. On 01/11/14 05:45, Michael Paquier wrote: Now here are a couple of comments at code level, this code seems not enough baked for a commit: 1) The following renaming should be done: - pg_get_transaction_committime to pg_get_transaction_commit_time -

Re: [HACKERS] tracking commit timestamps

2014-11-01 Thread Petr Jelinek
On 01/11/14 12:19, Petr Jelinek wrote: Hi, thanks for review. On 01/11/14 05:45, Michael Paquier wrote: Now here are a couple of comments at code level, this code seems not enough baked for a commit: 1) The following renaming should be done: - pg_get_transaction_committime to

Re: [HACKERS] tracking commit timestamps

2014-11-01 Thread Michael Paquier
On Sat, Nov 1, 2014 at 9:04 PM, Petr Jelinek p...@2ndquadrant.com wrote: On 01/11/14 12:19, Petr Jelinek wrote: Hi, thanks for review. On 01/11/14 05:45, Michael Paquier wrote: Now here are a couple of comments at code level, this code seems not enough baked for a commit: 1) The

Re: [HACKERS] tracking commit timestamps

2014-11-01 Thread Michael Paquier
On Sat, Nov 1, 2014 at 1:45 PM, Michael Paquier michael.paqu...@gmail.com wrote: I am still planning to do more extensive tests, and study a bit more committs.c (with even more comments) as it is the core part of the feature. More comments: - Heikki already mentioned it, but after reading the

Re: [HACKERS] tracking commit timestamps

2014-11-01 Thread Petr Jelinek
On 01/11/14 14:00, Michael Paquier wrote: More comments: - Heikki already mentioned it, but after reading the code I see little point in having the extra field implementing like that in core for many reasons even if it is *just* 4 bytes: 1) It is untested and actually there is no direct use for

Re: [HACKERS] pg_ctl non-idempotent behavior change

2014-11-01 Thread Peter Eisentraut
On 10/11/14 6:54 PM, Bruce Momjian wrote: Are we going to unrevert this patch for 9.5? Seems no one is thinking of restoring this patch and working on the issue. I had postponed work on this issue and set out to create a test infrastructure so that all the subtle behavioral dependencies

[HACKERS] Pipelining executions to postgresql server

2014-11-01 Thread Mikko Tiihonen
Hi, I created a proof of concecpt patch for postgresql JDBC driver that allows the caller to do pipelining of requests within a transaction. The pipelining here means same as for HTTP: the client can send the next execution already before waiting for the response of the previous request to be

Re: [HACKERS] Switch pg_basebackup to use -X stream instead of -X fetch by default?

2014-11-01 Thread Peter Eisentraut
On 10/13/14 3:16 PM, Bruce Momjian wrote: Where are we on this? I think we discovered that the pros and cons of the different options are not as clear-cut, and it's better to leave the default as is until additional features make one option a clear winner. -- Sent via pgsql-hackers mailing

Re: [HACKERS] Pipelining executions to postgresql server

2014-11-01 Thread Tom Lane
Mikko Tiihonen mikko.tiiho...@nitorcreations.com writes: I created a proof of concecpt patch for postgresql JDBC driver that allows the caller to do pipelining of requests within a transaction. The pipelining here means same as for HTTP: the client can send the next execution already before

Re: [HACKERS] _mdfd_getseg can be expensive

2014-11-01 Thread Tom Lane
Andres Freund and...@2ndquadrant.com writes: On 2014-10-31 18:48:45 -0400, Tom Lane wrote: While the basic idea is sound, this particular implementation seems pretty bizarre. What's with the md_seg_no stuff, and why is that array typed size_t? It stores the length of the array of _MdfdVec

Re: [HACKERS] Let's drop two obsolete features which are bear-traps for novices

2014-11-01 Thread Josh Berkus
On 10/31/2014 03:07 PM, Tom Lane wrote: I don't care one way or the other about the money type, but I will defend hash indexes, especially seeing that we've already added a pretty in-your-face warning as of 9.5: regression=# create table foo(f1 int); CREATE TABLE regression=# create index

Re: [HACKERS] [BUGS] ltree::text not immutable?

2014-11-01 Thread Tom Lane
Alvaro Herrera alvhe...@2ndquadrant.com writes: Tom Lane wrote: Not entirely sure what to do about this. It seems like for the purposes of contrib/chkpass, it's a good thing that chkpass_in() won't reuse the same salt. Weak as a 12-bit salt might be nowadays, it's still better than no salt.

Re: [HACKERS] _mdfd_getseg can be expensive

2014-11-01 Thread Andres Freund
On 2014-11-01 12:57:40 -0400, Tom Lane wrote: Andres Freund and...@2ndquadrant.com writes: On 2014-10-31 18:48:45 -0400, Tom Lane wrote: While the basic idea is sound, this particular implementation seems pretty bizarre. What's with the md_seg_no stuff, and why is that array typed

Re: [HACKERS] Let's drop two obsolete features which are bear-traps for novices

2014-11-01 Thread Andres Freund
On 2014-11-01 10:18:03 -0700, Josh Berkus wrote: On 10/31/2014 03:07 PM, Tom Lane wrote: I don't care one way or the other about the money type, but I will defend hash indexes, especially seeing that we've already added a pretty in-your-face warning as of 9.5: regression=# create table

Re: [HACKERS] Let's drop two obsolete features which are bear-traps for novices

2014-11-01 Thread Tom Lane
Josh Berkus j...@agliodbs.com writes: On 10/31/2014 03:07 PM, Tom Lane wrote: I don't care one way or the other about the money type, but I will defend hash indexes, especially seeing that we've already added a pretty in-your-face warning as of 9.5: regression=# create table foo(f1 int);

Re: [HACKERS] Pipelining executions to postgresql server

2014-11-01 Thread Andres Freund
On 2014-11-01 14:04:05 +, Mikko Tiihonen wrote: I created a proof of concecpt patch for postgresql JDBC driver that allows the caller to do pipelining of requests within a transaction. The pipelining here means same as for HTTP: the client can send the next execution already before waiting

Re: [HACKERS] tracking commit timestamps

2014-11-01 Thread Andres Freund
On 2014-11-01 13:45:44 +0900, Michael Paquier wrote: 14) I'd put the two checks in the reverse order: + Assert(xid != InvalidTransactionId); + + if (!commit_ts_enabled) + return; Please don't. The order is correct right now. Why you ask? This way the correctness of

Re: [HACKERS] tracking commit timestamps

2014-11-01 Thread Andres Freund
On 2014-11-01 22:00:40 +0900, Michael Paquier wrote: On Sat, Nov 1, 2014 at 1:45 PM, Michael Paquier michael.paqu...@gmail.com wrote: I am still planning to do more extensive tests, and study a bit more committs.c (with even more comments) as it is the core part of the feature. More

Re: [HACKERS] Let's drop two obsolete features which are bear-traps for novices

2014-11-01 Thread Andres Freund
On 2014-11-01 13:58:02 -0400, Tom Lane wrote: Yeah. When we last discussed this, the difficulty was around how to make that combination work. An unlogged index on an unlogged table is no problem: the init-fork mechanism is able to make them both go to empty after a crash. But for an

Re: [HACKERS] Let's drop two obsolete features which are bear-traps for novices

2014-11-01 Thread Andrew Dunstan
On 11/01/2014 01:26 PM, Andres Freund wrote: On 2014-11-01 10:18:03 -0700, Josh Berkus wrote: On 10/31/2014 03:07 PM, Tom Lane wrote: I don't care one way or the other about the money type, but I will defend hash indexes, especially seeing that we've already added a pretty in-your-face

Re: [HACKERS] Let's drop two obsolete features which are bear-traps for novices

2014-11-01 Thread Andres Freund
On 2014-11-01 14:13:02 -0400, Andrew Dunstan wrote: Yes, although there might not be much reason to use them either. I think Tom's first step of making hash indexes automatically unlogged makes sense. Longer term I'd like to see unlogged as an option for all AMs - especially btree. If we

Re: [HACKERS] Let's drop two obsolete features which are bear-traps for novices

2014-11-01 Thread Andrew Dunstan
On 11/01/2014 01:58 PM, Tom Lane wrote: Andres Freund and...@2ndquadrant.com writes: On 2014-11-01 10:18:03 -0700, Josh Berkus wrote: Yes, and I'm arguing that is the wrong decision. If hash indexes are discouraged, then they shouldn't be in core in the first place. Last time we discussed

Re: [HACKERS] Let's drop two obsolete features which are bear-traps for novices

2014-11-01 Thread Tom Lane
Andres Freund and...@2ndquadrant.com writes: On 2014-11-01 13:58:02 -0400, Tom Lane wrote: maybe we don't have to. What about having the init-fork mechanism restore a hash index into a state with the metapage marked as invalid? hashinsert etc could simply do nothing when they see this

Re: [HACKERS] Let's drop two obsolete features which are bear-traps for novices

2014-11-01 Thread Andres Freund
On 2014-11-01 14:19:22 -0400, Andrew Dunstan wrote: Isn't the planner still going to try to use the index in that case? If it's not then I'd be OK with it, but if it's going to make the table largely unusable until it's reindexed that would be rather sad. Both the planner (for querying) and

Re: [HACKERS] Let's drop two obsolete features which are bear-traps for novices

2014-11-01 Thread Tom Lane
Andres Freund and...@2ndquadrant.com writes: On 2014-11-01 14:19:22 -0400, Andrew Dunstan wrote: Isn't the planner still going to try to use the index in that case? If it's not then I'd be OK with it, but if it's going to make the table largely unusable until it's reindexed that would be

Re: [HACKERS] Let's drop two obsolete features which are bear-traps for novices

2014-11-01 Thread Andres Freund
On 2014-11-01 14:23:07 -0400, Tom Lane wrote: Andres Freund and...@2ndquadrant.com writes: On 2014-11-01 13:58:02 -0400, Tom Lane wrote: maybe we don't have to. What about having the init-fork mechanism restore a hash index into a state with the metapage marked as invalid? hashinsert

Re: [HACKERS] Let's drop two obsolete features which are bear-traps for novices

2014-11-01 Thread Tom Lane
Andres Freund and...@2ndquadrant.com writes: A REINDEX is imo unlikely to be acceptable. It takes long (why would you bother on a small table?) and locks the relation/indexes. I think the goalposts just took a vacation to Acapulco. What exactly do you think is going to make a crashed unlogged

Re: [HACKERS] Let's drop two obsolete features which are bear-traps for novices

2014-11-01 Thread Andres Freund
On 2014-11-01 14:39:21 -0400, Tom Lane wrote: Andres Freund and...@2ndquadrant.com writes: A REINDEX is imo unlikely to be acceptable. It takes long (why would you bother on a small table?) and locks the relation/indexes. I think the goalposts just took a vacation to Acapulco. I think

Re: [HACKERS] Let's drop two obsolete features which are bear-traps for novices

2014-11-01 Thread Andrew Dunstan
On 11/01/2014 02:34 PM, Andres Freund wrote: Yeah, if we were trying to duplicate the behavior of indisvalid, there'd need to be a way to detect the invalid index at plan time and not use it. But I'm not sure that that's actually an improvement from the user's standpoint: what they'd see is

Re: [HACKERS] Let's drop two obsolete features which are bear-traps for novices

2014-11-01 Thread Tom Lane
Andres Freund and...@2ndquadrant.com writes: On 2014-11-01 14:39:21 -0400, Tom Lane wrote: What exactly do you think is going to make a crashed unlogged index valid again without a REINDEX? Certainly the people who are currently using hash indexes in the way Andrew describes are expecting to

Re: [HACKERS] Let's drop two obsolete features which are bear-traps for novices

2014-11-01 Thread Andres Freund
On 2014-11-01 14:48:20 -0400, Tom Lane wrote: Andres Freund and...@2ndquadrant.com writes: On 2014-11-01 14:39:21 -0400, Tom Lane wrote: What exactly do you think is going to make a crashed unlogged index valid again without a REINDEX? Certainly the people who are currently using hash

Re: [HACKERS] Let's drop two obsolete features which are bear-traps for novices

2014-11-01 Thread Andrew Dunstan
On 11/01/2014 02:39 PM, Tom Lane wrote: Andres Freund and...@2ndquadrant.com writes: A REINDEX is imo unlikely to be acceptable. It takes long (why would you bother on a small table?) and locks the relation/indexes. I think the goalposts just took a vacation to Acapulco. What exactly do you

Re: [HACKERS] Let's drop two obsolete features which are bear-traps for novices

2014-11-01 Thread Andres Freund
On 2014-11-01 14:56:35 -0400, Andrew Dunstan wrote: On 11/01/2014 02:39 PM, Tom Lane wrote: Andres Freund and...@2ndquadrant.com writes: A REINDEX is imo unlikely to be acceptable. It takes long (why would you bother on a small table?) and locks the relation/indexes. I think the goalposts

Re: [HACKERS] Let's drop two obsolete features which are bear-traps for novices

2014-11-01 Thread Tom Lane
Andres Freund and...@2ndquadrant.com writes: On 2014-11-01 14:48:20 -0400, Tom Lane wrote: Sure. And as long as you aren't issuing queries that would want to scan the crashed index, it won't matter either way. The question is whether you'd rather that your inessential reporting queries fail

Re: [HACKERS] Let's drop two obsolete features which are bear-traps for novices

2014-11-01 Thread Tom Lane
Andrew Dunstan and...@dunslane.net writes: The real question here is whether the table should continue to be usable in a degraded state until it's reindexed. It certainly will be, as long as your notion of usable in a degraded state doesn't include issuing queries that would prefer to use the

Re: [HACKERS] Let's drop two obsolete features which are bear-traps for novices

2014-11-01 Thread Tom Lane
Andres Freund and...@2ndquadrant.com writes: One argument in that direction imo is HS. We certainly would just generally ignore unlogged indexes for querying while InRecovery, right? Because otherwise HS would become pretty useless. And I think it'd be pretty wierd if things worked on HS and

Re: [HACKERS] Let's drop two obsolete features which are bear-traps for novices

2014-11-01 Thread Andres Freund
On 2014-11-01 15:11:40 -0400, Tom Lane wrote: Andres Freund and...@2ndquadrant.com writes: One argument in that direction imo is HS. We certainly would just generally ignore unlogged indexes for querying while InRecovery, right? Because otherwise HS would become pretty useless. And I think

Re: [HACKERS] Temp tables, pg_class_temp and AccessExclusiveLocks

2014-11-01 Thread Pavel Stehule
Hi good idea. Mix of these both strategies can be a base for global temp tables implementation. Pavel 2014-10-31 15:53 GMT+01:00 Simon Riggs si...@2ndquadrant.com: While investigating how to reduce logging of AccessExclusiveLocks for temp tables, I came up with the attached patch. My

Re: [HACKERS] Let's drop two obsolete features which are bear-traps for novices

2014-11-01 Thread Tom Lane
Andres Freund and...@2ndquadrant.com writes: On 2014-11-01 15:11:40 -0400, Tom Lane wrote: I don't see how HS has anything to do with this discussion. Consider: SELECT * FROM tbl WHERE active AND value = $1; that can be satisfied by two indexes. One on (value), and an unlogged index on

Re: [HACKERS] Let's drop two obsolete features which are bear-traps for novices

2014-11-01 Thread Andres Freund
On 2014-11-01 15:33:27 -0400, Tom Lane wrote: With current usage of hash indexes, nobody would ever construct such an arrangement Do you think this should only be implemented for hash indexes? I'd think we'd do it for all existing index AMs? I'm not all that excited about unlogged hash

Re: [HACKERS] Let's drop two obsolete features which are bear-traps for novices

2014-11-01 Thread Tom Lane
Andres Freund and...@2ndquadrant.com writes: On 2014-11-01 15:33:27 -0400, Tom Lane wrote: With current usage of hash indexes, nobody would ever construct such an arrangement Do you think this should only be implemented for hash indexes? I'd think we'd do it for all existing index AMs? I

Re: [HACKERS] Let's drop two obsolete features which are bear-traps for novices

2014-11-01 Thread Peter Geoghegan
On Sat, Nov 1, 2014 at 12:40 PM, Andres Freund and...@2ndquadrant.com wrote: I'm not all that excited about unlogged hash indexes. Yes, that'll remove a annoying hazard, but I probably won't use them anyway. I am somewhat excited about the more general unlogged indexes feature. I strongly

Re: [HACKERS] group locking: incomplete patch, just for discussion

2014-11-01 Thread Robert Haas
On Sat, Nov 1, 2014 at 1:55 PM, Andres Freund and...@2ndquadrant.com wrote: Where will those preexisting cache entries come from, exactly? The postmaster is forking the parallel worker, not the user backend. Several things: 1) The relcache init files load a fair bit 2) There's cache entries

Re: [HACKERS] group locking: incomplete patch, just for discussion

2014-11-01 Thread Robert Haas
On Sat, Nov 1, 2014 at 1:55 PM, Andres Freund and...@2ndquadrant.com wrote: I doubt it. There's a whole bunch of problems that just exist by virtue of having a group leader. What if the leader dies but the worker backend isn't in a section of code that does a CHECK_FOR_INTERRUPTS()? In between

Re: [HACKERS] group locking: incomplete patch, just for discussion

2014-11-01 Thread Andres Freund
On 2014-11-01 17:00:59 -0400, Robert Haas wrote: On Sat, Nov 1, 2014 at 1:55 PM, Andres Freund and...@2ndquadrant.com wrote: I doubt it. There's a whole bunch of problems that just exist by virtue of having a group leader. What if the leader dies but the worker backend isn't in a section of

Re: [HACKERS] group locking: incomplete patch, just for discussion

2014-11-01 Thread Robert Haas
On Sat, Nov 1, 2014 at 5:06 PM, Andres Freund and...@2ndquadrant.com wrote: On 2014-11-01 17:00:59 -0400, Robert Haas wrote: On Sat, Nov 1, 2014 at 1:55 PM, Andres Freund and...@2ndquadrant.com wrote: I doubt it. There's a whole bunch of problems that just exist by virtue of having a group

Re: [HACKERS] group locking: incomplete patch, just for discussion

2014-11-01 Thread Robert Haas
On Sat, Nov 1, 2014 at 1:09 AM, Simon Riggs si...@2ndquadrant.com wrote: What are the specific restrictions you are suggesting we place on parallel workers? We need that definition clear so we can decide how to mark the functions. If we don't know, which is easily understandable, then the best

Re: [HACKERS] group locking: incomplete patch, just for discussion

2014-11-01 Thread Andres Freund
On 2014-11-01 16:59:34 -0400, Robert Haas wrote: I completely fail to see why you'd generally think it's safe for two backends to hold the same self conflicting lock. Yes, within carefully restricted sections of code that might be doable. But I don't think you can fully enforce that.

Re: [HACKERS] Let's drop two obsolete features which are bear-traps for novices

2014-11-01 Thread Josh Berkus
All, While there's argument about hash indexes, it looks like nobody minds if the MONEY type goes bye-bye. So, time for a patch ... -- Josh Berkus PostgreSQL Experts Inc. http://pgexperts.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your

[HACKERS] Locked stdio has noticeable performance cost for COPY

2014-11-01 Thread Andres Freund
Hi, Every now and then I've noticed that glibc's stdio functions show up prominently in profiles. Which seems somewhat odd, given they pretty much should delegate all work to the kernel and memcpy(). By accident I looked at a the disassembly and oops: A large part is due to locking. Especially in

Re: [HACKERS] group locking: incomplete patch, just for discussion

2014-11-01 Thread Robert Haas
On Sat, Nov 1, 2014 at 5:38 PM, Andres Freund and...@2ndquadrant.com wrote: That's just a mislabeled function. It's obviously not parallel safe at all. I see absolutely no problem with erroring out. I disagree. It's entirely parallel-safe, as long as you don't arbitrarily decide to have the

Re: [HACKERS] Let's drop two obsolete features which are bear-traps for novices

2014-11-01 Thread David Rowley
On Sun, Nov 2, 2014 at 12:59 PM, Josh Berkus j...@agliodbs.com wrote: All, While there's argument about hash indexes, it looks like nobody minds if the MONEY type goes bye-bye. So, time for a patch ... Will the patch move the feature to a contrib module? Or will our release notes state

[HACKERS] Silly coding in pgcrypto

2014-11-01 Thread Marko Tiikkaja
Hi, Patch attached to fix some sillyness in pgp_expect_packet_end() and pgp_skip_packet(). Will add to the next commitfest unless someone picks this one up before that. .marko *** a/contrib/pgcrypto/pgp-decrypt.c --- b/contrib/pgcrypto/pgp-decrypt.c *** *** 1069,1075