Re: [HACKERS] Support for REINDEX CONCURRENTLY

2013-02-07 Thread Tom Lane
Andres Freund and...@2ndquadrant.com writes: What about 3) Use reltoastidxid if != InvalidOid and manually build the list (using RelationGetIndexList) otherwise? Do we actually need reltoastidxid at all? I always thought having that field was a case of premature optimization. There might be

Re: [HACKERS] Support for REINDEX CONCURRENTLY

2013-02-07 Thread Andres Freund
On 2013-02-07 03:01:36 -0500, Tom Lane wrote: Andres Freund and...@2ndquadrant.com writes: What about 3) Use reltoastidxid if != InvalidOid and manually build the list (using RelationGetIndexList) otherwise? Do we actually need reltoastidxid at all? I always thought having that field

Re: [HACKERS] Support for REINDEX CONCURRENTLY

2013-02-07 Thread Michael Paquier
On Thu, Feb 7, 2013 at 5:01 PM, Tom Lane t...@sss.pgh.pa.us wrote: Andres Freund and...@2ndquadrant.com writes: What about 3) Use reltoastidxid if != InvalidOid and manually build the list (using RelationGetIndexList) otherwise? Do we actually need reltoastidxid at all? I always

Re: [HACKERS] Support for REINDEX CONCURRENTLY

2013-02-07 Thread Michael Paquier
On Thu, Feb 7, 2013 at 5:15 PM, Andres Freund and...@2ndquadrant.comwrote: On 2013-02-07 03:01:36 -0500, Tom Lane wrote: Andres Freund and...@2ndquadrant.com writes: What about 3) Use reltoastidxid if != InvalidOid and manually build the list (using RelationGetIndexList) otherwise?

Re: [HACKERS] Vacuum/visibility is busted

2013-02-07 Thread Simon Riggs
On 7 February 2013 05:39, Jeff Janes jeff.ja...@gmail.com wrote: While stress testing Pavan's 2nd pass vacuum visibility patch, I realized that vacuum/visibility was busted. But it wasn't his patch that busted it. As far as I can tell, the bad commit was in the range

Re: [HACKERS] [COMMITTERS] pgsql: Fast promote mode skips checkpoint at end of recovery.

2013-02-07 Thread Simon Riggs
On 6 February 2013 18:02, Robert Haas robertmh...@gmail.com wrote: So I would ask this question: why would someone want to turn off fast-promote mode, assuming for the sake of argument that it isn't buggy? You can write a question many ways, and lead people towards a conclusion as a result.

Re: [HACKERS] Vacuum/visibility is busted

2013-02-07 Thread Pavan Deolasee
On Thu, Feb 7, 2013 at 11:09 AM, Jeff Janes jeff.ja...@gmail.com wrote: While stress testing Pavan's 2nd pass vacuum visibility patch, I realized that vacuum/visibility was busted. But it wasn't his patch that busted it. As far as I can tell, the bad commit was in the range

Re: [HACKERS] [COMMITTERS] pgsql: Fast promote mode skips checkpoint at end of recovery.

2013-02-07 Thread Heikki Linnakangas
On 07.02.2013 10:41, Simon Riggs wrote: On 6 February 2013 18:02, Robert Haasrobertmh...@gmail.com wrote: So I would ask this question: why would someone want to turn off fast-promote mode, assuming for the sake of argument that it isn't buggy? You can write a question many ways, and lead

Re: [HACKERS] Vacuum/visibility is busted

2013-02-07 Thread Pavan Deolasee
On Thu, Feb 7, 2013 at 2:25 PM, Pavan Deolasee pavan.deola...@gmail.com wrote: Will look more into it, but thought this might be useful for others to spot the problem. And here is some more forensic info about one of the pages having duplicate tuples. jjanes=# select *, xmin, xmax, ctid

Re: [HACKERS] [COMMITTERS] pgsql: Fast promote mode skips checkpoint at end of recovery.

2013-02-07 Thread Simon Riggs
On 7 February 2013 09:04, Heikki Linnakangas hlinnakan...@vmware.com wrote: It makes me uncomfortable that we're adding switches to pg_ctl promote just because we're worried there might be bugs in our code. If we don't trust the code as it is, it needs more testing. We can analyze the code

Re: [HACKERS] sepgsql and materialized views

2013-02-07 Thread Kohei KaiGai
Thanks for your introduction. It made me almost clear. From selinux perspective, it does not switch user's privilege even when query scans underlying tables because it has no ownership concept. The current implementation does not make a problem because all the MV refresh shall be done in a

Re: [HACKERS] Considering Gerrit for CFs

2013-02-07 Thread Magnus Hagander
On Thu, Feb 7, 2013 at 8:20 AM, Daniel Farina dan...@heroku.com wrote: On Wed, Feb 6, 2013 at 3:00 PM, Joshua D. Drake j...@commandprompt.com wrote: On 02/06/2013 01:53 PM, Tom Lane wrote: ... if it's going to try to coerce us out of our email-centric habits, then I for one am very much

Re: [HACKERS] Considering Gerrit for CFs

2013-02-07 Thread Magnus Hagander
On Thu, Feb 7, 2013 at 8:29 AM, Brendan Jurd dire...@gmail.com wrote: On 7 February 2013 08:07, Josh Berkus j...@agliodbs.com wrote: The existing Gerrit community would be keen to have the PostgreSQL project as a major user, though, and would theoretically help with modification needs.

Re: [HACKERS] proposal: ANSI SQL 2011 syntax for named parameters

2013-02-07 Thread Simon Riggs
On 6 February 2013 20:31, Robert Haas robertmh...@gmail.com wrote: On Wed, Feb 6, 2013 at 1:06 PM, Simon Riggs si...@2ndquadrant.com wrote: On 6 February 2013 17:43, Robert Haas robertmh...@gmail.com wrote: On Mon, Feb 4, 2013 at 3:32 PM, Simon Riggs si...@2ndquadrant.com wrote: On 4 February

Re: [HACKERS] [COMMITTERS] pgsql: Fast promote mode skips checkpoint at end of recovery.

2013-02-07 Thread Robert Haas
On Thu, Feb 7, 2013 at 4:04 AM, Heikki Linnakangas hlinnakan...@vmware.com wrote: It makes me uncomfortable that we're adding switches to pg_ctl promote just because we're worried there might be bugs in our code. If we don't trust the code as it is, it needs more testing. We can analyze the

Re: [HACKERS] proposal: ANSI SQL 2011 syntax for named parameters

2013-02-07 Thread Robert Haas
On Thu, Feb 7, 2013 at 6:42 AM, Simon Riggs si...@2ndquadrant.com wrote: IMO the way to resolve that conflict is with a behaviour parameter to allow people to choose, rather than be forced to wait a year because some people still run an old version of an add-on package. A good way to do that

Re: [HACKERS] LDAP: bugfix and deprecated OpenLDAP API

2013-02-07 Thread Robert Haas
On Tue, Feb 5, 2013 at 4:39 AM, Albe Laurenz laurenz.a...@wien.gv.at wrote: I guess it's too late for something like that to go into 9.3. Should I add it to the next commitfest? Bug fixes can go in pretty much whenever, but adding it to the next CommitFest is a good way of backstopping it

[HACKERS] A question about the psql \copy command

2013-02-07 Thread Etsuro Fujita
Through the work on the patch [1], I had a question about the psql \copy command. We are permitted 1) but not permitted 2): 1) \copy foo from stdin ; 2) \copy foo from stdin; Is this intentional? I think it would be better to allow for 2). Attached is a patch. Thanks, Best regards, Etsuro

Re: [HACKERS] Identity projection

2013-02-07 Thread Amit Kapila
On Friday, December 14, 2012 5:11 PM Heikki Linnakangas wrote: On 12.11.2012 12:07, Kyotaro HORIGUCHI wrote: Hello, This is new version of identity projection patch. Reverted projectionInfo and ExecBuildProjectionInfo. Identity projection is recognized directly in ExecGroup, ExecResult,

[HACKERS] Re[2]: [HACKERS] standby, pg_basebackup and last xlog file

2013-02-07 Thread Миша Тюрин
Hello all and Heikki personally Thank you for your answer I have some new points: Понедельник, 21 января 2013, 10:08 +02:00 от Heikki Linnakangas hlinnakan...@vmware.com: On 21.01.2013 09:14, Миша Тюрин wrote: Is there any reason why pg_basebackup has limitation in an online backup

Re: [HACKERS] Vacuum/visibility is busted

2013-02-07 Thread Alvaro Herrera
Pavan Deolasee escribió: On Thu, Feb 7, 2013 at 2:25 PM, Pavan Deolasee pavan.deola...@gmail.com wrote: Will look more into it, but thought this might be useful for others to spot the problem. And here is some more forensic info about one of the pages having duplicate tuples.

Re: backend hangs at immediate shutdown (Re: [HACKERS] Back-branch update releases coming in a couple weeks)

2013-02-07 Thread MauMau
Hello, Tom-san, folks, From: Tom Lane t...@sss.pgh.pa.us I think if we want to make it bulletproof we'd have to do what the OP suggested and switch to SIGKILL. I'm not enamored of that for the reasons I mentioned --- but one idea that might dodge the disadvantages is to have the postmaster

Re: [HACKERS] Vacuum/visibility is busted

2013-02-07 Thread Alvaro Herrera
Alvaro Herrera escribió: Hm, if the foreign key patch is to blame, this sounds like these tuples had a different set of XMAX hint bits and a different Xmax, and they were clobbered by something like vacuum or page pruning. Hm, I think heap_freeze_tuple is busted, yes. -- Álvaro Herrera

Re: [HACKERS] [COMMITTERS] pgsql: Fast promote mode skips checkpoint at end of recovery.

2013-02-07 Thread Kevin Grittner
Heikki Linnakangas hlinnakan...@vmware.com wrote: On 07.02.2013 10:41, Simon Riggs wrote: Why would someone want to turn off safe-promote mode, assuming it was fast enough? Because faster is nicer, even if the slow mode would be fast enough. http://www.youtube.com/watch?v=H3R-rtWPyJY

[HACKERS] Record previous TLI in end-of-recovery record (was Re: [COMMITTERS] pgsql: Fast promote mode skips checkpoint at end of recovery.)

2013-02-07 Thread Heikki Linnakangas
(this is unrelated to the other discussion about this patch) On 29.01.2013 02:07, Simon Riggs wrote: Fast promote mode skips checkpoint at end of recovery. pg_ctl promote -m fast will skip the checkpoint at end of recovery so that we can achieve very fast failover when the apply delay is low.

[HACKERS] Re: Record previous TLI in end-of-recovery record (was Re: [COMMITTERS] pgsql: Fast promote mode skips checkpoint at end of recovery.)

2013-02-07 Thread Simon Riggs
On 7 February 2013 16:07, Heikki Linnakangas hlinnakan...@vmware.com wrote: It just occurred to me that it would be really nice if the end-of-recovery record, and the timeline-switching shutdown checkpoint record too for that matter, would include the previous timeline's ID that we forked

Re: [HACKERS] proposal: ANSI SQL 2011 syntax for named parameters

2013-02-07 Thread Dimitri Fontaine
Robert Haas robertmh...@gmail.com writes: $ Right now there is one and only one release in $ the field that contains hstore 1.1. If we go ahead and prohibit = as $ an operator name now, we're going to require everyone who is on 9.1 $ and uses hstore and wants to get to 9.3 to either (a) first

Re: [HACKERS] proposal: ANSI SQL 2011 syntax for named parameters

2013-02-07 Thread Tom Lane
Dimitri Fontaine dimi...@2ndquadrant.fr writes: Robert Haas robertmh...@gmail.com writes: I don't know what to add to that. There's no technical reason that I'm aware of for hstore 1.1 not to support all our maintained releases at the same time. That's exactly how we do it with non-core

Re: [HACKERS] split rm_name and rm_desc out of rmgr.c

2013-02-07 Thread Peter Eisentraut
On 2/5/13 3:47 PM, Alvaro Herrera wrote: Tom Lane wrote: Alvaro Herrera alvhe...@2ndquadrant.com writes: The approach in the second patch is to turn these into extern const RmgrId instead, and use a second inclusion of rmgrlist.h in rmgr.c that assigns them the values as consts. ... but

Re: [HACKERS] Vacuum/visibility is busted

2013-02-07 Thread Jeff Janes
On Thu, Feb 7, 2013 at 1:44 AM, Pavan Deolasee pavan.deola...@gmail.com wrote: On Thu, Feb 7, 2013 at 2:25 PM, Pavan Deolasee pavan.deola...@gmail.com wrote: Will look more into it, but thought this might be useful for others to spot the problem. And here is some more forensic info about

Re: [HACKERS] proposal: ANSI SQL 2011 syntax for named parameters

2013-02-07 Thread Dimitri Fontaine
Tom Lane t...@sss.pgh.pa.us writes: If you're suggesting that we should back-patch hstore 1.1 into 9.1, there might not be a technical reason why we couldn't do it, but there are certainly project-policy reasons. Removing operators, or indeed changing any SQL interface at all, is exactly the

Re: [HACKERS] Vacuum/visibility is busted

2013-02-07 Thread Jeff Janes
On Thu, Feb 7, 2013 at 12:55 AM, Pavan Deolasee pavan.deola...@gmail.com wrote: On Thu, Feb 7, 2013 at 11:09 AM, Jeff Janes jeff.ja...@gmail.com wrote: While stress testing Pavan's 2nd pass vacuum visibility patch, I realized that vacuum/visibility was busted. But it wasn't his patch that

Re: [HACKERS] split rm_name and rm_desc out of rmgr.c

2013-02-07 Thread Tom Lane
Peter Eisentraut pete...@gmx.net writes: On 2/5/13 3:47 PM, Alvaro Herrera wrote: Patch attached. This has broken cpluspluscheck: ./src/include/access/rmgrlist.h:28:8: error: expected constructor, destructor, or type conversion before '(' token cpluspluscheck has an explicit exclusion for

Re: [HACKERS] Vacuum/visibility is busted

2013-02-07 Thread Jeff Janes
On Thu, Feb 7, 2013 at 9:32 AM, Jeff Janes jeff.ja...@gmail.com wrote: On Thu, Feb 7, 2013 at 12:55 AM, Pavan Deolasee pavan.deola...@gmail.com wrote: Index scans do not return any duplicates and you need to force a seq scan to see them. Assuming that the page level VM bit might be

Re: [HACKERS] Vacuum/visibility is busted

2013-02-07 Thread Alvaro Herrera
Jeff Janes escribió: On Thu, Feb 7, 2013 at 12:55 AM, Pavan Deolasee pavan.deola...@gmail.com wrote: I don't see the assertion failure myself. If I do REINDEX INDEX it gives a duplicate key violation, and if I do REINDEX TABLE or REINDEX DATABASE I get claimed success. This is using

Re: [HACKERS] Vacuum/visibility is busted

2013-02-07 Thread Pavan Deolasee
On Thu, Feb 7, 2013 at 10:48 PM, Jeff Janes jeff.ja...@gmail.com wrote: On Thu, Feb 7, 2013 at 1:44 AM, Pavan Deolasee pavan.deola...@gmail.com wrote: jjanes=# select *, xmin, xmax, ctid from foo where index IN (select index from foo group by index having count(*) 1 ORDER by index) ORDER

Re: [HACKERS] Identity projection

2013-02-07 Thread Tom Lane
Amit Kapila amit.kap...@huawei.com writes: There can be 2 ways to remove result node a. Remove the Result plan node in case it is not required - This is same as currently it does for SubqueryScan. We can check if the result plan is trivial (with logic similar to trivial_subqueryscan()),

Re: [HACKERS] [COMMITTERS] pgsql: Fast promote mode skips checkpoint at end of recovery.

2013-02-07 Thread Simon Riggs
On 6 February 2013 17:43, Simon Riggs si...@2ndquadrant.com wrote: Here's what I think should be done: 1. Remove the check that prev checkpoint record exists. Agreed Done 2. Always do fast promotion if in standby mode. Remove the pg_ctl option. Disagreed, other viewpoints welcome.

Re: [HACKERS] Vacuum/visibility is busted

2013-02-07 Thread Jeff Janes
On Thu, Feb 7, 2013 at 10:09 AM, Pavan Deolasee pavan.deola...@gmail.com wrote: Right. I don't have the database handy at this moment, but earlier in the day I ran some queries against it and found that most of the duplicates which are not accessible via indexes have xmin very close to

Re: [HACKERS] Vacuum/visibility is busted

2013-02-07 Thread Andres Freund
On 2013-02-07 11:15:46 -0800, Jeff Janes wrote: On Thu, Feb 7, 2013 at 10:09 AM, Pavan Deolasee pavan.deola...@gmail.com wrote: Right. I don't have the database handy at this moment, but earlier in the day I ran some queries against it and found that most of the duplicates which are not

Re: [HACKERS] Vacuum/visibility is busted

2013-02-07 Thread Tom Lane
Jeff Janes jeff.ja...@gmail.com writes: Does anyone have suggestions on how to hack the system to make it fast-forward the current transaction id? What I've generally done is to stop the server then use pg_resetxlog to put the XID counter where I want it. I believe you'll need to manually

Re: [HACKERS] Vacuum/visibility is busted

2013-02-07 Thread Simon Riggs
On 7 February 2013 19:15, Jeff Janes jeff.ja...@gmail.com wrote: On Thu, Feb 7, 2013 at 10:09 AM, Pavan Deolasee pavan.deola...@gmail.com wrote: Right. I don't have the database handy at this moment, but earlier in the day I ran some queries against it and found that most of the duplicates

Re: [HACKERS] Vacuum/visibility is busted

2013-02-07 Thread Alvaro Herrera
Jeff Janes escribió: On Thu, Feb 7, 2013 at 10:09 AM, Pavan Deolasee pavan.deola...@gmail.com wrote: Right. I don't have the database handy at this moment, but earlier in the day I ran some queries against it and found that most of the duplicates which are not accessible via indexes have

[HACKERS] performance bug in instrument.c

2013-02-07 Thread Tomas Vondra
Hi, it seems there's a issue in instrument.c that may have significant performance impact for some plans when running explain analyze with TIMING OFF. There's this piece of code in InstrStartNode: if (instr-need_timer INSTR_TIME_IS_ZERO(instr-starttime))

Re: [HACKERS] sepgsql and materialized views

2013-02-07 Thread Kevin Grittner
Kohei KaiGai kai...@kaigai.gr.jp wrote: So, I'd like to review two options. 1) we uses db_table object class for materialized-views for a while, until selinux-side become ready. Probably, v9.3 will use db_table class then switched at v9.4. 2) we uses db_materialized_view object class from

Re: [HACKERS] performance bug in instrument.c

2013-02-07 Thread Tom Lane
Tomas Vondra t...@fuzzy.cz writes: There's this piece of code in InstrStartNode: if (instr-need_timer INSTR_TIME_IS_ZERO(instr-starttime)) INSTR_TIME_SET_CURRENT(instr-starttime); else elog(DEBUG2, InstrStartNode called twice in a row); but it should actually be

Re: [HACKERS] Considering Gerrit for CFs

2013-02-07 Thread Josh Berkus
Folks, First, thanks for the serious discussion of this. There are obvious tooling gaps (aren't there always?), but I don't really see the model as broken, and I don't think I've been around pgsql-hackers exclusively or extensively enough to have developed Stockholm syndrome. I don't see

Re: [HACKERS] issues with range types, btree_gist and constraints

2013-02-07 Thread Tomas Vondra
On 7.2.2013 01:10, Tom Lane wrote: The attached patch fixes these things, but not the buggy penalty function, because we ought to try to verify that these changes are enough to prevent creation of an incorrect index. I can't reproduce any misbehavior anymore with this patch applied. However,

Re: [HACKERS] performance bug in instrument.c

2013-02-07 Thread Tomas Vondra
Tom Lane t...@sss.pgh.pa.us wrote: Tomas Vondra tv(at)fuzzy(dot)cz writes: There's this piece of code in InstrStartNode: if (instr-need_timer INSTR_TIME_IS_ZERO(instr-starttime)) INSTR_TIME_SET_CURRENT(instr-starttime); else elog(DEBUG2, InstrStartNode called twice

Re: [HACKERS] performance bug in instrument.c

2013-02-07 Thread Tom Lane
Tomas Vondra t...@fuzzy.cz writes: Tom Lane t...@sss.pgh.pa.us wrote: A bigger question is why this is elog(DEBUG2) and not elog(ERROR). Had it been the latter, we'd have noticed the mistake immediately. The current choice might be masking any caller-logic errors that exist, too. Not sure

Re: [HACKERS] issues with range types, btree_gist and constraints

2013-02-07 Thread Tom Lane
Tomas Vondra t...@fuzzy.cz writes: I can't reproduce any of the issues anymore - I've tested both 9.2 and 9.3 branches (both containing the already commited fixes). And not only that the issues seem to be gone, but I'm actually getting significantly better performance. Cool. I noticed that

Re: [HACKERS] Vacuum/visibility is busted

2013-02-07 Thread Alvaro Herrera
Alvaro Herrera escribió: Alvaro Herrera escribió: Hm, if the foreign key patch is to blame, this sounds like these tuples had a different set of XMAX hint bits and a different Xmax, and they were clobbered by something like vacuum or page pruning. Hm, I think heap_freeze_tuple is

Re: [HACKERS] [JDBC] JPA + enum == Exception

2013-02-07 Thread Tom Dunstan
-Original Message- From: pgsql-jdbc-ow...@postgresql.org [mailto:pgsql-jdbc-ow...@postgresql.org] On Behalf Of Marc G. Fournier I'm trying to use enum's in a database, but the java guys are telling me that they are having problems with inserts ... reading from the database isn't a

Re: [HACKERS] Vacuum/visibility is busted

2013-02-07 Thread Tom Lane
Alvaro Herrera alvhe...@2ndquadrant.com writes: xid = HeapTupleHeaderGetRawXmax(tuple); ! if (((tuple-t_infomask HEAP_XMAX_IS_MULTI) ! MultiXactIdIsValid(xid) ! MultiXactIdPrecedes(xid, cutoff_multi)) || ! ((!(tuple-t_infomask

Re: [HACKERS] [JDBC] JPA + enum == Exception

2013-02-07 Thread Tom Lane
Tom Dunstan pg...@tomd.cc writes: ... That works ok, but when attempting to use a prepared statement: ps = con.prepareStatement(insert into enumcast values (?)); ps.setString(1, meh); ps.executeUpdate(); we get a org.postgresql.util.PSQLException: ERROR: column current_mood is

Re: [HACKERS] proposal: ANSI SQL 2011 syntax for named parameters

2013-02-07 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: On Thu, Feb 7, 2013 at 6:42 AM, Simon Riggs si...@2ndquadrant.com wrote: IMO the way to resolve that conflict is with a behaviour parameter to allow people to choose, rather than be forced to wait a year because some people still run an old version of

[HACKERS] Comment typo

2013-02-07 Thread Etsuro Fujita
I found a comment typo. Please find attached a patch. Best regards, Etsuro Fujita comment_typo.patch Description: Binary data -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

[HACKERS] Re[2]: [HACKERS] standby, pg_basebackup and last xlog file

2013-02-07 Thread Миша Тюрин
Hello all and Heikki personally Thank you for your answer I have some new points: 21.01.2013, 10:08 +02:00 от Heikki Linnakangas hlinnakan...@vmware.com: On 21.01.2013 09:14, Миша Тюрин wrote: Is there any reason why pg_basebackup has limitation in an online backup from the standby: The