Re: [HACKERS] Modify the DECLARE CURSOR command tag depending on the scrollable flag

2013-11-27 Thread Boszormenyi Zoltan
2013-11-27 20:49 keltezéssel, Alvaro Herrera írta: Boszormenyi Zoltan escribió: If you consider all these: - certain combinations of query and DECLARE stmt flags fail; - adding NO SCROLL is breaking backward compatibility; - the readahead code has to really know whether the cursor is scroll

Re: [HACKERS] Re: [BUGS] BUG #7873: pg_restore --clean tries to drop tables that don't exist

2013-11-27 Thread Pavel Stehule
Hello 2013/11/27 Tom Lane > Dean Rasheed writes: > > Actually the IF EXISTS in DROP TABLE now applies to the schema as > > well. Unfortunately there is currently no consistency across the > > various DROP commands --- some tolerate a non-existent schema, while > > others error out. > > Yeah.

Re: [HACKERS] GIN improvements part 1: additional information

2013-11-27 Thread Alexander Korotkov
On Wed, Nov 27, 2013 at 1:14 AM, Heikki Linnakangas wrote: > On 11/26/13 15:34, Alexander Korotkov wrote: > >> What's your plans about GIN now? I tried to rebase packed posting lists >> with head. But I found that you've changed interface of placeToPage >> function. That's conflicts with packed p

Re: [HACKERS] TODO: Split out pg_resetxlog output into pre- and post-sections

2013-11-27 Thread Rajeev rastogi
On 28 November 2013, Amit Kapila Wrote: > >> Further Review of this patch: > >> b. why to display 'First log segment after reset' in 'Currrent > >> pg_control values' section as now the same information > >> is available in new section "Values to be used after reset:" ? > > > > May not be alway

Re: [HACKERS] TODO: Split out pg_resetxlog output into pre- and post-sections

2013-11-27 Thread Amit Kapila
On Tue, Nov 26, 2013 at 5:33 PM, Rajeev rastogi wrote: > On 26 November 2013, Amit Kapila Wrote: >> Further Review of this patch: >> b. why to display 'First log segment after reset' in 'Currrent >> pg_control values' section as now the same information >> is available in new section "Values t

Re: [HACKERS] Get more from indices.

2013-11-27 Thread Etsuro Fujita
I wrote: > Kyotaro HORIGUCHI wrote: > > > * In get_relation_info(), the patch determines the branch condition > > > "keyattno != ObjectIdAttributeNumber". I fail to understand the > > > meaning of this branch condition. Could you explain about it? > > Literally answering, it means oid cannot be

Re: [HACKERS] Put json type into alphabetical order in manual table

2013-11-27 Thread Peter Eisentraut
Fixed. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] [GENERAL] pg_upgrade ?deficiency

2013-11-27 Thread Bruce Momjian
On Wed, Nov 27, 2013 at 03:36:12PM -0800, Kevin Grittner wrote: > Tom Lane wrote: > > > I'm inclined to agree with Kevin that this behavior is wrong and > > should be fixed (and back-patched), so far as pg_dumpall is concerned. > > pg_dumpall's charter is to be able to recreate a database cluster

Re: [HACKERS] [GENERAL] pg_upgrade ?deficiency

2013-11-27 Thread Bruce Momjian
On Wed, Nov 27, 2013 at 03:36:12PM -0800, Kevin Grittner wrote: > Of the people who posted on this thread supporting that, I think > Tom said it best: > > Tom Lane wrote: > > > I'm inclined to agree with Kevin that this behavior is wrong and > > should be fixed (and back-patched), so far as pg_d

Re: [HACKERS] Another bug introduced by fastpath patch

2013-11-27 Thread Tom Lane
I wrote: > We could still do this if we were willing to actually reject requests > for session level locks on fast-path-eligible lock types. At the moment > that costs us nothing really. If anyone ever did have a use case, we > could consider adding the extra logic to support it. Nope, that *sti

Re: [HACKERS] INSERT...ON DUPLICATE KEY LOCK FOR UPDATE

2013-11-27 Thread Peter Geoghegan
On Wed, Nov 27, 2013 at 1:09 AM, Peter Geoghegan wrote: > This, along with the already-discussed "attempted to update invisible > tuple" forms a full account of unexpected ERRORs seen during the > extended run of the test case, so far. Actually, it was slightly misleading of me to say it's the sa

[HACKERS] Proposed feature: Selective Foreign Keys

2013-11-27 Thread Tom Dunstan
Hi all! The Problem - One case that traditional SQL doesn't handle very well is when you have a child entity which can be attached to a number of different parent entities. Examples might be comments, tags or file attachments - we might have 20 different entities in the system t

Re: [HACKERS] doPickSplit stack buffer overflow in XLogInsert?

2013-11-27 Thread Peter Eisentraut
On 11/26/13, 5:14 PM, Kevin Grittner wrote: > I happened to build in a shell that was still set up for the clang > address sanitizer, and got the attached report. On a rerun it was > repeatable. XLogInsert() seems to read past the end of a variable > allocated on the stack in doPickSplit(). I hav

Re: [HACKERS] Incomplete freezing when truncating a relation during vacuum

2013-11-27 Thread Andres Freund
On 2013-11-27 18:18:02 -0500, Noah Misch wrote: > > I think the likelihood of the problem affecting !all-visible pages is > > close to zero. Each vacuum will try to clean those, so they surely will > > get vacuumed at some point. I think the only way that could happen is if > > the ConditionalLockB

Re: [HACKERS] Status of FDW pushdowns

2013-11-27 Thread Tom Lane
Dimitri Fontaine writes: > Atri Sharma writes: >> Can we add a function to the FDW API to define a SQL to foreign server >> side conversion? > I think that's not tenable. Even if you limit the discussion to the > postgres_fdw, some queries against past version will stop working > against new ver

Re: [HACKERS] Another bug introduced by fastpath patch

2013-11-27 Thread Tom Lane
Andres Freund writes: > On 2013-11-27 17:25:44 -0500, Tom Lane wrote: >> Or we >> could add a restriction to EligibleForRelationFastPath that restricts >> the fast-path mechanism to non-session locks, in which case we'd not >> need to make the zeroing contingent on allLocks either. I don't think

Re: [HACKERS] Status of FDW pushdowns

2013-11-27 Thread Dimitri Fontaine
Atri Sharma writes: > Can we add a function to the FDW API to define a SQL to foreign server > side conversion? I think that's not tenable. Even if you limit the discussion to the postgres_fdw, some queries against past version will stop working against new version (keywords changes, catalogs, de

Re: [HACKERS] MultiXact bugs

2013-11-27 Thread Andres Freund
On 2013-11-27 15:42:11 -0800, Kevin Grittner wrote: > Andres Freund wrote: > > What do you mean with "how far back"? > > What back-patching will be needed for a fix?  It sounds like 9.3? Yep. Greetings, Andres Freund -- Andres Freund http://www.2ndQuadrant.com/ PostgreS

Re: [HACKERS] MultiXact bugs

2013-11-27 Thread Kevin Grittner
Andres Freund wrote: > On 2013-11-27 15:14:11 -0800, Kevin Grittner wrote: >> ... however, I have not been able to trigger that Assert even with >> gdb breakpoints at what I think are the right spots.  Any >> suggestions?  How far back is it true that the above >> HeapTupleSatisfiesVacuum() can r

Re: [HACKERS] [GENERAL] pg_upgrade ?deficiency

2013-11-27 Thread Kevin Grittner
Bruce Momjian wrote: > On Wed, Nov 27, 2013 at 06:05:13AM -0800, Kevin Grittner wrote: >> Bruce Momjian wrote: >>> I am confused what we are patching.  Are we patching pg_dump, >>> pg_dumpall, or both? >> >> Just pg_dumpall.c. > > OK, there was a pg_dump patch earlier which we are not using now.

Re: [HACKERS] MultiXact pessmization in 9.3

2013-11-27 Thread Tom Lane
Andres Freund writes: > On 2013-11-27 19:24:35 -0300, Alvaro Herrera wrote: >> The other idea is to just not backpatch this. > I think backpatching is a good idea, I have seen GetMultiXactIdMembers() > + slru code take up 80% cpu in strange workloads. But it possibly might > be a good idea to wai

Re: [HACKERS] MultiXact bugs

2013-11-27 Thread Andres Freund
On 2013-11-27 15:14:11 -0800, Kevin Grittner wrote: > Andres Freund wrote: > > > HeapTupleHeaderGetUpdateXid() ignores aborted updaters > > and returns InvalidTransactionId in that case, but > > HeapTupleSatisfiesVacuum() returns HEAPTUPLE_DELETE_IN_PROGRESS... > > That sure *sounds* like it sho

Re: [HACKERS] Incomplete freezing when truncating a relation during vacuum

2013-11-27 Thread Noah Misch
On Wed, Nov 27, 2013 at 10:43:05PM +0100, Andres Freund wrote: > On 2013-11-27 14:53:27 -0500, Noah Misch wrote: > > How would you characterize the chances of this happening with default > > *vacuum_freeze_*_age settings? Offhand, it seems you would need to > > encounter > > this bug during each

Re: [HACKERS] Modify the DECLARE CURSOR command tag depending on the scrollable flag

2013-11-27 Thread Tom Lane
Peter Eisentraut writes: > On 11/27/13, 3:47 PM, Tom Lane wrote: >> Given these considerations, I think it'd be better to allow explicit >> application control over whether read-ahead happens for a particular >> query. And I have no problem whatsoever with requiring that the cursor >> be explicit

Re: [HACKERS] MultiXact bugs

2013-11-27 Thread Kevin Grittner
Andres Freund wrote: > HeapTupleHeaderGetUpdateXid() ignores aborted updaters > and returns InvalidTransactionId in that case, but > HeapTupleSatisfiesVacuum() returns HEAPTUPLE_DELETE_IN_PROGRESS... That sure *sounds* like it should cause a problem for this code in CheckForSerializableConflictO

Re: [HACKERS] Errors on missing pg_subtrans/ files with 9.3

2013-11-27 Thread Andres Freund
On 2013-11-27 13:57:52 -0300, Alvaro Herrera wrote: > Per bug report by J Smith in > cadfupgc5bmtv-yg9znxv-vcfkb+jprqs7m2oesqxam_4z1j...@mail.gmail.com > diagnosed by Andres Freund. Alvaro, do you see a way this could actually have caused J.'s problems? I thought about a few, but each

Re: [HACKERS] MultiXact pessmization in 9.3

2013-11-27 Thread Andres Freund
On 2013-11-27 19:24:35 -0300, Alvaro Herrera wrote: > One other thought is that MultiXactIdIsRunning and GetMultiXactIdMembers > are public functions, so this patch would represent an API change in > 9.3. I doubt any external modules would be relying on these functions, > but there's so many care

Re: [HACKERS] Another bug introduced by fastpath patch

2013-11-27 Thread Andres Freund
On 2013-11-27 17:25:44 -0500, Tom Lane wrote: > Or we > could add a restriction to EligibleForRelationFastPath that restricts > the fast-path mechanism to non-session locks, in which case we'd not > need to make the zeroing contingent on allLocks either. I don't think > we take any fast-path-eligi

Re: [HACKERS] [RFC] overflow checks optimized away

2013-11-27 Thread Bruce Momjian
On Mon, Sep 9, 2013 at 12:21:56PM -0400, Robert Haas wrote: > On Sat, Sep 7, 2013 at 6:55 PM, Greg Stark wrote: > >> Should these patches be applied? > > > > I have a copy of the program and was going to take care of this. > > When? 2.5 months later, status report? -- Bruce Momjian

Re: [HACKERS] Suggestion: Issue warning when calling SET TRANSACTION outside transaction block

2013-11-27 Thread Bruce Momjian
On Wed, Nov 27, 2013 at 04:44:02PM -0500, Bruce Momjian wrote: > I could live with this: > > Issuing ROLLBACK outside of a transaction > block has no effect except emitting a warning. Proposed doc patch attached. -- Bruce Momjian http://momjian.us EnterpriseDB

[HACKERS] Another bug introduced by fastpath patch

2013-11-27 Thread Tom Lane
In LockReleaseAll, we have this coding: for (partition = 0; partition < NUM_LOCK_PARTITIONS; partition++) { LWLockIdpartitionLock = FirstLockMgrLock + partition; SHM_QUEUE *procLocks = &(MyProc->myProcLocks[partition]); proclock = (PROCLOCK *) SHMQueueNext(pro

Re: [HACKERS] MultiXact pessmization in 9.3

2013-11-27 Thread Alvaro Herrera
Alvaro Herrera wrote: > Correct. The only difficulty here is that we would need to pass down > the fact that we know for certain that this is only a locking Multixact. > There are some callers that go to it indirectly via MultiXactIdWait or > MultiXactIdExpand, but now that I look I think it's fi

Re: [HACKERS] Suggestion: Issue warning when calling SET TRANSACTION outside transaction block

2013-11-27 Thread Bruce Momjian
On Wed, Nov 27, 2013 at 03:59:31PM -0500, Robert Haas wrote: > On Tue, Nov 26, 2013 at 5:02 PM, Bruce Momjian wrote: > > On Tue, Nov 26, 2013 at 01:58:04PM -0300, Alvaro Herrera wrote: > >> Bruce Momjian escribió: > >> > On Tue, Nov 26, 2013 at 11:22:39AM -0500, Tom Lane wrote: > >> > >> > > > Uh,

Re: [HACKERS] doPickSplit stack buffer overflow in XLogInsert?

2013-11-27 Thread Andres Freund
On 2013-11-27 15:29:24 -0500, Noah Misch wrote: > > If you are confident that neither of these is a real risk, I'll > > relax about this. > > If there is a real risk, I'm not seeing it. Me neither. Greetings, Andres Freund -- Andres Freund http://www.2ndQuadrant.com/ Pos

Re: [HACKERS] Incomplete freezing when truncating a relation during vacuum

2013-11-27 Thread Andres Freund
On 2013-11-27 14:53:27 -0500, Noah Misch wrote: > How would you characterize the chances of this happening with default > *vacuum_freeze_*_age settings? Offhand, it seems you would need to encounter > this bug during each of ~10 generations of autovacuum_freeze_max_age before > the old rows actual

Re: [HACKERS] [GENERAL] pg_upgrade ?deficiency

2013-11-27 Thread Bruce Momjian
On Wed, Nov 27, 2013 at 06:05:13AM -0800, Kevin Grittner wrote: > Bruce Momjian wrote: > > On Tue, Nov 26, 2013 at 03:25:44PM -0800, Kevin Grittner wrote: > >> Bruce Momjian wrote: > >> > >>> How are we handling breakage of pg_dump, not pg_dumpall? > >> > >> That was discussed.  Do you have somet

Re: [HACKERS] 9.2.1 & index-only scans : abnormal heap fetches after VACUUM FULL

2013-11-27 Thread Bruce Momjian
On Sat, Jan 12, 2013 at 02:14:03PM -0500, Kevin Grittner wrote: > Amit Kapila wrote: > > On Thursday, January 10, 2013 6:09 AM Josh Berkus wrote: > > >> Surely VACUUM FULL should rebuild the visibility map, and make > >> tuples in the new relation all-visible, no? > > Certainly it seems odd to me

Re: [HACKERS] Modify the DECLARE CURSOR command tag depending on the scrollable flag

2013-11-27 Thread Peter Eisentraut
On 11/27/13, 3:47 PM, Tom Lane wrote: > Given these considerations, I think it'd be better to allow explicit > application control over whether read-ahead happens for a particular > query. And I have no problem whatsoever with requiring that the cursor > be explicitly marked SCROLL or NO SCROLL be

Re: [HACKERS] Modify the DECLARE CURSOR command tag depending on the scrollable flag

2013-11-27 Thread Peter Eisentraut
On 11/27/13, 2:49 PM, Alvaro Herrera wrote: > Would it work to have a function of some sort to which you give a cursor > name and it returns whether it is scrollable or not? That might make sense. I think this case is similar to the question whether a view is updatable. You wouldn't put that inf

Re: [HACKERS] bytea_ouput = escape vs encode(byte, 'escape')

2013-11-27 Thread David Johnston
Jim Nasby-2 wrote > I'm wondering why bytes_output = escape produces different output than > encode(byte, 'escape') does. Is this intentional? If so, why? > > cnuapp_prod@postgres=# select e'\r'::bytea AS cr, e'\n'::bytea AS lf; > cr | lf > --+-- > \x0d | \x0a > (1 row) > > cnuapp_

Re: [HACKERS] Status of FDW pushdowns

2013-11-27 Thread David Fetter
On Thu, Nov 28, 2013 at 01:29:46AM +0530, Atri Sharma wrote: > On Thu, Nov 28, 2013 at 12:54 AM, David Fetter wrote: > > On Wed, Nov 27, 2013 at 10:29:34AM -0300, Alvaro Herrera wrote: > >> Shigeru Hanada escribió: > >> > >> > SQL/MED standard doesn't say much about PASS THROUGH mode, especially >

Re: [HACKERS] Suggestion: Issue warning when calling SET TRANSACTION outside transaction block

2013-11-27 Thread Robert Haas
On Tue, Nov 26, 2013 at 5:02 PM, Bruce Momjian wrote: > On Tue, Nov 26, 2013 at 01:58:04PM -0300, Alvaro Herrera wrote: >> Bruce Momjian escribió: >> > On Tue, Nov 26, 2013 at 11:22:39AM -0500, Tom Lane wrote: >> >> > > > Uh, I ended up mentioning "no effect" to highlight it does nothing, >> > > >

[HACKERS] bytea_ouput = escape vs encode(byte, 'escape')

2013-11-27 Thread Jim Nasby
I'm wondering why bytes_output = escape produces different output than encode(byte, 'escape') does. Is this intentional? If so, why? cnuapp_prod@postgres=# select e'\r'::bytea AS cr, e'\n'::bytea AS lf; cr | lf --+-- \x0d | \x0a (1 row) cnuapp_prod@postgres=# set bytea_output = es

Re: [HACKERS] Modify the DECLARE CURSOR command tag depending on the scrollable flag

2013-11-27 Thread Tom Lane
Boszormenyi Zoltan writes: > If you consider all these: > - certain combinations of query and DECLARE stmt flags fail; > - adding NO SCROLL is breaking backward compatibility; > - the readahead code has to really know whether the cursor is >scrollable so it can behave just like the server; I

Re: [HACKERS] Should we improve documentation on isolation levels?

2013-11-27 Thread Kevin Grittner
AK wrote: > the documentation is not incorrect, it is just incomplete. Yeah, that's exactly how I saw it.  :-) Docs changed on master only.  Thanks for the report! -- Kevin Grittner EDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company -- Sent via pgsql-hackers mailing list (

Re: [HACKERS] doPickSplit stack buffer overflow in XLogInsert?

2013-11-27 Thread Noah Misch
On Wed, Nov 27, 2013 at 11:38:23AM -0800, Kevin Grittner wrote: > Noah Misch wrote: > > The threat is that rounding the read size up to the next MAXALIGN > > would cross into an unreadable memory page, resulting in a > > SIGSEGV.  Every palloc chunk has MAXALIGN'd size under the hood, > > so the e

Re: [HACKERS] Status of FDW pushdowns

2013-11-27 Thread Atri Sharma
On Thu, Nov 28, 2013 at 12:54 AM, David Fetter wrote: > On Wed, Nov 27, 2013 at 10:29:34AM -0300, Alvaro Herrera wrote: >> Shigeru Hanada escribió: >> >> > SQL/MED standard doesn't say much about PASS THROUGH mode, especially >> > about interaction between client. Besides it, I think it would be

Re: [HACKERS] Should we improve documentation on isolation levels?

2013-11-27 Thread AK
I concur - the documentation is not incorrect, it is just incomplete. -- View this message in context: http://postgresql.1045698.n5.nabble.com/Should-we-improve-documentation-on-isolation-levels-tp5780629p5780636.html Sent from the PostgreSQL - hackers mailing list archive at Nabble.com. --

Re: [HACKERS] Incomplete freezing when truncating a relation during vacuum

2013-11-27 Thread Noah Misch
How would you characterize the chances of this happening with default *vacuum_freeze_*_age settings? Offhand, it seems you would need to encounter this bug during each of ~10 generations of autovacuum_freeze_max_age before the old rows actually become invisible. On Wed, Nov 27, 2013 at 02:14:53PM

Re: [HACKERS] Modify the DECLARE CURSOR command tag depending on the scrollable flag

2013-11-27 Thread Alvaro Herrera
Boszormenyi Zoltan escribió: > If you consider all these: > > - certain combinations of query and DECLARE stmt flags fail; > - adding NO SCROLL is breaking backward compatibility; > - the readahead code has to really know whether the cursor is > scrollable so it can behave just like the server;

Re: [HACKERS] Should we improve documentation on isolation levels?

2013-11-27 Thread Kevin Grittner
AK wrote: > The following is not precise: > > 13.2.1. Read Committed Isolation Level > > "Also note that two successive SELECT commands can see different data, even > though they are within a single transaction, if other transactions commit > changes during execution of the first SELECT." > > I t

Re: [HACKERS] Modify the DECLARE CURSOR command tag depending on the scrollable flag

2013-11-27 Thread Boszormenyi Zoltan
2013-11-27 19:16 keltezéssel, Tom Lane írta: Boszormenyi Zoltan writes: 2013-11-23 22:01 keltezéssel, Tom Lane írta: Boszormenyi Zoltan writes: Attached is the patch that modified the command tag returned by the DECLARE CURSOR command. It returns "DECLARE SCROLL CURSOR" or "DECLARE NO SCROLL

Re: [HACKERS] logical changeset generation v6.7

2013-11-27 Thread Fabrízio de Royes Mello
On Thu, Nov 14, 2013 at 11:46 AM, Andres Freund wrote: > > On 2013-11-12 18:50:33 +0100, Andres Freund wrote: > > > You've actually changed the meaning of this section (and not in a good way): > > > > > > be set at server start. wal_level must be set > > > -to archive or hot_stand

Re: [HACKERS] doPickSplit stack buffer overflow in XLogInsert?

2013-11-27 Thread Kevin Grittner
Noah Misch wrote: > (Kevin, I saw no attachment.) Apologies.  Trying again. > The threat is that rounding the read size up to the next MAXALIGN > would cross into an unreadable memory page, resulting in a > SIGSEGV.  Every palloc chunk has MAXALIGN'd size under the hood, > so the excess read of

[HACKERS] Should we improve documentation on isolation levels?

2013-11-27 Thread AK
I am not sure if i am posting in the right place - correct me if I am wrong. The following is not precise: 13.2.1. Read Committed Isolation Level "Also note that two successive SELECT commands can see different data, even though they are within a single transaction, if other transactions commit c

Re: [HACKERS] Status of FDW pushdowns

2013-11-27 Thread David Fetter
On Wed, Nov 27, 2013 at 10:29:34AM -0300, Alvaro Herrera wrote: > Shigeru Hanada escribió: > > > SQL/MED standard doesn't say much about PASS THROUGH mode, especially > > about interaction between client. Besides it, I think it would be > > nice to allow arbitrary FDW as backend of dblink interfa

Re: [HACKERS] INSERT...ON DUPLICATE KEY LOCK FOR UPDATE

2013-11-27 Thread Peter Geoghegan
On Wed, Nov 27, 2013 at 1:09 AM, Peter Geoghegan wrote: > Since it took me a relatively long time to recreate this, it may not > be trivial to do so. Unless you don't think it's useful to do so, I'm > going to give this test a full 24 hours, just in case it shows up > anything else like this. I s

Re: [HACKERS] Modify the DECLARE CURSOR command tag depending on the scrollable flag

2013-11-27 Thread Tom Lane
Boszormenyi Zoltan writes: > 2013-11-23 22:01 keltezéssel, Tom Lane írta: >> Boszormenyi Zoltan writes: >>> Attached is the patch that modified the command tag returned by >>> the DECLARE CURSOR command. It returns "DECLARE SCROLL CURSOR" >>> or "DECLARE NO SCROLL CURSOR" depending on the cursor'

Re: [HACKERS] Platform-dependent(?) failure in timeout handling

2013-11-27 Thread Tom Lane
Kevin Grittner writes: > Tom Lane wrote: >> 3. Establish a coding rule that if you catch an error with >> PG_TRY() and don't re-throw it, you have to unblock signals in >> your PG_CATCH block. > Could that be done in the PG_END_TRY macro? Interesting idea ... [ thinks for a bit ... ] but I'm no

Re: [HACKERS] Handling GIN incomplete splits

2013-11-27 Thread Heikki Linnakangas
On 11/22/13 15:04, Michael Paquier wrote: 2) post recovery cleanup: - OK, so roughly the soul of this patch is to change the update mechanism for a left child gin page so as the parent split is always done first before any new data is inserted in this child. And this ensures that we can remove th

Re: [HACKERS] Handling GIN incomplete splits

2013-11-27 Thread Jeff Janes
On Wed, Nov 13, 2013 at 8:49 AM, Heikki Linnakangas wrote: > Here's another part of my crusade against xlog cleanup routines. This > series of patches gets rid of the gin_cleanup() function, which is > currently used to finish splits of GIN b-tree pages, if the system crashes > (or an error occur

Re: [HACKERS] Status of FDW pushdowns

2013-11-27 Thread Atri Sharma
On Wed, Nov 27, 2013 at 11:08 PM, Dimitri Fontaine wrote: > Atri Sharma writes: >> This would work,but how can we do it for FDWs which do not parse SQL? >> Am I missing something here? > > Worst case: > > CREATE FOREIGN VIEW foo > AS $$ > whatever syntax is accepted on the other side Th

Re: [HACKERS] Status of FDW pushdowns

2013-11-27 Thread Dimitri Fontaine
Atri Sharma writes: > This would work,but how can we do it for FDWs which do not parse SQL? > Am I missing something here? Worst case: CREATE FOREIGN VIEW foo AS $$ whatever syntax is accepted on the other side $$; Regards, -- Dimitri Fontaine http://2ndQuadrant.fr PostgreSQL : E

Re: [HACKERS] Status of FDW pushdowns

2013-11-27 Thread Atri Sharma
> I guess the view query would have to be validated by the FDW, which > would just receive a text. +1 This is not exactly in context, but I and David Fetter discussed recently how we could do similar thing. This would work,but how can we do it for FDWs which do not parse SQL? Am I missing somet

Re: [HACKERS] lock on object is already held

2013-11-27 Thread Tom Lane
Daniel Wood writes: > Does the original version of my stress test not repro the problem on 9.2? [ tries it ... ] No, it doesn't, or at least the MTBF is a couple orders of magnitude better than on 9.3. Another odd thing (seen with my short version as well as your original) is that 9.3/HEAD run

Re: [HACKERS] Status of FDW pushdowns

2013-11-27 Thread Dimitri Fontaine
Shigeru Hanada writes: > I'm sorry but I don't see the point here. Do you mean that user > executes CREATE FOREIGN VIEW in advance and uses the view in a Yes that's what I mean. > I think it's nice to support executing ad-hoc remote query written in > the syntax which is valid only on remote da

Re: [HACKERS] Errors on missing pg_subtrans/ files with 9.3

2013-11-27 Thread Alvaro Herrera
Alvaro Herrera escribió: > Andres Freund escribió: > This seems simple to handle by adding the check you propose to the loop. > Basically if the xmax doesn't match the xmin, we reached the end, > there's nothing more to lock and we can return success without any > further work: As mentioned in th

Re: [HACKERS] doPickSplit stack buffer overflow in XLogInsert?

2013-11-27 Thread Noah Misch
On Wed, Nov 27, 2013 at 06:23:38AM -0800, Kevin Grittner wrote: > Andres Freund wrote: > > On 2013-11-26 14:14:38 -0800, Kevin Grittner wrote: > > > >> I happened to build in a shell that was still set up for the clang > >> address sanitizer, and got the attached report.  On a rerun it was (Kevin

Re: [HACKERS] Modify the DECLARE CURSOR command tag depending on the scrollable flag

2013-11-27 Thread Boszormenyi Zoltan
2013-11-23 22:01 keltezéssel, Tom Lane írta: Boszormenyi Zoltan writes: Attached is the patch that modified the command tag returned by the DECLARE CURSOR command. It returns "DECLARE SCROLL CURSOR" or "DECLARE NO SCROLL CURSOR" depending on the cursor's scrollable flag that can be determined i

Re: [HACKERS] Status of FDW pushdowns

2013-11-27 Thread Shigeru Hanada
2013/11/27 Dimitri Fontaine : > Alvaro Herrera writes: >> Seems to me that if you want to read remote tables without creating a >> foreign table, you could define them locally using something like the >> WITH syntax and then use them normally in the rest of the query. > > I guess what's needed her

Re: [HACKERS] Status of FDW pushdowns

2013-11-27 Thread Merlin Moncure
On Wed, Nov 27, 2013 at 4:20 AM, Shigeru Hanada wrote: > Hi Merlin, > > 2013/11/22 Merlin Moncure : >> On Thu, Nov 21, 2013 at 6:43 PM, Shigeru Hanada >> wrote: >>> 2013/11/22 Tom Lane : Merlin Moncure writes: > On Thu, Nov 21, 2013 at 9:05 AM, Bruce Momjian wrote: >> I know join p

Re: [HACKERS] Name type in postgres

2013-11-27 Thread Tom Lane
mohsen soodkhah mohammadi writes: > I want do understand that did can we have name without null-terminator? No. Possibly that was the original intention, but for many years the system has enforced that Name contains at least one trailing null byte, ie the maximum usable length of identifiers is

[HACKERS] [HITB-Announce] #HITB2014AMS Call for Papers Now Open

2013-11-27 Thread Hafez Kamal
Hi everyone - The Call for Papers for the 5th annual HITB Security Conference in Amsterdam is now open. #HITB2014AMS takes place at the Beurs van Berlage from the 27th - 30th of May 2014. The official conference hotel for the event is the Hilton DoubleTree. As always we start with 2-days of hands

Re: [HACKERS] Performance Improvement by reducing WAL for Update Operation

2013-11-27 Thread Amit Kapila
On Wed, Nov 27, 2013 at 7:35 PM, Robert Haas wrote: > On Wed, Nov 27, 2013 at 12:56 AM, Amit Kapila wrote: >>> The basic idea is that you use a rolling hash function to divide up >>> the history data into chunks of a given average size. So we scan the >>> history data, compute a rolling hash val

Re: [HACKERS] Platform-dependent(?) failure in timeout handling

2013-11-27 Thread Kevin Grittner
Tom Lane wrote: > 3. Establish a coding rule that if you catch an error with > PG_TRY() and don't re-throw it, you have to unblock signals in > your PG_CATCH block. Could that be done in the PG_END_TRY macro? -- Kevin Grittner EDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company

Re: [HACKERS] doPickSplit stack buffer overflow in XLogInsert?

2013-11-27 Thread Kevin Grittner
Andres Freund wrote: > On 2013-11-26 14:14:38 -0800, Kevin Grittner wrote: > >> I happened to build in a shell that was still set up for the clang >> address sanitizer, and got the attached report.  On a rerun it was >> repeatable.  XLogInsert() seems to read past the end of a variable >> allocate

Re: [HACKERS] Extension Templates S03E11

2013-11-27 Thread Vik Fearing
On 11/26/2013 10:07 PM, Dimitri Fontaine wrote: > What I now think we should do is only grant superusers the privileges to > install an extension from a template they own or is owned by another > superuser. Say what? Superusers bypass all privileges by definition. -- Vik -- Sent via pgsql-h

Re: [HACKERS] new unicode table border styles for psql

2013-11-27 Thread Pavel Stehule
2013/11/26 Pavel Stehule > > > > 2013/11/26 Peter Eisentraut > >> On 11/22/13, 3:26 AM, Pavel Stehule wrote: >> > website is related to patch for 9.3 (I add note there) >> > >> > patch for 9.4 is fixed - and now with small doc >> >> I think it would help if we considered the new border styles an

Re: [HACKERS] Shave a few instructions from child-process startup sequence

2013-11-27 Thread Robert Haas
On Tue, Nov 26, 2013 at 10:12 PM, Gurjeet Singh wrote: > On Tue, Nov 26, 2013 at 9:42 PM, Peter Eisentraut wrote: >> >> src/backend/postmaster/postmaster.c:2255: indent with spaces. >> +else >> src/backend/postmaster/postmaster.c:2267: indent with spaces. >> +break > > > Not s

Re: [HACKERS] [GENERAL] pg_upgrade ?deficiency

2013-11-27 Thread Kevin Grittner
Bruce Momjian wrote: > On Tue, Nov 26, 2013 at 03:25:44PM -0800, Kevin Grittner wrote: >> Bruce Momjian wrote: >> >>> How are we handling breakage of pg_dump, not pg_dumpall? >> >> That was discussed.  Do you have something to add? > > I am confused what we are patching.  Are we patching pg_dump,

Re: [HACKERS] Performance Improvement by reducing WAL for Update Operation

2013-11-27 Thread Robert Haas
On Wed, Nov 27, 2013 at 12:56 AM, Amit Kapila wrote: >> - There is a comment "TODO: It would be nice to behave like the >> history and the source strings were concatenated, so that you could >> compress using the new data, too." If we're not already doing that, >> then how are we managing to comp

Re: [HACKERS] Status of FDW pushdowns

2013-11-27 Thread Dimitri Fontaine
Alvaro Herrera writes: > Seems to me that if you want to read remote tables without creating a > foreign table, you could define them locally using something like the > WITH syntax and then use them normally in the rest of the query. I guess what's needed here is a kind of barrier that allows pus

Re: [HACKERS] Handling GIN incomplete splits

2013-11-27 Thread Heikki Linnakangas
On 11/22/13 15:04, Michael Paquier wrote: On Thu, Nov 21, 2013 at 9:44 PM, Heikki Linnakangas wrote: Here's a new version. To ease the review, I split the remaining patch again into two, where the first patch is just yet more refactoring. I also fixed some bugs in WAL logging and replay that I

Re: [HACKERS] Traffic jams in fn_extra

2013-11-27 Thread Greg Stark
On Sun, Nov 24, 2013 at 4:21 PM, Simon Riggs wrote: > Why do you need to do this dance with fn_extra? > > It's possible to allocate a hash table in a Transaction-lifetime > memory context on first call into a function then cache things there. > fn_extra gives a handle per function call site. It

Re: [HACKERS] Status of FDW pushdowns

2013-11-27 Thread Alvaro Herrera
Shigeru Hanada escribió: > SQL/MED standard doesn't say much about PASS THROUGH mode, especially > about interaction between client. Besides it, I think it would be > nice to allow arbitrary FDW as backend of dblink interface like this: > > postgres=> SELECT dblink_connect('con1', 'server name o

Re: [HACKERS] Incomplete freezing when truncating a relation during vacuum

2013-11-27 Thread Andres Freund
On 2013-11-27 14:45:25 +0200, Heikki Linnakangas wrote: > On 11/27/13 14:11, Andres Freund wrote: > >I don't think this warning is likely to be hit as the code stands - > >heap_page_prune() et. al. will have removed all dead tuples already, > >right and so has_dead_tuples won't be set. > > It migh

Re: [HACKERS] Incomplete freezing when truncating a relation during vacuum

2013-11-27 Thread Heikki Linnakangas
On 11/27/13 14:11, Andres Freund wrote: On 2013-11-27 13:56:58 +0200, Heikki Linnakangas wrote: Afaics the likelihood of the issue occuring on non-all-visible pages is pretty low, since they'd need to be skipped due to lock contention repeatedly. Hmm. If a page has its visibility-map flag set

Re: [HACKERS] Re: [BUGS] BUG #7873: pg_restore --clean tries to drop tables that don't exist

2013-11-27 Thread Pavel Stehule
I'll prepare patch 2013/11/27 Tom Lane > Dean Rasheed writes: > > Actually the IF EXISTS in DROP TABLE now applies to the schema as > > well. Unfortunately there is currently no consistency across the > > various DROP commands --- some tolerate a non-existent schema, while > > others error out

Re: [HACKERS] PL/Python: domain over array support

2013-11-27 Thread Heikki Linnakangas
On 11/27/13 14:15, Marko Kreen wrote: On Tue, Nov 26, 2013 at 07:12:00PM -0200, Rodolfo Campero wrote: 2013/11/26 Heikki Linnakangas Oops, sorry about that. Fixed. Maybe be you forgot to modify plpython_types_3.out Yes. Heikki, please fix plpython_types_3.out too. See attached patch. A

Re: [HACKERS] Incomplete freezing when truncating a relation during vacuum

2013-11-27 Thread Andres Freund
On 2013-11-27 13:56:58 +0200, Heikki Linnakangas wrote: > Ok, committed and backpatched that. Thanks. > >I wonder if we need to integrate any mitigating logic? Currently the > >corruption may only become apparent long after it occurred, that's > >pretty bad. And instructing people run a vacuum af

Re: [HACKERS] PL/Python: domain over array support

2013-11-27 Thread Marko Kreen
On Tue, Nov 26, 2013 at 07:12:00PM -0200, Rodolfo Campero wrote: > 2013/11/26 Heikki Linnakangas > > Oops, sorry about that. Fixed. > > Maybe be you forgot to modify > plpython_types_3.out Yes. Heikki, please fix plpython_types_3.out too. See attached patch. -- marko diff --git a/src/pl/plp

Re: [HACKERS] Incomplete freezing when truncating a relation during vacuum

2013-11-27 Thread Heikki Linnakangas
On 11/27/13 11:15, Andres Freund wrote: On 2013-11-27 11:01:55 +0200, Heikki Linnakangas wrote: On 11/27/13 01:21, Andres Freund wrote: On 2013-11-26 13:32:44 +0100, Andres Freund wrote: This seems to be the case since b4b6923e03f4d29636a94f6f4cc2f5cf6298b8c8. I suggest we go back to using sca

Re: [HACKERS] [PATCH] Add transforms feature

2013-11-27 Thread Hannu Krosing
On 11/15/2013 05:04 PM, Dimitri Fontaine wrote: > Hi, > > Peter Eisentraut writes: >> Rebased patch. No changes except that merge conflicts were resolved, >> and I had to add some Data::Dumper tweaks to the regression tests so >> that the results came out in consistent order on different version

Re: [HACKERS] PostgreSQL Service on Windows does not start. ~ "is not a valid Win32 application"

2013-11-27 Thread Rajeev rastogi
On 27 November 2013, Naoya Anzai wrote: > Hi, Rajeev > > > > I tested the latest patch. My observation is: > > > If we give relative data directory path while registering the > > > service, then service start fails. > > > But same works if the data directory is absolute path. > > > > > > Loo

Re: [HACKERS] Get more from indices.

2013-11-27 Thread Etsuro Fujita
Kyotaro HORIGUCHI wrote: > > * In get_relation_info(), the patch determines the branch condition > > "keyattno != ObjectIdAttributeNumber". I fail to understand the > > meaning of this branch condition. Could you explain about it? > Literally answering, it means oid cannot be NULL (if it exists)

Re: [HACKERS] Status of FDW pushdowns

2013-11-27 Thread Shigeru Hanada
Hi Merlin, 2013/11/22 Merlin Moncure : > On Thu, Nov 21, 2013 at 6:43 PM, Shigeru Hanada > wrote: >> 2013/11/22 Tom Lane : >>> Merlin Moncure writes: On Thu, Nov 21, 2013 at 9:05 AM, Bruce Momjian wrote: > I know join pushdowns seem insignificant, but it helps to restrict what > da

Re: [HACKERS] Assertions in PL/PgSQL

2013-11-27 Thread Pavel Stehule
2013/11/27 Peter Eisentraut > On Tue, 2013-11-19 at 10:40 -0500, Robert Haas wrote: > > I think the goal was to get to RAISE ASSERT > > WHEN ...; then, if assertions are off, you do nothing; if they're on, > > you error. IF condition THEN RAISE..." isn't a suitable surrogate in > > that case bec

[HACKERS] Name type in postgres

2013-11-27 Thread mohsen soodkhah mohammadi
hello. I want do understand that did can we have name without null-terminator? in src/backend/util/adt/name.c in nameout() function is: Names = PG_GETARG_NAME(0); PG_RETURN_CSTRING(pstrdup(NameStr(*s))); what do the pstrdup() function? do this function create string from name with null-term

Re: [HACKERS] INSERT...ON DUPLICATE KEY LOCK FOR UPDATE

2013-11-27 Thread Peter Geoghegan
On Wed, Nov 27, 2013 at 1:20 AM, Andres Freund wrote: > Even the "deadlock detected" errors might be a fkey-locking issue. Bug > #8434, but that's really hard to know without more details. Thanks, I was aware of that but didn't make the connection. I've written a test-case that is designed to ex

Re: [HACKERS] PostgreSQL Service on Windows does not start. ~ "is not a valid Win32 application"

2013-11-27 Thread Naoya Anzai
Hi, Rajeev > > I tested the latest patch. My observation is: > > If we give relative data directory path while registering the > > service, then service start fails. > > But same works if the data directory is absolute path. > > > > Looks like an existing issue. May be we need to inte

  1   2   >