Re: [HACKERS] Minor optimizations in lazy_scan_heap

On Tue, Dec 4, 2012 at 11:02 PM, Robert Haas wrote: > On Mon, Dec 3, 2012 at 1:23 AM, Pavan Deolasee > > > > Comments ? Anyone thinks any/all of above is useful ? > > I doubt that any of these things make enough difference to be worth > bothering with, You're right. These are not big ticket op

Re: [HACKERS] Review: Patch to compute Max LSN of Data Pages

Amit kapila writes: > On Friday, December 07, 2012 7:43 PM Muhammad Usama wrote: >> Although I am thinking why are you disallowing the absolute path of file. >> Any particular reason? > The reason to disallow absolute path is that, we need to test on multiple > platforms and to keep the scope l

Re: [HACKERS] Review: Patch to compute Max LSN of Data Pages

Hi Muhammad, On Friday, December 07, 2012 7:43 PM Muhammad Usama wrote: >Hi Amit On Mon, Dec 3, 2012 at 6:56 PM, Amit kapila mailto:amit.kap...@huawei.com>> wrote: >>I think we should expect provided path to be relative to current directory >> or may consider it to be relative to either one of D

Re: [HACKERS] Serious problem: media recovery fails after system or PostgreSQL crash

On Thu, Dec 6, 2012 at 3:52 PM, Tomas Vondra wrote: > Hi, > > On 6.12.2012 23:45, MauMau wrote: >> From: "Tom Lane" >>> Well, that's unfortunate, but it's not clear that automatic recovery is >>> possible. The only way out of it would be if an undamaged copy of the >>> segment was in pg_xlog/ ..

Re: [HACKERS] Commits 8de72b and 5457a1 (COPY FREEZE)

Jeff, * Jeff Davis (pg...@j-davis.com) wrote: > Most of your concerns seem to be related to freezing, and I'm mostly > interested in HEAP_XMIN_COMMITTED optimizations. So I think we're > talking past each other. My concern is about this patch/capability as a whole. I agree that the hint bit sett

Re: [HACKERS] Commits 8de72b and 5457a1 (COPY FREEZE)

On Thu, 2012-12-06 at 22:34 -0500, Stephen Frost wrote: > * Jeff Davis (pg...@j-davis.com) wrote: > > That is documented in the committed patch -- it's a trade, basically > > saying that you lose isolation but avoid extra writes. It seems > > reasonable that the user gets this behavior if specifica

Re: [HACKERS] ALTER TABLE ... NOREWRITE option

Robert Haas writes: > is needed there, or we may need to move some things from the exec > phase to the prep phase to make it all work out. I think it's totally > doable, but it's not going to be a 50-line patch. If you want to work on it, please be my guest :) Regards, -- Dimitri Fontaine http

Re: [HACKERS] pg_upgrade problem with invalid indexes

On Fri, Dec 7, 2012 at 10:38:39PM +0100, Andres Freund wrote: > On 2012-12-07 16:30:36 -0500, Bruce Momjian wrote: > > On Fri, Dec 7, 2012 at 04:21:48PM -0500, Tom Lane wrote: > > > Andres Freund writes: > > > > On 2012-12-07 13:59:41 -0500, Tom Lane wrote: > > > >> indisvalid should be sufficie

Re: [HACKERS] pg_upgrade problem with invalid indexes

On 2012-12-07 16:30:36 -0500, Bruce Momjian wrote: > On Fri, Dec 7, 2012 at 04:21:48PM -0500, Tom Lane wrote: > > Andres Freund writes: > > > On 2012-12-07 13:59:41 -0500, Tom Lane wrote: > > >> indisvalid should be sufficient. If you try to test more than that > > >> you're going to make the co

Re: [HACKERS] pg_upgrade problem with invalid indexes

Bruce Momjian writes: > On Fri, Dec 7, 2012 at 04:21:48PM -0500, Tom Lane wrote: >> Andres Freund writes: >>> Doesn't the check need to be at least indisvalid && indisready? Given >>> that 9.2 represents !indislive as indisvalid && !indisready? >> Um, good point. It's annoying that we had to d

Re: [HACKERS] pg_upgrade problem with invalid indexes

On Fri, Dec 7, 2012 at 04:21:48PM -0500, Tom Lane wrote: > Andres Freund writes: > > On 2012-12-07 13:59:41 -0500, Tom Lane wrote: > >> indisvalid should be sufficient. If you try to test more than that > >> you're going to make the code more version-specific, without actually > >> buying much.

Re: [HACKERS] pg_upgrade problem with invalid indexes

Andres Freund writes: > On 2012-12-07 13:59:41 -0500, Tom Lane wrote: >> indisvalid should be sufficient. If you try to test more than that >> you're going to make the code more version-specific, without actually >> buying much. > Doesn't the check need to be at least indisvalid && indisready? G

Re: [HACKERS] User control over psql error stream

On 11/21/2012 11:33:18 PM, Karl O. Pinc wrote: > On 11/21/2012 01:41:56 PM, Peter Eisentraut wrote: > > On 11/15/12 3:53 PM, Karl O. Pinc wrote: > > > This patch gives the end user control over psql's > > > error stream. This allows a single psql session > > > to use \o to send both errors and tab

Re: [HACKERS] pg_upgrade problem with invalid indexes

On Fri, Dec 7, 2012 at 11:57:34AM -0500, Bruce Momjian wrote: > On Fri, Dec 7, 2012 at 11:46:51AM -0500, Tom Lane wrote: > > Bruce Momjian writes: > > > On Fri, Dec 7, 2012 at 10:29:22AM -0500, Tom Lane wrote: > > >> On balance I am coming around to support the "just throw an error if > > >> th

Re: [HACKERS] pg_upgrade problem with invalid indexes

On 2012-12-07 13:59:41 -0500, Tom Lane wrote: > Bruce Momjian writes: > > On Fri, Dec 7, 2012 at 07:49:14PM +0100, Andres Freund wrote: > >> !indislive indexes are created during DROP INDEX CONCURRENTLY. Thats a > >> different case than CREATE INDEX CONCURRENTLY. Accessing their > >> definition i

Re: [HACKERS] pg_upgrade problem with invalid indexes

On 2012-12-07 13:54:44 -0500, Tom Lane wrote: > Andres Freund writes: > > !indislive indexes are created during DROP INDEX CONCURRENTLY. Thats a > > different case than CREATE INDEX CONCURRENTLY. Accessing their > > definition is actually problematic because i can vanish while youre > > examing it

Re: [HACKERS] pg_upgrade problem with invalid indexes

Bruce Momjian writes: > On Fri, Dec 7, 2012 at 07:49:14PM +0100, Andres Freund wrote: >> !indislive indexes are created during DROP INDEX CONCURRENTLY. Thats a >> different case than CREATE INDEX CONCURRENTLY. Accessing their >> definition is actually problematic because i can vanish while youre

Re: [HACKERS] pg_upgrade problem with invalid indexes

On Fri, Dec 7, 2012 at 07:49:14PM +0100, Andres Freund wrote: > On 2012-12-07 10:44:53 -0800, Josh Berkus wrote: > > > > > I wonder though if we shouldn't ignore !indislive indexes in pg_dump > > > (and the respective bw-compat hack). > > Obviously I wanted to ask whether we *should* ignore them

Re: [HACKERS] pg_upgrade problem with invalid indexes

Andres Freund writes: > !indislive indexes are created during DROP INDEX CONCURRENTLY. Thats a > different case than CREATE INDEX CONCURRENTLY. Accessing their > definition is actually problematic because i can vanish while youre > examing it which could cause errors both in the backend and in pg_

Re: [HACKERS] pg_upgrade problem with invalid indexes

On 2012-12-07 10:44:53 -0800, Josh Berkus wrote: > > > I wonder though if we shouldn't ignore !indislive indexes in pg_dump > > (and the respective bw-compat hack). Obviously I wanted to ask whether we *should* ignore them in the future. > Quite likely we shouldn't. However, that is why it wasn'

Re: [HACKERS] pg_upgrade problem with invalid indexes

> I wonder though if we shouldn't ignore !indislive indexes in pg_dump > (and the respective bw-compat hack). Quite likely we shouldn't. However, that is why it wasn't considered a problem. -- Josh Berkus PostgreSQL Experts Inc. http://pgexperts.com -- Sent via pgsql-hackers mailing list (p

Re: [HACKERS] Review of Row Level Security

On 5 December 2012 11:16, Kohei KaiGai wrote: >> Oracle defaults to putting VPD on all event types: INSERT, UPDATE, >> DELETE, SELECT. ISTM we should be doing the same, not just say "we can >> add an INSERT trigger if you want". >> >> Adding a trigger just begs the question as to why we are bothe

Re: [HACKERS] Review of Row Level Security

On 5 December 2012 11:16, Kohei KaiGai wrote: >> * TRUNCATE works, and allows you to remove all rows of a table, even >> ones you can't see to run a DELETE on. Er... >> > It was my oversight. My preference is to rewrite TRUNCATE command > with DELETE statement in case when row-security policy is

Re: [HACKERS] pg_upgrade problem with invalid indexes

On 2012-12-07 10:22:12 -0800, Josh Berkus wrote: > > > Yes, I thought of not dumping it. The problem is that we don't delete > > the index when it fails, so I assumed we didn't want to lose the index > > creation information. I need to understand why we did that. Why do we > > have pg_dump dump

Re: [HACKERS] pg_upgrade problem with invalid indexes

> Yes, I thought of not dumping it. The problem is that we don't delete > the index when it fails, so I assumed we didn't want to lose the index > creation information. I need to understand why we did that. Why do we > have pg_dump dump the index then? Because pg_restore will recreate the inde

Re: [HACKERS] Support for REINDEX CONCURRENTLY

On 7 December 2012 17:19, Andres Freund wrote: > On 2012-12-07 12:01:52 -0500, Tom Lane wrote: >> Simon Riggs writes: >> > On 7 December 2012 12:37, Michael Paquier >> > wrote: >> >> - There is still a problem with toast indexes. If the concurrent reindex >> >> of >> >> a toast index fails for

Re: [HACKERS] Proof of concept: auto updatable views [Review of Patch]

[ finally getting back to this patch, after many distractions ] Dean Rasheed writes: > On 8 November 2012 21:13, Tom Lane wrote: >> I think the most reasonable backwards-compatibility argument is that we >> shouldn't change the behavior if there are either INSTEAD rules or >> INSTEAD triggers.

Re: [HACKERS] Fix for pg_upgrade status display

On Wed, Dec 5, 2012 at 10:04:53PM -0500, Bruce Momjian wrote: > Pg_upgrade displays file names during copy and database names during > dump/restore. Andrew Dunstan identified three bugs: > > * long file names were being truncated to 60 _leading_ characters, which > often do not change for long

Re: [HACKERS] Support for REINDEX CONCURRENTLY

On 2012-12-07 12:01:52 -0500, Tom Lane wrote: > Simon Riggs writes: > > On 7 December 2012 12:37, Michael Paquier wrote: > >> - There is still a problem with toast indexes. If the concurrent reindex of > >> a toast index fails for a reason or another, pg_relation will finish with > >> invalid toa

Re: [HACKERS] Support for REINDEX CONCURRENTLY

Simon Riggs writes: > On 7 December 2012 12:37, Michael Paquier wrote: >> - There is still a problem with toast indexes. If the concurrent reindex of >> a toast index fails for a reason or another, pg_relation will finish with >> invalid toast index entries. I am still wondering about how to clea

Re: [HACKERS] pg_upgrade problem with invalid indexes

On Fri, Dec 7, 2012 at 11:46:51AM -0500, Tom Lane wrote: > Bruce Momjian writes: > > On Fri, Dec 7, 2012 at 10:29:22AM -0500, Tom Lane wrote: > >> On balance I am coming around to support the "just throw an error if > >> there are any invalid indexes" position. Adding extra complication in > >>

Re: [HACKERS] pg_upgrade problem with invalid indexes

Bruce Momjian writes: > On Fri, Dec 7, 2012 at 10:29:22AM -0500, Tom Lane wrote: >> On balance I am coming around to support the "just throw an error if >> there are any invalid indexes" position. Adding extra complication in >> pg_dump and pg_upgrade to handle ignoring them doesn't seem like a

Re: [HACKERS] ALTER TABLE ... NOREWRITE option

On Thu, Dec 6, 2012 at 3:34 PM, Dimitri Fontaine wrote: > I think we need to be solving that problem when we implement new firing > points for event trigger. The 'table_rewrite' event needs to fire at a > time when the code can cope with it. That's the main difficulty in > adding events in that sy

Re: [HACKERS] Commits 8de72b and 5457a1 (COPY FREEZE)

On 4 December 2012 20:38, Jeff Davis wrote: > Even if it is, I would > think that a sufficiently narrow case -- such as CTAS outside of a > transaction block -- would be safe CTAS would be safe to do that with. CLUSTER and VACUUM FULL are already done. Outside of a transaction block would be aut

Re: [HACKERS] pg_upgrade problem with invalid indexes

On Fri, Dec 7, 2012 at 10:29:22AM -0500, Tom Lane wrote: > Bruce Momjian writes: > > On Fri, Dec 7, 2012 at 03:32:51PM +0100, Andres Freund wrote: > >> ISTM that anybody who does DDL during or after pg_upgrade --check > >> deserves any pain. > >> > >> So throwing an error in both seems perfectl

Re: [HACKERS] Support for REINDEX CONCURRENTLY

On 2012-12-07 21:37:06 +0900, Michael Paquier wrote: > Hi all, > > Long time this thread has not been updated... > Please find attached the version 3 of the patch for support of REINDEX > CONCURRENTLY. > The code has been realigned with master up to commit da07a1e (6th December). > > Here are the t

Re: [HACKERS] Switching timeline over streaming replication

On 06.12.2012 15:39, Amit Kapila wrote: On Thursday, December 06, 2012 12:53 AM Heikki Linnakangas wrote: On 05.12.2012 14:32, Amit Kapila wrote: On Tuesday, December 04, 2012 10:01 PM Heikki Linnakangas wrote: After some diversions to fix bugs and refactor existing code, I've committed a coup

Re: [HACKERS] pg_upgrade problem with invalid indexes

Bruce Momjian writes: > On Fri, Dec 7, 2012 at 03:32:51PM +0100, Andres Freund wrote: >> ISTM that anybody who does DDL during or after pg_upgrade --check >> deserves any pain. >> >> So throwing an error in both seems perfectly fine for me. I agree with Andres on this. > I am just saying that

Re: [HACKERS] pg_upgrade problem with invalid indexes

On Fri, Dec 7, 2012 at 03:32:51PM +0100, Andres Freund wrote: > > Well, it is a CONCURRENT index creation, so locking would be minimal. > > I wouldn't call a ShareUpdateExclusiveLock minimal... > > > Do we want pg_upgrade to be groveling over the lock view to look for > > locks? I don't think s

Re: [HACKERS] pg_upgrade problem with invalid indexes

On 2012-12-07 09:27:39 -0500, Bruce Momjian wrote: > On Fri, Dec 7, 2012 at 03:24:46PM +0100, Andres Freund wrote: > > On 2012-12-07 09:21:58 -0500, Bruce Momjian wrote: > > > On Thu, Dec 6, 2012 at 10:27:21PM -0500, Stephen Frost wrote: > > > > * Bruce Momjian (br...@momjian.us) wrote: > > > > >

Re: [HACKERS] pg_upgrade problem with invalid indexes

On Fri, Dec 7, 2012 at 03:24:46PM +0100, Andres Freund wrote: > On 2012-12-07 09:21:58 -0500, Bruce Momjian wrote: > > On Thu, Dec 6, 2012 at 10:27:21PM -0500, Stephen Frost wrote: > > > * Bruce Momjian (br...@momjian.us) wrote: > > > > On Thu, Dec 6, 2012 at 09:45:11PM -0500, Stephen Frost wrot

Re: [HACKERS] WIP patch for hint bit i/o mitigation

On Thu, Dec 6, 2012 at 8:52 PM, Merlin Moncure wrote: >Thanks for that -- that's fairly comprehensive I'd say. I'm quite >interested in that benchmarking framework as well. Do you need help >setting up the scripts? Presently I am testing with pgbench custom query option & taking IO & VM statist

Re: [HACKERS] pg_upgrade problem with invalid indexes

On 2012-12-07 09:21:58 -0500, Bruce Momjian wrote: > On Thu, Dec 6, 2012 at 10:27:21PM -0500, Stephen Frost wrote: > > * Bruce Momjian (br...@momjian.us) wrote: > > > On Thu, Dec 6, 2012 at 09:45:11PM -0500, Stephen Frost wrote: > > > > Or preserve it as-is. I don't really like the 'make them fi

Re: [HACKERS] Serious problem: media recovery fails after system or PostgreSQL crash

Tomas wrote: So why don't you use an archive command that does not create such incomplete files? I mean something like this: archive_command = 'cp %p /arch/%f.tmp && mv /arch/%f.tmp /arch/%f' Until the file is renamed, it's considered 'incomplete'. That is a good idea. This would surely sol

Re: [HACKERS] pg_upgrade problem with invalid indexes

On Fri, Dec 7, 2012 at 09:16:56AM +, Simon Riggs wrote: > On 7 December 2012 04:02, Alvaro Herrera wrote: > > Andrew Dunstan wrote: > >> > >> On 12/06/2012 09:23 PM, Bruce Momjian wrote: > > > >> >As soon as pg_dump stopped dumping the CREATE INDEX, pg_upgrade would > >> >stop creating creati

Re: [HACKERS] pg_upgrade problem with invalid indexes

On Thu, Dec 6, 2012 at 10:27:21PM -0500, Stephen Frost wrote: > * Bruce Momjian (br...@momjian.us) wrote: > > On Thu, Dec 6, 2012 at 09:45:11PM -0500, Stephen Frost wrote: > > > Or preserve it as-is. I don't really like the 'make them fix it' > > > option, as a user could run into that in the mi

Re: [HACKERS] Review: Patch to compute Max LSN of Data Pages

Hi Amit On Mon, Dec 3, 2012 at 6:56 PM, Amit kapila wrote: > >I think we should expect provided path to be relative to current > directory > > or may consider it to be relative to either one of Data or CWD. > >Because normally we expect path to be relative to CWD if some program is > > asking f

Re: [HACKERS] pg_upgrade problem with invalid indexes

Bruce Momjian wrote: > On Thu, Dec 6, 2012 at 09:23:14PM -0500, Bruce Momjian wrote: > > On Thu, Dec 6, 2012 at 09:10:21PM -0500, Tom Lane wrote: > > > Bruce Momjian writes: > > > > On Thu, Dec 6, 2012 at 07:53:57PM -0500, Tom Lane wrote: > > > >> Because CREATE INDEX CONCURRENTLY can't drop th

Re: [HACKERS] pg_upgrade problem with invalid indexes

On Thu, Dec 6, 2012 at 09:23:14PM -0500, Bruce Momjian wrote: > On Thu, Dec 6, 2012 at 09:10:21PM -0500, Tom Lane wrote: > > Bruce Momjian writes: > > > On Thu, Dec 6, 2012 at 07:53:57PM -0500, Tom Lane wrote: > > >> Because CREATE INDEX CONCURRENTLY can't drop the index if it's already > > >>

Re: [HACKERS] Support for REINDEX CONCURRENTLY

On 7 December 2012 12:37, Michael Paquier wrote: > There are still two things that are missing in this patch, but I would like > to have more feedback before moving forward: > - REINDEX CONCURRENTLY needs tests in src/test/isolation Yes, it needs those > - There is still a problem with toast in

Re: [HACKERS] Setting visibility map in VACUUM's second phase

On Fri, Dec 7, 2012 at 9:21 AM, Pavan Deolasee wrote: > On Thu, Dec 6, 2012 at 11:31 PM, Tom Lane wrote: > >> >> I think taking a second whack at setting the visibility bit is a fine >> idea, but let's drop all the rest of this premature optimization. >> > > Fair enough. I thought about doing it

Re: [HACKERS] -DCLOBBER_CACHE_ALWAYS shows COPY FREEZE regression problem

On 7 December 2012 00:13, Tom Lane wrote: > Andrew Dunstan writes: >> On a new buildfarm member friarbird >> , >> configured with _DCLOBBER_CACHE_ALWAYS: > >>BEGIN; >>TRUNCATE vistest; >>

Re: [HACKERS] [v9.3] writable foreign tables

Kohei KaiGai wrote: > The attached patch is revised version. > > One most difference from the previous version is, it constructed > PoC features on Hanada-san's latest postgres-fdw.v5.patch. > Yesh, it looks to me working fine on RDBMS backend also. Cool. > Even though the filename of this patch

Re: [HACKERS] pg_upgrade problem with invalid indexes

On 7 December 2012 04:02, Alvaro Herrera wrote: > Andrew Dunstan wrote: >> >> On 12/06/2012 09:23 PM, Bruce Momjian wrote: > >> >As soon as pg_dump stopped dumping the CREATE INDEX, pg_upgrade would >> >stop creating creating it in the new cluster, and not transfer the index >> >files. >> >> So we

Re: [HACKERS] Performance Improvement by reducing WAL for Update Operation

Hello, I looked into the patch and have some comments. >From the restriction of the time for this rather big patch, please excuse that these comments are on a part of it. Others will follow in few days. heaptuple.c noncachegetattr(_with_len): - att_getlength should do strlen as worst cas