Re: [HACKERS] COMMIT NOWAIT Performance Option

2007-02-27 Thread Josh Berkus
Tom, > One of the things that's really attractive about the proposed mode is > that it does *not* create a risk of data corruption Oh, ok. That wasn't how I understood Simon's case. > I agree that we ought to look at some performance numbers before > accepting the patch, but I think Josh's a

Re: [HACKERS] COMMIT NOWAIT Performance Option

2007-02-27 Thread Joshua D. Drake
Gregory Stark wrote: > "Joshua D. Drake" <[EMAIL PROTECTED]> writes: > On 2/27/07, Josh Berkus wrote: > I see no reason to implement it if there is no performance gain. > >> However, I strongly concur that we need at least some evidence. It could >> easily be that a misstep in the code,

Re: [HACKERS] Resumable vacuum proposal and design overview

2007-02-27 Thread Galy Lee
Tom Lane wrote: >One problem with it is that a too-small target would result in vacuum >proceeding to scan indexes after having accumulated only a few dead >tuples, resulting in increases (potentially enormous ones) in the total >work needed to vacuum the table completely. Yeah. This is also my bi

Re: [HACKERS] COMMIT NOWAIT Performance Option

2007-02-27 Thread Bruce Momjian
Joshua D. Drake wrote: > Gregory Stark wrote: > > "Joshua D. Drake" <[EMAIL PROTECTED]> writes: > > > On 2/27/07, Josh Berkus wrote: > > I see no reason to implement it if there is no performance gain. > > > >> However, I strongly concur that we need at least some evidence. It could > >

Re: [HACKERS] COMMIT NOWAIT Performance Option

2007-02-27 Thread Josh Berkus
All, > > But we do don't we? fsync = off, full_page_writes = off? BTW, our testing seems to indicate that full_page_writes = off is safe on Solaris 10 on good hardware. At least, we haven't been able to break it yet. -- Josh Berkus PostgreSQL @ Sun San Francisco ---(

Re: [HACKERS] COMMIT NOWAIT Performance Option

2007-02-27 Thread Jeff Davis
On Tue, 2007-02-27 at 17:40 -0800, Josh Berkus wrote: > All, > > > > But we do don't we? fsync = off, full_page_writes = off? > > BTW, our testing seems to indicate that full_page_writes = off is safe on > Solaris 10 on good hardware. At least, we haven't been able to break it yet. > Is that

Re: [HACKERS] COMMIT NOWAIT Performance Option

2007-02-27 Thread Josh Berkus
Jeff, > Is that an OS-dependent parameter? I always assumed it depended entirely > on hardware. I have no way to test it for myself though, so I just leave > full_page_writes=on to be safe. It's a question of whether your HW+OS can guarentee no torn page writes for the xlog. Running on Sun hard

Re: [HACKERS] COMMIT NOWAIT Performance Option

2007-02-27 Thread Gregory Stark
"Josh Berkus" writes: > OK. I've seen no performance numbers yet though. It just seems to me that > any performance patch proposal should start a discussion of what amount of > performance we expect to gain. There exist proposals that can be prototyped and measured to see what potential they

Re: [HACKERS] [PATCHES]

2007-02-27 Thread Neil Conway
On Tue, 2007-02-27 at 16:20 -0800, Joshua D. Drake wrote: > Thus we may literally not have rights to the code. Do you really want to > go down the path of in 2 years, Fujitsu (No offense Fujitsu), but you > are the topic) decides that the code they provided is owned by them and > they didn't give u

Re: [HACKERS] autovacuum next steps, take 2

2007-02-27 Thread Galy Lee
Tom Lane wrote: > Saving the array is > expensive both in runtime and code complexity, and I don't believe we > can trust it later --- at least not without even more expensive-and- > complex measures, such as WAL-logging every such save :-( I don’t understand well the things you are worrying about

Re: [HACKERS] COMMIT NOWAIT Performance Option

2007-02-27 Thread Joshua D. Drake
Bruce Momjian wrote: > Joshua D. Drake wrote: >> Gregory Stark wrote: >>> "Joshua D. Drake" <[EMAIL PROTECTED]> writes: >>> >> On 2/27/07, Josh Berkus wrote: >>> I see no reason to implement it if there is no performance gain. However, I strongly concur that we need at least some evid

Re: [HACKERS] COMMIT NOWAIT Performance Option

2007-02-27 Thread Gregory Stark
"Josh Berkus" writes: > It's a question of whether your HW+OS can guarentee no torn page writes for > the xlog. no, the data files. torn pages in the xlog is also a problem but we protect ourselves with a CRC and stop replay if it the CRC doesn't match. So the cost there is a bit of cpu, not

Re: [HACKERS] [PATCHES]

2007-02-27 Thread Bruce Momjian
Neil Conway wrote: > On Tue, 2007-02-27 at 16:20 -0800, Joshua D. Drake wrote: > > Thus we may literally not have rights to the code. Do you really want to > > go down the path of in 2 years, Fujitsu (No offense Fujitsu), but you > > are the topic) decides that the code they provided is owned by th

Re: [HACKERS] COMMIT NOWAIT Performance Option

2007-02-27 Thread Jonah H. Harris
On 2/27/07, Gregory Stark <[EMAIL PROTECTED]> wrote: I suggested a while back implementing torn page detection by writing a sequential number ever 512 bytes in the blocks. (I was talking about WAL at the time but the same principle applies.) Do it at the smgr layer using readv/writev and the uppe

Re: [HACKERS] Resumable vacuum proposal and design overview

2007-02-27 Thread Tom Lane
Galy Lee <[EMAIL PROTECTED]> writes: > Tom Lane wrote: >> ... or set a flag to stop at the next cycle-completion point. > The extra cost to clean indexes may prevent this approach to work in > practices. Huh? There is no extra cost in what I suggested; it'll perform exactly the same number of in

Re: [HACKERS] Proposal for Implenting read-only queries during wal replay (SoC 2007)

2007-02-27 Thread Paul Silveira
Hello, I just wanted to voice my opinion for this feature... I've implemented a few Production applicaitons with PostgreSQL now and would die for that feature. Right now, I am constantly trying to find way's to make my data more available. I've even resulted to using pg_dump to create read onl

Re: [HACKERS] Resumable vacuum proposal and design overview

2007-02-27 Thread Galy Lee
Tom Lane wrote: > Huh? There is no extra cost in what I suggested; it'll perform > exactly the same number of index scans that it would do anyway. The things I wanted to say is that: If we can stop at any point, we can make maintenance memory large sufficient to contain all of the dead tuples,

Re: [HACKERS] Proposal for Implenting read-only queries during wal replay (SoC 2007)

2007-02-27 Thread Joshua D. Drake
Paul Silveira wrote: > Hello, > > I just wanted to voice my opinion for this feature... I've implemented a > few Production applicaitons with PostgreSQL now and would die for that > feature. Right now, I am constantly trying to find way's to make my data > more available. Paul unfortunately yo

Re: [HACKERS] Resumable vacuum proposal and design overview

2007-02-27 Thread Tom Lane
Galy Lee <[EMAIL PROTECTED]> writes: > If we can stop at any point, we can make maintenance memory large > sufficient to contain all of the dead tuples, then we only need to > clean index for once. No matter how many times vacuum stops, > indexes are cleaned for once. I beg your pardon? You're th

Re: [HACKERS] 5 Weeks till feature freeze or (do you know where your patch is?)

2007-02-27 Thread Henry B. Hotz
On Feb 23, 2007, at 1:24 PM, Joshua D. Drake wrote: Henry Hotz: GSSAPI (with Magnus) Progressing. Had hoped to have alpha patches by March 1, but I just got handed a proposal that I have to do by then. I trust it's OK to send the first version in next week? No real issues, except I ha

Re: [HACKERS] [COMMITTERS] pgsql: Update Solaris FAQ.

2007-02-27 Thread Henry B. Hotz
Would you like a krb5.h file for Solaris 9-10 that will allow you to "break into" the "hidden" library? Also S10u4 won't be out until this summer. I'd say the answer is currently "no". It's known by Sun that Postgres will compile against their Kerberos libraries though. On Feb 23, 2007,

Re: [HACKERS] 5 Weeks till feature freeze or (do you know where your patch is?)

2007-02-27 Thread Joshua D. Drake
Henry B. Hotz wrote: > > On Feb 23, 2007, at 1:24 PM, Joshua D. Drake wrote: > >> Henry Hotz: GSSAPI (with Magnus) > > Progressing. Had hoped to have alpha patches by March 1, but I just got > handed a proposal that I have to do by then. I trust it's OK to send > the first version in next week

Re: [HACKERS] Dead Space Map version 2

2007-02-27 Thread Tom Lane
Heikki Linnakangas <[EMAIL PROTECTED]> writes: > Tom Lane wrote: >> I'd be happier if the DSM content could be >> treated as just a hint. > If we don't have a frozen state, we can't use the DSM to implement > index-only scans. To implement index-only scans, the DSM would have to be expected to p

Re: [HACKERS] 5 Weeks till feature freeze or (do you know where your patch is?)

2007-02-27 Thread Kris Jurka
On Tue, 27 Feb 2007, Henry B. Hotz wrote: Question: are there any corresponding deadlines for the Java client code that I need to worry about? The JDBC driver will release a new version at the same time as the server, but we don't have nearly as strict rules about feature freeze/beta. We

Re: [HACKERS] Packed short varlenas, what next?

2007-02-27 Thread Tom Lane
I wrote: > Gregory Stark <[EMAIL PROTECTED]> writes: >> I'm really curious to know how people feel about the varlena patch. > One thing I think we could do immediately is apply the change to replace > "VARATT_SIZEP(x) = len" with "SET_VARSIZE(x, len)" --- that would > considerably reduce the size

Re: [HACKERS] COMMIT NOWAIT Performance Option

2007-02-27 Thread Jeroen T. Vermeulen
On Wed, February 28, 2007 06:59, Jim C. Nasby wrote: > On Tue, Feb 27, 2007 at 05:18:28PM -0500, Bruce Momjian wrote: >> I think we will remove fsync in favor of the new delay, and allow -1 to >> be the same behavior as fsync off. > > Well, presumably we'd still allow fsync for some number of vers

Re: [HACKERS] Dead Space Map version 2

2007-02-27 Thread ITAGAKI Takahiro
"Jim C. Nasby" <[EMAIL PROTECTED]> wrote: > > I'd like to add some kind of logical flavors to max_fsm_pages > > and max_dsm_pages. > > In the meantime, I'm not sure if it makes sense to tie the FSM size to > the DSM size, since each FSM page requires 48x the storage of a DSM > page. I think there

Re: [HACKERS] SCMS question

2007-02-27 Thread James Cloos
> "Warren" == Warren Turkal <[EMAIL PROTECTED]> writes: Warren> Is it possible to obtain a mirror of the CVS repository? I use CVSup to locally mirror the repo. They've had the repo available via CVSup for some years now. I use this .cvsup file: ,(/mirror/CvsUp/Postgresql/pgsql.cvsup)

Re: [HACKERS] Dead Space Map version 2

2007-02-27 Thread ITAGAKI Takahiro
"Jim C. Nasby" <[EMAIL PROTECTED]> wrote: > At some point it might make sense to convert the FSM into a bitmap; that > way everything just scales with database size. > In the meantime, I'm not sure if it makes sense to tie the FSM size to > the DSM size, since each FSM page requires 48x the stora

Re: [HACKERS] high CPU usage for stats collector in 8.2

2007-02-27 Thread Joshua D. Drake
Darcy Buskermolen wrote: > I'm observing high CPU usage (95%) of a 2.6GHz opteron by the stats collector > on an 8.2.3 box investigation has lead me to belive that the stats file is > written a lot more often that once every 500ms the following shows this > behavior. Any thoughts on the below

Re: [HACKERS] [PATCHES] - WIP Patch Updatable Cursor

2007-02-27 Thread John Bartlett
Hi, A list of ctids is stored in the file. The file is used to store the ctids during an updatable cursor transaction. It is set up as a permanent file as it has a potential lifetime of preserving data between crashes of the backend. Temporary files tend to be used for data that is defined wit

Re: [HACKERS] [PATCHES] WIP Patch - Updateable Cursors

2007-02-27 Thread FAST PostgreSQL
On Wed, 28 Feb 2007 09:48, Bruce Momjian wrote: [Added a subejct line] > FYI, I am not going to be comfortable accepting a final patch that > contains this email signature: > > This is an email from Fujitsu Australia Software Technology Pty Ltd, ABN > 27 003 693 481. It is confidentia

Re: [HACKERS] [PATCHES] - WIP Patch Updatable Cursor

2007-02-27 Thread Gavin Sherry
On Wed, 28 Feb 2007, John Bartlett wrote: > Hi, > > A list of ctids is stored in the file. I would have thought these would be stored in memory. If the set got large, you'd use a temporary file the way other systems which overflow to disk do? > > The file is used to store the ctids during an upd

Re: [HACKERS] [PATCHES]

2007-02-27 Thread Tom Lane
Bruce Momjian <[EMAIL PROTECTED]> writes: > Neil Conway wrote: >> For the case in question, sure, requiring some clarification from FJ >> would be reasonable. But more broadly, my point is that I think you're >> fooling yourself if you think that requiring a disclaimer or explicit >> transfer of co

Re: [HACKERS] [PATCHES]

2007-02-27 Thread Joshua D. Drake
>> Yes, I do. If there is an explicit claim, like an email footer or a >> copyright in the code, we do try to nail that down. > > AFAICT, the footer in question tries to make it illegal for us even to > have the message in our mail archives. If I were running the PG lists, > I would install fil

Re: [HACKERS] [PATCHES]

2007-02-27 Thread Tom Lane
"Joshua D. Drake" <[EMAIL PROTECTED]> writes: > ... In regards to your idea of a filter, there is no reason why > we couldn't install a filter that checks for signatures with specific > legal words and strips said signature automatically, responding to the > sender that we did so. The problem is t

Re: [HACKERS] [PATCHES]

2007-02-27 Thread Joshua D. Drake
> Not that I think that anyone owning both a law degree and a computer > in 2007 should legitimately be able to plead innocence here. FAST > Australia's lawyers are making themselves look like idiots, and the > same for every other company tacking on such notices. I think the > real bottom line

Re: [HACKERS] [PATCHES]

2007-02-27 Thread Stefan Kaltenbrunner
Tom Lane wrote: > "Joshua D. Drake" <[EMAIL PROTECTED]> writes: >> ... In regards to your idea of a filter, there is no reason why >> we couldn't install a filter that checks for signatures with specific >> legal words and strips said signature automatically, responding to the >> sender that we did

Re: [HACKERS] [PATCHES]

2007-02-27 Thread Tom Lane
"Joshua D. Drake" <[EMAIL PROTECTED]> writes: > Well the problem is, it isn't the guy that sent the patch that is the > idiot. That guys has zero control over the matter, the signature is > going to be tacked on at the MTA level. Sure, I know that and you know that. The problem we have to worry a

<    1   2