Re: [PATCHES] [HACKERS] Load distributed checkpoint

2007-02-27 Thread ITAGAKI Takahiro
Inaam Rana [EMAIL PROTECTED] wrote: One of the issues we had during testing with original patch was db stop not working properly. I think you coded something to do a stop checkpoint in immediately but if a checkpoint is already in progress at that time, it would take its own time to

Re: [HACKERS] Dead Space Map version 2

2007-02-27 Thread Simon Riggs
On Tue, 2007-02-27 at 12:05 +0900, ITAGAKI Takahiro wrote: If we combine this with the HOT patch, pages with HOT tuples are probably marked as UNFROZEN because we don't bother vacuuming HOT tuples. They can be removed incrementally and doesn't require explicit vacuums. Perhaps avoid DSM

Re: [HACKERS] Dead Space Map version 2

2007-02-27 Thread Simon Riggs
On Tue, 2007-02-27 at 00:55 -0500, Tom Lane wrote: Jim C. Nasby [EMAIL PROTECTED] writes: Yes, DSM would make FSM recovery more important, but I thought it was recoverable now? Or is that only on a clean shutdown? Currently we throw away FSM during any non-clean restart. This is probably

Re: [HACKERS] Dead Space Map version 2

2007-02-27 Thread ITAGAKI Takahiro
Jim C. Nasby [EMAIL PROTECTED] wrote: If we do UPDATE a tuple, the original page containing the tuple is marked as HIGH and the new page where the updated tuple is placed is marked as LOW. Don't you mean UNFROZEN? No, the new tuples are marked as LOW. I intend to use UNFROZEN and FROZEN

Re: [HACKERS] Dead Space Map version 2

2007-02-27 Thread ITAGAKI Takahiro
Tom Lane [EMAIL PROTECTED] wrote: Vacuum for XID wraparound would have to hit every page regardless. There is one problem at this point. If we want to guarantee that there are no tuples that XIDs are older than pg_class.relfrozenxid, we must scan all pages for XID wraparound for every vacuums.

Re: [HACKERS] Seeking Google SoC Mentors

2007-02-27 Thread Dave Page
Tom Lane wrote: Andrew Dunstan [EMAIL PROTECTED] writes: Well, here's a question. Given the recent discussion re full disjunction, I'd like to know what sort of commitment we are going to give people who work on proposed projects. Um, if you mean are we going to promise to accept a patch

Re: [HACKERS] Dead Space Map version 2

2007-02-27 Thread ITAGAKI Takahiro
Simon Riggs [EMAIL PROTECTED] wrote: If we combine this with the HOT patch, pages with HOT tuples are probably marked as UNFROZEN because we don't bother vacuuming HOT tuples. They can be removed incrementally and doesn't require explicit vacuums. Perhaps avoid DSM entries for HOT

Re: [HACKERS] Seeking Google SoC Mentors

2007-02-27 Thread Magnus Hagander
On Tue, Feb 27, 2007 at 09:21:42AM +, Dave Page wrote: Tom Lane wrote: Andrew Dunstan [EMAIL PROTECTED] writes: Well, here's a question. Given the recent discussion re full disjunction, I'd like to know what sort of commitment we are going to give people who work on proposed

Re: [HACKERS] Dead Space Map version 2

2007-02-27 Thread Heikki Linnakangas
Tom Lane wrote: The main problem with the levels proposed by Takahiro-san is that any transition from FROZEN to not-FROZEN *must* be exactly recovered, because vacuum will never visit an allegedly frozen page at all. This appears to require WAL-logging DSM state changes, which is a pretty

Re: [HACKERS] Seeking Google SoC Mentors

2007-02-27 Thread Dave Page
Magnus Hagander wrote: On Tue, Feb 27, 2007 at 09:21:42AM +, Dave Page wrote: Tom Lane wrote: Andrew Dunstan [EMAIL PROTECTED] writes: Well, here's a question. Given the recent discussion re full disjunction, I'd like to know what sort of commitment we are going to give people who work

Re: [HACKERS] Seeking Google SoC Mentors

2007-02-27 Thread Magnus Hagander
On Tue, Feb 27, 2007 at 09:53:41AM +, Dave Page wrote: Magnus Hagander wrote: On Tue, Feb 27, 2007 at 09:21:42AM +, Dave Page wrote: Tom Lane wrote: Andrew Dunstan [EMAIL PROTECTED] writes: Well, here's a question. Given the recent discussion re full disjunction, I'd like to

Re: [HACKERS] Seeking Google SoC Mentors

2007-02-27 Thread Dave Page
Magnus Hagander wrote: Right. We'll just have to live by Googles rule for that part, I'm talking about the discussions later. Once things are approved, they should all be handled on the standard mailinglists, IMHO. Oh, 100% agreed. Being able to make possibly controversial suggestiosn public

Re: [HACKERS] COMMIT NOWAIT Performance Option

2007-02-27 Thread Simon Riggs
On Mon, 2007-02-26 at 23:04 -0500, Tom Lane wrote: Simon Riggs [EMAIL PROTECTED] writes: COMMIT NOWAIT can co-exist with the normal form of COMMIT and does not threaten the consistency or robustness of other COMMIT modes. Read that again and think about it, before we go further, please.

Re: [HACKERS] [PATCHES]

2007-02-27 Thread Heikki Linnakangas
John Bartlett wrote: The community may wish to comment on the following issue: 1)At present the file that will contain the list of ctids is going into a new directory called pg_ctids, analogous to pg_twophase, and also stored in the pg_data directory. I don't understand this. What's

Re: [HACKERS] Developer TODO List as a PostgreSQL DB

2007-02-27 Thread Chad Wagner
On 2/26/07, Josh Berkus josh@agliodbs.com wrote: Just wondering after reading so many mails from Hackers List.(its 2.15AM now!!) Is there anybody working on something to create a DB from a) The TODO list http://www.postgresql.org/docs/faqs.TODO.html b) The sourcecode of PostgreSQL c) The

Re: [HACKERS] Compile libpq for pg8.2.3 with vc7

2007-02-27 Thread Magnus Hagander
On Mon, Feb 26, 2007 at 02:23:58PM -0500, Jeff McKenna wrote: Trying to compile 8.2.3 with VC 7.10.3077 (2003) on Win32, I get the following error: mypath\postgresql-8.2.3\src\include\c.h(88) : fatal error C1083: Cannot open include file: 'pg_config_os.h': No such file or directory

[HACKERS] Packed short varlenas, what next?

2007-02-27 Thread Gregory Stark
I'm really curious to know how people feel about the varlena patch. In particular I know these issues may elicit comment: 1) Do we really need a special case for little-endian machines? I think it would be trivial to add but having two code paths may be annoying to maintain. The flip side

Re: [HACKERS] Developer TODO List as a PostgreSQL DB

2007-02-27 Thread Devrim GÜNDÜZ
On Tue, 2007-02-27 at 08:41 -0500, Chad Wagner wrote: The TODO list is a bit outdated, Really? and it would make sense to make it into a web page and assign names to the tasks and which releases the TODO task will make it in. Perhaps it would even be useful for folks to post their WIP

Re: [HACKERS] Developer TODO List as a PostgreSQL DB

2007-02-27 Thread Andrew Dunstan
Chad Wagner wrote: On 2/26/07, *Josh Berkus* josh@agliodbs.com mailto:josh@agliodbs.com wrote: Just wondering after reading so many mails from Hackers List.(its 2.15 AM now!!) Is there anybody working on something to create a DB from a) The TODO list

Re: [HACKERS] Expanding DELETE/UPDATE returning

2007-02-27 Thread Florian G. Pflug
David Fetter wrote: On Mon, Feb 26, 2007 at 11:14:01PM -0500, Tom Lane wrote: Rusty Conover [EMAIL PROTECTED] writes: Or allow delete and update to be used in sub-queries? That's been discussed but the implementation effort seems far from trivial. One big problem is that a sub-query can

Re: [HACKERS] Proposal for Implenting read-only queries during wal replay (SoC 2007)

2007-02-27 Thread Florian G. Pflug
Merlin Moncure wrote: getting back on topic (ahem), florian: are you comfortable going ahead with this? is there anything you need help with? I'm currently updating my proposal, trying to incorporate the points people brought up in this thread. I realized that trying to use the same kind of

Re: [HACKERS] Developer TODO List as a PostgreSQL DB

2007-02-27 Thread Chad Wagner
On 2/27/07, Andrew Dunstan [EMAIL PROTECTED] wrote: Before we rehash recent debates, please, everybody, review them. Going over and over and over the same ground laboriously is really getting tiresome, and unfortunately it's also getting more frequent. It's *déjà vu *all over again. History

Re: [HACKERS] Packed short varlenas, what next?

2007-02-27 Thread Peter Eisentraut
Gregory Stark wrote: I'm really curious to know how people feel about the varlena patch. As I has mentioned earlier, I'm missing a plan to allow 8-byte varlena sizes. -- Peter Eisentraut http://developer.postgresql.org/~petere/ ---(end of

Re: [HACKERS] Developer TODO List as a PostgreSQL DB

2007-02-27 Thread Andrew Dunstan
Chad Wagner wrote: There is some point in putting it in a wiki where we can gather relevant documents, links to discussions etc. That's why the developers' wiki was established, AIUI. To be honest, it may be adequate to maintain this solely through the Wiki. The only

Re: [HACKERS] Developer TODO List as a PostgreSQL DB

2007-02-27 Thread Lukas Kahwe Smith
Andrew Dunstan wrote: Chad Wagner wrote: There is some point in putting it in a wiki where we can gather relevant documents, links to discussions etc. That's why the developers' wiki was established, AIUI. To be honest, it may be adequate to maintain this solely through

Re: [HACKERS] Developer TODO List as a PostgreSQL DB

2007-02-27 Thread Michael Glaesemann
On Feb 27, 2007, at 23:40 , Andrew Dunstan wrote: Maybe we need some extra FAQs, like: . Why do you still use CVS instead of insert favorite SCM system here? I just saw a patch from Robert Treat on just this topic. Doesn't look like its been applied yet. Michael Glaesemann grzm

Re: [HACKERS] Packed short varlenas, what next?

2007-02-27 Thread Tom Lane
Peter Eisentraut [EMAIL PROTECTED] writes: As I has mentioned earlier, I'm missing a plan to allow 8-byte varlena sizes. I don't think it's entirely fair to expect this patch to solve that problem. In the first place, that is not what the patch's goal is, but merely tangentially related to

Re: [HACKERS] Packed short varlenas, what next?

2007-02-27 Thread Gregory Stark
Tom Lane [EMAIL PROTECTED] writes: Tom Lane [EMAIL PROTECTED] writes: Peter Eisentraut [EMAIL PROTECTED] writes: As I has mentioned earlier, I'm missing a plan to allow 8-byte varlena sizes. Hm, change VARHDRSZ to 8 and change all the varlena data types to have an int64 leading field? I

Re: [HACKERS] Packed short varlenas, what next?

2007-02-27 Thread Tom Lane
Gregory Stark [EMAIL PROTECTED] writes: 2) The choice of encoding for toast pointers. Note that currently they don't actually save *any* space due to the alignment requirements of the OIDs. which seems kind of silly but I didn't see any reasonable way around that. I was expecting that

Re: [HACKERS] autovacuum next steps, take 2

2007-02-27 Thread Jim C. Nasby
On Tue, Feb 27, 2007 at 01:26:00AM -0500, Matthew T. O'Connor wrote: Tom Lane wrote: Matthew T. O'Connor matthew@zeut.net writes: I'm not sure what you are saying here, are you now saying that partial vacuum won't work for autovac? Or are you saying that saving state as Jim is describing

Re: [HACKERS] autovacuum next steps, take 2

2007-02-27 Thread Jim C. Nasby
On Tue, Feb 27, 2007 at 12:54:28AM -0500, Matthew T. O'Connor wrote: Jim C. Nasby wrote: On Mon, Feb 26, 2007 at 10:18:36PM -0500, Matthew T. O'Connor wrote: Jim C. Nasby wrote: Here is a worst case example: A DB with 6 tables all of which are highly active and will need to be vacuumed

Re: [HACKERS] autovacuum next steps, take 2

2007-02-27 Thread Jim C. Nasby
On Tue, Feb 27, 2007 at 12:00:41AM -0300, Alvaro Herrera wrote: Jim C. Nasby wrote: The advantage to keying this to autovac_naptime is that it means we don't need another GUC, but after I suggested that before I realized that's probably not the best idea. For example, I've seen clusters

Re: [HACKERS] Developer TODO List as a PostgreSQL DB

2007-02-27 Thread Joshua D. Drake
And, for the record, I do not see the slightest point in putting the TODO list on its own into a database. None, zilch, nada. As database professionals we should be adept at recognising when the use of a database is appropriate and when it isn't. If we put it into a tracking system

Re: [HACKERS] [PATCHES] HOT WIP Patch - version 3.2

2007-02-27 Thread Heikki Linnakangas
Pavan Deolasee wrote: - What do we do with the LP_DELETEd tuples at the VACUUM time ? In this patch, we are collecting them and vacuuming like any other dead tuples. But is that the best thing to do ? Since they don't need index cleanups, it's a waste of maintenance_work_mem to keep track of

Re: [HACKERS] Resumable vacuum proposal and design overview

2007-02-27 Thread Jim C. Nasby
On Tue, Feb 27, 2007 at 11:44:28AM +0900, Galy Lee wrote: For example, there is one table: - The table is a hundreds GBs table. - It takes 4-8 hours to vacuum such a large table. - Enabling cost-based delay may make it last for 24 hours. - It can be vacuumed during night time for

[HACKERS] 7.x horology regression test on Solaris buildfarm machines

2007-02-27 Thread Tom Lane
I see that kudu and dragonfly are now failing regression in the 7.3 and 7.4 branches, as a consequence of this patch: http://archives.postgresql.org/pgsql-committers/2007-02/msg00491.php Is it reasonable to assume that that machine will soon be patched to know about the new US DST rules? If not,

Re: [HACKERS] Seeking Google SoC Mentors

2007-02-27 Thread Jim C. Nasby
On Tue, Feb 27, 2007 at 12:47:14AM -0500, Tom Lane wrote: Jim C. Nasby [EMAIL PROTECTED] writes: Yes, but the list being discussed is SoC projects that the community would like to see done, which means most people would assume that #1 isn't an issue. We need to make sure that every

Re: [HACKERS] Resumable vacuum proposal and design overview

2007-02-27 Thread Simon Riggs
On Tue, 2007-02-27 at 10:37 -0600, Jim C. Nasby wrote: On Tue, Feb 27, 2007 at 11:44:28AM +0900, Galy Lee wrote: For example, there is one table: - The table is a hundreds GBs table. - It takes 4-8 hours to vacuum such a large table. - Enabling cost-based delay may make it last

Re: [HACKERS] 7.x horology regression test on Solaris buildfarm machines

2007-02-27 Thread Andrew Dunstan
Tom Lane wrote: I see that kudu and dragonfly are now failing regression in the 7.3 and 7.4 branches, as a consequence of this patch: http://archives.postgresql.org/pgsql-committers/2007-02/msg00491.php Is it reasonable to assume that that machine will soon be patched to know about the new US

Re: [HACKERS] Dead Space Map version 2

2007-02-27 Thread Jim C. Nasby
On Tue, Feb 27, 2007 at 12:55:21AM -0500, Tom Lane wrote: Jim C. Nasby [EMAIL PROTECTED] writes: Yes, DSM would make FSM recovery more important, but I thought it was recoverable now? Or is that only on a clean shutdown? Currently we throw away FSM during any non-clean restart. This is

Re: [HACKERS] 7.x horology regression test on Solaris buildfarm machines

2007-02-27 Thread Kris Jurka
On Tue, 27 Feb 2007, Tom Lane wrote: I see that kudu and dragonfly are now failing regression in the 7.3 and 7.4 branches, as a consequence of this patch: http://archives.postgresql.org/pgsql-committers/2007-02/msg00491.php Is it reasonable to assume that that machine will soon be patched to

Re: [HACKERS] Dead Space Map version 2

2007-02-27 Thread Jim C. Nasby
On Tue, Feb 27, 2007 at 05:38:39PM +0900, ITAGAKI Takahiro wrote: Jim C. Nasby [EMAIL PROTECTED] wrote: If we do UPDATE a tuple, the original page containing the tuple is marked as HIGH and the new page where the updated tuple is placed is marked as LOW. Don't you mean UNFROZEN?

Re: [HACKERS] Resumable vacuum proposal and design overview

2007-02-27 Thread Tom Lane
Simon Riggs [EMAIL PROTECTED] writes: On Tue, 2007-02-27 at 10:37 -0600, Jim C. Nasby wrote: ... The idea would be to give vacuum a target run time, and it would monitor how much time it had remaining, taking into account how long it should take to scan the indexes based on how long it's been

Re: [HACKERS] autovacuum next steps, take 2

2007-02-27 Thread Matthew T. O'Connor
Jim C. Nasby wrote: On Tue, Feb 27, 2007 at 12:00:41AM -0300, Alvaro Herrera wrote: Jim C. Nasby wrote: The advantage to keying this to autovac_naptime is that it means we don't need another GUC, but after I suggested that before I realized that's probably not the best idea. For example, I've

Re: [HACKERS] 7.x horology regression test on Solaris buildfarm machines

2007-02-27 Thread Kris Jurka
On Tue, 27 Feb 2007, Kris Jurka wrote: I'll look at an upgrade. Eel is failing as well, but surprisingly canary is not. Canary hasn't had any updates applied, so why isn't it failing as well: Shouldn't all of the buildfarm members be failing either before or after the patch? That

Re: [HACKERS] 7.x horology regression test on Solaris buildfarm machines

2007-02-27 Thread Tom Lane
Kris Jurka [EMAIL PROTECTED] writes: Shouldn't all of the buildfarm members be failing either before or after the patch? That doesn't seem to be the case for any of them. No, because for the standard regression behavior we have variant result files both with and without the DST law change:

Re: [HACKERS] Resumable vacuum proposal and design overview

2007-02-27 Thread Matthew T. O'Connor
Tom Lane wrote: Simon Riggs [EMAIL PROTECTED] writes: On Tue, 2007-02-27 at 10:37 -0600, Jim C. Nasby wrote: ... The idea would be to give vacuum a target run time, and it would monitor how much time it had remaining, taking into account how long it should take to scan the indexes based on how

Re: [HACKERS] Seeking Google SoC Mentors

2007-02-27 Thread Josh Berkus
Well, the HOT discussion hasn't yet led to an accepted patch ... and I'd say its authors still did way too much work before getting the community involved.  But certainly it's a better model to look at than what the FD author did. That's pretty much the mentor's job. I don't remember who

[HACKERS] bug in gist hstore?

2007-02-27 Thread Gregory Stark
In the following code from hstore_io.c, is HStore a varlena? In which case is the following code buggy because it omits to subtract VARHDRSZ from in-size and therefore is not handling the empty hstore and also starting the loop from the varlena header instead of the first data byte? Or if HStore

Re: [HACKERS] Seeking Google SoC Mentors

2007-02-27 Thread Robert Treat
On Monday 26 February 2007 18:46, Josh Berkus wrote: Demian, Could you also please share your thoughts on what would be a good student profile- for instance, how much theoretical background and practical experience, for working on a SoC project? Well, it shouldn't be the student's first

Re: [HACKERS] Seeking Google SoC Mentors

2007-02-27 Thread Andrew Dunstan
Jim C. Nasby wrote: I agree we certainly don't want to go designing these projects in advance, but I think we could at least ensure that the community buys into the concept of each project. Yes, at least for those that go on a suggestion list. And that was my worry about Warren's suggestion

Re: [HACKERS] Resumable vacuum proposal and design overview

2007-02-27 Thread Tom Lane
Matthew T. O'Connor [EMAIL PROTECTED] writes: Tom Lane wrote: It occurs to me that we may be thinking about this the wrong way entirely. Perhaps a more useful answer to the problem of using a defined maintenance window is to allow VACUUM to respond to changes in the vacuum cost delay

Re: [HACKERS] autovacuum next steps, take 2

2007-02-27 Thread Jim C. Nasby
On Tue, Feb 27, 2007 at 12:12:22PM -0500, Matthew T. O'Connor wrote: Jim C. Nasby wrote: On Tue, Feb 27, 2007 at 12:00:41AM -0300, Alvaro Herrera wrote: Jim C. Nasby wrote: The advantage to keying this to autovac_naptime is that it means we don't need another GUC, but after I suggested that

Re: [HACKERS] COMMIT NOWAIT Performance Option

2007-02-27 Thread Jim C. Nasby
On Tue, Feb 27, 2007 at 10:49:32AM +, Simon Riggs wrote: I dislike introducing new nonstandard syntax (Oracle compatible is not standard). If we did this I'd vote for control via a GUC setting only; I think that is more useful anyway, as an application can be made to run with such a

Re: [HACKERS] COMMIT NOWAIT Performance Option

2007-02-27 Thread Josh Berkus
Simon, One of the things I love about doing informal online user support in the PostgreSQL community, and formal user support for Sun's customers, is the almost-ironclad guarentee that if a user has a corrupt database or data loss, one of three things is true: a) they didn't apply some

Re: [HACKERS] Packed short varlenas, what next?

2007-02-27 Thread Josh Berkus
Greg, I'm really curious to know how people feel about the varlena patch. In particular I know these issues may elicit comment: Haven't tested yet. Will let you know when I do. -- Josh Berkus PostgreSQL @ Sun San Francisco ---(end of

Re: [HACKERS] COMMIT NOWAIT Performance Option

2007-02-27 Thread Tom Lane
Jim C. Nasby [EMAIL PROTECTED] writes: So would you set commit_fsync_delay on a per-transaction basis? That doesn't make much sense to me... I guess I'm not seeing how you would explicitly mark transactions that you didn't want to fsync immediately. My assumption was that most of the time

[HACKERS] Implicit casts with generic arrays

2007-02-27 Thread Peter Eisentraut
I've looked into cutting back on the implicit casts to text, which exposed the following little gem. The expressions 'abc' || 34 34 || 'abc' would no longer work, with the following error message: ERROR: 22P02: array value must start with { or dimension information That's because the best

Re: [HACKERS] Seeking Google SoC Mentors

2007-02-27 Thread Martijn van Oosterhout
On Tue, Feb 27, 2007 at 10:49:18AM -0600, Jim C. Nasby wrote: I agree we certainly don't want to go designing these projects in advance, but I think we could at least ensure that the community buys into the concept of each project. ISTM one of the big issues with FD is that most people didn't

Re: [HACKERS] Packed short varlenas, what next?

2007-02-27 Thread Tom Lane
Gregory Stark [EMAIL PROTECTED] writes: I'm really curious to know how people feel about the varlena patch. One thing I think we could do immediately is apply the change to replace VARATT_SIZEP(x) = len with SET_VARSIZE(x, len) --- that would considerably reduce the size of the patch and allow

Re: [HACKERS] Resumable vacuum proposal and design overview

2007-02-27 Thread Simon Riggs
On Tue, 2007-02-27 at 12:23 -0500, Tom Lane wrote: Matthew T. O'Connor [EMAIL PROTECTED] writes: Tom Lane wrote: It occurs to me that we may be thinking about this the wrong way entirely. Perhaps a more useful answer to the problem of using a defined maintenance window is to allow VACUUM

Re: [HACKERS] bug in gist hstore?

2007-02-27 Thread Gregory Stark
Gregory Stark [EMAIL PROTECTED] writes: In the following code from hstore_io.c, is HStore a varlena? Sorry, thinko, it is a varlena but size isn't the first struct field, there's a len field first. -- Gregory Stark EnterpriseDB http://www.enterprisedb.com

Re: [HACKERS] [PATCHES] HOT WIP Patch - version 3.2

2007-02-27 Thread Pavan Deolasee
On 2/27/07, Heikki Linnakangas [EMAIL PROTECTED] wrote: Pavan Deolasee wrote: - What do we do with the LP_DELETEd tuples at the VACUUM time ? In this patch, we are collecting them and vacuuming like any other dead tuples. But is that the best thing to do ? Since they don't need index

Re: [HACKERS] Seeking Google SoC Mentors

2007-02-27 Thread Peter Eisentraut
Martijn van Oosterhout wrote: I think it might be useful to at least encourage people wanting to an SoC project to create page on the developer wiki selling their idea. You know, questions like: why do we want it? Where do you expect it to be included? etc. They are expected to do as much

Re: [HACKERS] COMMIT NOWAIT Performance Option

2007-02-27 Thread Simon Riggs
On Tue, 2007-02-27 at 11:32 -0600, Jim C. Nasby wrote: On Tue, Feb 27, 2007 at 10:49:32AM +, Simon Riggs wrote: I dislike introducing new nonstandard syntax (Oracle compatible is not standard). If we did this I'd vote for control via a GUC setting only; I think that is more useful

Re: [HACKERS] Implicit casts with generic arrays

2007-02-27 Thread Tom Lane
Peter Eisentraut [EMAIL PROTECTED] writes: I've looked into cutting back on the implicit casts to text, which exposed the following little gem. The expressions 'abc' || 34 34 || 'abc' would no longer work, with the following error message: ERROR: 22P02: array value must start with { or

Re: [HACKERS] 7.x horology regression test on Solaris buildfarm machines

2007-02-27 Thread Kris Jurka
On Tue, 27 Feb 2007, Tom Lane wrote: No, because for the standard regression behavior we have variant result files both with and without the DST law change: see horology and horology_1. The issue only comes up for machines that were matching to horology-no-DST-before-1970.out (which may be

Re: [HACKERS] autovacuum next steps, take 2

2007-02-27 Thread Casey Duncan
On Feb 26, 2007, at 12:49 PM, Alvaro Herrera wrote: Jim C. Nasby wrote: That's why I'm thinking it would be best to keep the maximum size of stuff for the second worker small. It probably also makes sense to tie it to time and not size, since the key factor is that you want it to hit

Re: [HACKERS] COMMIT NOWAIT Performance Option

2007-02-27 Thread Zeugswetter Andreas ADI SD
There are 2 GUCs that would control the behaviour here: transaction_guarantee = on | off has been enabled. Use this parameter with care; if you find yourself wanting to use this parameter all of the time you should consult a psychiatrist or change open source databases.

Re: conversion efforts (Re: [HACKERS] SCMS question)

2007-02-27 Thread Bruce Momjian
Warren Turkal wrote: On Monday 26 February 2007 10:13, Bruce Momjian wrote: Warren Turkal wrote: On Saturday 24 February 2007 16:47, Chad Wagner wrote: head pgsql/src/interfaces/perl5/Attic/test.pl.oldstyle,v head ? ?1.3; access; symbols ? ? ? ? Release-1-6-0:1.1.1.1

Re: [HACKERS] COMMIT NOWAIT Performance Option

2007-02-27 Thread Jonah H. Harris
On 2/27/07, Josh Berkus josh@agliodbs.com wrote: It seriously narrows down the problem space to know that PostgreSQL does *not* allow data loss if it's physically possible to prevent it. Seems like we're trying to protect users from themselves again. This is not a PostgreSQL database issue;

Re: [HACKERS] COMMIT NOWAIT Performance Option

2007-02-27 Thread Josh Berkus
Jonah, Under Oracle, NOWAIT is an asynchronous commit... anyone that uses it should understand that it's still not on-disk and that they can lose it in the event of a failure. That's what Oracle's docs even say. It's just a risk vs. reward trade off. You're missing my point, which is that

Re: [HACKERS] COMMIT NOWAIT Performance Option

2007-02-27 Thread Joshua D. Drake
Josh Berkus wrote: Simon, One of the things I love about doing informal online user support in the PostgreSQL community, and formal user support for Sun's customers, is the almost-ironclad guarentee that if a user has a corrupt database or data loss, one of three things is true: a) they

Re: conversion efforts (Re: [HACKERS] SCMS question)

2007-02-27 Thread Warren Turkal
On Tuesday 27 February 2007 12:26, Bruce Momjian wrote: OK, I definately had added the semicolons, so I am confused why you don't see them.  Anyway, I have remove the duplicate 'creation:' lines, so now there is only one line in each file.  Let me know how that works. Everything looks good

Re: [HACKERS] COMMIT NOWAIT Performance Option

2007-02-27 Thread Dave Page
Joshua D. Drake wrote: 2. We have to accept that not everyone wants IRON clad data integrity. We have many, many options for dealing with that now, including PITR and REPLICATION. 100% agreed - our own stats collector is extremely similar (in that it may drop data under high load) to a system

Re: [HACKERS] COMMIT NOWAIT Performance Option

2007-02-27 Thread Jordan Henderson
I am not sure about some of this. The Oracle option does not change the engine fsync behavior I believe. All that is changed is whether the client side waits for the complete of the fsync or not. If this is true, the data store, logs, etc, are all protected. The user may still experience a

Re: conversion efforts (Re: [HACKERS] SCMS question)

2007-02-27 Thread Andrew Dunstan
Warren Turkal wrote: On Tuesday 27 February 2007 12:26, Bruce Momjian wrote: OK, I definately had added the semicolons, so I am confused why you don't see them. Anyway, I have remove the duplicate 'creation:' lines, so now there is only one line in each file. Let me know how that works.

Re: [HACKERS] COMMIT NOWAIT Performance Option

2007-02-27 Thread Tom Lane
Joshua D. Drake [EMAIL PROTECTED] writes: Josh Berkus wrote: It seriously narrows down the problem space to know that PostgreSQL does *not* allow data loss if it's physically possible to prevent it. But we do don't we? fsync = off, full_page_writes = off? One of the things that's really

Re: [HACKERS] COMMIT NOWAIT Performance Option

2007-02-27 Thread Jonah H. Harris
On 2/27/07, Josh Berkus josh@agliodbs.com wrote: You're missing my point, which is that nobody has demonstrated that there is any real performance gain from this. I see no reason to implement it if there is no performance gain. While I'll back your request for results, it seems nearly

[HACKERS] high CPU usage for stats collector in 8.2

2007-02-27 Thread Darcy Buskermolen
I'm observing high CPU usage (95%) of a 2.6GHz opteron by the stats collector on an 8.2.3 box investigation has lead me to belive that the stats file is written a lot more often that once every 500ms the following shows this behavior. PostgreSQL 8.2.3 on x86_64-redhat-linux-gnu, compiled by

Re: [HACKERS] COMMIT NOWAIT Performance Option

2007-02-27 Thread Bruce Momjian
Tom Lane wrote: Simon Riggs [EMAIL PROTECTED] writes: COMMIT NOWAIT can co-exist with the normal form of COMMIT and does not threaten the consistency or robustness of other COMMIT modes. Read that again and think about it, before we go further, please. I read that, and thought about it,

Re: [HACKERS] COMMIT NOWAIT Performance Option

2007-02-27 Thread Bruce Momjian
Jonah H. Harris wrote: On 2/27/07, Josh Berkus josh@agliodbs.com wrote: You're missing my point, which is that nobody has demonstrated that there is any real performance gain from this. I see no reason to implement it if there is no performance gain. While I'll back your request for

Re: [HACKERS] COMMIT NOWAIT Performance Option

2007-02-27 Thread Bruce Momjian
Jim C. Nasby wrote: On Mon, Feb 26, 2007 at 10:56:58PM +, Simon Riggs wrote: 2. remove fsync parameter Why? Wouldn't fsync=off still speed up checkpoints? ISTM you'd still want this for things like database restores. I think we will remove fsync in favor of the new delay, and allow -1

Re: [HACKERS] COMMIT NOWAIT Performance Option

2007-02-27 Thread Joshua D. Drake
Bruce Momjian wrote: Jonah H. Harris wrote: On 2/27/07, Josh Berkus josh@agliodbs.com wrote: You're missing my point, which is that nobody has demonstrated that there is any real performance gain from this. I see no reason to implement it if there is no performance gain. While I'll back

Re: conversion efforts (Re: [HACKERS] SCMS question)

2007-02-27 Thread Warren Turkal
On Tuesday 27 February 2007 13:50, Andrew Dunstan wrote: You know, you can prune what is rsynced. I am not sure why you brought this up, but yes I did know this. my rsync line looks like this:   rsync -avzCH --delete --exclude-from=/home/cvsmirror/pg-exclude

Re: conversion efforts (Re: [HACKERS] SCMS question)

2007-02-27 Thread Joshua D. Drake
Warren Turkal wrote: On Tuesday 27 February 2007 12:26, Bruce Momjian wrote: OK, I definately had added the semicolons, so I am confused why you don't see them. Anyway, I have remove the duplicate 'creation:' lines, so now there is only one line in each file. Let me know how that works.

Re: [HACKERS] [PATCHES]

2007-02-27 Thread Bruce Momjian
FYI, I am not going to be comfortable accepting a final patch that contains this email signature: This is an email from Fujitsu Australia Software Technology Pty Ltd, ABN 27 003 693 481. It is confidential to the ordinary user of the email address to which it was

Re: [HACKERS] [PATCHES]

2007-02-27 Thread Joshua D. Drake
Bruce Momjian wrote: FYI, I am not going to be comfortable accepting a final patch that contains this email signature: This is an email from Fujitsu Australia Software Technology Pty Ltd, ABN 27 003 693 481. It is confidential to the ordinary user of the email address to

[HACKERS] Synchronized Scan update

2007-02-27 Thread Jeff Davis
I have found some interesting results from my tests with the Synchronized Scan patch I'm working on. The two benefits that I hope to achieve with the patch are: (1) Better caching behavior with multiple sequential scans running in parallel (2) Faster sequential reads from disk and less seeking

Re: [HACKERS] No ~ operator for box, point

2007-02-27 Thread Jim C. Nasby
On Tue, Feb 20, 2007 at 04:22:27PM -0500, Bruce Momjian wrote: Added to TODO: * Add missing operators for geometric data types Some geometric types do not have the full suite of geometric operators, e.g. box @ point I've started looking at this, and ISTM

Re: [HACKERS] [PATCHES]

2007-02-27 Thread Neil Conway
On Tue, 2007-02-27 at 14:52 -0800, Joshua D. Drake wrote: Gonna have to concur with that. Not that the sig is legally binding anyway, we do need to have a disclaimer in the email stating that you are assigning to PGDG I think it's pretty silly to start caring about this now. Do you think that

Re: [HACKERS] [PATCHES]

2007-02-27 Thread Bruce Momjian
Neil Conway wrote: On Tue, 2007-02-27 at 14:52 -0800, Joshua D. Drake wrote: Gonna have to concur with that. Not that the sig is legally binding anyway, we do need to have a disclaimer in the email stating that you are assigning to PGDG I think it's pretty silly to start caring about

Re: [HACKERS] COMMIT NOWAIT Performance Option

2007-02-27 Thread Jim C. Nasby
On Tue, Feb 27, 2007 at 05:18:28PM -0500, Bruce Momjian wrote: Jim C. Nasby wrote: On Mon, Feb 26, 2007 at 10:56:58PM +, Simon Riggs wrote: 2. remove fsync parameter Why? Wouldn't fsync=off still speed up checkpoints? ISTM you'd still want this for things like database restores.

Re: [HACKERS] No ~ operator for box, point

2007-02-27 Thread Tom Lane
Jim C. Nasby [EMAIL PROTECTED] writes: Is there any reason not to make these casts implicit? To the extent that you're trying to provide operators that should be indexable, that won't solve the problem. I'm unconvinced that these casts should be implicit anyway, as the types are really

Re: [HACKERS] COMMIT NOWAIT Performance Option

2007-02-27 Thread Bruce Momjian
Jim C. Nasby wrote: On Tue, Feb 27, 2007 at 05:18:28PM -0500, Bruce Momjian wrote: Jim C. Nasby wrote: On Mon, Feb 26, 2007 at 10:56:58PM +, Simon Riggs wrote: 2. remove fsync parameter Why? Wouldn't fsync=off still speed up checkpoints? ISTM you'd still want this for

Re: [HACKERS] [PATCHES]

2007-02-27 Thread Joshua D. Drake
Neil Conway wrote: On Tue, 2007-02-27 at 14:52 -0800, Joshua D. Drake wrote: Gonna have to concur with that. Not that the sig is legally binding anyway, we do need to have a disclaimer in the email stating that you are assigning to PGDG I think it's pretty silly to start caring about this

Re: [HACKERS] COMMIT NOWAIT Performance Option

2007-02-27 Thread Jim C. Nasby
On Tue, Feb 27, 2007 at 07:17:37PM -0500, Bruce Momjian wrote: Actually, I don't know that combining both settings is a wise move. The delay should still provide crash protection, whereas with fsync=off you've got absolutely no protection from anything. That's a huge difference, and one

Re: conversion efforts (Re: [HACKERS] SCMS question)

2007-02-27 Thread Joshua D. Drake
Warren Turkal wrote: On Tuesday 27 February 2007 12:26, Bruce Momjian wrote: OK, I definately had added the semicolons, so I am confused why you don't see them. Anyway, I have remove the duplicate 'creation:' lines, so now there is only one line in each file. Let me know how that works.

Re: conversion efforts (Re: [HACKERS] SCMS question)

2007-02-27 Thread Andrew Dunstan
Warren Turkal wrote: On Tuesday 27 February 2007 13:50, Andrew Dunstan wrote: You know, you can prune what is rsynced. I am not sure why you brought this up, but yes I did know this. Well I thought it might be useful to prune that directory you were having trouble with. But we

Re: [HACKERS] COMMIT NOWAIT Performance Option

2007-02-27 Thread Gregory Stark
Joshua D. Drake [EMAIL PROTECTED] writes: On 2/27/07, Josh Berkus josh@agliodbs.com wrote: I see no reason to implement it if there is no performance gain. However, I strongly concur that we need at least some evidence. It could easily be that a misstep in the code, causes a loop over the

  1   2   >