Re: [PATCHES] still alive?

2008-10-01 Thread Alvaro Herrera
Bruce Momjian wrote: > > Marc, care to do the honors? > Note: 1. there are several lists to kill, not just pgsql-patches. The database says: pgsql-chat pgsql-benchmarks pgsql-hackers-win32 pgsql-hackers-pitr pgsql-cygwin pgsql-ports 2. The archives must, obviously, survive the kill, an

Re: [PATCHES] still alive?

2008-10-01 Thread Joshua Drake
On Wed, 1 Oct 2008 20:05:00 -0400 (EDT) Bruce Momjian <[EMAIL PROTECTED]> wrote: > > Marc, care to do the honors? KILL IT!! :P > > --- > > Peter Eisentraut wrote: > > Simon Riggs wrote: > > > On Thu, 2008-09-11 at 15:

Re: [PATCHES] still alive?

2008-10-01 Thread Bruce Momjian
Marc, care to do the honors? --- Peter Eisentraut wrote: > Simon Riggs wrote: > > On Thu, 2008-09-11 at 15:39 +0300, Peter Eisentraut wrote: > >> Bruce Momjian wrote: > >>> Abhijit Menon-Sen wrote: > I thought -patches

Re: [PATCHES] still alive?

2008-10-01 Thread Bruce Momjian
Peter Eisentraut wrote: > Simon Riggs wrote: > > On Thu, 2008-09-11 at 15:39 +0300, Peter Eisentraut wrote: > >> Bruce Momjian wrote: > >>> Abhijit Menon-Sen wrote: > I thought -patches was supposed to die. What happened? > >>> I was wondering the same thing. Peter? > >> Hmm, let's try this:

Re: [PATCHES] still alive?

2008-10-01 Thread Peter Eisentraut
Simon Riggs wrote: On Thu, 2008-09-11 at 15:39 +0300, Peter Eisentraut wrote: Bruce Momjian wrote: Abhijit Menon-Sen wrote: I thought -patches was supposed to die. What happened? I was wondering the same thing. Peter? Hmm, let's try this: Anyone who thinks the patches list should remain as

Re: [PATCHES] libpq not linked against libgssapi

2008-10-01 Thread Magnus Hagander
Markus Schaaf wrote: > src/interfaces/libpq/Makefile is missing a reference to libgssapi. > The problem occurred while compiling 8.3.3 on NetBSD --with-gssapi, > and is still present in HEAD. A patch is attached. Applied and backpatched to 8.3. Thanks! //Magnus -- Sent via pgsql-patches mailin

Re: [PATCHES] still alive?

2008-09-30 Thread Simon Riggs
On Thu, 2008-09-11 at 15:39 +0300, Peter Eisentraut wrote: > Bruce Momjian wrote: > > Abhijit Menon-Sen wrote: > >> I thought -patches was supposed to die. What happened? > > > > I was wondering the same thing. Peter? > > Hmm, let's try this: > > Anyone who thinks the patches list should remai

Re: [PATCHES] Proposed patch to change TOAST compression strategy for 8.3.4

2008-09-30 Thread Simon Riggs
On Mon, 2008-09-29 at 12:32 +0200, Jérôme Jouanin wrote: > Upgrade 8.3.4 is available. Before compiling, I have to apply the > optimized toast patch : bin7hetTGkMRL.bin. > > There are differences in 1 of the 3 files patched : tuptoaster.c > > The patch runs successfully but before installing on

Re: [PATCHES] [HACKERS] Infrastructure changes for recovery

2008-09-29 Thread Simon Riggs
On Mon, 2008-09-29 at 11:24 -0400, Tom Lane wrote: > Simon Riggs <[EMAIL PROTECTED]> writes: > > On Mon, 2008-09-29 at 10:13 -0400, Tom Lane wrote: > >> ... If we crash and restart, we'll have to get to the end > >> of this file before we start letting backends in; which might be further > >> than

Re: [PATCHES] [HACKERS] Infrastructure changes for recovery

2008-09-29 Thread Tom Lane
Simon Riggs <[EMAIL PROTECTED]> writes: > On Mon, 2008-09-29 at 10:13 -0400, Tom Lane wrote: >> ... If we crash and restart, we'll have to get to the end >> of this file before we start letting backends in; which might be further >> than we actually got before the crash, but not too much further be

Re: [PATCHES] [HACKERS] Infrastructure changes for recovery

2008-09-29 Thread Simon Riggs
On Mon, 2008-09-29 at 10:13 -0400, Tom Lane wrote: > Simon Riggs <[EMAIL PROTECTED]> writes: > > I think we can get away with writing the LSN value to disk, as you > > suggested, but only every so often. No need to do it after every WAL > > record, just consistently every so often, so it gives us

Re: [PATCHES] [HACKERS] Infrastructure changes for recovery

2008-09-29 Thread Tom Lane
Simon Riggs <[EMAIL PROTECTED]> writes: > I think we can get away with writing the LSN value to disk, as you > suggested, but only every so often. No need to do it after every WAL > record, just consistently every so often, so it gives us a point at > which we know we are safe. Huh? How does that

Re: [PATCHES] [HACKERS] Infrastructure changes for recovery

2008-09-29 Thread Simon Riggs
On Mon, 2008-09-29 at 08:46 -0400, Tom Lane wrote: > Simon Riggs <[EMAIL PROTECTED]> writes: > > ... That kinda works, but the problem is that restartpoints are time based, > > not log based. We need them to be deterministic for us to rely upon them > > in the above way. > > Right, but the perfor

Re: [PATCHES] [HACKERS] Infrastructure changes for recovery

2008-09-29 Thread Tom Lane
Simon Riggs <[EMAIL PROTECTED]> writes: > ... That kinda works, but the problem is that restartpoints are time based, > not log based. We need them to be deterministic for us to rely upon them > in the above way. Right, but the performance disadvantages of making them strictly log-distance-based a

Re: [PATCHES] [HACKERS] Infrastructure changes for recovery

2008-09-29 Thread Simon Riggs
On Sun, 2008-09-28 at 21:16 -0400, Tom Lane wrote: > Simon Riggs <[EMAIL PROTECTED]> writes: > >> It does nothing AFAICS for the > >> problem that when restarting archive recovery from a restartpoint, > >> it's not clear when it is safe to start letting in backends. You need > >> to get past the

Re: [PATCHES] [PgFoundry] Unsigned Data Types [1 of 2]

2008-09-28 Thread Ryan Bradetich
Hello all, Just wanted to let everyone know I have committed this patch to the PgFoundry uint project. I have also updated the commit-fest wiki with this status. Thanks to everyone (especially Jaime) for the feedback and reviews. - Ryan -- Sent via pgsql-patches mailing list (pgsql-patches@pos

Re: [PATCHES] [HACKERS] Infrastructure changes for recovery

2008-09-28 Thread Tom Lane
Simon Riggs <[EMAIL PROTECTED]> writes: >> It does nothing AFAICS for the >> problem that when restarting archive recovery from a restartpoint, >> it's not clear when it is safe to start letting in backends. You need >> to get past the highest LSN that has made it out to disk, and there is >> no g

Re: [PATCHES] [HACKERS] Infrastructure changes for recovery

2008-09-28 Thread Simon Riggs
On Sun, 2008-09-28 at 14:02 -0400, Tom Lane wrote: > It does nothing AFAICS for the > problem that when restarting archive recovery from a restartpoint, > it's not clear when it is safe to start letting in backends. You need > to get past the highest LSN that has made it out to disk, and there i

Re: [PATCHES] [HACKERS] get_relation_stats_hook()

2008-09-28 Thread Tom Lane
Simon Riggs <[EMAIL PROTECTED]> writes: > New version of Postgres patch, v5. Implements suggested changes. > Ready for review and apply. Applied with some revisions. The method for passing back freefunc didn't work, so I made it pass the whole VariableStatsData struct instead; this might allow so

Re: [PATCHES] [HACKERS] Infrastructure changes for recovery

2008-09-28 Thread Tom Lane
Simon Riggs <[EMAIL PROTECTED]> writes: > On Thu, 2008-09-25 at 18:28 -0400, Tom Lane wrote: >> After reading this for awhile, I realized that there is a rather >> fundamental problem with it: it switches into "consistent recovery" >> mode as soon as it's read WAL beyond ControlFile->minRecoveryPoi

Re: [PATCHES] [HACKERS] get_relation_stats_hook()

2008-09-26 Thread Simon Riggs
On Wed, 2008-08-06 at 23:38 +0100, Simon Riggs wrote: > On Wed, 2008-08-06 at 16:37 +0100, Simon Riggs wrote: > > > I'll submit the fully working plugin once we've stabilised the API. It's > > designed as a contrib module, so it can go in pgfoundry or contrib. > > OK, here's fully working plugin

Re: [PATCHES] [HACKERS] Infrastructure changes for recovery

2008-09-26 Thread Simon Riggs
On Fri, 2008-09-26 at 11:20 +0100, Simon Riggs wrote: > > After reading this for awhile, I realized that there is a rather > > fundamental problem with it: it switches into "consistent recovery" > > mode as soon as it's read WAL beyond ControlFile->minRecoveryPoint. > > In a crash recovery situat

Re: [PATCHES] [HACKERS] Infrastructure changes for recovery

2008-09-26 Thread Simon Riggs
On Thu, 2008-09-25 at 18:28 -0400, Tom Lane wrote: > Simon Riggs <[EMAIL PROTECTED]> writes: > > Version 7 > Anyway, that's sufficiently bad that I'm bouncing the patch for > reconsideration. No problem, I understand this needs discussion. There's less detail here than first appears. There a

Re: [PATCHES] [HACKERS] Infrastructure changes for recovery

2008-09-25 Thread Tom Lane
Simon Riggs <[EMAIL PROTECTED]> writes: > Version 7 After reading this for awhile, I realized that there is a rather fundamental problem with it: it switches into "consistent recovery" mode as soon as it's read WAL beyond ControlFile->minRecoveryPoint. In a crash recovery situation that typically

Re: [PATCHES] [HACKERS] Subtransaction commits and Hot Standby

2008-09-24 Thread Simon Riggs
On Wed, 2008-09-24 at 13:48 +0100, Simon Riggs wrote: > On Tue, 2008-09-23 at 22:47 +0100, Simon Riggs wrote: > > > I've tested this some more and am much happier with it now. > > The concept is fine, but I've found a coding bug in further testing. > Please wait now for new version before review

Re: [PATCHES] hash index improving v3

2008-09-24 Thread Simon Riggs
On Wed, 2008-09-24 at 12:04 -0400, Bruce Momjian wrote: > Can we consider this hash thread closed/completed? Please -- Simon Riggs www.2ndQuadrant.com PostgreSQL Training, Services and Support -- Sent via pgsql-patches mailing list (pgsql-patches@postgresql.org) To make changes

Re: [PATCHES] hash index improving v3

2008-09-24 Thread Bruce Momjian
Can we consider this hash thread closed/completed? --- Tom Lane wrote: > Simon Riggs <[EMAIL PROTECTED]> writes: > > Thinks: Why not just sort all of the time and skip the debate entirely? > > The sort is demonstrably a los

Re: [PATCHES] Solve a problem of LC_TIME of windows.

2008-09-24 Thread Hiroshi Saito
Hi. - Original Message - From: "Alvaro Herrera" <[EMAIL PROTECTED]> What about this line? #define STRLEN_MAX 255 Are we really sure the strftime format cannot be longer than that? Ahh, although the place to replace is here, is it said that MAX_L10 N_DATA is suitable? -- cache_loca

Re: [PATCHES] Solve a problem of LC_TIME of windows.

2008-09-24 Thread Hiroshi Saito
Hi. - Original Message - From: "Magnus Hagander" <[EMAIL PROTECTED]> In principle, I think this patch looks good. Do you (or somebody else) have an example where this breaks in an encoding where I can actually understand the characters, though ;-) That would make testing a whole lot

Re: [PATCHES] Solve a problem of LC_TIME of windows.

2008-09-24 Thread Alvaro Herrera
Magnus Hagander wrote: > Also, the patch needs error checking. strftime() can fail, and the > multibyte conversion functions can certainly fail. That will need to be > added. What about this line? #define STRLEN_MAX 255 Are we really sure the strftime format cannot be longer than that? -- Alv

Re: [PATCHES] [HACKERS] Subtransaction commits and Hot Standby

2008-09-24 Thread Simon Riggs
On Tue, 2008-09-23 at 22:47 +0100, Simon Riggs wrote: > I've tested this some more and am much happier with it now. The concept is fine, but I've found a coding bug in further testing. Please wait now for new version before review. -- Simon Riggs www.2ndQuadrant.com PostgreSQL Trai

Re: [PATCHES] Solve a problem of LC_TIME of windows.

2008-09-24 Thread Magnus Hagander
Hiroshi Saito wrote: > Hi. > > I have problem of LC_TIME of windows.(CVS-HEAD) > > As for Version 8.3.3. It is edited by wrong gettext and is. (But, it is > expressed.) > http://winpg.jp/~saito/pg_work/LC_MESSAGE_CHECK/LC_TIME_PATCH/pg8.3.3-to_char_gettext_format.png > > > As for Version 8.4. I

Re: [PATCHES] [HACKERS] Subtransaction commits and Hot Standby

2008-09-23 Thread Simon Riggs
On Thu, 2008-09-18 at 15:59 +0100, Simon Riggs wrote: > On Tue, 2008-09-16 at 10:11 -0400, Alvaro Herrera wrote: > > > I wonder if the improved clog API required to mark multiple > > transactions as committed at once would be also useful to > > TransactionIdCommitTree which is used in regular tra

Re: [PATCHES] [HACKERS] Infrastructure changes for recovery

2008-09-23 Thread Simon Riggs
On Mon, 2008-09-22 at 23:06 +0100, Simon Riggs wrote: > On Thu, 2008-09-18 at 10:09 -0400, Tom Lane wrote: > > Simon Riggs <[EMAIL PROTECTED]> writes: > > > On Thu, 2008-09-18 at 09:06 -0400, Tom Lane wrote: > > >> Do we really need a checkpoint there at all? > > > > > "Timelines only change at s

Re: [PATCHES] hash index improving v3

2008-09-23 Thread Simon Riggs
On Tue, 2008-09-23 at 09:13 -0400, Tom Lane wrote: > Simon Riggs <[EMAIL PROTECTED]> writes: > > Thinks: Why not just sort all of the time and skip the debate entirely? > > The sort is demonstrably a loser for smaller indexes. Admittedly, > if the index is small then the sort can't cost all that

Re: [PATCHES] hash index improving v3

2008-09-23 Thread Simon Riggs
On Tue, 2008-09-23 at 09:13 -0400, Tom Lane wrote: > The other side of that coin is that it's not clear this is really worth > arguing about, much less exposing a separate parameter for. Agreed. -- Simon Riggs www.2ndQuadrant.com PostgreSQL Training, Services and Support -- Sent

Re: [PATCHES] hash index improving v3

2008-09-23 Thread Tom Lane
Simon Riggs <[EMAIL PROTECTED]> writes: > Thinks: Why not just sort all of the time and skip the debate entirely? The sort is demonstrably a loser for smaller indexes. Admittedly, if the index is small then the sort can't cost all that much, but if the (correct) threshold is some large fraction o

Re: [PATCHES] hash index improving v3

2008-09-23 Thread Simon Riggs
On Tue, 2008-09-23 at 08:16 -0400, Tom Lane wrote: > Simon Riggs <[EMAIL PROTECTED]> writes: > > maintenance_work_mem is already used for 3 separate operations that bear > > little resemblance to each other. If it's appropriate for all of those > > then its appropriate for this usage also. > > No

Re: [PATCHES] hash index improving v3

2008-09-23 Thread Tom Lane
Simon Riggs <[EMAIL PROTECTED]> writes: > maintenance_work_mem is already used for 3 separate operations that bear > little resemblance to each other. If it's appropriate for all of those > then its appropriate for this usage also. No, it isn't. The fundamental point here is that this isn't a mem

Re: [PATCHES] hash index improving v3

2008-09-22 Thread Simon Riggs
On Tue, 2008-09-23 at 00:48 -0400, Tom Lane wrote: > Simon Riggs <[EMAIL PROTECTED]> writes: > > I wasn't very happy with effective_cache_size and not happy with > > shared_buffers either. If building hash indexes is memory critical then > > we just need to say so and encourage others to set memor

Re: [PATCHES] hash index improving v3

2008-09-22 Thread Tom Lane
Simon Riggs <[EMAIL PROTECTED]> writes: > I wasn't very happy with effective_cache_size and not happy with > shared_buffers either. If building hash indexes is memory critical then > we just need to say so and encourage others to set memory use correctly. > People are already aware that maintenance

Re: [PATCHES] hash index improving v3

2008-09-22 Thread Simon Riggs
On Tue, 2008-09-23 at 00:05 -0400, Tom Lane wrote: > "Jonah H. Harris" <[EMAIL PROTECTED]> writes: > > On Mon, Sep 22, 2008 at 11:25 PM, Alex Hunsaker <[EMAIL PROTECTED]> wrote: > >> On Sun, Sep 14, 2008 at 8:16 PM, Tom Lane <[EMAIL PROTECTED]> wrote: > >>> I'm considering changing hashbuild to sw

Re: [PATCHES] hash index improving v3

2008-09-22 Thread Tom Lane
"Jonah H. Harris" <[EMAIL PROTECTED]> writes: > On Mon, Sep 22, 2008 at 11:25 PM, Alex Hunsaker <[EMAIL PROTECTED]> wrote: >> On Sun, Sep 14, 2008 at 8:16 PM, Tom Lane <[EMAIL PROTECTED]> wrote: >>> I'm considering changing hashbuild to switch over at shared_buffers instead >>> of effective_cache_s

Re: [PATCHES] hash index improving v3

2008-09-22 Thread Alex Hunsaker
On Mon, Sep 22, 2008 at 7:57 PM, Alex Hunsaker <[EMAIL PROTECTED]> wrote: > 50,000,000 rows and 32,768,000 collisions I should mention thats 50 million rows + 32 million more or 62,768,000 rows which explains some of the added index creation time... > index time: > head: 576600.967 ms > v5:

Re: [PATCHES] hash index improving v3

2008-09-22 Thread Jonah H. Harris
On Mon, Sep 22, 2008 at 11:25 PM, Alex Hunsaker <[EMAIL PROTECTED]> wrote: > On Sun, Sep 14, 2008 at 8:16 PM, Tom Lane <[EMAIL PROTECTED]> wrote: >> BTW, one thing I noticed was that the hash index build time for the >> "wide column" case got a lot worse after applying the patch (from 56 to >> 237

Re: [PATCHES] hash index improving v3

2008-09-22 Thread Alex Hunsaker
On Sun, Sep 14, 2008 at 8:16 PM, Tom Lane <[EMAIL PROTECTED]> wrote: > BTW, one thing I noticed was that the hash index build time for the > "wide column" case got a lot worse after applying the patch (from 56 to > 237 sec). The reason for this turned out to be that with the smaller > predicted in

Re: [PATCHES] hash index improving v3

2008-09-22 Thread Alex Hunsaker
On Fri, Sep 12, 2008 at 8:29 AM, Kenneth Marshall <[EMAIL PROTECTED]> wrote: > On Thu, Sep 11, 2008 at 08:51:53PM -0600, Alex Hunsaker wrote: >> On Thu, Sep 11, 2008 at 9:24 AM, Kenneth Marshall <[EMAIL PROTECTED]> wrote: >> > Alex, >> > >> > I meant to check the performance with increasing numbers

Re: [PATCHES] [HACKERS] Infrastructure changes for recovery

2008-09-22 Thread Simon Riggs
On Thu, 2008-09-18 at 10:09 -0400, Tom Lane wrote: > Simon Riggs <[EMAIL PROTECTED]> writes: > > On Thu, 2008-09-18 at 09:06 -0400, Tom Lane wrote: > >> Do we really need a checkpoint there at all? > > > "Timelines only change at shutdown checkpoints". > > Hmm. I *think* that that is just a deb

Re: [PATCHES] [PgFoundry] Unsigned Data Types [1 of 2]

2008-09-21 Thread Ryan Bradetich
Hello Jaime, > i'm still seeing the failures in the copy commands (the ones about the paths) I just tested this on a different machine (to get it away from my development environment) I was able to duplicate the failures. It looks like I need to update the expected/ files as well. I will get fix

Re: [PATCHES] [PgFoundry] Unsigned Data Types [1 of 2]

2008-09-21 Thread Jaime Casanova
On Mon, Sep 15, 2008 at 9:45 PM, Ryan Bradetich <[EMAIL PROTECTED]> wrote: >> >> I have the code and regression tests updated to solve the problems you >> initially >> discovered. After code reading, stepping through with the debugger, and >> help from RhodiumToad on irc I was able to implement n

Re: [PATCHES] [HACKERS] Infrastructure changes for recovery

2008-09-18 Thread Simon Riggs
On Thu, 2008-09-18 at 10:09 -0400, Tom Lane wrote: > Simon Riggs <[EMAIL PROTECTED]> writes: > > On Thu, 2008-09-18 at 09:06 -0400, Tom Lane wrote: > >> Do we really need a checkpoint there at all? > > > "Timelines only change at shutdown checkpoints". > > Hmm. I *think* that that is just a deb

Re: [PATCHES] [HACKERS] Infrastructure changes for recovery

2008-09-18 Thread Tom Lane
Simon Riggs <[EMAIL PROTECTED]> writes: > On Thu, 2008-09-18 at 09:06 -0400, Tom Lane wrote: >> Do we really need a checkpoint there at all? > "Timelines only change at shutdown checkpoints". Hmm. I *think* that that is just a debugging crosscheck rather than a critical property. But yeah, it w

Re: [PATCHES] [HACKERS] Infrastructure changes for recovery

2008-09-18 Thread Simon Riggs
On Thu, 2008-09-18 at 09:06 -0400, Tom Lane wrote: > Simon Riggs <[EMAIL PROTECTED]> writes: > > Having some trouble trying to get a clean state change from recovery to > > normal mode. > > > Startup needs to be able to write WAL at the end of recovery so it can > > write a ShutdownCheckpoint, ye

Re: [PATCHES] [HACKERS] Infrastructure changes for recovery

2008-09-18 Thread Tom Lane
Simon Riggs <[EMAIL PROTECTED]> writes: > Having some trouble trying to get a clean state change from recovery to > normal mode. > Startup needs to be able to write WAL at the end of recovery so it can > write a ShutdownCheckpoint, yet must not be allowed to write WAL before > that. Other services

Re: [PATCHES] [HACKERS] Infrastructure changes for recovery

2008-09-18 Thread Simon Riggs
On Thu, 2008-09-18 at 09:05 +0100, Simon Riggs wrote: > Feels like I should shutdown the bgwriter after recovery and then > allow it to be cranked up again after normal processing starts, and do > all of this through postmaster state changes. That way bgwriter > doesn't need to do a dynamic state

Re: [PATCHES] [HACKERS] Infrastructure changes for recovery

2008-09-18 Thread Simon Riggs
On Tue, 2008-09-16 at 15:45 -0400, Tom Lane wrote: > Simon Riggs <[EMAIL PROTECTED]> wrote: > > Testing takes a while on this, I probably won't complete it until > > Friday. So enclosed patch is for eyeballs only at this stage. > > What's the status on that patch? Having some trouble trying to g

Re: [PATCHES] still alive?

2008-09-17 Thread Bernd Helmle
--On Mittwoch, September 17, 2008 22:37:29 +0300 Heikki Linnakangas <[EMAIL PROTECTED]> wrote: Just send them to pgsql-hackers. Oh, that's fine, since discussions are already moved from -patches and are continued there. The only disadvantage i see is that -hackers is a little more frequente

Re: [PATCHES] still alive?

2008-09-17 Thread Heikki Linnakangas
Bernd Helmle wrote: --On Donnerstag, September 11, 2008 15:39:01 +0300 Peter Eisentraut <[EMAIL PROTECTED]> wrote: Anyone who thinks the patches list should remain as separate from hackers, shout now (with rationale)! Seems i've missed something, what's then supposed to hold patches now? Jus

Re: [PATCHES] still alive?

2008-09-17 Thread Alvaro Herrera
Bernd Helmle wrote: > --On Donnerstag, September 11, 2008 15:39:01 +0300 Peter Eisentraut > <[EMAIL PROTECTED]> wrote: > >> >> Hmm, let's try this: >> >> Anyone who thinks the patches list should remain as separate from >> hackers, shout now (with rationale)! > > Seems i've missed something, what

Re: [PATCHES] still alive?

2008-09-17 Thread Bernd Helmle
--On Donnerstag, September 11, 2008 15:39:01 +0300 Peter Eisentraut <[EMAIL PROTECTED]> wrote: Hmm, let's try this: Anyone who thinks the patches list should remain as separate from hackers, shout now (with rationale)! Seems i've missed something, what's then supposed to hold patches now?

Re: [PATCHES] [HACKERS] Infrastructure changes for recovery

2008-09-17 Thread Simon Riggs
On Tue, 2008-09-16 at 15:45 -0400, Tom Lane wrote: > Simon Riggs <[EMAIL PROTECTED]> wrote: > > Testing takes a while on this, I probably won't complete it until > > Friday. So enclosed patch is for eyeballs only at this stage. > > What's the status on that patch? Checking EXEC_BACKEND case and

Re: [PATCHES] [HACKERS] Infrastructure changes for recovery

2008-09-16 Thread Tom Lane
Simon Riggs <[EMAIL PROTECTED]> wrote: > Testing takes a while on this, I probably won't complete it until > Friday. So enclosed patch is for eyeballs only at this stage. What's the status on that patch? regards, tom lane -- Sent via pgsql-patches mailing list (pgsql-pat

Re: [PATCHES] [PgFoundry] Unsigned Data Types [1 of 2]

2008-09-15 Thread Ryan Bradetich
Message Resend. I forgot to spit the attachments so they did not make it through the list. Patch 1 of 2 : Base uint type. Patch 2 of 2 : Regression tests. - Ryan On Mon, Sep 15, 2008 at 8:13 AM, Ryan Bradetich <[EMAIL PROTECTED]> wrote: > Hello Jaime, > > I have the code and regression tests u

Re: [PATCHES] [PgFoundry] Unsigned Data Types [1 of 2]

2008-09-15 Thread Jaime Casanova
On 9/15/08, Ryan Bradetich <[EMAIL PROTECTED]> wrote: > Hello Jaime, > > I have the code and regression tests updated to solve the problems you > initially > discovered. great, i will test during this week... -- regards, Jaime Casanova Soporte y capacitación de PostgreSQL Asesoría y desarrollo

Re: [PATCHES] hash index improving v3

2008-09-14 Thread Tom Lane
I did some testing of my own on the hash index patch, and mostly seem to have confirmed Alex's results. I used a table like this: create table tab (f1 serial primary key, f2 some-datatype); create index ind on tab using hash (f2); and populated it with 2 million rows of data; the

Re: [PATCHES] [HACKERS] Infrastructure changes for recovery

2008-09-12 Thread Simon Riggs
On Fri, 2008-09-12 at 15:31 -0400, Alvaro Herrera wrote: > Simon Riggs wrote: > > > > All I changed was the word "restartpoint"... its otherwise identical to > > existing message. I'd rather not change that. > > The new message is not translatable, the original was. OK, perhaps the word "restar

Re: [PATCHES] [HACKERS] Infrastructure changes for recovery

2008-09-12 Thread Alvaro Herrera
Simon Riggs wrote: > > On Fri, 2008-09-12 at 14:14 -0400, Alvaro Herrera wrote: > > Simon Riggs wrote: > > > > > --- 5716,5725 > > > > > > CheckpointStats.ckpt_sync_end_t, > > > &sync_secs,

Re: [PATCHES] [HACKERS] Infrastructure changes for recovery

2008-09-12 Thread Simon Riggs
On Fri, 2008-09-12 at 14:14 -0400, Alvaro Herrera wrote: > Simon Riggs wrote: > > > --- 5716,5725 > > CheckpointStats.ckpt_sync_end_t, > > &sync_secs, &sync_usecs); > > > > ! elog(LOG, "%s complete:

Re: [PATCHES] [HACKERS] Infrastructure changes for recovery

2008-09-12 Thread Alvaro Herrera
Simon Riggs wrote: > --- 5716,5725 > CheckpointStats.ckpt_sync_end_t, > &sync_secs, &sync_usecs); > > ! elog(LOG, "%s complete: wrote %d buffers (%.1f%%); " >"%d transaction log

Re: [PATCHES] hash index improving v3

2008-09-12 Thread Alex Hunsaker
On Fri, Sep 12, 2008 at 3:14 AM, Zdenek Kotala <[EMAIL PROTECTED]> wrote: > Alex Hunsaker napsal(a): >> >> On Wed, Sep 10, 2008 at 10:27 AM, Alex Hunsaker <[EMAIL PROTECTED]> wrote: >>> >>> On Wed, Sep 10, 2008 at 8:47 AM, Zdenek Kotala <[EMAIL PROTECTED]> >>> wrote: What locale did you u

Re: [PATCHES] hash index improving v3

2008-09-12 Thread Kenneth Marshall
On Thu, Sep 11, 2008 at 08:51:53PM -0600, Alex Hunsaker wrote: > On Thu, Sep 11, 2008 at 9:24 AM, Kenneth Marshall <[EMAIL PROTECTED]> wrote: > > Alex, > > > > I meant to check the performance with increasing numbers of collisions, > > not increasing size of the hashed item. In other words, somethi

Re: [PATCHES] hash index improving v3

2008-09-12 Thread Zdenek Kotala
Alex Hunsaker napsal(a): On Wed, Sep 10, 2008 at 10:27 AM, Alex Hunsaker <[EMAIL PROTECTED]> wrote: On Wed, Sep 10, 2008 at 8:47 AM, Zdenek Kotala <[EMAIL PROTECTED]> wrote: What locale did you use? It would be nice to have also comparing between C and any UTF8 locale. I think we should see big

Re: [PATCHES] hash index improving v3

2008-09-11 Thread Alex Hunsaker
On Thu, Sep 11, 2008 at 9:24 AM, Kenneth Marshall <[EMAIL PROTECTED]> wrote: > Alex, > > I meant to check the performance with increasing numbers of collisions, > not increasing size of the hashed item. In other words, something like > this: > > for ($coll=500; $i<=100; $i=$i*2) { > for ($i=0;

Re: [PATCHES] hash index improving v3

2008-09-11 Thread Kenneth Marshall
On Wed, Sep 10, 2008 at 10:17:31PM -0600, Alex Hunsaker wrote: > On Wed, Sep 10, 2008 at 9:49 PM, Alex Hunsaker <[EMAIL PROTECTED]> wrote: > > On Wed, Sep 10, 2008 at 7:04 AM, Kenneth Marshall <[EMAIL PROTECTED]> wrote: > >> On Tue, Sep 09, 2008 at 07:23:03PM -0600, Alex Hunsaker wrote: > >>> On Tu

Re: [PATCHES] still alive?

2008-09-11 Thread Peter Eisentraut
Bruce Momjian wrote: Abhijit Menon-Sen wrote: I thought -patches was supposed to die. What happened? I was wondering the same thing. Peter? Hmm, let's try this: Anyone who thinks the patches list should remain as separate from hackers, shout now (with rationale)! -- Sent via pgsql-patch

Re: [PATCHES] hash index improving v3

2008-09-10 Thread Alex Hunsaker
On Wed, Sep 10, 2008 at 9:49 PM, Alex Hunsaker <[EMAIL PROTECTED]> wrote: > On Wed, Sep 10, 2008 at 7:04 AM, Kenneth Marshall <[EMAIL PROTECTED]> wrote: >> On Tue, Sep 09, 2008 at 07:23:03PM -0600, Alex Hunsaker wrote: >>> On Tue, Sep 9, 2008 at 7:48 AM, Kenneth Marshall <[EMAIL PROTECTED]> wrote:

Re: [PATCHES] hash index improving v3

2008-09-10 Thread Alex Hunsaker
On Wed, Sep 10, 2008 at 7:04 AM, Kenneth Marshall <[EMAIL PROTECTED]> wrote: > On Tue, Sep 09, 2008 at 07:23:03PM -0600, Alex Hunsaker wrote: >> On Tue, Sep 9, 2008 at 7:48 AM, Kenneth Marshall <[EMAIL PROTECTED]> wrote: >> > I think that the glacial speed for generating a big hash index is >> > th

Re: [PATCHES] hash index improving v3

2008-09-10 Thread Alex Hunsaker
On Wed, Sep 10, 2008 at 10:27 AM, Alex Hunsaker <[EMAIL PROTECTED]> wrote: > On Wed, Sep 10, 2008 at 8:47 AM, Zdenek Kotala <[EMAIL PROTECTED]> wrote: >> What locale did you use? It would be nice to have also comparing between C >> and any UTF8 locale. I think we should see big benefit when non C l

Re: [PATCHES] [HACKERS] Infrastructure changes for recovery

2008-09-10 Thread Simon Riggs
On Mon, 2008-09-08 at 13:34 -0400, Tom Lane wrote: > ISTM that it would probably be better if there were exactly one InRedo > flag in shared memory, probably in xlog.c's shared state, with the > postmaster not being responsible for setting or clearing it; rather > the startup process should do th

Re: [PATCHES] hash index improving v3

2008-09-10 Thread Alex Hunsaker
On Wed, Sep 10, 2008 at 8:47 AM, Zdenek Kotala <[EMAIL PROTECTED]> wrote: > What locale did you use? It would be nice to have also comparing between C > and any UTF8 locale. I think we should see big benefit when non C locale is > used. Err yes this was UTF8, Ill see about doing a C locale. -- S

Re: [PATCHES] [PgFoundry] Unsigned Data Types [1 of 2]

2008-09-10 Thread Ryan Bradetich
Hello Jaime, It is taking longer than I expected to implement the scalarltsel and scalargtsel functions for the unsigned integer data type. I am still working on this solution and hope to have an updated patch later this week (or over the weekend at the latest). Just wanted to keep you updated on

Re: [PATCHES] hash index improving v3

2008-09-10 Thread Zdenek Kotala
Alex Hunsaker napsal(a): wide: # NOTE not on the same machine as the "narrow" test was run! # spit out 2, 000, 000 random 100 length strings perl gen.pl > data.sql create table test_hash (wide text); copy test_hash from './data.sql'; create index test_hash_num_idx on test_hash using hash (wide

Re: [PATCHES] hash index improving v3

2008-09-10 Thread Kenneth Marshall
On Tue, Sep 09, 2008 at 07:23:03PM -0600, Alex Hunsaker wrote: > On Tue, Sep 9, 2008 at 7:48 AM, Kenneth Marshall <[EMAIL PROTECTED]> wrote: > > I think that the glacial speed for generating a big hash index is > > the same problem that the original code faced. > > Yeah sorry, I was not saying it

Re: [PATCHES] hash index improving v3

2008-09-09 Thread Alex Hunsaker
On Tue, Sep 9, 2008 at 7:23 PM, Alex Hunsaker <[EMAIL PROTECTED]> wrote: > BTW Im still planning on doing a wide vs narrow test... sometime... :) narrow: (exactly the same as what I just did in the other post) create table test_hash(num int8); insert into test_hash (num) select generate_series(1,

Re: [PATCHES] hash index improving v3

2008-09-09 Thread Alex Hunsaker
On Tue, Sep 9, 2008 at 7:23 PM, Alex Hunsaker <[EMAIL PROTECTED]> wrote: > create table test_hash(num int8); > insert into test_hash (num) select generate_series(1, 200); > create index test_hash_num_idx on test_hash (num); > > pgbench -c1 -n -t1 -f bench_index.sql > cvs head: tps = 3161.50

Re: [PATCHES] hash index improving v3

2008-09-09 Thread Alex Hunsaker
On Tue, Sep 9, 2008 at 7:48 AM, Kenneth Marshall <[EMAIL PROTECTED]> wrote: > I think that the glacial speed for generating a big hash index is > the same problem that the original code faced. Yeah sorry, I was not saying it was a new problem with the patch. Err at least not trying to :) *Both* o

Re: [PATCHES] [PgFoundry] Unsigned Data Types [1 of 2]

2008-09-09 Thread Ryan Bradetich
Hello Tom, On Tue, Sep 9, 2008 at 5:11 AM, Tom Lane <[EMAIL PROTECTED]> wrote: > "Ryan Bradetich" <[EMAIL PROTECTED]> writes: >> I am assuming you are seeing this error in the uint_test1.sql: >> ERROR: could not find hash function for hash operator 16524 >> I can bypass the error in uint_test

Re: [PATCHES] hash index improving v3

2008-09-09 Thread Kenneth Marshall
On Sat, Sep 06, 2008 at 08:23:05PM -0600, Alex Hunsaker wrote: > On Sat, Sep 6, 2008 at 1:09 PM, Tom Lane <[EMAIL PROTECTED]> wrote: > >For the convenience of anyone intending to test, here is an updated > >patch against CVS HEAD that incorporates Alex's fix. > > Here are the results for a table c

Re: [PATCHES] [PgFoundry] Unsigned Data Types [1 of 2]

2008-09-09 Thread Tom Lane
"Ryan Bradetich" <[EMAIL PROTECTED]> writes: > I am assuming you are seeing this error in the uint_test1.sql: > ERROR: could not find hash function for hash operator 16524 > I can bypass the error in uint_test1.sql by disabling the hash joins. > I am going to dig in and figure out why the hash

Re: [PATCHES] [PgFoundry] Unsigned Data Types [1 of 2]

2008-09-08 Thread Ryan Bradetich
Hello Jaime, Thank you for the test cases! > mmm... i rebuild my test env and it works for me this time... until i > execute an analyze. I guess autovacuum executed an auto analyze last > time... I am able to duplicate the error you saw in the uint_test2.sql. I am assuming you are seeing this e

Re: [PATCHES] [PgFoundry] Unsigned Data Types [1 of 2]

2008-09-08 Thread Jaime Casanova
On Mon, Sep 8, 2008 at 10:08 PM, Ryan Bradetich <[EMAIL PROTECTED]> wrote: > > Can you send me the test case that generates this error? > My regression tests do not include a table t1 so I was not able > to reproduce this error directly. > yeah! that table is mine! here are the scripts... > contr

Re: [PATCHES] [PgFoundry] Unsigned Data Types [1 of 2]

2008-09-08 Thread Ryan Bradetich
Hello Jaime, > why i need the cast in this case? even if the cast is really necesary > (the message seems realy ugly) > > contrib_regression=# select * from t1 where f1 > 35; > ERROR: unsupported type: 16486 > > contrib_regression=# select * from t1 where f1 > 35::uint4; > f1 > - > 36 > 37

Re: [PATCHES] [HACKERS] TODO item: Implement Boyer-Moore searching (First time hacker)

2008-09-08 Thread David Rowley
Heikki Linnakangas Wrote: > Tom Lane wrote: > > Heikki Linnakangas <[EMAIL PROTECTED]> writes: > >> Also, it would be nice to use B-M(-H) for LIKE as well. > > > > Right offhand, that seems impossible, at least in patterns with %. > > Or were you thinking of trying to separate out the fixed substr

Re: [PATCHES] [HACKERS] Infrastructure changes for recovery

2008-09-08 Thread Tom Lane
Simon Riggs <[EMAIL PROTECTED]> writes: > On Mon, 2008-09-08 at 13:34 -0400, Tom Lane wrote: >> Hmm, I dunno, it seems like that might be a bad choice. Are you sure >> it's not cleaner to just use the regular checkpoint code? > When I tried to write it, it just looked to my eyes like every single

Re: [PATCHES] [HACKERS] Infrastructure changes for recovery

2008-09-08 Thread Simon Riggs
On Mon, 2008-09-08 at 13:34 -0400, Tom Lane wrote: > Simon Riggs <[EMAIL PROTECTED]> writes: > > Included patch with the following changes: > > > * new postmaster mode known as consistent recovery, entered only when > > recovery passes safe/consistent point. InRedo is now set in all > > processes

Re: [PATCHES] [HACKERS] Infrastructure changes for recovery

2008-09-08 Thread Tom Lane
Simon Riggs <[EMAIL PROTECTED]> writes: > Included patch with the following changes: > * new postmaster mode known as consistent recovery, entered only when > recovery passes safe/consistent point. InRedo is now set in all > processes when started/connected in consistent recovery mode. I looked a

Re: [PATCHES] hash index improving v3

2008-09-08 Thread Tom Lane
Zdenek Kotala <[EMAIL PROTECTED]> writes: > I attach cleanup patch which we discussed last commit fest. It introduce new > macros HashGetMetaPage and HashGetBitmap and of course, it break backward on > disk format compatibility which we will break it anyway when Xiao's patch > will > be committ

Re: [PATCHES] [PgFoundry] Unsigned Data Types [1 of 2]

2008-09-08 Thread Ryan Bradetich
Hello Jaime, > the same problem happens in joins, unions, hash, etc... so you have to > look at those functions as well Great! Added to the list to check. I am planning to build regression tests for these types to catch these errors in the future. Thanks again for your testing and review! > P

Re: [PATCHES] hash index improving v3

2008-09-08 Thread Zdenek Kotala
Tom Lane napsal(a): "Alex Hunsaker" <[EMAIL PROTECTED]> writes: On Fri, Sep 5, 2008 at 2:21 PM, Alex Hunsaker <[EMAIL PROTECTED]> wrote: Ok now that I made it so it actually *test* collisions, with the patch it always returns all rows that matched the hashed "key". And here is the fix, we ju

Re: [PATCHES] hash index improving v3

2008-09-08 Thread Zdenek Kotala
It is nice test case. It should be part of hash index regression test. Zdenek Alex Hunsaker napsal(a): Ok now that I made it so it actually *test* collisions, with the patch it always returns all rows that matched the hashed "key". For example (non lobotomized inthash8, I just brute f

  1   2   3   4   5   6   7   8   9   10   >