Re: [HACKERS] tsearch2 in 8.3

2007-04-25 Thread Jeremy Drake
On Tue, 24 Apr 2007, Bruce Momjian wrote: Naz Gassiep wrote: A few of us on IRC were wondering what the status of tsearch2 is in 8.3 ? Was it decided to include it in core or did we decide to keep FTS as a plugin? Some brief comments from anyone on the inside of the whole FTS issue

Re: [PATCHES] [HACKERS] Full page writes improvement, code update

2007-04-25 Thread Zeugswetter Andreas ADI SD
1) To deal with partial/inconsisitent write to the data file at crash recovery, we need full page writes at the first modification to pages after each checkpoint. It consumes much of WAL space. We need to find a way around this someday. Other DBs don't do this; it may be becuase

[HACKERS] Buildfarm: Stage logs not available for MSVC builds

2007-04-25 Thread Dave Page
I just noticed that the stage logs aren't displayed against MSVC build hosts as they are for regular hosts, eg: http://www.pgbuildfarm.org/cgi-bin/show_log.pl?nm=mastodondt=2007-04-25%2001:00:02 vs. http://www.pgbuildfarm.org/cgi-bin/show_log.pl?nm=narwhaldt=2007-04-25%2002:00:03 Is this WIP,

[HACKERS] autovacuum does not start in HEAD

2007-04-25 Thread ITAGAKI Takahiro
I found that autovacuum launcher does not launch any workers in HEAD. AFAICS, we track the time to be vaccumed of each database in the following way: 1. In rebuild_database_list(), we initialize avl_dbase-adl_next_worker with (current_time + autovacuum_naptime / nDBs). 2. In

[HACKERS] ECPG failure on BF member Vaquita (Windows Vista)

2007-04-25 Thread Dave Page
I'm seeing an ECPG-Check failure on Windows Vista - any ideas what might be causing this? http://www.pgbuildfarm.org/cgi-bin/show_log.pl?nm=vaquitadt=2007-04-24%2020:00:05 The only other Vista buildfarm member (baiji, on the same physical box) is running MSVC builds which don't yet test ECPG

Re: [HACKERS] Buildfarm: Stage logs not available for MSVC builds

2007-04-25 Thread Dave Page
Andrew Dunstan wrote: The problem not beacuse of MSVC, but because of member misconfiguration, by the look of it. The tar command string will need to be set in the config file and tar installed. I found that I needed bsdtar for Windows for this to work. See Ah, OK, thanks - there was a typo

[HACKERS] Avoiding unnecessary reads in recovery

2007-04-25 Thread Heikki Linnakangas
In recovery, with full_pages_writes=on, we read in each page only to overwrite the contents with a full page image. That's a waste of time, and can have a surprisingly large effect on recovery time. As a quick test on my laptop, I initialized a DBT-2 test with 5 warehouses, and let it run for

Re: [PATCHES] [HACKERS] Full page writes improvement, code update

2007-04-25 Thread Kenneth Marshall
On Wed, Apr 25, 2007 at 10:00:16AM +0200, Zeugswetter Andreas ADI SD wrote: 1) To deal with partial/inconsisitent write to the data file at crash recovery, we need full page writes at the first modification to pages after each checkpoint. It consumes much of WAL space. We need

Re: [HACKERS] autovacuum does not start in HEAD

2007-04-25 Thread Alvaro Herrera
ITAGAKI Takahiro wrote: I found that autovacuum launcher does not launch any workers in HEAD. AFAICS, we track the time to be vaccumed of each database in the following way: 1. In rebuild_database_list(), we initialize avl_dbase-adl_next_worker with (current_time + autovacuum_naptime

Re: [HACKERS] Avoiding unnecessary reads in recovery

2007-04-25 Thread Heikki Linnakangas
Heikki Linnakangas wrote: While working on this, this comment in ReadBuffer caught my eye: /* * During WAL recovery, the first access to any data page should * overwrite the whole page from the WAL; so a clobbered page * header is not reason to fail. Hence, when InRecovery

Re: [HACKERS] Avoiding unnecessary reads in recovery

2007-04-25 Thread Gregory Stark
Heikki Linnakangas [EMAIL PROTECTED] writes: While working on this, this comment in ReadBuffer caught my eye: /* * During WAL recovery, the first access to any data page should * overwrite the whole page from the WAL; so a clobbered page * header is not reason to

Re: [HACKERS] ECPG failure on BF member Vaquita (Windows Vista)

2007-04-25 Thread Michael Meskes
On Wed, Apr 25, 2007 at 10:47:57AM +0100, Dave Page wrote: I'm seeing an ECPG-Check failure on Windows Vista - any ideas what might be causing this? Hmm, first glance suggests some permission problems. I never touched a Vista system so far, so I'm at a loss as far as details are concerned. I

[HACKERS] database size estimates

2007-04-25 Thread Francois Deliege
Hi, I am trying to estimate the size of a table composed of 51754000 rows. Each row has 31 attributes: 16 x bit(24) and 15 x bit(8) So, the payload should be: 51754000 x ( 16 x 24 + 15 x 24 ) bits = 3115 MB Now, from what I understand from postgresql manual is that the overhead is composed of

Re: [HACKERS] ECPG failure on BF member Vaquita (Windows Vista)

2007-04-25 Thread Dave Page
Michael Meskes wrote: On Wed, Apr 25, 2007 at 10:47:57AM +0100, Dave Page wrote: I'm seeing an ECPG-Check failure on Windows Vista - any ideas what might be causing this? Hmm, first glance suggests some permission problems. Yes, that was my thought as well, however I ran cacls down the

Re: [HACKERS] database size estimates

2007-04-25 Thread Heikki Linnakangas
Francois Deliege wrote: Hi, I am trying to estimate the size of a table composed of 51754000 rows. Each row has 31 attributes: 16 x bit(24) and 15 x bit(8) So, the payload should be: 51754000 x ( 16 x 24 + 15 x 24 ) bits = 3115 MB What data types are those exactly? If those 24-bit fields are

[HACKERS] Kill a Long Running Query

2007-04-25 Thread Mageshwaran
Hi , Any body tell me how to kill a long running query in postgresql, is there any statement to kill a query, and also tell me how to log slow queries to a log file. Regards J Mageshwaran ** DISCLAIMER ** Information contained and transmitted by this E-MAIL is proprietary to

Re: [HACKERS] [ADMIN] Kill a Long Running Query

2007-04-25 Thread Aaron Bono
On 4/25/07, Mageshwaran [EMAIL PROTECTED] wrote: Hi , Any body tell me how to kill a long running query in postgresql, is there any statement to kill a query, and also tell me how to log slow queries to a log file. Regards J Mageshwaran See if this helps:

Re: [HACKERS] Kill a Long Running Query

2007-04-25 Thread Andrew Dunstan
Mageshwaran wrote: Hi , Any body tell me how to kill a long running query in postgresql, is there any statement to kill a query, and also tell me how to log slow queries to a log file. First. please do not cross-post like this. Pick the correct list and use it. Second, this query

Re: [HACKERS] Kill a Long Running Query

2007-04-25 Thread Heikki Linnakangas
Please don't cross-post to multiple mailing lists. And pgsql-hackers is not the correct list for basic usage questions. And long end-of-mail disclaimers are not generally appreciated. Mageshwaran wrote: Any body tell me how to kill a long running query in postgresql, is there any statement to

Re: [HACKERS] Avoiding unnecessary reads in recovery

2007-04-25 Thread Heikki Linnakangas
Gregory Stark wrote: So in short I think with your patch this piece of code no longer has a role. Either your patch kicks in and we never even look at the damaged page at all, or we should be treating it as corrupt data and just check zero_damaged_pages alone and not do anything special in

Re: [HACKERS] ECPG failure on BF member Vaquita (Windows Vista)

2007-04-25 Thread Andrew Dunstan
Dave Page wrote: Michael Meskes wrote: On Wed, Apr 25, 2007 at 10:47:57AM +0100, Dave Page wrote: I'm seeing an ECPG-Check failure on Windows Vista - any ideas what might be causing this? Hmm, first glance suggests some permission problems. Yes, that was my thought as

Re: [HACKERS] ECPG failure on BF member Vaquita (Windows Vista)

2007-04-25 Thread Dave Page
Andrew Dunstan wrote: Please don't do that on your buildfarm repo copy (if that's what you did). You should not touch *anything* inside it. If need to you do this, make a copy (see later) and alter that. If you did do this to the buildfarm repo copy, please blow it away so that buildfarm

Re: [HACKERS] ECPG failure on BF member Vaquita (Windows Vista)

2007-04-25 Thread Andrew Dunstan
Dave Page wrote: Andrew Dunstan wrote: Please don't do that on your buildfarm repo copy (if that's what you did). You should not touch *anything* inside it. If need to you do this, make a copy (see later) and alter that. If you did do this to the buildfarm repo copy, please blow it away so

Re: [HACKERS] Avoiding unnecessary reads in recovery

2007-04-25 Thread Simon Riggs
On Wed, 2007-04-25 at 13:48 +0100, Heikki Linnakangas wrote: I was surprised how big a difference it makes, but when you think about it it's logical. Without the patch, it's doing roughly the same I/O as the test itself, reading in pages, modifying them, and writing them back. With the

Re: [HACKERS] temporal variants of generate_series()

2007-04-25 Thread Neil Conway
On Thu, 2007-04-12 at 14:56 -0700, Andrew Hammond wrote: I've written the following function definitions to extend generate_series to support some temporal types (timestamptz, date and time). Please include them if there's sufficient perceived need or value. I could see these being useful,

Re: [HACKERS] ECPG failure on BF member Vaquita (Windows Vista)

2007-04-25 Thread Mark Wong
On 4/25/07, Michael Meskes [EMAIL PROTECTED] wrote: I also saw that wombat is segfaulting in ecpg tests but not only with CVS HEAD but also trying to test 8.2. Any idea what's going on with this machine? I generated a stack trace for REL8_2_STABLE, but I'm not sure how helpful it is. Let me

Re: [HACKERS] ECPG failure on BF member Vaquita (Windows Vista)

2007-04-25 Thread Andrew Dunstan
Mark Wong wrote: On 4/25/07, Michael Meskes [EMAIL PROTECTED] wrote: I also saw that wombat is segfaulting in ecpg tests but not only with CVS HEAD but also trying to test 8.2. Any idea what's going on with this machine? I generated a stack trace for REL8_2_STABLE, but I'm not sure how

Re: [HACKERS] Avoiding unnecessary reads in recovery

2007-04-25 Thread Tom Lane
Simon Riggs [EMAIL PROTECTED] writes: As regards the zero_damaged_pages question, I raised that some time ago but we didn't arrive at an explicit answer. All I would say is we can't allow invalid pages in the buffer manager at any time, whatever options we have requested, otherwise other code

Re: [HACKERS] ECPG failure on BF member Vaquita (Windows Vista)

2007-04-25 Thread Tom Lane
Andrew Dunstan [EMAIL PROTECTED] writes: I think you'll need to compile with optimisation turned off and then try running the test under debugger control, putting a breakpoint in ECPGget_variable() and then stepping through it. I wonder what value of var-ind_pointer it is getting? You

Re: [PATCHES] [HACKERS] Full page writes improvement, code update

2007-04-25 Thread Josh Berkus
Andreas, Writing to a different area was considered in pg, but there were more negative issues than positive. So imho pg_compresslog is the correct path forward. The current discussion is only about whether we want a more complex pg_compresslog and no change to current WAL, or an increased

Re: [HACKERS] ECPG failure on BF member Vaquita (Windows Vista)

2007-04-25 Thread Mark Wong
On 4/25/07, Tom Lane [EMAIL PROTECTED] wrote: Andrew Dunstan [EMAIL PROTECTED] writes: I think you'll need to compile with optimisation turned off and then try running the test under debugger control, putting a breakpoint in ECPGget_variable() and then stepping through it. I wonder what value

Re: [HACKERS] [BUGS] BUG #3245: PANIC: failed to re-find shared loc k o b j ect

2007-04-25 Thread Tom Lane
I wrote: Heikki Linnakangas [EMAIL PROTECTED] writes: We could have two kinds of seq scans, with and without support for concurrent inserts. Yeah, I considered that too, but it just seems too error-prone. We could maybe make it trustworthy by having hash_seq_search complain if it noticed

Re: [HACKERS] strange buildfarm failures

2007-04-25 Thread Stefan Kaltenbrunner
Stefan Kaltenbrunner wrote: two of my buildfarm members had different but pretty weird looking failures lately: http://www.pgbuildfarm.org/cgi-bin/show_log.pl?nm=quaggadt=2007-04-25%2002:03:03 and http://www.pgbuildfarm.org/cgi-bin/show_log.pl?nm=emudt=2007-04-24%2014:35:02 any

Re: [HACKERS] ECPG failure on BF member Vaquita (Windows Vista)

2007-04-25 Thread Mark Wong
On 4/25/07, Tom Lane [EMAIL PROTECTED] wrote: Mark Wong [EMAIL PROTECTED] writes: Does this help? (gdb) p var-ind_pointer $8 = (void *) 0x0 Well, that seems to be the reason why it's failing to indirect through ind_pointer ... but why is it only failing on your machine and not everyone

Re: [HACKERS] ECPG failure on BF member Vaquita (Windows Vista)

2007-04-25 Thread Andrew Dunstan
Tom Lane wrote: Hmm, and I don't have to look far to find a smoking gun: #if defined(__GNUC__) (defined (__powerpc__) || defined(__amd64__) || defined(__x86_64__)) if (create_statement(lineno, compat, force_indicator, con, stmt, query, args) == false) #else if

[HACKERS] My upcoming travel

2007-04-25 Thread Bruce Momjian
I am leaving Friday for a two week trip to Sydney and Melbourne, Australia, Mumbai and Pune, India, and London, England. I will be attending Open Cebit in Sydney, and a Sydney PostgreSQL Users Group meeting. London is planning to put together a user group meeting. I expect both events to appear

Re: [HACKERS] ECPG failure on BF member Vaquita (Windows Vista)

2007-04-25 Thread Tom Lane
Andrew Dunstan [EMAIL PROTECTED] writes: But I also see that my amd64/FC6 machine does pass these tests with gcc. Yeah, but the typedef represented by va_list can and probably does vary between amd64 and ppc64. I haven't an easy way to check, but I wonder whether it's not an array type on ppc.

Re: [HACKERS] Fragmentation project

2007-04-25 Thread Gustavo Tonini
Josh, On 4/23/07, Josh Berkus [EMAIL PROTECTED] wrote: Gustavo, Oh, you're talking about distributing partitions across different nodes and parallelizing queries. No, we don't do that today. Yes.This is the goal. Well, I will try it. I'll send the project reports to this list. Comments

Re: [HACKERS] Fragmentation project

2007-04-25 Thread Gustavo Tonini
Marko, On 4/24/07, Marko Kreen [EMAIL PROTECTED] wrote: On 4/23/07, Heikki Linnakangas [EMAIL PROTECTED] wrote: Oh, you're talking about distributing partitions across different nodes and parallelizing queries. No, we don't do that today. PL/Proxy actually works like that, only in smaller

[HACKERS] Re: [BUGS] BUG #3245: PANIC: failed to re-find shared loc k o b j ect

2007-04-25 Thread Heikki Linnakangas
Tom Lane wrote: I wrote: Heikki Linnakangas [EMAIL PROTECTED] writes: We could have two kinds of seq scans, with and without support for concurrent inserts. Yeah, I considered that too, but it just seems too error-prone. We could maybe make it trustworthy by having hash_seq_search complain

Re: [HACKERS] Avoiding unnecessary reads in recovery

2007-04-25 Thread Heikki Linnakangas
Tom Lane wrote: Simon Riggs [EMAIL PROTECTED] writes: As regards the zero_damaged_pages question, I raised that some time ago but we didn't arrive at an explicit answer. All I would say is we can't allow invalid pages in the buffer manager at any time, whatever options we have requested,

Re: [HACKERS] [BUGS] BUG #3245: PANIC: failed to re-find shared loc k o b j ect

2007-04-25 Thread Tom Lane
Heikki Linnakangas [EMAIL PROTECTED] writes: Tom Lane wrote: I still don't want to introduce more checking overhead into hash_seq_search, though, so what I'm now thinking about is a new dynahash primitive named something like hash_freeze, which'd mark a hashtable as disallowing insertions.

Re: [HACKERS] Avoiding unnecessary reads in recovery

2007-04-25 Thread Tom Lane
Heikki Linnakangas [EMAIL PROTECTED] writes: Tom Lane wrote: but it'd make it safe to use in non-WAL contexts (I think there are other places where we know we are going to init the page and so a physical read is a waste of time). Is there? I can't think of any. Extending a relation doesn't

Re: [PATCHES] [HACKERS] Full page writes improvement, code update

2007-04-25 Thread Tom Lane
Josh Berkus [EMAIL PROTECTED] writes: Andreas, So imho pg_compresslog is the correct path forward. The current discussion is only about whether we want a more complex pg_compresslog and no change to current WAL, or an increased WAL size for a less complex implementation. Both would be able

Re: [HACKERS] strange buildfarm failures

2007-04-25 Thread Tom Lane
Stefan Kaltenbrunner [EMAIL PROTECTED] writes: Stefan Kaltenbrunner wrote: two of my buildfarm members had different but pretty weird looking failures lately: http://www.pgbuildfarm.org/cgi-bin/show_log.pl?nm=quaggadt=2007-04-25%2002:03:03 and

Re: [HACKERS] database size estimates

2007-04-25 Thread Gregory Stark
Heikki Linnakangas [EMAIL PROTECTED] writes: Francois Deliege wrote: Hi, I am trying to estimate the size of a table composed of 51754000 rows. Each row has 31 attributes: 16 x bit(24) and 15 x bit(8) So, the payload should be: 51754000 x ( 16 x 24 + 15 x 24 ) bits = 3115 MB What data

Re: [HACKERS] Fragmentation project

2007-04-25 Thread Josh Berkus
Gustavo, The pgpool is an interesting approach to this, but I think that the funcionality of inserting a record at a backend which will be redirectioned to other and verifying deadlocks under network demands in acquiring locks on the referenced records/tables in several hosts. Then, IMO, this

Re: [PATCHES] [HACKERS] Full page writes improvement, code update

2007-04-25 Thread Koichi Suzuki
Hi, Zeugswetter Andreas ADI SD wrote: I don't insist the name and the default of the GUC parameter. I'm afraid wal_fullpage_optimization = on (default) makes some confusion because the default behavior becomes a bit different on WAL itself. Seems my wal_fullpage_optimization is not a good

Re: [HACKERS] autovacuum does not start in HEAD

2007-04-25 Thread ITAGAKI Takahiro
I wrote: I found that autovacuum launcher does not launch any workers in HEAD. The attached autovacuum-fix.patch could fix the problem. I changed to use 'greater or equal' instead of 'greater' at the decision of next autovacuum target. The point was in the resolution of timer; There is a

Re: [HACKERS] strange buildfarm failures

2007-04-25 Thread Alvaro Herrera
Tom Lane wrote: Stefan Kaltenbrunner [EMAIL PROTECTED] writes: Stefan Kaltenbrunner wrote: two of my buildfarm members had different but pretty weird looking failures lately: http://www.pgbuildfarm.org/cgi-bin/show_log.pl?nm=quaggadt=2007-04-25%2002:03:03 and

Re: [HACKERS] strange buildfarm failures

2007-04-25 Thread Stefan Kaltenbrunner
Alvaro Herrera wrote: Tom Lane wrote: Stefan Kaltenbrunner [EMAIL PROTECTED] writes: Stefan Kaltenbrunner wrote: two of my buildfarm members had different but pretty weird looking failures lately: http://www.pgbuildfarm.org/cgi-bin/show_log.pl?nm=quaggadt=2007-04-25%2002:03:03 and