Re: [PATCHES] [HACKERS] ARC Memory Usage analysis

2004-10-26 Thread Curt Sampson
On Tue, 26 Oct 2004, Greg Stark wrote: I see mmap or O_DIRECT being the only viable long-term stable states. My natural inclination was the former but after the latest thread on the subject I suspect it'll be forever out of reach. That makes O_DIRECT And a Postgres managed cache the only real

Re: [HACKERS] Using ALTER TABLESPACE in pg_dump

2004-10-26 Thread Fabien COELHO
Dear Tom, ISTM that the core business of a database is to help organize and protect data, and it is plainly that. You just wish you won't need it, so it is somehow abstract, but when and if you need it, it is not second-order at all;-) and it is much too late to redo the dump. So you create some

Re: [PATCHES] [HACKERS] ARC Memory Usage analysis

2004-10-26 Thread Simon Riggs
On Mon, 2004-10-25 at 16:34, Jan Wieck wrote: The problem is, with a too small directory ARC cannot guesstimate what might be in the kernel buffers. Nor can it guesstimate what recently was in the kernel buffers and got pushed out from there. That results in a way too small B1 list, and

Re: [PATCHES] [HACKERS] ARC Memory Usage analysis

2004-10-26 Thread Simon Riggs
On Tue, 2004-10-26 at 06:53, Tom Lane wrote: Greg Stark [EMAIL PROTECTED] writes: Tom Lane [EMAIL PROTECTED] writes: Another issue is what we do with the effective_cache_size value once we have a number we trust. We can't readily change the size of the ARC lists on the fly. Huh? I

[HACKERS]

2004-10-26 Thread Bernd Helmle
[sorry if this mail appears more than once, but it seems the others didn't make it through the list] This is a short preview on the view update code i'm currently working on. It is far away from being ready, but i want to share the current code and get some hints, what things have to be made

Re: [HACKERS] plans for bitmap indexes?

2004-10-26 Thread Hannu Krosing
On K, 2004-10-20 at 03:03, Simon Riggs wrote: Well, thats the best one yet. That's the solution, if ever I heard it. The reduction in bitmap size makes their use much safer. Size matters, since we're likely to start using these techniques on very large databases, which imply obviously have

Re: [HACKERS] Automatic view update rules

2004-10-26 Thread Bernd Helmle
--On Dienstag, Oktober 26, 2004 13:02:27 +0200 Bernd Helmle [EMAIL PROTECTED] wrote: [sorry if this mail appears more than once, but it seems the others didn't make it through the list] This is a short preview on the view update code i'm currently working on. It is far away from being ready, but

Re: [HACKERS] plans for bitmap indexes?

2004-10-26 Thread Hannu Krosing
On K, 2004-10-20 at 01:52, Mark Kirkwood wrote: I don't believe that read only is required. The update/insert performance impact of bimap indexes is however very high (in Oracle's implementation anyway) - to the point where many sites drop them before adding in new data, and recreated 'em

Re: [PATCHES] [HACKERS] ARC Memory Usage analysis

2004-10-26 Thread Simon Riggs
On Tue, 2004-10-26 at 09:49, Simon Riggs wrote: On Mon, 2004-10-25 at 16:34, Jan Wieck wrote: The problem is, with a too small directory ARC cannot guesstimate what might be in the kernel buffers. Nor can it guesstimate what recently was in the kernel buffers and got pushed out from

Re: [HACKERS] Possible make_oidjoins_check Security Issue

2004-10-26 Thread Bruce Momjian
Tom Lane wrote: Neil Conway [EMAIL PROTECTED] writes: On Wed, 2004-10-20 at 06:18, Rod Taylor wrote: http://secunia.com/advisories/12860/ This seems like a rather inconsequential problem, Indeed, since ordinary users have no use for make_oidjoins_check. It's surely very implausible

Re: [HACKERS] rmtree() failure on Windows

2004-10-26 Thread Andrew Dunstan
Here is some more info. Below is a trace from dropdb. There is a loop around the rmdir() calls which I have set to time out at 600 seconds. The call eventually succeeds after around 300 seconds (I've seen this several times). It looks like we are the victim of some caching - the directory

Re: [HACKERS] Possible make_oidjoins_check Security Issue

2004-10-26 Thread Tom Lane
Bruce Momjian [EMAIL PROTECTED] writes: I believe the proper way to handle this is a new directory under /tmp. It's definitely not worth the trouble. I looked at what configure does to make /tmp subdirectories portably, and it is spectacularly ugly (not to mention long). If

[HACKERS] making pdf of docs

2004-10-26 Thread Dennis Bjorklund
Is there something wrong that makes it impossible to build the doc as a pdf? I started a build 4 hours ago, and it has still not finished (stuck at 100% CPU on my old 800Mhz 1G RAM machine). I know that openjade is very slow so for the first 3 hours I didn't worry. Now I'm starting to think

Re: [HACKERS] making pdf of docs

2004-10-26 Thread Kris Jurka
On Tue, 26 Oct 2004, Dennis Bjorklund wrote: Is there something wrong that makes it impossible to build the doc as a pdf? My experience is that the latest openjade crashes. The latest jade takes about 10 days on an Athlon 1600, but I can build it in a very reasonable timeframe with an

Re: [PATCHES] [HACKERS] ARC Memory Usage analysis

2004-10-26 Thread Greg Stark
Curt Sampson [EMAIL PROTECTED] writes: On Tue, 26 Oct 2004, Greg Stark wrote: I see mmap or O_DIRECT being the only viable long-term stable states. My natural inclination was the former but after the latest thread on the subject I suspect it'll be forever out of reach. That makes

Re: [HACKERS] making pdf of docs

2004-10-26 Thread Mike Mascari
Dennis Bjorklund wrote: Is there something wrong that makes it impossible to build the doc as a pdf? I started a build 4 hours ago, and it has still not finished (stuck at 100% CPU on my old 800Mhz 1G RAM machine). I know that openjade is very slow so for the first 3 hours I didn't worry. Now

Re: [HACKERS] plans for bitmap indexes?

2004-10-26 Thread Greg Stark
Hannu Krosing [EMAIL PROTECTED] writes: I repeat here my earlier proposal of making the bitmap indexes page-level and clustering data automatically on AND of all defined bitmap indexes. The problem with page-level bitmaps is that they could be much less effective. Consider a query like

[HACKERS] Postgres performs a Seq Scan instead of an Index Scan!

2004-10-26 Thread Jos van Roosmalen
Hello, I have a little question. Why performs Postgresql a Seq. Scan in the next Select statement instead of a Index Read? I have an index on (ATTR1,ATTR2,ATTR3), so why is postgresql not performing a Index Keyed Read in the SELECT? I agree that the tables are empty so maybe this influence the

Re: [HACKERS] Postgres performs a Seq Scan instead of an Index Scan!

2004-10-26 Thread James Robinson
On Oct 26, 2004, at 12:12 PM, Jos van Roosmalen wrote: ATTR1 INT8 Looks like your column is int8, yet your query is sending in an int4. Therefore the index is not used. This is fixed in PG 8.0. In the mean time, you can: SELECT * FROM TESTTABLE WHERE ATTR1=1::INT8 ... which explicitly

Re: [HACKERS] Postgres performs a Seq Scan instead of an Index Scan!

2004-10-26 Thread Jochem van Dieten
On Tue, 26 Oct 2004 18:12:36 +0200, Jos van Roosmalen wrote: I have a little question. Why performs Postgresql a Seq. Scan in the next Select statement instead of a Index Read? That is a FAQ: http://www.postgresql.org/docs/faqs/FAQ.html#4.8 Please direct any further questions of this nature

Re: [HACKERS] Postgres performs a Seq Scan instead of an Index Scan!

2004-10-26 Thread Kurt Roeckx
On Tue, Oct 26, 2004 at 06:12:36PM +0200, Jos van Roosmalen wrote: CREATE TABLE TESTTABLE (ATTR1 INT8,ATTR2 INT4,ATTR3 TIMESTAMP); CREATE UNIQUE INDEX TESTINDEX ON TESTTABLE(ATTR1,ATTR2,ATTR3); EXPLAIN SELECT * FROM TESTTABLE WHERE ATTR1=1 AND ATTR2=2 AND ATTR3='2004-01-01'; try: explain

Re: [HACKERS] rmtree() failure on Windows

2004-10-26 Thread Tom Lane
Andrew Dunstan [EMAIL PROTECTED] writes: Here is some more info. Below is a trace from dropdb. There is a loop around the rmdir() calls which I have set to time out at 600 seconds. The call eventually succeeds after around 300 seconds (I've seen this several times). It looks like we are the

Re: [HACKERS] to_char/to_number loses sign

2004-10-26 Thread Tom Lane
Karel Zak [EMAIL PROTECTED] writes: Yes, you're right. It strange, but NUM_S missing there. The conversion from string to number is less stable part of formatting.c... The patch is in the attachment. This patch causes the regression tests to fail. I think you need to consider the to_char()

Re: [HACKERS] Postresql 8.0 Beta 3 - SELECT ... FOR UPDATE

2004-10-26 Thread Bruce Momjian
There is a statement_timeout that will control how long a statement can execute before being cancelled. We have never agreed that controlling how long we wait for an individual lock is valuable. --- Robert Treat wrote: On

[HACKERS] New compile warnings in CVS

2004-10-26 Thread Bruce Momjian
I just updated my CVS copy and am seeing four new warnings from pgstat.c: pgstat.c:2352: warning: variable `dbentry' might be clobbered by `longjmp' or `vfork' pgstat.c:2360: warning: variable `havebackends' might be clobbered by `longjmp' or `vfork'

Re: [HACKERS] New compile warnings in CVS

2004-10-26 Thread Bruce Momjian
Bruce Momjian wrote: I just updated my CVS copy and am seeing four new warnings from pgstat.c: pgstat.c:2352: warning: variable `dbentry' might be clobbered by `longjmp' or `vfork' pgstat.c:2360: warning: variable `havebackends' might be clobbered by `longjmp'

Re: [HACKERS] rmtree() failure on Windows

2004-10-26 Thread Andrew Dunstan
Tom Lane wrote: Andrew Dunstan [EMAIL PROTECTED] writes: Here is some more info. Below is a trace from dropdb. There is a loop around the rmdir() calls which I have set to time out at 600 seconds. The call eventually succeeds after around 300 seconds (I've seen this several times). It looks

Re: [HACKERS] New compile warnings in CVS

2004-10-26 Thread Tom Lane
Bruce Momjian [EMAIL PROTECTED] writes: I just updated my CVS copy and am seeing four new warnings from pgstat.c: pgstat.c:2352: warning: variable `dbentry' might be clobbered by `longjmp' or `vfork' pgstat.c:2360: warning: variable `havebackends' might be clobbered

[HACKERS] 8.0b4: COMMIT outside of a transaction echoes ROLLBACK

2004-10-26 Thread Ian Barwick
just wondering: test= select version(); version -- PostgreSQL 8.0.0beta4 on i686-pc-linux-gnu, compiled by GCC gcc (GCC) 3.3.3 (SuSE Linux) (1 row) test= begin; BEGIN

Re: [HACKERS] timestamp with time zone a la sql99

2004-10-26 Thread Bruce Momjian
Added to TODO: * Once we expand timestamptz to bigger than 8 bytes, there's essentially --- Tom Lane wrote: Dennis Bjorklund [EMAIL PROTECTED] writes: So if I understand you correctly you are planning to extend

Re: [HACKERS]

2004-10-26 Thread Hannu Krosing
On T, 2004-10-26 at 14:02, Bernd Helmle wrote: - gram.y is only an ugly hack to get the CHECK OPTION working. needs deeper efforts, because it makes WITH a reserved keyword IMHO it should be a reserved keyword. I once wrangled to fit ANSI SQL recursive queries into postgres grammar and

[HACKERS] DBT-3 Query 2 EXPLAIN ANALYZE differences

2004-10-26 Thread Mark Wong
I was doing some testing with DBT-3 on our 8-way STP systems and noticed a significant difference in the execution of Query 2 using 8.0beta3. Here is the query template we're using: select s_acctbal, s_name, n_name, p_partkey, p_mfgr, s_address, s_phone, s_comment from part, supplier,

Re: [HACKERS] plans for bitmap indexes?

2004-10-26 Thread Hannu Krosing
On T, 2004-10-26 at 18:42, Greg Stark wrote: Hannu Krosing [EMAIL PROTECTED] writes: I repeat here my earlier proposal of making the bitmap indexes page-level and clustering data automatically on AND of all defined bitmap indexes. The problem with page-level bitmaps is that they could

Re: [HACKERS] plans for bitmap indexes?

2004-10-26 Thread Hannu Krosing
On T, 2004-10-26 at 23:53, Hannu Krosing wrote: On T, 2004-10-26 at 18:42, Greg Stark wrote: Hannu Krosing [EMAIL PROTECTED] writes: I repeat here my earlier proposal of making the bitmap indexes page-level and clustering data automatically on AND of all defined bitmap indexes.

Re: [HACKERS] plans for bitmap indexes?

2004-10-26 Thread Andre Maasikas
Hannu Krosing wrote: the per-page clustering would make sure that all the tuples would be on 1 (or on a few) pages. I understand that You can cluster on one column, but how do you do it for indexes on other columns? BTW, lossy variants also lose count(), group by only from index and what comes to

Re: [HACKERS] Unixware 714 pthreads

2004-10-26 Thread Bruce Momjian
The only help I can be is that on Unixware (only) the backend is compiled with threading enabled. This might be showing some thread bugs. --- [EMAIL PROTECTED] wrote: Hi every one, I need help to debug the problem I

Re: [HACKERS] rmtree() failure on Windows

2004-10-26 Thread Reini Urban
Andrew Dunstan schrieb: Here is some more info. Below is a trace from dropdb. There is a loop around the rmdir() calls which I have set to time out at 600 seconds. The call eventually succeeds after around 300 seconds (I've seen this several times). It looks like we are the victim of some

[HACKERS] Should bgwriter log checkpoint start/end?

2004-10-26 Thread Tom Lane
In previous releases it was possible to observe whether an automatic checkpoint was in progress by looking to see if there was a postmaster child process doing one. In 8.0 this will not work because the bgwriter is always there. I am thinking that for tasks such as performance debugging it would

Re: [HACKERS] DBT-3 Query 2 EXPLAIN ANALYZE differences

2004-10-26 Thread Josh Berkus
Mark, The plans are different and I suspect thats where the differences lie. For brevity (and readability) I won't copy the plans here but I'll provide the links. Search for 'PERF1.POWER.Q2' in the file, it's the second query executed and you'll notice the differences under the SubPlan:

Re: [PERFORM] [PATCHES] [HACKERS] ARC Memory Usage analysis

2004-10-26 Thread Josh Berkus
Thomas, As a result, I was intending to inflate the value of effective_cache_size to closer to the amount of unused RAM on some of the machines I admin (once I've verified that they all have a unified buffer cache). Is that correct? Currently, yes. Right now, e_c_s is used just to inform

Re: [HACKERS] New compile warnings in CVS

2004-10-26 Thread Neil Conway
On Wed, 2004-10-27 at 03:57, Tom Lane wrote: No doubt this is from the PG_TRY that Neil added a couple days ago. I think he is going to take it out again in favor of using AllocateFile, so ignore the warnings for now (they're obviously bogus anyway). Sorry, I didn't see those compile warnings

Re: [PATCHES] [HACKERS] ARC Memory Usage analysis

2004-10-26 Thread Curt Sampson
On Wed, 26 Oct 2004, Greg Stark wrote: I don't see why mmap is any more out of reach than O_DIRECT; it's not all that much harder to implement, and mmap (and madvise!) is more widely available. Because there's no way to prevent a write-out from occurring and no way to be notified by mmap

Re: [HACKERS] 8.0b4: COMMIT outside of a transaction echoes ROLLBACK

2004-10-26 Thread Bruce Momjian
Ian Barwick wrote: just wondering: test= select version(); version -- PostgreSQL 8.0.0beta4 on i686-pc-linux-gnu, compiled by GCC gcc (GCC) 3.3.3 (SuSE Linux)

Re: [PATCHES] [HACKERS] ARC Memory Usage analysis

2004-10-26 Thread Scott Marlowe
On Mon, 2004-10-25 at 23:53, Tom Lane wrote: Greg Stark [EMAIL PROTECTED] writes: Tom Lane [EMAIL PROTECTED] writes: Another issue is what we do with the effective_cache_size value once we have a number we trust. We can't readily change the size of the ARC lists on the fly. Huh? I

[HACKERS] pg_dump test success

2004-10-26 Thread Christopher Kings-Lynne
Hi guys, I just thought I'd let you know that i just dumped our production database with 8.0's pg_dumpall and reloaded it into a test 8.0 database. It worked. No errors. For the first time in our company's history with PostgreSQL, we can upgrade without editing the dump file!!! I feel like my