[HACKERS] Experimental ARC implementation

2003-11-03 Thread Jan Wieck
Attached is a first trial implementation of the Adaptive Replacement Cache (ARC). The patch is against CVS HEAD of 7.4. The algorithm is based on what's described in these papers: http://www.almaden.ibm.com/StorageSystems/autonomic_storage/ARC/arcfast.pdf

Re: [HACKERS] Experimental patch for inter-page delay in VACUUM

2003-11-03 Thread Jan Wieck
Christopher Browne wrote: The world rejoiced as [EMAIL PROTECTED] (Hannu Krosing) wrote: Christopher Browne kirjutas E, 03.11.2003 kell 02:15: Well, actually, the case where it _would_ be troublesome would be where there was a combination of huge tables needing vacuuming and smaller ones that are

Re: [HACKERS] Experimental ARC implementation

2003-11-03 Thread Jan Wieck
Jan Wieck wrote: I will follow up shortly with an approach that integrates Tom's delay mechanism plus my first READ_BY_VACUUM hack into one combined experiement. Okay, the attached patch contains the 3 already discussed and one additional change. I also made a few changes. 1) ARC policy. Has

Re: [HACKERS] 7.4RC1 failed to build on Linux

2003-11-03 Thread Jan Wieck
Marc G. Fournier wrote: what version of tcl do you have installed? Looks like 8.0. As per recent discussion we are actually require 8.1 exactly because of those missing functions in all versions prior to that. Jan On Tue, 4 Nov 2003, Kenji Sugita wrote: Failed to build on Linux. $ uname

Re: [HACKERS] Experimental patch for inter-page delay in VACUUM

2003-11-03 Thread Jan Wieck
Ang Chin Han wrote: Christopher Browne wrote: Centuries ago, Nostradamus foresaw when Stephen [EMAIL PROTECTED] would write: As it turns out. With vacuum_page_delay = 0, VACUUM took 1m20s (80s) to complete, with vacuum_page_delay = 1 and vacuum_page_delay = 10, both VACUUMs completed in 18m3s

Re: [HACKERS] Experimental ARC implementation

2003-11-04 Thread Jan Wieck
Jan Wieck wrote: Jan Wieck wrote: I will follow up shortly with an approach that integrates Tom's delay mechanism plus my first READ_BY_VACUUM hack into one combined experiement. Okay, the attached patch contains the 3 already discussed and one additional change. Ooopsy the B1/B2 queue

Re: [HACKERS] bufmgr code question

2003-11-04 Thread Jan Wieck
Neil Conway wrote: In the BufferDesc struct, there seem to be two ways to mark a buffer page as dirty: setting the BM_DIRTY bit mask in the 'flags' field of the struct, and setting the 'cntxDirty' field to true. What is the difference between these two indications of a page's dirtiness? I don't

Re: [HACKERS] Experimental patch for inter-page delay in VACUUM

2003-11-04 Thread Jan Wieck
Tom Lane wrote: Jan Wieck [EMAIL PROTECTED] writes: What still needs to be addressed is the IO storm cause by checkpoints. I see it much relaxed when stretching out the BufferSync() over most of the time until the next one should occur. But the kernel sync at it's end still pushes the system

Re: [HACKERS] Experimental patch for inter-page delay in VACUUM

2003-11-04 Thread Jan Wieck
Tom Lane wrote: Jan Wieck [EMAIL PROTECTED] writes: Tom Lane wrote: I have never been happy with the fact that we use sync(2) at all. Sure does it do too much. But together with the other layer of indirection, the virtual file descriptor pool, what is the exact guaranteed behaviour

Re: [HACKERS] Experimental ARC implementation

2003-11-04 Thread Jan Wieck
Jan Wieck wrote: Jan Wieck wrote: Jan Wieck wrote: I will follow up shortly with an approach that integrates Tom's delay mechanism plus my first READ_BY_VACUUM hack into one combined experiement. Okay, the attached patch contains the 3 already discussed and one additional change. Ooopsy

Re: [HACKERS] Experimental ARC implementation

2003-11-05 Thread Jan Wieck
Zeugswetter Andreas SB SD wrote: My plan is to create another background process very similar to the checkpointer and to let that run forever basically looping over that BufferSync() with a bool telling that it's the bg_writer. Why not use the checkpointer itself inbetween checkpoints ? use a

[HACKERS] Performance features the 4th

2003-11-05 Thread Jan Wieck
I've just uploaded http://developer.postgresql.org/~wieck/all_performance.v4.74.diff.gz This patch contains the still not yet ready performance improvements discussed over the couple last days. _Shared buffer replacement_: The buffer replacement strategy is a slightly modified version of ARC.

Re: [HACKERS] Experimental ARC implementation

2003-11-05 Thread Jan Wieck
Zeugswetter Andreas SB SD wrote: Why not use the checkpointer itself inbetween checkpoints ? use a min and a max dirty setting like Informix. Start writing when more than max are dirty stop when at min. This avoids writing single pages (which is slow, since it cannot be grouped together by

Re: [HACKERS] Performance features the 4th

2003-11-05 Thread Jan Wieck
Manfred Spraul wrote: Jan Wieck wrote: _Vacuum page delay_: Tom Lane's napping during vacuums with another tuning option. I replaced the usleep() call with a PG_DELAY(msec) macro in miscadmin.h, which does use select(2) instead. That should address the possible portability problems. What

Re: [HACKERS] Performance features the 4th

2003-11-05 Thread Jan Wieck
Neil Conway wrote: Jan Wieck [EMAIL PROTECTED] writes: This patch contains the still not yet ready performance improvements discussed over the couple last days. Cool stuff! The buffer replacement strategy is a slightly modified version of ARC. BTW Jan, I got your message about taking a look

Re: [HACKERS] Performance features the 4th

2003-11-05 Thread Jan Wieck
Andrew Dunstan wrote: Jan Wieck wrote: How portable is getrusage()? Could the postmaster issue that frequently for RUSAGE_CHILDREN and leave the result somewhere in the shared memory for whoever is concerned? SVr4, BSD4.3, SUS2 and POSIX1003.1, I believe. I also believe there is a M$ dll

Re: [HACKERS] Schema boggle...

2003-11-05 Thread Jan Wieck
Chris Bowlby wrote: Hi All, I work with Marc Fournier, for those who don't know, and have been working extensively with the schemas feature for Hub.Org's new Account Management package. Each client's data is stored in a seperate schema set asside just for them (though they will never have

Re: [HACKERS] Changes to Contributor List

2003-11-05 Thread Jan Wieck
contributors are missing from the list? B) What contributors are listed under Major Developers who haven't contributed any code since 7.1.0? C) Who needs their e-mail address updated? D) Who needs their description updated? (Text, please ... I won't write it) Jan Wieck in Philadelphia

Re: [HACKERS] Experimental ARC implementation

2003-11-07 Thread Jan Wieck
Bruce Momjian wrote: Greg Stark wrote: Bruce Momjian [EMAIL PROTECTED] writes: Have you considered having the background writer check the pages it is about to write to see if they can be added to the FSM, thereby reducing the need for vacuum? Seems we would need to add a statistics parameter

Re: [HACKERS] Experimental ARC implementation

2003-11-07 Thread Jan Wieck
Bruce Momjian wrote: Jan Wieck wrote: If the system is write-bound, the checkpointer will find that many dirty blocks that he has no time to nap and will burst them out as fast as possible anyway. Well, at least that's the theory. PostgreSQL with the non-overwriting storage concept can never

Re: [HACKERS] Performance features the 4th

2003-11-07 Thread Jan Wieck
Christopher Browne wrote: A long time ago, in a galaxy far, far away, [EMAIL PROTECTED] (Bruce Momjian) wrote: Tom Lane wrote: Jan Wieck [EMAIL PROTECTED] writes: As a matter of fact, people who have performance problems are likely to be the same who have upgrade problems. And as Gaetano

Re: [HACKERS] Experimental ARC implementation

2003-11-07 Thread Jan Wieck
Bruce Momjian wrote: Jan Wieck wrote: What doing frequent fdatasync/fsync during a constant ongoing checkpoint will cause is to significantly lower the physical write storm happening at the sync(), which is causing huge problems right now. I don't see that frankly because sync() is syncing

Re: [HACKERS] Performance features the 4th

2003-11-07 Thread Jan Wieck
Tom Lane wrote: Jan Wieck [EMAIL PROTECTED] writes: However, I have not seen much evidence yet that the vacuum delay alone does that much. Gaetano and a couple of other people did experiments that seemed to show it was useful. I think we'd want to change the shape of the knob per later

Re: [HACKERS] stats collector causes shared-memory-block leakage

2003-11-07 Thread Jan Wieck
Tom Lane wrote: While investigating Scott Goodwin's recent report of trouble on Mac OS X, I have realized that we have an unpleasant little misbehavior in our last several releases. After a backend crash, the postmaster will attempt to recycle (delete and recreate) the old shared memory segment.

Re: [HACKERS] Experimental patch for inter-page delay in VACUUM

2003-11-09 Thread Jan Wieck
Tom Lane wrote: Jan Wieck [EMAIL PROTECTED] writes: How I can see the background writer operating is that he's keeping the buffers in the order of the LRU chain(s) clean, because those are the buffers that most likely get replaced soon. In my experimental ARC code it would traverse the T1

Re: [HACKERS] Performance features the 4th

2003-11-09 Thread Jan Wieck
scott.marlowe wrote: On Fri, 7 Nov 2003, Matthew T. O'Connor wrote: - Original Message - From: Jan Wieck [EMAIL PROTECTED] Tom Lane wrote: Gaetano and a couple of other people did experiments that seemed to show it was useful. I think we'd want to change the shape of the knob per

Re: [HACKERS] Performance features the 4th

2003-11-09 Thread Jan Wieck
Matthew T. O'Connor wrote: - Original Message - From: Jan Wieck [EMAIL PROTECTED] Tom Lane wrote: Gaetano and a couple of other people did experiments that seemed to show it was useful. I think we'd want to change the shape of the knob per later suggestions (sleep 10 ms every N

Re: [HACKERS] 7.4 and 7.3.5 showstopper

2003-10-30 Thread Jan Wieck
Jan Wieck wrote: Confirmed, that's a bug - pgsql-hackers CC'd and scipt for full reproduction attached. Assumptions where correct, bug fixed in REL7_3_STABLE and HEAD. I also added a slightly modified version of the script that reproduced the bug to the foreign_key regression test. Jan

[HACKERS] 7.4 and 7.3.5 showstopper (was: Re: [SQL] Bug in Rule+Foreing key constrain?)

2003-10-30 Thread Jan Wieck
trigger in PL/pgSQL that tries to UPDATE the row and if GET DIAGNOSTICS tells it it succeeded, returns NULL to suppress the INSERT. That should work around the bug for the time being. Jan Michele Bendazzoli wrote: On Thu, 2003-10-30 at 18:29, Jan Wieck wrote: Not entirely. On which table(s

Re: [HACKERS] Experimental patch for inter-page delay in VACUUM

2003-11-10 Thread Jan Wieck
--- Jan Wieck wrote: Tom Lane wrote: Jan Wieck [EMAIL PROTECTED] writes: How I can see the background writer operating is that he's keeping the buffers in the order of the LRU chain(s) clean, because those are the buffers that most likely get replaced soon. In my experimental ARC code

Re: [HACKERS] Experimental patch for inter-page delay in VACUUM

2003-11-10 Thread Jan Wieck
Bruce Momjian wrote: Now, O_SYNC is going to force every write to the disk. If we have a transaction that has to write lots of buffers (has to write them to reuse the shared buffer) So make the background writer/checkpointer keeping the LRU head clean. I explained that 3 times now. Jan --

Re: [HACKERS] Experimental patch for inter-page delay in VACUUM

2003-11-10 Thread Jan Wieck
before it got released. Did any PostgreSQL developer other than you ever look at the SRA code? Jan Bruce Momjian wrote: scott.marlowe wrote: On Tue, 4 Nov 2003, Tom Lane wrote: Jan Wieck [EMAIL PROTECTED] writes: What still needs to be addressed is the IO storm cause by checkpoints. I see

Re: [HACKERS] Experimental patch for inter-page delay in VACUUM

2003-11-10 Thread Jan Wieck
Bruce Momjian wrote: Tom Lane wrote: Andrew Sullivan [EMAIL PROTECTED] writes: On Sun, Nov 02, 2003 at 01:00:35PM -0500, Tom Lane wrote: real traction we'd have to go back to the take over most of RAM for shared buffers approach, which we already know to have a bunch of severe disadvantages.

Re: [HACKERS] Experimental patch for inter-page delay in VACUUM

2003-11-10 Thread Jan Wieck
Bruce Momjian wrote: Jan Wieck wrote: Bruce Momjian wrote: Now, O_SYNC is going to force every write to the disk. If we have a transaction that has to write lots of buffers (has to write them to reuse the shared buffer) So make the background writer/checkpointer keeping the LRU head clean. I

Re: [HACKERS] Experimental patch for inter-page delay in VACUUM

2003-11-10 Thread Jan Wieck
Bruce Momjian wrote: Jan Wieck wrote: Bruce Momjian wrote: I would be interested to know if you have the background write process writing old dirty buffers to kernel buffers continually if the sync() load is diminished. What this does is to push more dirty buffers into the kernel cache

Re: [HACKERS] Experimental patch for inter-page delay in VACUUM

2003-11-10 Thread Jan Wieck
Tom Lane wrote: Bruce Momjian [EMAIL PROTECTED] writes: Now, if we are sure that writes will happen only in the checkpoint process, O_SYNC would be OK, I guess, but will we ever be sure of that? This is a performance issue, not a correctness issue. It's okay for backends to wait for writes as

Re: [HACKERS] Question for the developers.

2003-11-10 Thread Jan Wieck
[EMAIL PROTECTED] wrote: Hi, I am Suchindra Katageri and am working as a Software Engineer at Linuxlabs, Atlanta, GA. I am presently working on developing libraries to make postgresql run on a cluster. I was wondering if it was possible to force Database writes to stable storage, without

Re: [HACKERS] Experimental patch for inter-page delay in VACUUM

2003-11-10 Thread Jan Wieck
Zeugswetter Andreas SB SD wrote: One problem with O_SYNC would be, that the OS does not group writes any more. So the code would need to eighter do it's own sorting and grouping (256k) or use aio, or you won't be able to get the maximum out of the disks. Or just run multiple writer

Re: [HACKERS] Experimental patch for inter-page delay in VACUUM

2003-11-10 Thread Jan Wieck
Bruce Momjian wrote: Jan Wieck wrote: Bruce Momjian wrote: Jan Wieck wrote: Bruce Momjian wrote: Now, O_SYNC is going to force every write to the disk. If we have a transaction that has to write lots of buffers (has to write them to reuse the shared buffer) So make the background

Re: [HACKERS] Experimental patch for inter-page delay in VACUUM

2003-11-10 Thread Jan Wieck
Bruce Momjian wrote: Jan Wieck wrote: Zeugswetter Andreas SB SD wrote: One problem with O_SYNC would be, that the OS does not group writes any more. So the code would need to eighter do it's own sorting and grouping (256k) or use aio, or you won't be able to get the maximum out

Re: [HACKERS] Experimental patch for inter-page delay in VACUUM

2003-11-10 Thread Jan Wieck
Bruce Momjian wrote: Jan Wieck wrote: If the background cleaner has to not just write() but write/fsync or write/O_SYNC, it isn't going to be able to clean them fast enough. It creates a bottleneck where we didn't have one before. We are trying to eliminate an I/O storm during

Re: [HACKERS] Experimental patch for inter-page delay in VACUUM

2003-11-10 Thread Jan Wieck
Andrew Sullivan wrote: On Sun, Nov 09, 2003 at 08:54:25PM -0800, Joe Conway wrote: two servers, mounted to the same data volume, and some kind of coordination between the writer processes. Anyone know if this is similar to how Oracle handles RAC? It is similar, yes, but there's some mighty

Re: [HACKERS] Experimental patch for inter-page delay in VACUUM

2003-11-11 Thread Jan Wieck
Shridhar Daithankar wrote: On Tuesday 11 November 2003 00:50, Neil Conway wrote: Jan Wieck [EMAIL PROTECTED] writes: We can't resize shared memory because we allocate the whole thing in one big hump - which causes the shmmax problem BTW. If we allocate that in chunks of multiple blocks, we only

Re: [HACKERS] Experimental patch for inter-page delay in VACUUM

2003-11-11 Thread Jan Wieck
Shridhar Daithankar wrote: On Tuesday 11 November 2003 18:55, Jan Wieck wrote: And how does a newly mmap'ed segment propagate into a running backend? It wouldn't. Just like we allocate fixed amount of shared memory at startup now, we would do same for mmaped segments. Allocate maximum

Re: [HACKERS] bugzilla (Was: What do you want me to do?)

2003-11-11 Thread Jan Wieck
Marc G. Fournier wrote: On Tue, 11 Nov 2003, Shridhar Daithankar wrote: --- Hello, My name is and I work with postgresql group. Recently we/postgresql advocay/general group have decided to form a group of people who would help other projects w.r.t postgresql. So feel free to ask me

[HACKERS] Proposal for a cascaded master-slave replication system

2003-11-11 Thread Jan Wieck
Dear community, for some reason the post I sent yesterday night still did not show up on the mailing lists. I have set up some links on the developers side under http://developer.postgresql.org/~wieck/slony1.html The concept will be the base for some of my work as a Software Engineer here at

Re: [HACKERS] [GENERAL] Proposal for a cascaded master-slave replication system

2003-11-11 Thread Jan Wieck
Joe Conway wrote: Jan Wieck wrote: http://developer.postgresql.org/~wieck/slony1.html Very interesting read. Nice work! We want to build this system as a community project. The plan was from the beginning to release the product under the BSD license. And we think it is best to start

Re: [HACKERS] 7.4 hot backup capabilities?

2003-11-12 Thread Jan Wieck
Austin Gonyou wrote: On Wed, 2003-11-12 at 01:00, Shridhar Daithankar wrote: Austin Gonyou wrote: What facilities are/will be available for hot(online) backups with the 7.4 release? PITR, something else? TIA. pg_dump? Did you mean hot failover or hot backup? Postgresql does hot backup for a

Re: [HACKERS] Proposal for a cascaded master-slave replication system

2003-11-12 Thread Jan Wieck
Hans-Jürgen Schönig wrote: Jan, First of all we really appreciate that this is going to be an Open Source project. There is something I wanted to add from a marketing point of view: I have done many public talks in the 2 years or so. There is one question people keep asking me: How about the

Re: [HACKERS] Proposal for a cascaded master-slave replication system

2003-11-12 Thread Jan Wieck
allways wanted to have. I think your right on by not thinking smaller!! Thanks Jan Jordan Henderson On Wednesday 12 November 2003 10:45, Jan Wieck wrote: Hans-Jürgen Schönig wrote: Jan, First of all we really appreciate that this is going to be an Open Source project. There is something I

[HACKERS] ARC buffer strategy committed

2003-11-12 Thread Jan Wieck
I have just committed the implementation of ARC into the 7.5devel tree. This doesn't contain the vacuum page delay, or the test implementation of the background writer based on the checkpoint process. These two need some more discussion and bouncing around. Jan --

Re: [HACKERS] ARC buffer strategy committed

2003-11-12 Thread Jan Wieck
Tom Lane wrote: Jan Wieck [EMAIL PROTECTED] writes: I have just committed the implementation of ARC into the 7.5devel tree. I'm seeing a whole bunch of regression test failures that weren't there half an hour ago ... regards, tom lane Oh ... hmmm ... will check Jan

Re: [HACKERS] ARC buffer strategy committed

2003-11-12 Thread Jan Wieck
Jan Wieck wrote: Tom Lane wrote: Jan Wieck [EMAIL PROTECTED] writes: I have just committed the implementation of ARC into the 7.5devel tree. I'm seeing a whole bunch of regression test failures that weren't there half an hour ago ... regards, tom lane Oh ... hmmm ... will check That's strange

Re: [HACKERS] ARC buffer strategy committed

2003-11-12 Thread Jan Wieck
Tom Lane wrote: Jan Wieck [EMAIL PROTECTED] writes: That's strange, if I reverse apply my patch I have buffer leak's and all kinds of crap. Can't even initdb. Who else applied patches tonight? FWIW, I can initdb --- that seems fine --- but the regression tests spew lots of small fragments. What

Re: [HACKERS] ARC buffer strategy committed

2003-11-13 Thread Jan Wieck
Jan Wieck wrote: Tom Lane wrote: Jan Wieck [EMAIL PROTECTED] writes: That's strange, if I reverse apply my patch I have buffer leak's and all kinds of crap. Can't even initdb. Who else applied patches tonight? FWIW, I can initdb --- that seems fine --- but the regression tests spew lots of small

Re: [HACKERS] ARC buffer strategy committed

2003-11-13 Thread Jan Wieck
Tom Lane wrote: Jan Wieck [EMAIL PROTECTED] writes: Yeah, adding a buffer multiple times to the list of unused buffers ensures that it later on gets used for multiple contents simultaneously. Hm. Looking at the patch last night, I was wondering why you had removed all the guard logic from

[HACKERS] Background writer process

2003-11-13 Thread Jan Wieck
The attached diff is another attempt for distributing the write IO. It is a separate background process much like the checkpointer. It's purpose is to keep the number of dirty blocks in the buffer cache at a reasonable level and try that the buffers returned by the strategy for replacement are

Re: [HACKERS] Background writer process

2003-11-13 Thread Jan Wieck
Bruce Momjian wrote: Kurt Roeckx wrote: On Thu, Nov 13, 2003 at 04:35:31PM -0500, Jan Wieck wrote: For sure the sync() needs to be replaced by the discussed fsync() of recently written files. And I think the algorithm how much and how often to flush can be significantly improved. But after

Re: [HACKERS] Background writer process

2003-11-13 Thread Jan Wieck
Kurt Roeckx wrote: On Thu, Nov 13, 2003 at 05:39:32PM -0500, Bruce Momjian wrote: Jan Wieck wrote: Bruce Momjian wrote: He found that write() itself didn't encourage the kernel to write the buffers to disk fast enough. I think the final solution will be to use fsync or O_SYNC. write

Re: [HACKERS] cvs head? initdb?

2003-11-13 Thread Jan Wieck
Christopher Browne wrote: [EMAIL PROTECTED] (elein) writes: What is the status of CVS head? Isn't it in sync with 7.4.RC2? I just upgraded from CVS and rebuilt clean and initdb now gives this lovely informative initdb failed message. No, I believe that they have started applying 7.5 patches,

Re: [HACKERS] cvs head? initdb?

2003-11-14 Thread Jan Wieck
Bruce Momjian wrote: Jan Wieck wrote: Christopher Browne wrote: [EMAIL PROTECTED] (elein) writes: What is the status of CVS head? Isn't it in sync with 7.4.RC2? I just upgraded from CVS and rebuilt clean and initdb now gives this lovely informative initdb failed message. No, I believe

Re: [HACKERS] Background writer process

2003-11-14 Thread Jan Wieck
Shridhar Daithankar wrote: On Friday 14 November 2003 03:05, Jan Wieck wrote: For sure the sync() needs to be replaced by the discussed fsync() of recently written files. And I think the algorithm how much and how often to flush can be significantly improved. But after all, this does not change

Re: [HACKERS] cvs head? initdb?

2003-11-14 Thread Jan Wieck
Robert Treat wrote: On Fri, 2003-11-14 at 10:32, Jan Wieck wrote: Bruce Momjian wrote: Jan Wieck wrote: Christopher Browne wrote: [EMAIL PROTECTED] (elein) writes: What is the status of CVS head? Isn't it in sync with 7.4.RC2? I just upgraded from CVS and rebuilt clean and initdb now

Re: [HACKERS] cvs head? initdb?

2003-11-14 Thread Jan Wieck
Bruce Momjian wrote: Jan Wieck wrote: Yeah, there was a problem with *extreme* sharing ... the code tried to use the same buffer for multiple disk blocks at the same time, and somehow the backends did not agree on the correct content. But it's fixed and back in. You can see ARC working

Re: [HACKERS] cvs head? initdb?

2003-11-15 Thread Jan Wieck
--- Robert Treat wrote: On Friday 14 November 2003 14:23, Neil Conway wrote: Jan Wieck [EMAIL PROTECTED] writes: Robert Treat wrote: people would always want to have those choices (especially for doing development/testing/benchmarking

Re: [HACKERS] Proposal for a cascaded master-slave replication system

2003-11-13 Thread Jan Wieck
Bruce Momjian wrote: Hans-J?rgen Sch?nig wrote: Meanwhile we seem to be in a situation where PostgreSQL is rather competing against Oracle than against MySQL. In our case there are more people asking for Oracle - Pg migration than for MySQL - Pg. MySQL does not seem to be the great enemy

Re: [HACKERS] cvs head? initdb?

2003-11-16 Thread Jan Wieck
Bruce Momjian wrote: I figured it should begin with debug_ or log_, maybe: debug_shared_buffers = 10 # seconds If it's just that and since nobody else seemed to care ... changed. Jan -- #==# # It's easier to get forgiveness

Re: [HACKERS] cvs head? initdb?

2003-11-16 Thread Jan Wieck
Robert Treat wrote: but how do you test this if you cant run them both against each other to compare? (initally running vs 7.4 does tell you something, but even now, 7.5 improved cross datatype index improvments could skew the results of any comparisons) Right. But with the current two

Re: [HACKERS] Release cycle length

2003-11-19 Thread Jan Wieck
Marc G. Fournier wrote: On Tue, 18 Nov 2003, Peter Eisentraut wrote: The time from release 7.3 to release 7.4 was 355 days, an all-time high. We really need to shorten that. We already have a number of significant improvements in 7.5 now, and several good ones coming up in the next few weeks.

[HACKERS] Background writer committed

2003-11-19 Thread Jan Wieck
I committed the first part of the background writer process. We had a consensus on attempting to avoid write() calls from regular backends, but did no come to any conclusions what to do to force the kernel to actually do some IO. Consequently, this patch is a separate process launched by

Re: [HACKERS] Background writer committed

2003-11-20 Thread Jan Wieck
Shridhar Daithankar wrote: Jan Wieck wrote: I committed the first part of the background writer process. We had a consensus on attempting to avoid write() calls from regular backends, but did no come to any conclusions what to do to force the kernel to actually do some IO. Consequently

Re: [HACKERS] Release cycle length

2003-11-21 Thread Jan Wieck
Alvaro Herrera wrote: On Fri, Nov 21, 2003 at 09:38:50AM +0800, Christopher Kings-Lynne wrote: Yeah, I think the main issue in all this is that for real production sites, upgrading Postgres across major releases is *painful*. We have to find a solution to that before it makes sense to speed up

Re: [HACKERS] pg_restore and create FK without verification check

2003-11-27 Thread Jan Wieck
ow wrote: --- Tom Lane [EMAIL PROTECTED] wrote: People might be more interested in debating this topic with you if we hadn't discussed it at length just a couple months back. There wasn't consensus then that we had to offer an escape hatch, and you've not offered any argument that wasn't made

Re: [HACKERS] pg_restore and create FK without verification check

2003-11-28 Thread Jan Wieck
Oli Sennhauser wrote: People might be more interested in debating this topic with you if we hadn't discussed it at length just a couple months back. There wasn't consensus then that we had to offer an escape hatch, and you've not offered any argument that wasn't made before. I'm simply

Re: [HACKERS] ERROR: Index pg_toast_8443892_index is not a btree

2003-12-08 Thread Jan Wieck
strk wrote: I get the following error when vacuuming a db or inserting a big value in a column of a toastable datatype (GEOMETRY). ERROR: Index pg_toast_8443892_index is not a btree My last action has been killing a psql that was getting mad about receiving too much input and beeping as hell

Re: [HACKERS] ERROR: Index pg_toast_8443892_index is not a btree

2003-12-08 Thread Jan Wieck
strk wrote: JanWieck wrote: strk wrote: I get the following error when vacuuming a db or inserting a big value in a column of a toastable datatype (GEOMETRY). ERROR: Index pg_toast_8443892_index is not a btree My last action has been killing a psql that was getting mad about receiving

Re: [HACKERS] CVS HEAD compile failure

2003-12-08 Thread Jan Wieck
Kurt Roeckx wrote: On Mon, Dec 08, 2003 at 01:27:35PM -0500, Tom Lane wrote: Andrew Dunstan [EMAIL PROTECTED] writes: Tom Lane wrote: I've been able to reproduce this on one of my machines, and it's nasty. In that case I'm confused about why this code compiles on my machine: What compiler are

Re: [HACKERS] CVS HEAD compile failure

2003-12-08 Thread Jan Wieck
Tom Lane wrote: Kurt Roeckx [EMAIL PROTECTED] writes: I can compile current cvs with gcc 2.95.3, openssl 0.9.7b and zlib 1.2.1. current CVS meaning since I fixed the include order ? Good question. Using my cvsup tree here, which I did sup today already. So what -D would trigger the failure? Jan

Re: [HACKERS] Minor (very) feature request...

2003-12-08 Thread Jan Wieck
Steve Wampler wrote: Would it be (is it?) possible to add timestamp to the log messages put out by postgresql? I've got several databases running in an environment where users have this annoying habit of coming up to me with (Oh yes, three days ago around 4pm our instrument had trouble writing

Re: [HACKERS] ERROR: Index pg_toast_8443892_index is not a btree

2003-12-09 Thread Jan Wieck
Tom Lane wrote: Christopher Kings-Lynne [EMAIL PROTECTED] writes: Is there anything stopping us going through the code and finding all ereports that can be fixed by a REINDEX, and issue a HINT with all of them saying that they should REINDEX the broken index? How would you know which ones

Re: [HACKERS] ERROR: Index pg_toast_8443892_index is not a btree

2003-12-09 Thread Jan Wieck
Christopher Kings-Lynne wrote: I couldn't agree more. Look at this very instance. He now found the right reindex command and the corrupted file is gone. We don't have the slightest clue what happened to that file. Was it truncated? Did some other process scribble around in the shared memory?

Re: [HACKERS] ERROR: Index pg_toast_8443892_index is not a btree

2003-12-10 Thread Jan Wieck
Tom Lane wrote: strk [EMAIL PROTECTED] writes: The question now is: what does that message mean ? It means that the magic number that should be on the first page of the btree index isn't right. We can deduce that something has clobbered the first page of the index, but guessing what and how

Re: [HACKERS] pljava revisited

2003-12-10 Thread Jan Wieck
Andrew Rawnsley wrote: Other pl* (perl, python, tcl) languages have vanilla C glue code. Might be better to stick to this. If you aren't using advanced C++ features that shouldn't be too hard - well structured C can be just as readable as well structured C++. At the very lowest level, about

Re: [HACKERS] Resurrecting pg_upgrade

2003-12-16 Thread Jan Wieck
Tom Lane wrote: Marc G. Fournier [EMAIL PROTECTED] writes: Then again, in the case of pg_upgrade, wouldn't just disabling access from anywhere except localhost prevent others from getting in? Not if your normal operating mode includes connections from clients running locally. I really don't see

Re: [HACKERS] Bug in new buffer freelist code

2003-12-23 Thread Jan Wieck
Tom Lane wrote: I just had the parallel regression tests hang up due to what appears to be a bug in the new ARC code. The CLUSTER test gets into an infinite loop trying to do CLUSTER clstr_1;. The loop is in StrategyInvalidateBuffer's check that the buffer is already in the freelist; it isn't,

Re: [HACKERS] Bug in new buffer freelist code

2003-12-23 Thread Jan Wieck
Tom Lane wrote: BTW, I just managed to reproduce the hang, after a whole afternoon of trying ... only it was with a non-debug build. Sigh. Anyway, it seems my HP machine has a significantly higher probability of showing the problem than my Linux machine --- I have been unable to see the problem

Re: [HACKERS] Bug in new buffer freelist code

2003-12-23 Thread Jan Wieck
Tom Lane wrote: Jan Wieck [EMAIL PROTECTED] writes: It seems to me that buffers that are thrown away via StrategyInvalidateBuffer() do not get their relnode and blocknum cleaned out. Mmmm. They definitely should be; if you look at the prior version of buf_table.c, BufTableDelete did

Re: [HACKERS] PostgreSQL port to pure Java?

2003-12-25 Thread Jan Wieck
Ivelin Ivanov wrote: That was uncalled for. Statements like this do not make the Postgres community any healthier. You don't have any benefit of pushing back Java users. On the other hand, can you imagine the reaction of the Java camp to an idea like why not rewrite JBoss in Tcl and PL/Tcl so

Re: [HACKERS] IEEE 754

2003-12-29 Thread Jan Wieck
Sai Hertz And Control Systems wrote: Dear all , I would like to share my concerns about the IEEE 754 specification and floating point handling by PostgreSQL . Also I would like to learn how professional users of PostgreSQL work with rounding of monetary terms . For all monetary values the

Re: [HACKERS] [PATCHES] update i386 spinlock for hyperthreading

2003-12-29 Thread Jan Wieck
Tom Lane wrote: Bruce Momjian [EMAIL PROTECTED] writes: Manfred Spraul wrote: I remember there were patches that tried other algorithms instead of the simple LRU for the buffer manager. Has anyone tried to change the locking of the buffer manager? CVS HEAD already has an

Re: [HACKERS] [PATCHES] update i386 spinlock for hyperthreading

2003-12-30 Thread Jan Wieck
Manfred Spraul wrote: Bruce Momjian wrote: Anyone see an attack path here? Should we have one lock per hash bucket rather than one for the entire hash? That's the simple part. The problem is the aging strategy: we need a strategy that doesn't rely on a global list that's updated

Re: [HACKERS] [PATCHES] update i386 spinlock for hyperthreading

2003-12-30 Thread Jan Wieck
Tom Lane wrote: Manfred Spraul [EMAIL PROTECTED] writes: Are there strategies that do not rely on a global lock? The Linux kernel uses a lazy LRU with referenced bits: on access, the referenced bit is set. The freespace logic takes pages from the end of a linked list, and checks that bit: if

Re: [HACKERS] cache in plpgsql

2003-12-31 Thread Jan Wieck
ivan wrote: as new know plpgsql has special cache which remember too long (event non-existing tables (i mean old oid)) so i suggest to create same function only in plpgsql which would clear this cache, or sth like this ? for ie, where i drop table i would do plpgsql_clear_cache (); and when i

Re: [HACKERS] cache in plpgsql

2003-12-31 Thread Jan Wieck
drop, create etc in function, but using only EXECUTE .. Do you have any numbers about how that would affect performance? Jan there must be same solution .. no ? On Wed, 31 Dec 2003, Jan Wieck wrote: ivan wrote: as new know plpgsql has special cache which remember too long (event non-existing

Re: [HACKERS] cache in plpgsql

2003-12-31 Thread Jan Wieck
ivan wrote: but , all in all, do you think that now it is ok ? No, I don't. I just prefer complete solutions over patchwork. Jan On Wed, 31 Dec 2003, Jan Wieck wrote: ivan wrote: why all backend can not using one cache, which would be always Variable sized shared memory with garbage collection

[HACKERS] PL/Java issues

2003-12-31 Thread Jan Wieck
I have included the JDBC mailing list since I guess most Java developers are around here, but not necessarily on Hackers. Dave Cramer and I where discussing a few issues about the PL/Java implementation last night and would like to get more input and suggestions on the matter. The basic

Re: [HACKERS] cache in plpgsql

2004-01-01 Thread Jan Wieck
that particular catalog object or not? Jan On Wed, 31 Dec 2003, Jan Wieck wrote: ivan wrote: why all backend can not using one cache, which would be always Variable sized shared memory with garbage collection for SPI plans? in real state ... or i can just clear only my cache, at first

Re: [HACKERS] cache in plpgsql

2004-01-02 Thread Jan Wieck
Tom Lane wrote: Jan Wieck [EMAIL PROTECTED] writes: Tom Lane wrote: Another little problem is that plpgsql doesn't really have any mechanism for invalidating cached stuff at all; it will leak memory like there's no tomorrow if we start dropping cached subplans. Everyone seems to look

Re: [HACKERS] cache in plpgsql

2004-01-02 Thread Jan Wieck
Tom Lane wrote: Jan Wieck [EMAIL PROTECTED] writes: But can you tell me how to actually check if a saved SPI plan references that particular catalog object or not? This is certainly doable in principle. recordDependencyOnExpr() contains much of the logic that would be needed -- it would need

Re: [JDBC] [HACKERS] PL/Java issues

2004-01-02 Thread Jan Wieck
each postgres connection having a unique thread in that jvm and having each connection thread run with its own class loader instance so that separate classes (and thus static members) are loaded for each connection. thanks, --Barry Jan Wieck wrote: I have included the JDBC mailing list since I

  1   2   3   4   5   6   7   8   9   10   >