Re: [HACKERS] Bgwriter behavior
Added to TODO: * Improve the background writer Allow the background writer to more efficiently write dirty buffers from the end of the LRU cache and use a clock sweep algorithm to write other dirty buffers to reduced checkpoint I/O --- Simon Riggs wrote: On Wed, 2004-12-22 at 04:43, Tom Lane wrote: Bruce Momjian pgman@candle.pha.pa.us writes: So what are we doing for 8.0? Well, it looks like RC2 has already crashed and burned --- I can't imagine that Marc will let us release without an RC3 given what was committed today, never mind the btree bug that Mark Wong seems to have found. So maybe we should just bite the bullet and do something real about this. I'm willing to code up a proposed patch for the two-track idea I suggested, and if anyone else has a favorite maybe they could write something too. But do we have the resources to test such patches and make a decision in the next few days? At the moment my inclination is to sit on what we have. I've not seen any indication that 8.0 is really worse than earlier releases; the most you could argue against it is that it's not as much better as we hoped. That's not grounds to muck around at the RC3 stage. Agreed, if somewhat reluctantly. We may have the time to test, but it is clear that we do not have the time to validate those tests, then discuss and agree on the results. Time to go with what we have. [Mark's possible bug seems a higher priority for me.] -- Best Regards, Simon Riggs ---(end of broadcast)--- TIP 3: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly -- Bruce Momjian| http://candle.pha.pa.us pgman@candle.pha.pa.us | (610) 359-1001 + If your life is a hard drive, | 13 Roberts Road + Christ can be your backup.| Newtown Square, Pennsylvania 19073 ---(end of broadcast)--- TIP 9: the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Re: [HACKERS] displaying contents
On Mon, Dec 27, 2004 at 03:11:11PM -0600, Jaime Casanova wrote: Jaime, there is way to display all the values (fields) in a tree node like this? for debug purpouses. Query*query; Have you tried gdb already? You attach it to a running backend and can query the (err) Query struct. Query is a Node, so it can be stringified by nodeToString(); you can elog/ereport that or use elog_node_display(). -- Alvaro Herrera ([EMAIL PROTECTED]) Postgres is bloatware by design: it was built to house PhD theses. (Joey Hellerstein, SIGMOD annual conference 2002) ---(end of broadcast)--- TIP 8: explain analyze is your friend
Re: [HACKERS] Ready for RC3?
What are the open issues before we can release an RC3? I don't know of any except the btree problem OSDL found. Is that fixed? Are their others? There is the issue with shared mem on Win2003 that came up the other day. I don't think it's a general error, but rather something environment specific. (I've had several reports of ppl with fairly large databases in production on win2003 that don't have this problem). But it'd be good to have confirmation on that. If we could give that one a couple of days, it would be good, but it's not a showstopper. //Magnus ---(end of broadcast)--- TIP 7: don't forget to increase your free space map settings
Re: [HACKERS] Ready for RC3?
Bruce Momjian wrote: What are the open issues before we can release an RC3? I don't know of any except the btree problem OSDL found. Is that fixed? Are their others? There sill may be an issue with the statistics collector resetting randomly on a win32 server under heavy load. I have noticed this on multiple servers but it hasn't been confirmed by anybody else. Merlin ---(end of broadcast)--- TIP 7: don't forget to increase your free space map settings
Re: [HACKERS] Ready for RC3?
There is the issue with shared mem on Win2003 that came up the other day. I don't think it's a general error, but rather something environment specific. (I've had several reports of ppl with fairly large databases in production on win2003 that don't have this problem). But it'd be good to have confirmation on that. If we could give that one a couple of days, it would be good, but it's not a showstopper. I just confirmed this an a dual Opteron box running *win2k* on a realease candidate built less than a week ago. :-(. I was trying to upgrade a production server to retest the statistics collector...I'm going to try a fresh compile but may have to revert back to beta5. FATAL: could not attach to proper memory at fixed address: shmget(key=5432001, addr=00BB) failed: Invalid argument ---(end of broadcast)--- TIP 5: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faqs/FAQ.html
Re: [HACKERS] Ready for RC3?
Bruce Momjian pgman@candle.pha.pa.us writes: What are the open issues before we can release an RC3? I don't know of any except the btree problem OSDL found. Is that fixed? No; we haven't seen a test case yet, or even any details except the error message. FWIW, I think it's highly likely that whatever bug Mark has found exists also in 7.4 --- so it's unclear that it is a reason to hold up RC3. I think that the error must be coming out of btree page deletion during VACUUM, and that code hasn't changed materially since 7.4. Aside from the Windows issues mentioned, I'm somewhat concerned about the schemas missing after initdb problem that the one guy in Hungary reported. But considering that no one else has seen it, it might be a bug in his compiler or some such. He was running suse 7 on s390, which is a pretty old release of suse I believe, and likely not one with well wrung out s390 support. (I tried to reproduce using current Red Hat on s390 with same locale, but no dice.) regards, tom lane ---(end of broadcast)--- TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]
Re: [HACKERS] Ready for RC3?
Magnus Hagander [EMAIL PROTECTED] writes: There is the issue with shared mem on Win2003 that came up the other day. I don't think it's a general error, but rather something environment specific. (I've had several reports of ppl with fairly large databases in production on win2003 that don't have this problem). I doubt that size of database has anything at all to do with this. Is there a way to determine current end of ordinary memory in Windows? (Something akin to sbrk(0) on Unix.) It would be interesting to get the complainant to try it with some code added that prints out the memory end address both just before the postmaster creates the shmem segment, and just before the child process tries to attach to it. (Maybe these are the same place in the code, don't recall for sure.) regards, tom lane ---(end of broadcast)--- TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]
Re: [HACKERS] Ready for RC3?
Magnus Hagander wrote: What are the open issues before we can release an RC3? I don't know of any except the btree problem OSDL found. Is that fixed? Are their others? There is the issue with shared mem on Win2003 that came up the other day. I don't think it's a general error, but rather something environment specific. (I've had several reports of ppl with fairly large databases in production on win2003 that don't have this problem). But it'd be good to have confirmation on that. If we could give that one a couple of days, it would be good, but it's not a showstopper. Added to open items: * Win2003 shared memory address conflict -- Bruce Momjian| http://candle.pha.pa.us pgman@candle.pha.pa.us | (610) 359-1001 + If your life is a hard drive, | 13 Roberts Road + Christ can be your backup.| Newtown Square, Pennsylvania 19073 ---(end of broadcast)--- TIP 5: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faqs/FAQ.html
Re: [HACKERS] Ready for RC3?
Merlin Moncure wrote: Bruce Momjian wrote: What are the open issues before we can release an RC3? I don't know of any except the btree problem OSDL found. Is that fixed? Are their others? There sill may be an issue with the statistics collector resetting randomly on a win32 server under heavy load. I have noticed this on multiple servers but it hasn't been confirmed by anybody else. Added to open items: * Random stats processor resets on Win32 -- Bruce Momjian| http://candle.pha.pa.us pgman@candle.pha.pa.us | (610) 359-1001 + If your life is a hard drive, | 13 Roberts Road + Christ can be your backup.| Newtown Square, Pennsylvania 19073 ---(end of broadcast)--- TIP 4: Don't 'kill -9' the postmaster
[HACKERS] Multiple language support
Are there any plans for modifying the message outputs from the programs to support multiple languages? Most of the messages (at least in things like the database engine and utilities like initdb etc) have messages hard coded in English. This does not affect me very much, but I have at least 6 beta testers for the postgreSQL OS/2 port that are based in Germany and one in Italy, and another in the Netherlands. Or am I missing something somewhere? Lorne Sunley -- --- [EMAIL PROTECTED] --- ---(end of broadcast)--- TIP 6: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] Multiple language support
[EMAIL PROTECTED] wrote: Are there any plans for modifying the message outputs from the programs to support multiple languages? Most of the messages (at least in things like the database engine and utilities like initdb etc) have messages hard coded in English. This does not affect me very much, but I have at least 6 beta testers for the postgreSQL OS/2 port that are based in Germany and one in Italy, and another in the Netherlands. Or am I missing something somewhere? You are missing more than just something. What do you think all those .po files are sitting in your source tree? Why do you think we have a topic on this on the developers' web site: http://developer.postgresql.org/~petere/nlsstatus/ cheers andrew ---(end of broadcast)--- TIP 8: explain analyze is your friend
Re: [HACKERS] Bgwriter behavior
John Hansen wrote: I ran some tests last week and can report results similar on Tom's test: pgbench -i -s 10 bench pgbench -c 10 -t 1 bench The tests were on a machine with a single SCSI drive that doesn't lie about fsync. I found 7.4.X got around 75tps while 8.0 got 100tps, very similar to the 65/107 numbers Tom had. You do realize, that pgbench result comparisons are about as useful as a fork for eating soup? On another note, how do you know for sure, that your drive does not lie about fsync? Did you run the tests with fsync turned off vs fsync on? I just tried and got 115tps with fsync off vs 100 with fsync on, so fsync is certainly doing something. -- Bruce Momjian| http://candle.pha.pa.us pgman@candle.pha.pa.us | (610) 359-1001 + If your life is a hard drive, | 13 Roberts Road + Christ can be your backup.| Newtown Square, Pennsylvania 19073 ---(end of broadcast)--- TIP 5: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faqs/FAQ.html
Re: [HACKERS] Bgwriter behavior
Bruce Momjian pgman@candle.pha.pa.us writes: John Hansen wrote: On another note, how do you know for sure, that your drive does not lie about fsync? I just tried and got 115tps with fsync off vs 100 with fsync on, so fsync is certainly doing something. [ raised eyebrow... ] Something is wrong with that. I'd expect a *much* higher difference. It's difficult to credit a tps rate higher than your disk's RPM rating with fsync on, but most modern CPUs can do a lot better than that with fsync off. If you have a 7200 RPM drive then I'd believe the 100 figure, but not the other ... regards, tom lane ---(end of broadcast)--- TIP 4: Don't 'kill -9' the postmaster
Re: [HACKERS] Bgwriter behavior
Tom Lane wrote: Bruce Momjian pgman@candle.pha.pa.us writes: John Hansen wrote: On another note, how do you know for sure, that your drive does not lie about fsync? I just tried and got 115tps with fsync off vs 100 with fsync on, so fsync is certainly doing something. [ raised eyebrow... ] Something is wrong with that. I'd expect a *much* higher difference. It's difficult to credit a tps rate higher than your disk's RPM rating with fsync on, but most modern CPUs can do a lot better than that with fsync off. If you have a 7200 RPM drive then I'd believe the 100 figure, but not the other ... I think it is a 10k RPM drive, Seagate Cheteetah ST336607LW. -- Bruce Momjian| http://candle.pha.pa.us pgman@candle.pha.pa.us | (610) 359-1001 + If your life is a hard drive, | 13 Roberts Road + Christ can be your backup.| Newtown Square, Pennsylvania 19073 ---(end of broadcast)--- TIP 5: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faqs/FAQ.html
[HACKERS] Windows default to localhost is in the wrong place
Isn't this fix in the wrong place? 2004-03-23 22:10 momjian * src/bin/psql/startup.c: Also, what is the default connection mode of psql? It should probably be equivalent to -h localhost, shouldn't it? Now that is something I had not thought of. Seems we can assume a Win32 psql can never use unix domain sockets, so defaulting that to localhost is a good solution too. Andrew Dunstan ISTM that libpq itself ought to default to localhost, rather than failing, on machines that don't have Unix sockets. regards, tom lane ---(end of broadcast)--- TIP 6: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] Windows default to localhost is in the wrong place
Tom Lane wrote: Isn't this fix in the wrong place? 2004-03-23 22:10 momjian * src/bin/psql/startup.c: Also, what is the default connection mode of psql? It should probably be equivalent to -h localhost, shouldn't it? Now that is something I had not thought of. Seems we can assume a Win32 psql can never use unix domain sockets, so defaulting that to localhost is a good solution too. Andrew Dunstan ISTM that libpq itself ought to default to localhost, rather than failing, on machines that don't have Unix sockets. So then it would be picked up by pg_dump, createdb etc., plus third party clients? Not a bad idea. Are you going to fix it for 8.0? cheers andrew ---(end of broadcast)--- TIP 8: explain analyze is your friend
Re: [HACKERS] Windows default to localhost is in the wrong place
Andrew Dunstan [EMAIL PROTECTED] writes: Tom Lane wrote: ISTM that libpq itself ought to default to localhost, rather than failing, on machines that don't have Unix sockets. So then it would be picked up by pg_dump, createdb etc., plus third party clients? Not a bad idea. Are you going to fix it for 8.0? If I don't hear any objections in the next few hours ... regards, tom lane ---(end of broadcast)--- TIP 2: you can get off all lists at once with the unregister command (send unregister YourEmailAddressHere to [EMAIL PROTECTED])
Re: [HACKERS] Bgwriter behavior
On Tue, 2004-12-28 at 07:23, John Hansen wrote: I ran some tests last week and can report results similar on Tom's test: pgbench -i -s 10 bench pgbench -c 10 -t 1 bench The tests were on a machine with a single SCSI drive that doesn't lie about fsync. I found 7.4.X got around 75tps while 8.0 got 100tps, very similar to the 65/107 numbers Tom had. You do realize, that pgbench result comparisons are about as useful as a fork for eating soup? I'd have to agree. I find it hard to get comparable results on my test server, let alone discuss other people's findings. The only tests I have reasonable faith in these days are those performed to a rigorous test method, which is also published, visible and challengeable. OSDL is the nearest thing to that we have to that. -- Best Regards, Simon Riggs ---(end of broadcast)--- TIP 3: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
[HACKERS] buildfarm NetBSD/m68k tsearch regression failure
This result is now seen for HEAD on buildfarm member osprey: = pgsql.23138/contrib/tsearch/regression.diffs === *** ./expected/tsearch.out Tue Dec 28 12:05:27 2004 --- ./results/tsearch.out Tue Dec 28 19:52:47 2004 *** *** 312,322 (1 row) SELECT '1'::mquery_txt; ! mquery_txt ! ! '1' ! (1 row) ! SELECT '1 '::mquery_txt; mquery_txt --- 312,318 (1 row) SELECT '1'::mquery_txt; ! ERROR: cache lookup failed for type 3095621458 SELECT '1 '::mquery_txt; mquery_txt == Note that this is the only failure on buildfarm for HEAD except for 2 machines where the failure is known to be related to old hardware / emulator software. cheers andrew ---(end of broadcast)--- TIP 3: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] Bgwriter behavior
I ran some tests last week and can report results similar on Tom's test: pgbench -i -s 10 bench pgbench -c 10 -t 1 bench don't you have to specify the scaling factor for the benchmark as well? as in pgbench -c 10 -t 1 -s 10 bench ? I just tried and got 115tps with fsync off vs 100 with fsync on, so fsync is certainly doing something. well, I usually get results that differ by that much from run to run. Probably you ran in to more checkpoints on the second test. Also, did you reinitialize the bench database with pgbench -i ? ... John ---(end of broadcast)--- TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]
Re: [HACKERS] Bgwriter behavior
John Hansen wrote: I ran some tests last week and can report results similar on Tom's test: pgbench -i -s 10 bench pgbench -c 10 -t 1 bench don't you have to specify the scaling factor for the benchmark as well? as in pgbench -c 10 -t 1 -s 10 bench ? I just tried and got 115tps with fsync off vs 100 with fsync on, so fsync is certainly doing something. well, I usually get results that differ by that much from run to run. Probably you ran in to more checkpoints on the second test. Also, did you reinitialize the bench database with pgbench -i ? I destroyed the database and recreated it. -- Bruce Momjian| http://candle.pha.pa.us pgman@candle.pha.pa.us | (610) 359-1001 + If your life is a hard drive, | 13 Roberts Road + Christ can be your backup.| Newtown Square, Pennsylvania 19073 ---(end of broadcast)--- TIP 3: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] buildfarm NetBSD/m68k tsearch regression failure
Andrew Dunstan [EMAIL PROTECTED] writes: This result is now seen for HEAD on buildfarm member osprey: Yeah, I was wondering about that. It should be easy to reproduce the failure by hand (just run the tsearch regression test and then re-execute that query) --- can we see a debugger stack trace from the errfinish call? regards, tom lane ---(end of broadcast)--- TIP 5: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faqs/FAQ.html
Re: [HACKERS] Bgwriter behavior
Bruce Momjian wrote: well, I usually get results that differ by that much from run to run. Probably you ran in to more checkpoints on the second test. Also, did you reinitialize the bench database with pgbench -i ? I destroyed the database and recreated it. The only way I managed to control the variability in Pgbench was to *reboot the machine* and recreate the database for each test. In addition it seems that using a larger scale factor (e.g 200) helped as well. Having said that, on FreeBSD 5.3 with hw.ata.wc=0 (i.e no write cache) my results for s=200, t=1 and c=4 were 49 (+/- 0.5) tps for both 7.4.6 and 8.0.0RC1 - no measurable difference. If I reduced the number of transactions to t=1000, then 7.4.6 jumped ahead by about 10 tps. Bruce - are you able to try s=200? It would be interesting to see what your setup does. regards Mark ---(end of broadcast)--- TIP 3: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] race condition for drop schema cascade?
Tom Lane wrote: Andrew Dunstan [EMAIL PROTECTED] writes: I have seen this failure several times, but not consistently, on the buildfarm member otter (Debian/MIPS) and possible on others, and am wondering if it indicates a possible race condition on DROP SCHEMA CASCADE. Hard to see what, considering that there's only one backend touching that tablespace in the test. I'd be inclined to wonder if there's a filesystem-level problem on that platform. What filesystem are you running on anyway? I have just seen this error again, this time on Cygwin. I did a trawl thought the buildfarm history looking for other occurrences and found it happening on many platforms: pgbuildfarm=# select name, operating_system, stage, count from buildsystems b, (select sysname, stage, count(*) as count from build_status where log ~ 'tablespace testspace is not empty' group by sysname, stage) as s where s.sysname=b.name; name| operating_system |stage | count --+--+--+--- spoonbill | OpenBSD | Check| 2 lionfish | Linux| Check| 9 kudu | Solaris | InstallCheck | 1 kudu | Solaris | Check| 5 emu | OpenBSD | Check| 137 loris | Windows | Check| 2 gibbon| Cygwin | InstallCheck | 1 panda | Linux Debian | Check| 3 otter | Debian Linux | Check| 2 hare | Debian Linux | Check| 3 dog | Fedora Core | Check|17 fantail | Linux| Check| 3 osprey| NetBSD | Check|15 (13 rows) cheers andrew ---(end of broadcast)--- TIP 3: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] race condition for drop schema cascade?
Andrew Dunstan [EMAIL PROTECTED] writes: I have just seen this error again, this time on Cygwin. I did a trawl thought the buildfarm history looking for other occurrences and found it happening on many platforms: [ yawning... ] I've got to go to bed now, but so far tonight my Fedora Core 3 machine has completed 314 iterations of make check on CVS tip with no such error. So whatever this is, there must be some platform-specific issue involved... regards, tom lane ---(end of broadcast)--- TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]
