Re: [HACKERS] bgwriter changes

2004-12-15 Thread Simon Riggs
On Tue, 2004-12-14 at 13:30, Neil Conway wrote: In recent discussion[1] with Simon Riggs, there has been some talk of making some changes to the bgwriter. To summarize the problem, the bgwriter currently scans the entire T1+T2 buffer lists and returns a list of all the currently dirty

Re: [HACKERS] [Testperf-general] BufferSync and bgwriter

2004-12-15 Thread Jan Wieck
On 12/12/2004 5:08 PM, Simon Riggs wrote: On Sun, 2004-12-12 at 05:46, Neil Conway wrote: Simon Riggs wrote: If the bgwriter_percent = 100, then we should actually do the sensible thing and prepare the list that we need, i.e. limit StrategyDirtyBufferList to finding at most bgwriter_maxpages.

Re: [HACKERS] [Testperf-general] BufferSync and bgwriter

2004-12-15 Thread Jan Wieck
On 12/12/2004 9:43 PM, Neil Conway wrote: On Sun, 2004-12-12 at 22:08 +, Simon Riggs wrote: On Sun, 2004-12-12 at 05:46, Neil Conway wrote: Is the plan to make bgwriter_percent = 100 the default setting? Hmm...must confess that my only plan is: i) discover dynamic behaviour of bgwriter ii)

Re: [HACKERS] production server down

2004-12-15 Thread Joe Conway
Tom Lane wrote: My advice is to backup the $PGDATA tree (which you said was in progress), then pg_resetxlog, then cross-check the hell out of the data you see. Only if you can detect some data problems can we guess at something else to do ... Before running pg_resetxlog, a couple of questions: 1.

Re: [HACKERS] bgwriter changes

2004-12-15 Thread Jan Wieck
On 12/14/2004 2:40 PM, Tom Lane wrote: Zeugswetter Andreas DAZ SD [EMAIL PROTECTED] writes: Is it possible to do a patch that produces a dirty buffer list in LRU order and stops early when eighter maxpages is reached or bgwriter_percent pages are scanned ? Only if you redefine the meaning of

Re: [HACKERS] [INTERFACES] postgresql-7.4.5

2004-12-15 Thread Dave Cramer
That driver should work, however you might have something else misconfigured. What error are you getting ? You should post this to the jdbc list Dave ElayaRaja S wrote: Hi, I am using postgresql-7.4.5. I nee to use the jdbc connection So i downloaded 4 versions of driver( pg74.215.jdbc1.jar,

Re: [HACKERS] production server down

2004-12-15 Thread Tom Lane
Joe Conway [EMAIL PROTECTED] writes: I don't trust it at all. So does that imply that I should override next transaction id and WAL starting address per the manpage? Yes, override everything there's a switch for. Also check that the other values shown by pg_controldata look reasonable (the

Re: [HACKERS] production server down

2004-12-15 Thread Greg Stark
Tom Lane [EMAIL PROTECTED] writes: The server experienced a hang (as yet unexplained) yesterday and was restarted at 2004-12-13 16:38:49 according to syslog. I'm told by the network admin that there was a problem with the network card on restart, so the nfs mount most probably

Re: Re: [HACKERS] [Testperf-general] BufferSync and bgwriter

2004-12-15 Thread simon
Josh Berkus [EMAIL PROTECTED] wrote on 15.12.2004, 18:36:53: Hmmm, I've not seen this. For example, with people who are having trouble with checkpoint spikes on Linux, I've taken to recommending that they call sync() (via cron) every 5-10 seconds (thanks, Bruce, for suggestion!).

Re: [HACKERS] production server down

2004-12-15 Thread Joe Conway
Tom Lane wrote: Joe Conway [EMAIL PROTECTED] writes: I don't trust it at all. So does that imply that I should override next transaction id and WAL starting address per the manpage? Yes, override everything there's a switch for. Also check that the other values shown by pg_controldata look

[HACKERS] Identifying time of last stat reset via sql

2004-12-15 Thread Ed L.
I asked this on general, but didn't receive any responses. Is it possible via SQL to identify the time of the last stat reset (or pg_stat_reset() call)? This is what I'm lacking to be able to measure query activity volume over time via SQL. Maybe a function similar to the fictitious

Re: [HACKERS] possible wierd boolean bug?

2004-12-15 Thread Merlin Moncure
Merlin Moncure [EMAIL PROTECTED] writes: I confirmed the problem on a linux server running beta3...so this problem is quite reproducible by running the attached scripts on a freshly loaded database. The attached patch fixes the problem for me. regards, tom lane

Re: [HACKERS] race condition for drop schema cascade?

2004-12-15 Thread Tom Lane
Andrew Dunstan [EMAIL PROTECTED] writes: I have seen this failure several times, but not consistently, on the buildfarm member otter (Debian/MIPS) and possible on others, and am wondering if it indicates a possible race condition on DROP SCHEMA CASCADE. Hard to see what, considering that

Re: [HACKERS] strange regression failure

2004-12-15 Thread Andrew Dunstan
Tom Lane wrote: Andrew Dunstan [EMAIL PROTECTED] writes: Has anyone got any idea on why I see things like this from time to time? It seems to be intermittent, which is odd. I have a suspicion that the unexpected result occurs when the client encoding is UTF8, because psql/mbprint.c's

Re: [HACKERS] strange regression failure

2004-12-15 Thread Andrew Dunstan
Tom Lane wrote: Andrew Dunstan [EMAIL PROTECTED] writes: Has anyone got any idea on why I see things like this from time to time? It seems to be intermittent, which is odd. I have a suspicion that the unexpected result occurs when the client encoding is UTF8, In fact, I was

Re: [HACKERS] production server down

2004-12-15 Thread Joe Conway
Greg Stark wrote: Tom Lane [EMAIL PROTECTED] writes: I've always felt that running a database across NFS was a Bad Idea ;-) Well not that I disagree with that sentiment, but NFS was specifically designed to handle this particular scenario. *UNLESS* you use the soft option. As popular as it is,

Re: [HACKERS] strange regression failure

2004-12-15 Thread Tom Lane
Andrew Dunstan [EMAIL PROTECTED] writes: Has anyone got any idea on why I see things like this from time to time? It seems to be intermittent, which is odd. I have a suspicion that the unexpected result occurs when the client encoding is UTF8, In fact, I was able to duplicate the failure

Re: [HACKERS] strange regression failure

2004-12-15 Thread Tom Lane
Alvaro Herrera [EMAIL PROTECTED] writes: On Wed, Dec 15, 2004 at 05:20:28PM -0500, Andrew Dunstan wrote: Our emails just crossed. You got it. I will have buildfarm force SQL_ASCII. But then you will lose reports using other encodings ... make check forces C locale anyway. It's only the

Re: [HACKERS] somebody working on: Prevent default re-use of sysids

2004-12-15 Thread Bruce Momjian
Matthias Schmidt wrote: Hi Tom, after beeing offline because of a chrashed box, I able to mail again. I would like to volunteer for the uptime() function. Is that OK? Sure. -- Bruce Momjian| http://candle.pha.pa.us [EMAIL PROTECTED] | (610)

Re: [HACKERS] possible wierd boolean bug?

2004-12-15 Thread Tom Lane
Merlin Moncure [EMAIL PROTECTED] writes: Anyways, it would be nice to be able to use the sql row constructor to do equality/comparison...wouldn't get caught writing such silly sql statements :) You mean like this? regression=# select row(1,2,3) = row(1,2,3); ?column? -- t (1 row)

[HACKERS] getting 'order by' working with unicode locale? ICU?

2004-12-15 Thread Palle Girgensohn
Hi! I'm using Postgresql on FreeBSD, and would like to get order by to work with unicode. The OS does have collation implemented for unicode (UTF-8) locales. Some freebsd people point me towards IBM:s ICU kit. How much effort would be required to get postgresql to sort properly, mainly using

Re: [HACKERS] strange regression failure

2004-12-15 Thread Andrew Dunstan
Tom Lane wrote: Alvaro Herrera [EMAIL PROTECTED] writes: On Wed, Dec 15, 2004 at 05:20:28PM -0500, Andrew Dunstan wrote: Our emails just crossed. You got it. I will have buildfarm force SQL_ASCII. But then you will lose reports using other encodings ... make check forces C

Re: [HACKERS] strange regression failure

2004-12-15 Thread Neil Conway
On Wed, 2004-12-15 at 21:33 -0500, Andrew Dunstan wrote: Also, currently buildfarm just runs postgres' own test suites. I'm not averse to supporting a more extensive test suite just for farm members, if people think that's a good idea. I think you'd get more mileage out of expanding the

Re: [HACKERS] bgwriter changes

2004-12-15 Thread Zeugswetter Andreas DAZ SD
and stops early when eighter maxpages is reached or bgwriter_percent pages are scanned ? Only if you redefine the meaning of bgwriter_percent. At present it's defined by reference to the total number of dirty pages, and that can't be known without collecting them all. If it

Re: Re: [HACKERS] bgwriter changes

2004-12-15 Thread simon
Zeugswetter Andreas DAZ SD [EMAIL PROTECTED] wrote on 15.12.2004, 11:39:44: and stops early when eighter maxpages is reached or bgwriter_percent pages are scanned ? Only if you redefine the meaning of bgwriter_percent. At present it's defined by reference to the total number

Re: [HACKERS] bgwriter changes

2004-12-15 Thread Zeugswetter Andreas DAZ SD
The two alternative algorithms are similar, but have these differences: The former (option (2)) finds a constant number of dirty pages, though has varying search time. This has the disadvantage of converging against 0 dirty pages. A system that has less than maxpages dirty will write every

Re: [HACKERS] production server down

2004-12-15 Thread Alvaro Herrera
On Tue, Dec 14, 2004 at 09:22:42PM -0800, Joe Conway wrote: # pg_controldata /replica/pgdata Current log file ID: 0 Next log file segment:1 Latest checkpoint location: 0/9B0B8C Prior checkpoint location:0/9AA1B4 Latest checkpoint's

Re: RE: Re: [HACKERS] bgwriter changes

2004-12-15 Thread simon
Zeugswetter Andreas DAZ SD [EMAIL PROTECTED] wrote on 15.12.2004, 15:33:16: The two alternative algorithms are similar, but have these differences: The former (option (2)) finds a constant number of dirty pages, though has varying search time. This has the disadvantage of converging

[HACKERS] strange regression failure

2004-12-15 Thread Andrew Dunstan
Has anyone got any idea on why I see things like this from time to time? It seems to be intermittent, which is odd. You can see it on occasional ContribCheck failures on buildfarm. cheers andrew = pgsql.2428/contrib/tsearch2/regression.diffs === ***

Re: [HACKERS] production server down

2004-12-15 Thread Tom Lane
Joe Conway [EMAIL PROTECTED] writes: Before running pg_resetxlog, a couple of questions: 1. Since it appears that pg_control is suspect, should I force it to be rebuilt, and if so, how? pg_resetxlog will rebuild it in any case. However it will re-use the existing contents as much as it

Re: [HACKERS] bgwriter changes

2004-12-15 Thread Tom Lane
Jan Wieck [EMAIL PROTECTED] writes: Still, we need to avoid scanning over all the clean blocks of a large buffer pool, so there is need for a separate dirty-LRU. That's not happening, unless you want to undo the cntxDirty stuff, with unknown implications for performance and deadlock safety.

Re: [HACKERS] production server down

2004-12-15 Thread Joe Conway
Tom Lane wrote: pg_resetxlog will rebuild it in any case. However it will re-use the existing contents as much as it can (if you don't use any of the command line options to override values). Given Alvaro's observation that the existing file looks suspiciously close to a freshly-initdb'd one, I

Re: [HACKERS] bgwriter changes

2004-12-15 Thread Jan Wieck
On 12/15/2004 12:10 PM, Tom Lane wrote: Jan Wieck [EMAIL PROTECTED] writes: Still, we need to avoid scanning over all the clean blocks of a large buffer pool, so there is need for a separate dirty-LRU. That's not happening, unless you want to undo the cntxDirty stuff, with unknown implications

Re: [HACKERS] [Testperf-general] BufferSync and bgwriter

2004-12-15 Thread Josh Berkus
Jan, I too don't think that this approach will retain the checkpoing smooting effect, the current implementation has. The real problem is that the cleaner the buffer pool is, the longer the scan for dirty buffers will take because the dirty blocks tend to be at the very end of the scan

Re: [HACKERS] [Testperf-general] BufferSync and bgwriter

2004-12-15 Thread Josh Berkus
Folks, To allow DBT2 to be used for real bgwriter benchmarking, Mark would need to change the following: 1) Randomize the timing of the commits, so that sometimes there is only 30 writes/minute, and other times there is 300. A timing pattern that would produce a sine wave with occasional

Re: [HACKERS] [Testperf-general] BufferSync and bgwriter

2004-12-15 Thread Josh Berkus
Simon, Clearly, OSDL-DBT2 is not a real world test! That is its benefit, since it is heavily instrumented and we are able to re-run it many times without different parameter settings. The application is well known and doesn't suffer that badly from factors that would allow certain effects to

Re: [HACKERS] possible wierd boolean bug?

2004-12-15 Thread Tom Lane
Merlin Moncure [EMAIL PROTECTED] writes: I confirmed the problem on a linux server running beta3...so this problem is quite reproducible by running the attached scripts on a freshly loaded database. The attached patch fixes the problem for me. regards, tom lane ***

Re: [HACKERS] somebody working on: Prevent default re-use of sysids for dropped users and groups?

2004-12-15 Thread Matthias Schmidt
Hi Tom, after beeing offline because of a chrashed box, I able to mail again. I would like to volunteer for the uptime() function. Is that OK? cheers, Matthias Am 13.12.2004 um 03:31 schrieb Bruce Momjian: Matthias Schmidt wrote: Am 07.12.2004 um 19:24 schrieb Tom Lane: Matthias Schmidt [EMAIL

Re: [HACKERS] production server down

2004-12-15 Thread Joe Conway
Tom Lane wrote: Yes, override everything there's a switch for. Also check that the other values shown by pg_controldata look reasonable (the locale settings are probably the only ones you might get burned on). What if anything have you got in $PGDATA/pg_xlog? -rw--- 1 postgres postgres

[HACKERS] race condition for drop schema cascade?

2004-12-15 Thread Andrew Dunstan
I have seen this failure several times, but not consistently, on the buildfarm member otter (Debian/MIPS) and possible on others, and am wondering if it indicates a possible race condition on DROP SCHEMA CASCADE. == pgsql.30167/src/test/regress/regression.diffs

Re: [HACKERS] production server down

2004-12-15 Thread Michael Fuhr
On Wed, Dec 15, 2004 at 11:41:02AM -0800, Joe Conway wrote: Just wanted to close the loop for the sake of the list archives. With Tom's xlog dump tool I was able (with a bunch of his help off-list) to identify the needed parameters for pg_resetxlog. Running pg_resetxlog got us back a

Re: [HACKERS] bgwriter changes

2004-12-15 Thread Mark Kirkwood
Simon Riggs wrote: 100pct.patch (SR) Test results to date: 1. Mark Kirkwood ([HACKERS] [Testperf-general] BufferSync and bgwriter) pgbench 1xCPU 1xDisk shared_buffers=1 showed 8.0RC1 had regressed compared with 7.4.6, but patch improved performance significantly against 8.0RC1 It occurs to

Re: [HACKERS] strange regression failure

2004-12-15 Thread Tom Lane
Andrew Dunstan [EMAIL PROTECTED] writes: Has anyone got any idea on why I see things like this from time to time? It seems to be intermittent, which is odd. I have a suspicion that the unexpected result occurs when the client encoding is UTF8, because psql/mbprint.c's ucs_wcwidth() function

Re: [HACKERS] strange regression failure

2004-12-15 Thread Alvaro Herrera
On Wed, Dec 15, 2004 at 05:20:28PM -0500, Andrew Dunstan wrote: Our emails just crossed. You got it. I will have buildfarm force SQL_ASCII. But then you will lose reports using other encodings ... -- Alvaro Herrera ([EMAIL PROTECTED]) Cuando mañana llegue pelearemos segun lo que mañana exija

Re: [HACKERS] getting 'order by' working with unicode locale? ICU?

2004-12-15 Thread Tom Lane
Palle Girgensohn [EMAIL PROTECTED] writes: I'm using Postgresql on FreeBSD, and would like to get order by to work with unicode. What makes you think it doesn't? Use the right locale and you're set. regards, tom lane ---(end of

Re: [HACKERS] getting 'order by' working with unicode locale? ICU?

2004-12-15 Thread Palle Girgensohn
--On onsdag, december 15, 2004 23.21.13 -0500 Tom Lane [EMAIL PROTECTED] wrote: Palle Girgensohn [EMAIL PROTECTED] writes: I'm using Postgresql on FreeBSD, and would like to get order by to work with unicode. What makes you think it doesn't? Use the right locale and you're set. Not on FreeBSD,