Re: [HACKERS] Bgwriter behavior
On Fri, 2004-12-31 at 01:14, Bruce Momjian wrote: Simon Riggs wrote: On Mon, 2004-12-27 at 22:21, Bruce Momjian wrote: Should we consider at least adjusting the meaning of bgwriter_percent? Yes. As things stand, this is the only change that seems safe. Here's a very short patch that implements this change within BufferSync in bufmgr.c - No algorithm changes - No error message changes - Only change is the call to StrategyDirtyBufferList is made using the maximum number of buffers that will be cleaned, rather than uselessly trawling through all of shared_buffers This changes the meaning of bgwriter_percent from percent of dirty buffers to percent of shared_buffers. The default settings of 1% of 1000 buffers gives up to 10 dirty block writes every 250ms Benefit: allows performance tuning by increases options for setting bgwriter_delay which would otherwise have an ineffectually high minimum setting Risk: low 1-line doc patch to follow, if this is approved. I am not objecting to the patch, but what value is there in having both bgwriter_percent and bgwriter_maxpages? Seems both are redundant and that one would be enough. In brief: i) for now: as little change as possible is good ii) the two parameters are OK iii) trying to decide an alternative takes time, which we do not have iv) what is presented here is simply a performance bug fix, not the best long term alternative... I'd like to move quickly: if we do this (or an alternative), it has to be done soon and it would be easy to discuss this until we run out of time. Could we vote: in RC3, or not? In more detail... The value of having both is: i) as little change as possible at this stage of RC - the main one ...which gives us stability ...and also avoids having to re-discuss what they *should* be ii) Having two isn't that bad. bgwriter_percent auto adjusts the length of the to-be-cleaned-list, so it is roughly useful anywhere between 500 and 1 shared_buffers. That is IMHO slightly more useful than a hard definition set via bgwriter_maxpages, since that is likely to be set wrong anyway - but has some value as an outside limit on the number of pages. [You may wish to set shared_buffers 1 even on smaller servers, since many now have 2GB RAM and yet a relatively poor I/O subsystem. Having maxpages set separately allows the majority of people to set shared_buffers higher without swamping their I/O subsystems because they didn't know about the r8.0 bgwriter feature/parameters] iii) changing the parameters might tempt us towards changing the algorithm, which is not a topic we have reached agreement on iv) I see it as a goal to remove all of those parameters anyway, as well as explore some of the many options and ideas everybody has presented, so further change is likely at the next release whatever is done now. The patch is as simple as I can make it and yet remove the unnecessary performance effect in the existing code. Thanks to Neil and others for showing that this was possible...I see this patch as a team effort. I've already spoken against larger change and would do so again now: if we don't agree this change, then I would vote for no-change simply because this patch is minimal change. We *suspect* further change is beneficial but we have no evidence to support what that change should be, amongst the large range of possible solutions proposed. -- Best Regards, Simon Riggs ---(end of broadcast)--- TIP 3: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] RC3 in ... ~12 hours ...
On Fri, 2004-12-31 at 01:38, Marc G. Fournier wrote: Baring any cries of *STOP* ... I'm going to try and get it packaged first thing in the morning ... Could I ask the Core team to vote on the bgwriter patch before RC3? I'm in the wrong timezone to discuss this quickly. -- Best Regards, Simon Riggs ---(end of broadcast)--- TIP 6: Have you searched our list archives? http://archives.postgresql.org
[HACKERS] uptime() for postmaster
Hi Bruce, I started to work on the uptime() for the postmaster yesterday. A couple of questions: a) is the name uptime() OK? b) is the return-type 'Interval' OK? c) does it make sense (... fit in the scheme?) to place the code here: src/backend/utils/misc/uptime.c d) Can I piggy-back on 'BackendParameters' to get postmasters start-time to the backends? happy new year, Matthias -- Matthias Schmidt Viehtriftstr. 49 67346 Speyer Tel.: +49 6232 4867 Fax.: +49 6232 640089 ---(end of broadcast)--- TIP 7: don't forget to increase your free space map settings
[HACKERS] 'COPY ... FROM' inserts to btree, blocks on buffer writeout
Hi, two concurrent clients try to 'COPY ... FROM ...' to the same table, feature_link. The second one (pid 17983) is waiting for an ExclusiveLock on the table's primary key index, key__idpk__flink. The first one (pid 17980) is inserting into the index's btree, waiting for a buffer lock. This locking state seems to persist. Do you have any idea what could cause this scenario and what kind of debugging information I could extract (the processes are still running) to find out who is currently holding the buffer lock in question? Thanks for your help, best wishes, Mike [fuchs:~/build/postgresql-8.0.0beta2] vi +1004 ./src/backend/storage/buffer/bufmgr.c 999 /* 1000* Protect buffer content against concurrent update. (Note that 1001* hint-bit updates can still occur while the write is in progress, 1002* but we assume that that will not invalidate the data written.) 1003*/ 1004 LockBuffer(buffer, BUFFER_LOCK_SHARE); 1005 [fuchs:~/build/postgresql-8.0.0beta2] gdb src/backend/postgres [...] (gdb) attach 17980 [...] (gdb) where #0 0x002a96181279 in semop () from /lib64/tls/libc.so.6 #1 0x00511707 in PGSemaphoreLock (sema=0x2aa16de2f0, interruptOK=0 '\0') at pg_sema.c:418 #2 0x0053549a in LWLockAcquire (lockid=32741, mode=LW_SHARED) at lwlock.c:315 #3 0x00528a3d in FlushBuffer (buf=0x2a9953ea40, reln=0x900640) at bufmgr.c:1004 #4 0x0052806d in BufferAlloc (reln=0x2768021, blockNum=3220402560, foundPtr=0x7fbff3728f ) at bufmgr.c:365 #5 0x00527edc in ReadBufferInternal (reln=0x2aa24786b8, blockNum=20055, bufferLockHeld=0 '\0') at bufmgr.c:153 #6 0x0044a930 in _bt_getbuf (rel=0x2aa24786b8, blkno=3220402560, access=2) at nbtpage.c:492 #7 0x00448c87 in _bt_split (rel=0x2aa24786b8, buf=673, firstright=89, newitemoff=164, newitemsz=40, newitem=0x911d40, newitemonleft=0 '\0', itup_off=0x7fbff374c6, itup_blkno=0x7fbff374c8) at nbtinsert.c:683 #8 0x004486ba in _bt_insertonpg (rel=0x2aa24786b8, buf=673, stack=0x9924f0, keysz=1, scankey=0x911d90, btitem=0x911d40, afteritem=0, split_only_page=0 '\0') at nbtinsert.c:500 #9 0x004481eb in _bt_doinsert (rel=0x2aa24786b8, btitem=0x911d40, index_is_unique=1 '\001', heapRel=0x2aa2477c38) at nbtinsert.c:141 #10 0x0044bb71 in btinsert (fcinfo=0x2768021) at nbtree.c:257 #11 0x005a44aa in OidFunctionCall6 (functionId=41320481, arg1=183111222968, arg2=548681250960, arg3=548681250928, arg4=9515332, arg5=183111220280, arg6=18446744073709551612) at fmgr.c:1487 #12 0x004476a5 in index_insert (indexRelation=0x2aa24786b8, datums=0x7fbff37890, nulls=0x7fbff37870 î, heap_t_ctid=0x913144, heapRelation=0x2aa2477c38, check_uniqueness=1 '\001') at indexam.c:226 #13 0x004cf3cf in ExecInsertIndexTuples (slot=0x0, tupleid=0x7fbff37180, estate=0x990450, is_vacuum=0 '\0') at execUtils.c:859 #14 0x0049c1d1 in CopyFrom (rel=0x2aa2477c38, attnumlist=0x990060, binary=0 '\0', oids=0 '\0', delim=0x63e4cd \t, null_print=0x608614 \\N, csv_mode=0 '\0', quote=0x0, escape=0x0, force_notnull_atts=0x911cb0) at copy.c:1958 #15 0x0049a34f in DoCopy (stmt=0x2768021) at copy.c:1043 #16 0x0053d7b9 in PortalRunUtility (portal=0x906b40, query=0x8fa5f0, dest=0x8fa910, completionTag=0x7fbff37f60 ) at pquery.c:839 #17 0x0053da41 in PortalRunMulti (portal=0x906b40, dest=0x8fa910, altdest=0x8fa910, completionTag=0x7fbff37f60 ) at pquery.c:902 ---Type return to continue, or q return to quit--- #18 0x0053d230 in PortalRun (portal=0x906b40, count=9223372036854775807, dest=0x8fa910, altdest=0x8fa910, completionTag=0x7fbff37f60 ) at pquery.c:543 #19 0x00539c59 in exec_simple_query ( query_string=0x8fa700 COPY feature_link from '/anniedev1/impseb/datastore/results/2004/12/29/22/38/bio:query:ncbi-blast:10562062.out_featurelink') at postgres.c:924 #20 0x0053bf6d in PostgresMain (argc=9414400, argv=0x86f028, username=0x86eff0 annieseb) at postgres.c:2970 #21 0x00514db7 in BackendRun (port=0x89c650) at postmaster.c:2848 #22 0x00514850 in BackendStartup (port=0x89c650) at postmaster.c:2470 #23 0x00512fde in ServerLoop () at postmaster.c:1215 #24 0x00512446 in PostmasterMain (argc=1, argv=0x804850) at postmaster.c:898 #25 0x004e3206 in main (argc=1, argv=0x804850) at main.c:265 [fuchs:/anniedev1/impseb/work/annie] ps -ef | grep impseb2 | grep -v grep | sort anniedb2 17976 21254 0 Dec29 ?00:03:50 postgres: annieseb impseb2 10.44.1.20(33855) idle in transaction anniedb2 17977 21254 0 Dec29 ?00:01:38 postgres: annieseb impseb2 10.44.1.20(33856) idle in transaction anniedb2 17979 21254 0 Dec29 ?00:03:21 postgres: annieseb impseb2 10.44.1.20(33857) idle in transaction anniedb2 17980 21254 0 Dec29 ?00:02:26 postgres: annieseb impseb2 10.44.1.20(33858) COPY anniedb2 17981 21254 0 Dec29 ?
Re: [HACKERS] uptime() for postmaster
Matthias Schmidt [EMAIL PROTECTED] writes: a) is the name uptime() OK? Probably should use pg_uptime(), or something else starting with pg_. b) is the return-type 'Interval' OK? It might be better to return the actual postmaster start time (as timestamptz) and let the user do whatever arithmetic he wants. With an interval, there's immediately a question of interpretation --- what current timestamp did you use in the computation? I'm not dead set on this, but it feels cleaner. c) does it make sense (... fit in the scheme?) to place the code here: src/backend/utils/misc/uptime.c No. This sort of stuff should go into utils/adt/. I'd be inclined to drop the function into one of the existing timestamp-related files rather than make a whole new file just for it. Someplace near the now() function would make sense, for instance. d) Can I piggy-back on 'BackendParameters' to get postmasters start-time to the backends? AFAICS you have no other choice. regards, tom lane ---(end of broadcast)--- TIP 4: Don't 'kill -9' the postmaster
[HACKERS] contrib regression on old versions
We only have a couple of buildfarm members reporting on versions older than 7.4. Both are RedHat based systems, and both fail the contrib regression suites. I'm inclined to have write these off as not worth fixing, and inhibit contrib regression tests for versions older than 7.4. in the next release of the buildfarm code. And unless we want to fix them maybe we should also say the same for 7.4. Thoughts? cheers andrew ---(end of broadcast)--- TIP 7: don't forget to increase your free space map settings
Re: [HACKERS] 'COPY ... FROM' inserts to btree, blocks on buffer writeout
Michael Wildpaner [EMAIL PROTECTED] writes: two concurrent clients try to 'COPY ... FROM ...' to the same table, feature_link. The second one (pid 17983) is waiting for an ExclusiveLock on the table's primary key index, key__idpk__flink. You didn't show a stack trace for this one ... regards, tom lane ---(end of broadcast)--- TIP 7: don't forget to increase your free space map settings
Re: [HACKERS] contrib regression on old versions
Andrew Dunstan [EMAIL PROTECTED] writes: We only have a couple of buildfarm members reporting on versions older than 7.4. Both are RedHat based systems, and both fail the contrib regression suites. I'm inclined to have write these off as not worth fixing, and inhibit contrib regression tests for versions older than 7.4. in the next release of the buildfarm code. And unless we want to fix them maybe we should also say the same for 7.4. I'm inclined to wonder why people want to run daily tests on dead branches in the first place ;-) I don't believe in having the code hide the failures, though. regards, tom lane ---(end of broadcast)--- TIP 5: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faqs/FAQ.html
[HACKERS] TSearch still in contrib?
Hello, Is there any reason why TSearch (not TSearch2) is still in contrib? TSearch is a 7.3.x module... Sincerely, Joshua D. Drake -- Command Prompt, Inc., home of Mammoth PostgreSQL - S/ODBC and S/JDBC Postgresql support, programming shared hosting and dedicated hosting. +1-503-667-4564 - [EMAIL PROTECTED] - http://www.commandprompt.com PostgreSQL Replicator -- production quality replication for PostgreSQL begin:vcard fn:Joshua Drake n:Drake;Joshua org:Command Prompt, Inc. adr:;;PO Box 215 ;Cascade Locks;OR;97014;US email;internet:[EMAIL PROTECTED] title:Consultant tel;work:503-667-4564 tel;fax:503-210-0334 x-mozilla-html:FALSE url:http://www.commandprompt.com version:2.1 end:vcard ---(end of broadcast)--- TIP 2: you can get off all lists at once with the unregister command (send unregister YourEmailAddressHere to [EMAIL PROTECTED])
Re: [HACKERS] contrib regression on old versions
Tom Lane wrote: I'm inclined to wonder why people want to run daily tests on dead branches in the first place ;-) Unless you explicitly force it, buildfarm will not run a test unless there is a change on the branch being tested. I am trying gently to persuade the buildfarm owners not to use the force flags. I don't believe in having the code hide the failures, though. Ok. cheers andrew ---(end of broadcast)--- TIP 9: the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Re: [HACKERS] 'COPY ... FROM' inserts to btree, blocks on buffer writeout
Michael Wildpaner [EMAIL PROTECTED] writes: two concurrent clients try to 'COPY ... FROM ...' to the same table, feature_link. The second one (pid 17983) is waiting for an ExclusiveLock on the table's primary key index, key__idpk__flink. The first one (pid 17980) is inserting into the index's btree, waiting for a buffer lock. This locking state seems to persist. After staring at this for a little bit I have a theory. The stack trace for 17980 indicates that it is trying to flush a dirty buffer so that it can take over the buffer for a new index page. It's blocked trying to acquire a shared buffer lock on that buffer, which means that someone else must have an exclusive buffer lock, which the code is not expecting because the buffer just came off the free list and therefore was not pinned by anyone. However ... FlushBuffer releases the BufMgrLock before trying to acquire the per-buffer lock. If 17980 lost its time slice during that interval, it'd be possible for someone else to come in and re-pin the chosen victim page and then lock the buffer before 17980 could. Normally this wouldn't be any big problem, but if the someone else later blocked on some lock held by 17980, you'd have a deadlock. I think that's exactly what we're seeing here. The victim buffer page must be the same one that 17983 is trying to split; so it's got exclusive lock (and pin) on that page, and is now stuck waiting for the lock that would give it the privilege to extend the index. A possible fix for this is to reorder the operations in FlushBuffer so that we share-lock the buffer before releasing BufMgrLock ... but I'm not sure that doesn't introduce other deadlock risks. It needs some thought. If this is the correct explanation, the bug has been around for a good while; but it's got to be a very low-probability scenario. Congrats to Michael for taking the time to dig into it when he saw it. regards, tom lane ---(end of broadcast)--- TIP 3: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
[HACKERS] exception handling in plpgsql
hello I am using the following sytex to handle exceptions in plpgsql (I am using postgres 8 rc1) some code EXCEPTION WHEN NO_DATA THEN RAISE NOTICE 'NO DATA'; WHEN OTHERS THEN RAISE NOTICE 'An exception occurred'; RETURN emp_rec; and i receive the following error ERROR: unrecognized exception condition no_data How can i rectify this error? __ Do you Yahoo!? Yahoo! Mail - Find what you need with new enhanced search. http://info.mail.yahoo.com/mail_250 ---(end of broadcast)--- TIP 5: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faqs/FAQ.html
[HACKERS] sysv_shmem potential problem
Hi I am using the sysv_shmem.c shared memory allocation api for os/2 and I ran into a problem when OS/2 allocates shared memory over the 2 gigabyte address boundary. The existing sysv_shmem.c tests for the return address of the segment as less than 0 and determines that a negative indicates an error. I have this patch (below) ifdef'd for OS/2 but I thought that there may be a problem on other platforms that can allocate shared memory over the 2 gig boundary The existing code is if (shmid 0) Index: sysv_shmem.c === RCS file: /projects/cvsroot/pgsql/src/backend/port/sysv_shmem.c,v retrieving revision 1.41 diff -r1.41 sysv_shmem.c 80a81,85 #ifdef __OS2__ /* shared memory address may be allocated over 2 gig and will negative */ /* so test for the explicit -1 return */ if (shmid == -1) #else 81a87 #endif -- --- [EMAIL PROTECTED] --- ---(end of broadcast)--- TIP 7: don't forget to increase your free space map settings
Re: [HACKERS] sysv_shmem potential problem
[EMAIL PROTECTED] writes: I am using the sysv_shmem.c shared memory allocation api for os/2 and I ran into a problem when OS/2 allocates shared memory over the 2 gigabyte address boundary. The existing sysv_shmem.c tests for the return address of the segment as less than 0 and determines that a negative indicates an error. shmget returns an ID, not an address. I quote from the Single Unix Spec: Upon successful completion, shmget() returns a non-negative integer, namely a shared memory identifier; otherwise, it returns -1 and errno will be set to indicate the error. While your change might be harmless, it should not be necessary, and it certainly shouldn't have anything to do with 2gig address boundaries. regards, tom lane ---(end of broadcast)--- TIP 6: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] sysv_shmem potential problem
I see, The shmem.c implementation I am using returns the OS/2 memory ID which also happens to be the base address of the allocated memory. Bug in shmem.c code then Thanks Lorne In [EMAIL PROTECTED], on 12/31/04 at 03:53 PM, Tom Lane [EMAIL PROTECTED] said: [EMAIL PROTECTED] writes: I am using the sysv_shmem.c shared memory allocation api for os/2 and I ran into a problem when OS/2 allocates shared memory over the 2 gigabyte address boundary. The existing sysv_shmem.c tests for the return address of the segment as less than 0 and determines that a negative indicates an error. shmget returns an ID, not an address. I quote from the Single Unix Spec: Upon successful completion, shmget() returns a non-negative integer, namely a shared memory identifier; otherwise, it returns -1 and errno will be set to indicate the error. While your change might be harmless, it should not be necessary, and it certainly shouldn't have anything to do with 2gig address boundaries. regards, tom lane -- --- [EMAIL PROTECTED] --- ---(end of broadcast)--- TIP 4: Don't 'kill -9' the postmaster
Re: [HACKERS] exception handling in plpgsql
On Fri, Dec 31, 2004 at 03:18:39PM -0500, Andrew Dunstan wrote: It has told you what the problem is. Use a handler for an exception that actually exists. To see what these are, read http://developer.postgresql.org/docs/postgres/plpgsql-errors-and-messages.html As the PL/pgSQL Trapping Errors documentation says, The _condition_ names can be any of those shown in Appendix A, so a more useful link would be: http://developer.postgresql.org/docs/postgres/errcodes-appendix.html -- Michael Fuhr http://www.fuhr.org/~mfuhr/ ---(end of broadcast)--- TIP 3: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] exception handling in plpgsql
Michael Fuhr wrote: On Fri, Dec 31, 2004 at 03:18:39PM -0500, Andrew Dunstan wrote: It has told you what the problem is. Use a handler for an exception that actually exists. To see what these are, read http://developer.postgresql.org/docs/postgres/plpgsql-errors-and-messages.html As the PL/pgSQL Trapping Errors documentation says, The _condition_ names can be any of those shown in Appendix A, so a more useful link would be: http://developer.postgresql.org/docs/postgres/errcodes-appendix.html You are right. My humble apologies. andrew ---(end of broadcast)--- TIP 7: don't forget to increase your free space map settings
Re: [HACKERS] 'COPY ... FROM' inserts to btree, blocks on buffer writeout
I wrote: A possible fix for this is to reorder the operations in FlushBuffer so that we share-lock the buffer before releasing BufMgrLock ... but I'm not sure that doesn't introduce other deadlock risks. That's too simplistic, since at least one caller of FlushBuffer is trying to write pages that may be in active use. If the target page is already exclusive-locked by another process then a deadlock between the per-buffer lock and BufMgrLock is entirely possible. I think that it would work for BufferAlloc to share-lock the victim buffer before calling FlushBuffer; we'd have to add a bool parameter to FlushBuffer telling it the lock was already acquired. This is safe in this particular path because a buffer that was just taken from the freelist should not be exclusive-locked by anyone. (It could possibly be share-locked by the bgwriter, but that won't cause a deadlock.) A process that wanted exclusive lock would first have to pin the buffer, which can't happen before we release the BufMgrLock. By adding the parameter we'd avoid changing the behavior for other callers. Comments, better ideas? BTW, it looks to me like this deadlock potential has existed at least since 7.0. I seem to recall one or two reports of unexplainable apparent deadlocks, which perhaps are now explained. regards, tom lane ---(end of broadcast)--- TIP 8: explain analyze is your friend
Re: [HACKERS] 'COPY ... FROM' inserts to btree, blocks on buffer
On Fri, 31 Dec 2004, Tom Lane wrote: Michael Wildpaner [EMAIL PROTECTED] writes: two concurrent clients try to 'COPY ... FROM ...' to the same table, feature_link. The second one (pid 17983) is waiting for an ExclusiveLock on the table's primary key index, key__idpk__flink. You didn't show a stack trace for this one ... Here it is: [fuchs:/people/mike/build/postgresql-8.0.0beta2] gdb src/backend/postgres [...] (gdb) attach 17983 [...] (gdb) where #0 0x002a96181279 in semop () from /lib64/tls/libc.so.6 #1 0x00511707 in PGSemaphoreLock (sema=0x2aa16d8f90, interruptOK=1 '\001') at pg_sema.c:418 #2 0x00533e76 in ProcSleep (lockMethodTable=0x274801d, lockmode=7, lock=0x2aa1773878, proclock=0x2aa18ee398) at proc.c:725 #3 0x00532a7c in WaitOnLock (lockmethodid=32797, locallock=0x894180, owner=0x975130) at lock.c:1037 #4 0x0053246f in LockAcquire (lockmethodid=1, locktag=0x1, xid=2710496152, lockmode=7, dontWait=0 '\0') at lock.c:754 #5 0x00531aa3 in LockPage (relation=0x274801d, blkno=3220402448, lockmode=7) at lmgr.c:267 #6 0x0044a98d in _bt_getbuf (rel=0x2aa2466168, blkno=3220402448, access=2) at nbtpage.c:490 #7 0x00448c87 in _bt_split (rel=0x2aa2466168, buf=16355, firstright=89, newitemoff=119, newitemsz=40, newitem=0x985828, newitemonleft=0 '\0', itup_off=0x7fbff374c6, itup_blkno=0x7fbff374c8) at nbtinsert.c:683 #8 0x004486ba in _bt_insertonpg (rel=0x2aa2466168, buf=16355, stack=0x9a2a70, keysz=1, scankey=0x984a60, btitem=0x985828, afteritem=0, split_only_page=0 '\0') at nbtinsert.c:500 #9 0x004481eb in _bt_doinsert (rel=0x2aa2466168, btitem=0x985828, index_is_unique=1 '\001', heapRel=0x2aa24656e8) at nbtinsert.c:141 #10 0x0044bb71 in btinsert (fcinfo=0x274801d) at nbtree.c:257 #11 0x005a44aa in OidFunctionCall6 (functionId=41189405, arg1=18347880, arg2=548681250960, arg3=548681250928, arg4=10030788, arg5=18345192, arg6=18446744073709551612) at fmgr.c:1487 #12 0x004476a5 in index_insert (indexRelation=0x2aa2466168, datums=0x7fbff37890, nulls=0x7fbff37870 î, heap_t_ctid=0x990ec4, heapRelation=0x2aa24656e8, check_uniqueness=1 '\001') at indexam.c:226 #13 0x004cf3cf in ExecInsertIndexTuples (slot=0x0, tupleid=0x7fbff37110, estate=0x9828a0, is_vacuum=0 '\0') at execUtils.c:859 #14 0x0049c1d1 in CopyFrom (rel=0x2aa24656e8, attnumlist=0x9a25f0, binary=0 '\0', oids=0 '\0', delim=0x63e4cd \t, null_print=0x608614 \\N, csv_mode=0 '\0', quote=0x0, escape=0x0, force_notnull_atts=0x9849d0) at copy.c:1958 #15 0x0049a34f in DoCopy (stmt=0x274801d) at copy.c:1043 #16 0x0053d7b9 in PortalRunUtility (portal=0x906b70, query=0x8fa620, dest=0x8fa940, completionTag=0x7fbff37f60 ) at pquery.c:839 #17 0x0053da41 in PortalRunMulti (portal=0x906b70, dest=0x8fa940, altdest=0x8fa940, completionTag=0x7fbff37f60 ) at pquery.c:902 #18 0x0053d230 in PortalRun (portal=0x906b70, count=9223372036854775807, dest=0x8fa940, altdest=0x8fa940, completionTag=0x7fbff37f60 ) at pquery.c:543 #19 0x00539c59 in exec_simple_query ( query_string=0x8fa730 COPY feature_link from '/anniedev1/impseb/datastore/results/2004/12/29/22/38/bio:query:ncbi-blast:10562060.out_featurelink') at postgres.c:924 #20 0x0053bf6d in PostgresMain (argc=9414448, argv=0x86f028, username=0x86eff0 annieseb) at postgres.c:2970 #21 0x00514db7 in BackendRun (port=0x89c650) at postmaster.c:2848 #22 0x00514850 in BackendStartup (port=0x89c650) at postmaster.c:2470 #23 0x00512fde in ServerLoop () at postmaster.c:1215 #24 0x00512446 in PostmasterMain (argc=1, argv=0x804850) at postmaster.c:898 #25 0x004e3206 in main (argc=1, argv=0x804850) at main.c:265 Happy new year! (nearly 2 AM MET here ;) Best wishes, Mike -- Do not feel safe. The poet remembers. DI Michael Wildpaner You can kill one, but another is born. Ph.D. Student The words are written down, the deed, the date. (Czeslaw Milosz) ---(end of broadcast)--- TIP 5: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faqs/FAQ.html
Re: [HACKERS] TODO item: make world safe for spaces in build/install
Tom Lane wrote: Pursuant to Theodore Petrosky's recent trouble report, I thought I would see what happens if you try to build Postgres in a directory whose path contains spaces, or if the install prefix contains spaces. It doesn't work, not even close. It looks to me like there are a couple of hundred places in the Makefiles that would need to be fixed by adding quotes around various references to $(libdir) and related variables. I'm afraid that VPATH builds with spaces in the path to the original source tree are not realistically fixable at all; certainly prep_buildtree cannot easily be fixed (it misparses the output of find) and it looks like we'd need upstream Autoconf changes as well to get configure to work fully in that scenario. I don't think it's appropriate to try to fix this at RC3 stage, seeing that it's not a regression --- this never worked before either. But maybe we should add it to the TODO list to consider working on someday. I've just reluctantly managed to convince myself that it's worth a TODO. In 99% of cases it's a matter of don't do that. If we didn't have relocatable installs it would matter a lot. Maybe for now it's worth checking for and causing an error at the configure stage. That would be nicer than some arbitrary failure later in the process. cheers andrew ---(end of broadcast)--- TIP 4: Don't 'kill -9' the postmaster
[HACKERS] RC3 Bundled a wee bit later then planned ...
Busy day, forgot all about it :( I've up'd all occurances of rc2 - rc3 (so I got the win32 file in include) ... as well as upgraded the Copyright to be 1996-2005 ... the commit was so large that it wouldn't go through pgsql-committers, but is in the logs ... And I upgrade the dsssl stylesheets to 1.79 before building this one ... So ... it *should* be a clean build ... Will announce it more generally on Sat ... Marc G. Fournier Hub.Org Networking Services (http://www.hub.org) Email: [EMAIL PROTECTED] Yahoo!: yscrappy ICQ: 7615664 ---(end of broadcast)--- TIP 8: explain analyze is your friend
[HACKERS] Copyright update
With 8.0 coming out in 2005, I think I should update the copyrights on the files. -- Bruce Momjian| http://candle.pha.pa.us pgman@candle.pha.pa.us | (610) 359-1001 + If your life is a hard drive, | 13 Roberts Road + Christ can be your backup.| Newtown Square, Pennsylvania 19073 ---(end of broadcast)--- TIP 7: don't forget to increase your free space map settings
Re: [HACKERS] Copyright update
Bruce Momjian wrote: With 8.0 coming out in 2005, I think I should update the copyrights on the files. I see Marc has already done the update. I am checking with the src/tools/copyright to make sure he got them all. He updated legal.sgml so I bet he got them all. -- Bruce Momjian| http://candle.pha.pa.us pgman@candle.pha.pa.us | (610) 359-1001 + If your life is a hard drive, | 13 Roberts Road + Christ can be your backup.| Newtown Square, Pennsylvania 19073 ---(end of broadcast)--- TIP 2: you can get off all lists at once with the unregister command (send unregister YourEmailAddressHere to [EMAIL PROTECTED])
Re: [HACKERS] Copyright update
At 2004-12-31 23:49:35 -0500, pgman@candle.pha.pa.us wrote: With 8.0 coming out in 2005, I think I should update the copyrights on the files. I don't think it actually makes any difference. -- ams ---(end of broadcast)--- TIP 3: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [PATCHES] [HACKERS] Bgwriter behavior
This change isn't going to make it for RC3, and it probably not something we want to rush. I think there are a few issues involved: o everyone agrees the current meaning of bgwriter_percent is useless (percent of dirty buffers) o removal of bgwriter_percent will cause problems because postgresql.conf is only installed via initdb, so beta users will have to have some workaround so their existing postgresql.conf files work. o bgwriter_percent and bgwriter_maxpages are duplicate for a given number of buffers and it isn't clear which one takes precedence. o 8.1 might use these variables with different meanings, causing slight upgrade confusion. o Another idea is for bgwriter_percent to control how much of the buffer is scanned. Tom feels bgwriter_maxpages is good because it allows the user to specify the I/O traffic, while bgwriter_percent as total pages (not just dirty ones) is perhaps easier to set a default (I/O load varies based on buffer cache size) and perhaps easier to understand. I am not sure what to suggest at this point but whatever solution we use should take the above issues into account. --- Simon Riggs wrote: On Fri, 2004-12-31 at 01:14, Bruce Momjian wrote: Simon Riggs wrote: On Mon, 2004-12-27 at 22:21, Bruce Momjian wrote: Should we consider at least adjusting the meaning of bgwriter_percent? Yes. As things stand, this is the only change that seems safe. Here's a very short patch that implements this change within BufferSync in bufmgr.c - No algorithm changes - No error message changes - Only change is the call to StrategyDirtyBufferList is made using the maximum number of buffers that will be cleaned, rather than uselessly trawling through all of shared_buffers This changes the meaning of bgwriter_percent from percent of dirty buffers to percent of shared_buffers. The default settings of 1% of 1000 buffers gives up to 10 dirty block writes every 250ms Benefit: allows performance tuning by increases options for setting bgwriter_delay which would otherwise have an ineffectually high minimum setting Risk: low 1-line doc patch to follow, if this is approved. I am not objecting to the patch, but what value is there in having both bgwriter_percent and bgwriter_maxpages? Seems both are redundant and that one would be enough. In brief: i) for now: as little change as possible is good ii) the two parameters are OK iii) trying to decide an alternative takes time, which we do not have iv) what is presented here is simply a performance bug fix, not the best long term alternative... I'd like to move quickly: if we do this (or an alternative), it has to be done soon and it would be easy to discuss this until we run out of time. Could we vote: in RC3, or not? In more detail... The value of having both is: i) as little change as possible at this stage of RC - the main one ...which gives us stability ...and also avoids having to re-discuss what they *should* be ii) Having two isn't that bad. bgwriter_percent auto adjusts the length of the to-be-cleaned-list, so it is roughly useful anywhere between 500 and 1 shared_buffers. That is IMHO slightly more useful than a hard definition set via bgwriter_maxpages, since that is likely to be set wrong anyway - but has some value as an outside limit on the number of pages. [You may wish to set shared_buffers 1 even on smaller servers, since many now have 2GB RAM and yet a relatively poor I/O subsystem. Having maxpages set separately allows the majority of people to set shared_buffers higher without swamping their I/O subsystems because they didn't know about the r8.0 bgwriter feature/parameters] iii) changing the parameters might tempt us towards changing the algorithm, which is not a topic we have reached agreement on iv) I see it as a goal to remove all of those parameters anyway, as well as explore some of the many options and ideas everybody has presented, so further change is likely at the next release whatever is done now. The patch is as simple as I can make it and yet remove the unnecessary performance effect in the existing code. Thanks to Neil and others for showing that this was possible...I see this patch as a team effort. I've already spoken against larger change and would do so again now: if we don't agree this change, then I would vote for no-change simply because this patch is minimal change. We *suspect* further change is beneficial but we have no evidence to support what that change should be, amongst the large range of possible solutions proposed. -- Best Regards, Simon Riggs ---(end of broadcast)--- TIP 2: you can get
