Re: [HACKERS] tuple concurrently updated
On Thu, 25 Jul 2002, Tom Lane wrote: Kangmo, Kim [EMAIL PROTECTED] writes: If the index on the same class, two concurrent CREATE INDEX command can update pg_class.relpages at the same time. Or try to, anyway. The problem here is that the code that updates system catalogs is not prepared to cope with concurrent updates to the same tuple. I see. So the error is basically harmless, and can be rectified anyway with an ANALYZE, right? cjs -- Curt Sampson [EMAIL PROTECTED] +81 90 7737 2974 http://www.netbsd.org Don't you know, in this new Dark Age, we're all light. --XTC ---(end of broadcast)--- TIP 6: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] Question about LWLockAcquire's use of semaphores instead of spinlocks
Hi Bob: We're have been working with an sproc version of postgres and it has improve performance over a NUMA3 origin 3000 due to IRIX implements round_robin by default on memory placement instead of first touch as it did on fork. We're been wondering about replacing IPC shmem with a shared arena to help performance improve on IRIX. I dont´know if people here in postgres are interested on specifical ports but it could help you improve your performance. Regards - Original Message - From: Robert E. Bruccoleri [EMAIL PROTECTED] To: Tom Lane [EMAIL PROTECTED] Cc: [EMAIL PROTECTED]; [EMAIL PROTECTED] Sent: Sunday, July 28, 2002 5:45 AM Subject: Re: [HACKERS] Question about LWLockAcquire's use of semaphores instead of spinlocks Tom Lane writes: Robert E. Bruccoleri [EMAIL PROTECTED] writes: On SGI multiprocessor machines, I suspect that a spinlock implementation of LWLockAcquire would give better performance than using IPC semaphores. Is there any specific reason that a spinlock could not be used in this context? Are you confusing LWLockAcquire with TAS spinlocks? No. If you're saying that we don't have an implementation of TAS for SGI hardware, then feel free to contribute one. If you are wanting to replace LWLocks with spinlocks, then you are sadly mistaken, IMHO. This touches on my question. Why am I mistaken? I don't understand. BTW, about 5 years ago, I rewrote the TAS spinlocks for the SGI platform to make it work correctly. The current implementation is fine. +-++ | Robert E. Bruccoleri, Ph.D. | email: [EMAIL PROTECTED]| | P.O. Box 314| URL: http://www.congen.com/~bruc | | Pennington, NJ 08534|| +-++ ---(end of broadcast)--- TIP 4: Don't 'kill -9' the postmaster ---(end of broadcast)--- TIP 3: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] tuple concurrently updated
Curt Sampson [EMAIL PROTECTED] writes: On Thu, 25 Jul 2002, Tom Lane wrote: Kangmo, Kim [EMAIL PROTECTED] writes: If the index on the same class, two concurrent CREATE INDEX command can update pg_class.relpages at the same time. Or try to, anyway. The problem here is that the code that updates system catalogs is not prepared to cope with concurrent updates to the same tuple. I see. So the error is basically harmless, and can be rectified anyway with an ANALYZE, right? Other than the fact that the second CREATE INDEX fails and rolls back, there's no problem ;-) I was thinking that it might help to have UpdateStats not try to update the pg_class tuple if the correct value is already present. This will just narrow the window for failure, not eliminate it; but it'd be a simple change and would probably help some. Another thing to look at is merging that routine with setRelhasindex so that a CREATE INDEX involves only one update to the table's pg_class row, not two. regards, tom lane ---(end of broadcast)--- TIP 4: Don't 'kill -9' the postmaster
Re: [HACKERS] tuple concurrently updated
On Sun, 28 Jul 2002, Tom Lane wrote: Other than the fact that the second CREATE INDEX fails and rolls back, there's no problem ;-) Agh! So what, in the current version of postgres, are my options for doing parallel index builds? cjs -- Curt Sampson [EMAIL PROTECTED] +81 90 7737 2974 http://www.netbsd.org Don't you know, in this new Dark Age, we're all light. --XTC ---(end of broadcast)--- TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]
Re: [HACKERS] Question about LWLockAcquire's use of semaphores instead of spinlocks
Robert E. Bruccoleri [EMAIL PROTECTED] writes: Tom Lane writes: If you're saying that we don't have an implementation of TAS for SGI hardware, then feel free to contribute one. If you are wanting to replace LWLocks with spinlocks, then you are sadly mistaken, IMHO. This touches on my question. Why am I mistaken? I don't understand. Because we just got done replacing spinlocks with LWLocks ;-). I don't believe that reverting that change will improve matters. It will certainly hurt on SMP machines, and I believe that it would at best be a breakeven proposition on uniprocessors. See the discussions last fall that led up to development of the LWLock mechanism. The problem with TAS spinlocks is that they are suitable only for implementing locks that will be held for *very short* periods, ie, actual contention is rare. Over time we had allowed that mechanism to be abused for locking fairly large and complex shared-memory data structures (eg, the lock manager, the buffer manager). The next step up, a lock-manager lock, is very expensive and certainly can't be used by the lock manager itself anyway. LWLocks are an intermediate mechanism that is marginally more expensive than a spinlock but behaves much more gracefully in the presence of contention. LWLocks also allow us to distinguish shared and exclusive lock modes, thus further reducing contention in some cases. BTW, now that I reread the title of your message, I wonder if you haven't just misunderstood what's happening in lwlock.c. There is no IPC semaphore call in the fast (no-contention) path of control. A semaphore call occurs only when we are forced to wait, ie, yield the processor. Substituting a spinlock for that cannot improve matters; it would essentially result in wasting the remainder of our timeslice in a busy-loop, rather than yielding the CPU at once to some other process that can get some useful work done. regards, tom lane ---(end of broadcast)--- TIP 5: Have you checked our extensive FAQ? http://www.postgresql.org/users-lounge/docs/faq.html
Re: [HACKERS] Virus Emails
That may be true with some variants. However my mail server has rejected the relay of several mails sent pretending to be from me (envelope sender) to other parties and I think these could be klez variants or another such virus. Since my server rejected them I cannot be sure of the contents. On Sunday 28 July 2002 04:06 am, Curt Sampson wrote: On Sat, 27 Jul 2002, Tom Lane wrote: One of the nastier aspects of the Klez virus However, even a trivial look at the detail mail headers (Received: etc) will convince you that the spam did not originate from the claimed From: address. If you care to post a few sets of complete headers, we can probably triangulate pretty quickly on the virus-infected loser who's originating these messages. It appears to me that the envelope sender is not forged by Klez.H, assuming that that's the virus I'm getting all the time. So you could check for the Return-Path: header, or maybe From (note: no colon) if you're using a Berkeley-mailbox style system, and find out the e-mail address of the real sender. cjs ---(end of broadcast)--- TIP 6: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] Question about LWLockAcquire's use of semaphores instead of spinlocks
Dear Tom, Thank you for the explanation. I did not understand what was going on in lwlock.c. My systems are all SGI Origins having between 8 and 32 processors, and I've been running PostgreSQL on them for about 5 years. These machines do provide a number of good mechanisms for high performance shared memory parallelism that I don't think are found elsewhere. I wish that I had the time to understand and tune PostgreSQL to run really well on them. I have a question for you and other developers with regard to my SGI needs. If I made a functional Origin 2000 system available to you with hardware support, would the group be willing to tailor the SGI port for better performance? Sincerely, Bob Robert E. Bruccoleri [EMAIL PROTECTED] writes: Tom Lane writes: If you're saying that we don't have an implementation of TAS for SGI hardware, then feel free to contribute one. If you are wanting to replace LWLocks with spinlocks, then you are sadly mistaken, IMHO. This touches on my question. Why am I mistaken? I don't understand. Because we just got done replacing spinlocks with LWLocks ;-). I don't believe that reverting that change will improve matters. It will certainly hurt on SMP machines, and I believe that it would at best be a breakeven proposition on uniprocessors. See the discussions last fall that led up to development of the LWLock mechanism. The problem with TAS spinlocks is that they are suitable only for implementing locks that will be held for *very short* periods, ie, actual contention is rare. Over time we had allowed that mechanism to be abused for locking fairly large and complex shared-memory data structures (eg, the lock manager, the buffer manager). The next step up, a lock-manager lock, is very expensive and certainly can't be used by the lock manager itself anyway. LWLocks are an intermediate mechanism that is marginally more expensive than a spinlock but behaves much more gracefully in the presence of contention. LWLocks also allow us to distinguish shared and exclusive lock modes, thus further reducing contention in some cases. BTW, now that I reread the title of your message, I wonder if you haven't just misunderstood what's happening in lwlock.c. There is no IPC semaphore call in the fast (no-contention) path of control. A semaphore call occurs only when we are forced to wait, ie, yield the processor. Substituting a spinlock for that cannot improve matters; it would essentially result in wasting the remainder of our timeslice in a busy-loop, rather than yielding the CPU at once to some other process that can get some useful work done. regards, tom lane +-++ | Robert E. Bruccoleri, Ph.D. | email: [EMAIL PROTECTED]| | P.O. Box 314| URL: http://www.congen.com/~bruc | | Pennington, NJ 08534|| +-++ ---(end of broadcast)--- TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]
Re: [HACKERS] Question about LWLockAcquire's use of semaphores instead of spinlocks
Dear Luis, I would be very interested. Replacing the IPC shared memory with an arena make a lot of sense. --Bob Hi Bob: We're have been working with an sproc version of postgres and it has improve performance over a NUMA3 origin 3000 due to IRIX implements round_robin by default on memory placement instead of first touch as it did on fork. We're been wondering about replacing IPC shmem with a shared arena to help performance improve on IRIX. I dont´know if people here in postgres are interested on specifical ports but it could help you improve your performance. Regards +-++ | Robert E. Bruccoleri, Ph.D. | email: [EMAIL PROTECTED]| | P.O. Box 314| URL: http://www.congen.com/~bruc | | Pennington, NJ 08534|| +-++ ---(end of broadcast)--- TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]
Re: [HACKERS] Virus Emails
On Sat, 27 Jul 2002, Tom Lane wrote: One of the nastier aspects of the Klez virus However, even a trivial look at the detail mail headers (Received: etc) will convince you that the spam did not originate from the claimed From: address. If you care to post a few sets of complete headers, we can probably triangulate pretty quickly on the virus-infected loser who's originating these messages. It appears to me that the envelope sender is not forged by Klez.H, assuming that that's the virus I'm getting all the time. So you could check for the Return-Path: header, or maybe From (note: no colon) if you're using a Berkeley-mailbox style system, and find out the e-mail address of the real sender. cjs -- Curt Sampson [EMAIL PROTECTED] +81 90 7737 2974 http://www.netbsd.org Don't you know, in this new Dark Age, we're all light. --XTC ---(end of broadcast)--- TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]
Re: [HACKERS] [PATCHES] prepareable statements
Neil Conway writes: On Thu, Jul 25, 2002 at 10:54:04PM +0200, Peter Eisentraut wrote: I'm not sure I like that. It seems too confusing. Why not keep it as the standard says? (After all, it is the PREPARE part that we're adjusting, not EXECUTE.) I think it's both, isn't it? My understanding of Tom's post is that the features described by SQL92 are somewhat similar to the patch, but not directly related. What I was trying to say is this: There is one prepared statement facility in the standards that allows you to prepare a statement defined in a host variable, whereas you are proposing one that specifies the statement explicitly. However, both of these are variants of the same concept, so the EXECUTE command doesn't need to be different. -- Peter Eisentraut [EMAIL PROTECTED] ---(end of broadcast)--- TIP 2: you can get off all lists at once with the unregister command (send unregister YourEmailAddressHere to [EMAIL PROTECTED])
Re: [HACKERS] Virus Emails
God, I go through 200+ of those almost daily as moderator ... imagine if we had the lists open? :) On Sat, 27 Jul 2002, Christopher Kings-Lynne wrote: Hi guys, I seem to be getting virus emails that pretend to be one of your guys. eg. I get them from T.Ishii and N.Conway, etc. Anyone out there on the list who should perhaps scan their computer? :) Chris ---(end of broadcast)--- TIP 5: Have you checked our extensive FAQ? http://www.postgresql.org/users-lounge/docs/faq.html ---(end of broadcast)--- TIP 3: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] Virus Emails
On Sun, 2002-07-28 at 20:10, Marc G. Fournier wrote: God, I go through 200+ of those almost daily as moderator ... imagine if we had the lists open? :) I picked up a copy of McAfee's vscan for FreeBSD from one of my contract people, and have amavisd-milter running to prevent them from even getting in the door. Mayhaps pgsql.org should do the same? On Sat, 27 Jul 2002, Christopher Kings-Lynne wrote: Hi guys, I seem to be getting virus emails that pretend to be one of your guys. eg. I get them from T.Ishii and N.Conway, etc. Anyone out there on the list who should perhaps scan their computer? :) Chris ---(end of broadcast)--- TIP 5: Have you checked our extensive FAQ? http://www.postgresql.org/users-lounge/docs/faq.html ---(end of broadcast)--- TIP 3: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly -- Larry Rosenman http://www.lerctr.org/~ler Phone: +1 972-414-9812 E-Mail: [EMAIL PROTECTED] US Mail: 1905 Steamboat Springs Drive, Garland, TX 75044-6749 ---(end of broadcast)--- TIP 3: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] [GENERAL] Stats Collector
Looks to me, someone forgot something. That would be me and now I
remember that I originally wanted to add some utility command for that.
What you need in the meantime is a little C function that calls
void pgstat_reset_counters(void);
I might find the time tomorrow to write one for you if you don't know
how.
Is this the kind of thing you mean?
#include postgres.h
#include fmgr.h
extern Datum pg_reset_stats(PG_FUNCTION_ARGS);
PG_FUNCTION_INFO_V1(pg_reset_stats);
Datum
pg_reset_stats(PG_FUNCTION_ARGS)
{
void pgstat_reset_counters(void);
PG_RETURN_VOID();
}
With this code I get this:
test=# select pg_reset_stats();
ERROR: Unable to look up type id 0
I'm creating it like this:
create or replace function pg_reset_stats() returns opaque as
'/home/chriskl/local/lib/postgresql/pg_reset_stats.so'
language 'C';
Is it something to do with the return type being declared wrongly?
Hmm...the manual indicates that opaque functions cannot be called directly -
so what the heck do I do?
Also, where would I put this function in the main postgres source and how
would I modify initdb?
Chris
---(end of broadcast)---
TIP 3: if posting/reading through Usenet, please send an appropriate
subscribe-nomail command to [EMAIL PROTECTED] so that your
message can get through to the mailing list cleanly
Re: [HACKERS] [GENERAL] Stats Collector
Christopher Kings-Lynne [EMAIL PROTECTED] writes: Is it something to do with the return type being declared wrongly? Yup. Make it return a useless '1' or 'true' or some such. regards, tom lane ---(end of broadcast)--- TIP 4: Don't 'kill -9' the postmaster
