Re: [HACKERS] Pre-forking backend
At 04:50 PM 9/29/01 -0400, Tom Lane wrote: Bruce Momjian [EMAIL PROTECTED] writes: On some operating systems, only one child at a time can accept() on the socket. On these, you have to lock around the call to accept(). But how do you know the client wants the database you have forked? They could want a different one. This approach would only work as far as saving the fork() call itself, not the backend setup time. Not sure it's worth the trouble. I doubt that the fork itself is a huge component of our start time; it's setting up all the catalog caches and so forth that's expensive. I don't think there's much benefit as well. For most cases where preforking would help, you could just simply not disconnect. Get the app to connect to the correct DB on startup and then just wait, do stuff then don't disconnect either rollback or commit. Or have a DB connection pool. What would be good is a DB that can handle lots of connections well. That would help almost any case. Preforking is good for web servers but for DB servers it doesn't seem as useful. Cheerio, Link. ---(end of broadcast)--- TIP 4: Don't 'kill -9' the postmaster
Re: [HACKERS] Pre-forking backend
Bruce Momjian [EMAIL PROTECTED] writes: How hard would it be to pre-fork an extra backend How are you going to pass the connection socket to an already-forked child process? AFAIK there's no remotely portable way ... Umm... Apache? They use a preforking model and it works quite well for every *NIX that Apache runs on. ;) Maybe RSE can comment on this further... -sc -- Sean Chittenden ---(end of broadcast)--- TIP 2: you can get off all lists at once with the unregister command (send unregister YourEmailAddressHere to [EMAIL PROTECTED])
Re: [HACKERS] Spinlock performance improvement proposal
Vadim Mikheev wrote: I have committed changes to implement this proposal. I'm not seeing any significant performance difference on pgbench on my single-CPU system ... but pgbench is I/O bound anyway on this hardware, so that's not very surprising. I'll be interested to see what other people observe. (Tatsuo, care to rerun that 1000-client test?) What is your system? CPU, memory, IDE/SCSI, OS? Scaling factor and # of clients? BTW1 - shouldn't we rewrite pgbench to use threads instead of libpq async queries? At least as option. I'd say that with 1000 clients current pgbench implementation is very poor. Would it be useful to run a test like the AS3AP benchmark on this to look for performance measurements? On linux the Open Source Database Benchmark (osdb.sf.net) does this, and it's multi-threaded to simulate multiple clients hitting the database at once. The only inconvenience is having to download a separate program to generate the test data, as OSDB doesn't generate this itself yet. I can supply the test program (needs to be run through Wine) and a script if anyone wants. ??? BTW2 - shouldn't we learn if there are really portability/performance issues in using POSIX mutex-es (and cond. variables) in place of TAS (and SysV semaphores)? Vadim ---(end of broadcast)--- TIP 3: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly -- My grandfather once told me that there are two kinds of people: those who work and those who take the credit. He told me to try to be in the first group; there was less competition there. - Indira Gandhi ---(end of broadcast)--- TIP 3: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] Glitch in handling of postmaster -o options
Tom Lane wrote: snip I wonder whether we should retire -o. Or change it so that the postmaster parses the given options for itself (consequently adjusting its copies of GUC variables) instead of passing them on to backends for parsing at backend start time. Retiring -o would seem like a good idea. Just about every person I bump into that's new to PostgreSQL doesn't get -o right for some time. It's simple in concept, but different from how every other package works, so it confuses newcomers who don't know the difference between the different parts of PostgreSQL. It would be good if we could just having options that replace each -o option (i.e. -F instead of -o '-F', -x -y instead of -o '-x -y') so it's similar to how other programs command line arguments work. Regards and best wishes, Justin Clift regards, tom lane ---(end of broadcast)--- TIP 3: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly -- My grandfather once told me that there are two kinds of people: those who work and those who take the credit. He told me to try to be in the first group; there was less competition there. - Indira Gandhi ---(end of broadcast)--- TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]
Re: [HACKERS] Glitch in handling of postmaster -o options
Justin Clift [EMAIL PROTECTED] writes: Retiring -o would seem like a good idea. That was what I was thinking too. I can think of ways to reimplement -o options so that they work safely ... but is it worth the trouble? AFAICS, -o options confuse both people and machines, and have no redeeming value beyond supporting old startup scripts. Which could easily be updated. Some judicious feature removal may be the best path here. regards, tom lane ---(end of broadcast)--- TIP 5: Have you checked our extensive FAQ? http://www.postgresql.org/users-lounge/docs/faq.html
Re: [HACKERS] multibyte performance
I did some benchmarking with/without multibyte support using current. (1) regression test (2) pgbench pgbench unfortunately seems quite irrelevant to this issue, since it performs no textual operations whatsoever. It'd be interesting to modify pgbench so that it updates the filler column somehow on each update (perhaps store a text copy of the new balance there), and then repeat the tests. Ok. Here is the result: Without multibyte: 1 50.190473 2 65.943052 4 74.908752 8 62.589973 16 87.546988 32 94.448773 64 88.019452 128 64.107839 With multibyte: 1 47.473237 2 61.435628 4 83.047684 8 95.556846 16 92.157352 32 95.879001 64 91.486652 128 66.926568 a graph is attached. bench2.png Description: PNG image ---(end of broadcast)--- TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]
Re: [HACKERS] Spinlock performance improvement proposal
I wrote: The following proposal should improve performance substantially when there is contention for a lock, but it creates no portability risks ... I have committed changes to implement this proposal. I'm not seeing any significant performance difference on pgbench on my single-CPU system ... but pgbench is I/O bound anyway on this hardware, so that's not very surprising. I'll be interested to see what other people observe. (Tatsuo, care to rerun that 1000-client test?) regards, tom lane ---(end of broadcast)--- TIP 6: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] Spinlock performance improvement proposal
I have committed changes to implement this proposal. I'm not seeing any significant performance difference on pgbench on my single-CPU system ... but pgbench is I/O bound anyway on this hardware, so that's not very surprising. I'll be interested to see what other people observe. (Tatsuo, care to rerun that 1000-client test?) What is your system? CPU, memory, IDE/SCSI, OS? Scaling factor and # of clients? BTW1 - shouldn't we rewrite pgbench to use threads instead of libpq async queries? At least as option. I'd say that with 1000 clients current pgbench implementation is very poor. BTW2 - shouldn't we learn if there are really portability/performance issues in using POSIX mutex-es (and cond. variables) in place of TAS (and SysV semaphores)? Vadim ---(end of broadcast)--- TIP 3: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] Spinlock performance improvement proposal
On Thursday 27 September 2001 04:09, you wrote: This depends on your system. Solaris has a huge difference between thread and process context switch times, whereas Linux has very little difference (and in fact a Linux process context switch is about as fast as a Solaris thread switch on the same hardware--Solaris is just a pig when it comes to process context switching). I have never worked on any big systems but from what (little) I have seen, I think there should be a hybrid model. This whole discussion started off, from poor performance on SMP machines. If I am getting this correctly, threads can be spread on multiple CPUs if available but process can not. So I would suggest to have threaded approach for intensive tasks such as sorting/searching etc. IMHO converting entire paradigm to thread based is a huge task and may not be required in all cases. I think of an approach. Threads are created when they are needed but they are kept dormant when not needed. So that there is no recreation overhead(if that's a concern). So at any given point of time, one back end connection has as many threads as number of CPUs. More than that may not yield much of performance improvement. Say a big task like sorting is split and given to different threads so that it can use them all. It should be easy to switch the threading function and arguments on the fly, restricting number of threads and there will not be much of thread switching as each thread handles different parts of task and later the results are merged. Number of threads should be equal to or twice that of number of CPUs. I don't think more than those many threads would yield any performance improvement. And with this approach we can migrate one functionality at a time to threaded one, thus avoiding big effort at any given time. Just a suggestion. Shridhar _ Do You Yahoo!? Get your free @yahoo.com address at http://mail.yahoo.com ---(end of broadcast)--- TIP 5: Have you checked our extensive FAQ? http://www.postgresql.org/users-lounge/docs/faq.html
Re: [HACKERS] Spinlock performance improvement proposal
Vadim Mikheev [EMAIL PROTECTED] writes: I have committed changes to implement this proposal. I'm not seeing any significant performance difference on pgbench on my single-CPU system ... but pgbench is I/O bound anyway on this hardware, so that's not very surprising. I'll be interested to see what other people observe. (Tatsuo, care to rerun that 1000-client test?) What is your system? CPU, memory, IDE/SCSI, OS? Scaling factor and # of clients? HP C180, SCSI-2 disks, HPUX 10.20. I used scale factor 10 and between 1 and 10 clients. Now that I think about it, I was running with the default NBuffers (64), which probably constrained performance too. BTW1 - shouldn't we rewrite pgbench to use threads instead of libpq async queries? At least as option. I'd say that with 1000 clients current pgbench implementation is very poor. Well, it uses select() to wait for activity, so as long as all query responses arrive as single packets I don't see the problem. Certainly rewriting pgbench without making libpq thread-friendly won't help a bit. BTW2 - shouldn't we learn if there are really portability/performance issues in using POSIX mutex-es (and cond. variables) in place of TAS (and SysV semaphores)? Sure, that'd be worth looking into on a long-term basis. regards, tom lane ---(end of broadcast)--- TIP 4: Don't 'kill -9' the postmaster
Re: [HACKERS] O_DIRECT
The O_DIRECT flag has been added in FreeBSD 4.4 (386 Alpha) also. From the release notes: Kernel Changes The O_DIRECT flag has been added to open(2) and fcntl(2). Specifying this flag for open files will attempt to minimize the cache effects of reading and writing. I wonder if using this for WAL would be good. -- Bruce Momjian| http://candle.pha.pa.us [EMAIL PROTECTED] | (610) 853-3000 + If your life is a hard drive, | 830 Blythe Avenue + Christ can be your backup.| Drexel Hill, Pennsylvania 19026 ---(end of broadcast)--- TIP 6: Have you searched our list archives? http://archives.postgresql.org
[HACKERS] Big update for timestamp etc.
I've committed (most of?) the changes to implement a timestamp without time zone type. To help with backward compatibility and upgrading, I've made the *default* timestamp the one with time zone. That is, if you want a timestamp without time zone you will have to fully specify it verbatim. SQL99 calls for without time zone to be the default, and we can make it so in the next release. I have not made any changes to pg_dump or psql or any other utility or client-side driver to recognize the new types or to do conversions to SQL99 names, but these should be done before we exit beta. I can not actually *check* my changes to the cvs tree, since cvsup.postgresql.org apparently does not yet see the changes to cvs.postgresql.org. This is pretty annoying from my pov; hopefully the gain in distributing the load offsets the problems in syncing from a slave machine rather than truth. Details from the cvs log follow. - Thomas Measure the current transaction time to milliseconds. Define a new function, GetCurrentTransactionStartTimeUsec() to get the time to this precision. Allow now() and timestamp 'now' to use this higher precision result so we now have fractional seconds in this constant. Add timestamp without time zone type. Move previous timestamp type to timestamp with time zone. Accept another ISO variant for date/time values: -mm-ddThh:mm:ss (note the T separating the day from hours information). Remove 'current' from date/time types; convert to 'now' in input. Separate time and timetz regression tests. Separate timestamp and timestamptz regression test. ---(end of broadcast)--- TIP 6: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] Spinlock performance improvement proposal
Good summary. I agree checkpoint should look like as normal a Proc as possible. At the just-past OSDN database conference, Bruce and I were annoyed by some benchmark results showing that Postgres performed poorly on an 8-way SMP machine. Based on past discussion, it seems likely that the culprit is the known inefficiency in our spinlock implementation. After chewing on it for awhile, we came up with an idea for a solution. The following proposal should improve performance substantially when there is contention for a lock, but it creates no portability risks because it uses the same system facilities (TAS and SysV semaphores) that we have always relied on. Also, I think it'd be fairly easy to implement --- I could probably get it done in a day. Comments anyone? regards, tom lane Plan: Replace most uses of spinlocks with lightweight locks (LW locks) implemented by a new lock manager. The principal remaining use of true spinlocks (TAS locks) will be to provide mutual exclusion of access to LW lock structures. Therefore, we can assume that spinlocks are never held for more than a few dozen instructions --- and never across a kernel call. It's pretty easy to rejigger the spinlock code to work well when the lock is never held for long. We just need to change the spinlock retry code so that it does a tight spin (continuous retry) for a few dozen cycles --- ideally, the total delay should be some small multiple of the max expected lock hold time. If lock still not acquired, yield the CPU via a select() call (10 msec minimum delay) and repeat. Although this looks inefficient, it doesn't matter on a uniprocessor because we expect that backends will only rarely be interrupted while holding the lock, so in practice a held lock will seldom be encountered. On SMP machines the tight spin will win since the lock will normally become available before we give up and yield the CPU. Desired properties of the LW lock manager include: * very fast fall-through when no contention for lock * waiting proc does not spin * support both exclusive and shared (read-only) lock modes * grant lock to waiters in arrival order (no starvation) * small lock structure to allow many LW locks to exist. Proposed contents of LW lock structure: spinlock mutex (protects LW lock state and PROC queue links) count of exclusive holders (always 0 or 1) count of shared holders (0 .. MaxBackends) queue head pointer (NULL or ptr to PROC object) queue tail pointer (could do without this to save space) If a backend sees it must wait to acquire the lock, it adds its PROC struct to the end of the queue, releases the spinlock mutex, and then sleeps by P'ing its per-backend wait semaphore. A backend releasing the lock will check to see if any waiter should be granted the lock. If so, it will update the lock state, release the spinlock mutex, and finally V the wait semaphores of any backends that it decided should be released (which it removed from the lock's queue while holding the sema). Notice that no kernel calls need be done while holding the spinlock. Since the wait semaphore will remember a V occurring before P, there's no problem if the releaser is fast enough to release the waiter before the waiter reaches its P operation. We will need to add a few fields to PROC structures: * Flag to show whether PROC is waiting for an LW lock, and if so whether it waits for read or write access * Additional PROC queue link field. We can't reuse the existing queue link field because it is possible for a PROC to be waiting for both a heavyweight lock and a lightweight one --- this will occur when HandleDeadLock or LockWaitCancel tries to acquire the LockMgr module's lightweight lock (formerly spinlock). It might seem that we also need to create a second wait semaphore per backend, one to wait on HW locks and one to wait on LW locks. But I believe we can get away with just one, by recognizing that a wait for an LW lock can never be interrupted by a wait for a HW lock, only vice versa. After being awoken (V'd), the LW lock manager must check to see if it was actually granted the lock (easiest way: look at own PROC struct to see if LW lock wait flag has been cleared). If not, the V must have been to grant us a HW lock --- but we still have to sleep to get the LW lock. So remember this happened, then loop back and P again. When we finally get the LW lock, if there was an extra P operation then V the semaphore once before returning. This will allow ProcSleep to exit the wait for the HW lock when we return to it. Fine points: While waiting for an LW lock, we need to show in our PROC struct whether we are waiting for read or write access. But we don't need to remember this after getting the lock; if we know we have the lock, it's easy to see by
Re: [HACKERS] O_DIRECT and performance
Well, O_DIRECT has finally made it into the Linux kernel. It lets you open a file in such a way that reads and writes don't go to the buffer cache but straight to the disk. Accesses must be aligned on filesystem block boundaries. Is there any case where PG would benefit from this? I can see it reducing memory pressure and duplication of data between PG shared buffers and the block buffer cache. OTOH, it does require that writes be batched up for decent performance, since each write has an implicit fsync() involved (just as with O_SYNC). Anyone played with this on systems that already support it (Solaris?) I have heard there are many cases there O_DIRECT on Solaris is slower for databases than normal I/O. I think bulk copy was faster but not normal operation. Probably not something we are going to get into soon. -- Bruce Momjian| http://candle.pha.pa.us [EMAIL PROTECTED] | (610) 853-3000 + If your life is a hard drive, | 830 Blythe Avenue + Christ can be your backup.| Drexel Hill, Pennsylvania 19026 ---(end of broadcast)--- TIP 5: Have you checked our extensive FAQ? http://www.postgresql.org/users-lounge/docs/faq.html
Re: [HACKERS] Spinlock performance improvement proposal
* Doug McNaught [EMAIL PROTECTED] wrote: | | Depends on what you mean. For scaling well with many connections and | simultaneous queries, there's no reason IMHO that the current | process-per-backend model won't do, assuming the locking issues are | addressed. Wouldn't a threading model allow you to share more data across different connections ? I'm thinking in terms of introducing more cache functionality to improve performance. What is shared memory used for today ? -- Gunnar Rønning - [EMAIL PROTECTED] Senior Consultant, Polygnosis AS, http://www.polygnosis.com/ ---(end of broadcast)--- TIP 2: you can get off all lists at once with the unregister command (send unregister YourEmailAddressHere to [EMAIL PROTECTED])
Re: [HACKERS] Spinlock performance improvement proposal
Chamanya wrote: On Thursday 27 September 2001 04:09, you wrote: This depends on your system. Solaris has a huge difference between thread and process context switch times, whereas Linux has very little difference (and in fact a Linux process context switch is about as fast as a Solaris thread switch on the same hardware--Solaris is just a pig when it comes to process context switching). I have never worked on any big systems but from what (little) I have seen, I think there should be a hybrid model. This whole discussion started off, from poor performance on SMP machines. If I am getting this correctly, threads can be spread on multiple CPUs if available but process can not. Different processes will be on handled evenly across all CPUs in an SMP machine, unless you set process affinity for a process and a CPU. So I would suggest to have threaded approach for intensive tasks such as sorting/searching etc. IMHO converting entire paradigm to thread based is a huge task and may not be required in all cases. Dividing a query into multiple threads is an amazing task. I wish I had a couple years and someone willing to pay me to try it. I think of an approach. Threads are created when they are needed but they are kept dormant when not needed. So that there is no recreation overhead(if that's a concern). So at any given point of time, one back end connection has as many threads as number of CPUs. More than that may not yield much of performance improvement. Say a big task like sorting is split and given to different threads so that it can use them all. This is a huge undertaking, and quite frankly, if I understand PostgreSQL, a complete redesign of the entire system. It should be easy to switch the threading function and arguments on the fly, restricting number of threads and there will not be much of thread switching as each thread handles different parts of task and later the results are merged. That is not what I would consider easy. Number of threads should be equal to or twice that of number of CPUs. I don't think more than those many threads would yield any performance improvement. That isn't true at all. One of the problems I see when when people discuss performance on an SMP machine, is that they usually think from the perspective of a single task. If you are doing data mining, one sql query may take a very long time. Which may be a problem, but in the grander scheme of things there are usually multiple concurrent performance issues to be considered. Threading the back end for parallel query processing will probably not help this. More often than not a database has much more to do than one thing at a time. Also, if you are threading query processing, you have to analyze what your query needs to do with the threads. If your query is CPU bound, then you will want to use fewer threads, if your query is I/O bound, you should have as many threads as you have I/O requests, and have each thread block on the I/O. And with this approach we can migrate one functionality at a time to threaded one, thus avoiding big effort at any given time. Perhaps I am being over dramatic, but I have moved a number of systems from fork() to threaded (for ports to Windows NT from UNIX), and if my opinion means anything on this mailing list, I STRONGLY urge against it. PostgreSQL is a huge system, over a decade old. The original developers are no longer working on it, and in fact, probably wouldn't recognize it. There are nooks and crannys that no one knows about. It has also been my experience going from separate processes to separate threads does not do much for performance, simply because the operation of your system does not change, only the methods by which you share memory. If you want to multithread a single query, that's a different story and a good RD project in itself. ---(end of broadcast)--- TIP 5: Have you checked our extensive FAQ? http://www.postgresql.org/users-lounge/docs/faq.html
Re: [HACKERS] path for contrib/intarray (current CVS)
Your patch has been added to the PostgreSQL unapplied patches list at: http://candle.pha.pa.us/cgi-bin/pgpatches I will try to apply it within the next 48 hours. Please apply attached patch to current CVS tree. Changes: 1. gist__int_ops is now without lossy 2. added sort entry in picksplit Regards, Oleg _ Oleg Bartunov, sci.researcher, hostmaster of AstroNet, Sternberg Astronomical Institute, Moscow University (Russia) Internet: [EMAIL PROTECTED], http://www.sai.msu.su/~megera/ phone: +007(095)939-16-83, +007(095)939-23-83 Content-Description: [ Attachment, skipping... ] ---(end of broadcast)--- TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED] -- Bruce Momjian| http://candle.pha.pa.us [EMAIL PROTECTED] | (610) 853-3000 + If your life is a hard drive, | 830 Blythe Avenue + Christ can be your backup.| Drexel Hill, Pennsylvania 19026 ---(end of broadcast)--- TIP 5: Have you checked our extensive FAQ? http://www.postgresql.org/users-lounge/docs/faq.html
Re: [HACKERS] Pre-forking backend
Bruce Momjian [EMAIL PROTECTED] writes: How hard would it be to pre-fork an extra backend How are you going to pass the connection socket to an already-forked child process? AFAIK there's no remotely portable way ... No idea but it seemed like a nice optimization if we could do it. -- Bruce Momjian| http://candle.pha.pa.us [EMAIL PROTECTED] | (610) 853-3000 + If your life is a hard drive, | 830 Blythe Avenue + Christ can be your backup.| Drexel Hill, Pennsylvania 19026 ---(end of broadcast)--- TIP 4: Don't 'kill -9' the postmaster
Re: [HACKERS] Pre-forking backend
Bruce Momjian [EMAIL PROTECTED] writes: How hard would it be to pre-fork an extra backend How are you going to pass the connection socket to an already-forked child process? AFAIK there's no remotely portable way ... regards, tom lane ---(end of broadcast)--- TIP 5: Have you checked our extensive FAQ? http://www.postgresql.org/users-lounge/docs/faq.html
Re: [HACKERS] doc build error
Tatsuo Ishii writes: Has anyone successfully built docs recently? I got: cd sgml /bin/tar -c -f ../admin.tar -T HTML.manifest *.gif *.css /bin/tar: *.gif: Cannot stat: No such file or directory I'll fix it. -- Peter Eisentraut [EMAIL PROTECTED] http://funkturm.homeip.net/~peter ---(end of broadcast)--- TIP 4: Don't 'kill -9' the postmaster
[HACKERS] iscachable settings for datetime functions
I've been looking at the iscachable changes you committed recently, and I think a lot of them need to be adjusted still. One consideration I hadn't thought of recently (though I think we did take it into account for the 7.0 release) is that any function whose output varies depending on the TimeZone variable has to be marked noncachable. This certainly means that some (all?) of the datetime output functions need to be noncachable. I am wondering also if any of the type conversion functions depend on TimeZone --- for example, what are the rules for conversion between timestamptz and timestamp? The functions that convert between type TEXT and the datetime types need to be treated the same as the corresponding I/O conversion functions. For example, text_date is currently marked cachable which is wrong --- as evidenced by the fact that CURRENT_DATE is folded prematurely: regression=# create table foo (f1 date default current_date); CREATE regression=# \d foo Table foo Column | Type | Modifiers +--+ f1 | date | default '2001-09-29'::date The two single-parameter age() functions need to be noncachable since they depend on today's date. I also suspect that their implementation should be revised: writing 'today' with no qualifier exposes you to premature constant folding. Probably select age(current_date::timestamp, $1) (or ::timestamptz respectively) would work better. Why are only some of the date_part functions cachable? Is this a timezone dependency issue, or just an oversight? Surely the abstime comparison functions must be cachable (if they can't be, then indexes on abstime are nonsensical...). Ditto for all other within-type comparison functions. regards, tom lane ---(end of broadcast)--- TIP 4: Don't 'kill -9' the postmaster
Re: [HACKERS] Spinlock performance improvement proposal
Bruce Momjian [EMAIL PROTECTED] writes: No scale factor, as I illustrated from the initialization command I used. Standard buffers too. Let me know what values I should use for testing. Scale factor has to be = max number of clients you use, else you're just measuring serialization on the branch rows. I think the default NBuffers (64) is too low to give meaningful performance numbers, too. I've been thinking that maybe we should raise it to 1000 or so by default. This would trigger startup failures on platforms with small SHMMAX, but we could tell people to use -B until they get around to fixing their kernel settings. It's been a long time since we fit into a 1-MB shared memory segment at the default settings anyway, so maybe it's time to select somewhat-realistic defaults. What we have now is neither very useful nor the lowest common denominator... regards, tom lane ---(end of broadcast)--- TIP 5: Have you checked our extensive FAQ? http://www.postgresql.org/users-lounge/docs/faq.html
Re: [HACKERS] Preparation for Beta
Bruce Momjian [EMAIL PROTECTED] writes: OK, I think we are on track for Monday beta. One thing that I think absolutely *must* happen before we can go beta is to fix the documentation build process at hub.org. Until the online developer docs are up-to-date, how are beta testers going to know what to look at? (For that matter, what are the odds that the doc tarball generation process will succeed?) Personally I'd really like to see CVSWeb working again too... regards, tom lane ---(end of broadcast)--- TIP 2: you can get off all lists at once with the unregister command (send unregister YourEmailAddressHere to [EMAIL PROTECTED])
Re: [HACKERS] Spinlock performance improvement proposal
OK, testing now with 1000 backends and 2000 buffers. Will report. Bruce Momjian [EMAIL PROTECTED] writes: No scale factor, as I illustrated from the initialization command I used. Standard buffers too. Let me know what values I should use for testing. Scale factor has to be = max number of clients you use, else you're just measuring serialization on the branch rows. I think the default NBuffers (64) is too low to give meaningful performance numbers, too. I've been thinking that maybe we should raise it to 1000 or so by default. This would trigger startup failures on platforms with small SHMMAX, but we could tell people to use -B until they get around to fixing their kernel settings. It's been a long time since we fit into a 1-MB shared memory segment at the default settings anyway, so maybe it's time to select somewhat-realistic defaults. What we have now is neither very useful nor the lowest common denominator... regards, tom lane -- Bruce Momjian| http://candle.pha.pa.us [EMAIL PROTECTED] | (610) 853-3000 + If your life is a hard drive, | 830 Blythe Avenue + Christ can be your backup.| Drexel Hill, Pennsylvania 19026 ---(end of broadcast)--- TIP 3: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] Preparation for Beta
On Sat, 29 Sep 2001, Tom Lane wrote: Bruce Momjian [EMAIL PROTECTED] writes: OK, I think we are on track for Monday beta. One thing that I think absolutely *must* happen before we can go beta is to fix the documentation build process at hub.org. Until the online developer docs are up-to-date, how are beta testers going to know what to look at? (For that matter, what are the odds that the doc tarball generation process will succeed?) Personally I'd really like to see CVSWeb working again too... So would I but it can't happen till Marc finishes some configuring I requested on the developers site. Vince. -- == Vince Vielhaber -- KA8CSHemail: [EMAIL PROTECTED]http://www.pop4.net 56K Nationwide Dialup from $16.00/mo at Pop4 Networking Online Campground Directoryhttp://www.camping-usa.com Online Giftshop Superstorehttp://www.cloudninegifts.com == ---(end of broadcast)--- TIP 2: you can get off all lists at once with the unregister command (send unregister YourEmailAddressHere to [EMAIL PROTECTED])
Re: [HACKERS] Pre-forking backend
Bruce Momjian wrote: Tom Lane wrote: Bruce Momjian [EMAIL PROTECTED] writes: How hard would it be to pre-fork an extra backend How are you going to pass the connection socket to an already-forked child process? AFAIK there's no remotely portable way ... No idea but it seemed like a nice optimization if we could do it. What can be done is to have the parent process open and listen() on the socket, then have each child do an accept() on the socket. That way you don't have to pass the socket. The function of the parent process would then be only to decide when to start new children. On some operating systems, only one child at a time can accept() on the socket. On these, you have to lock around the call to accept(). ---(end of broadcast)--- TIP 5: Have you checked our extensive FAQ? http://www.postgresql.org/users-lounge/docs/faq.html
Re: [HACKERS] Pre-forking backend
Bruce Momjian wrote: Tom Lane wrote: Bruce Momjian [EMAIL PROTECTED] writes: How hard would it be to pre-fork an extra backend How are you going to pass the connection socket to an already-forked child process? AFAIK there's no remotely portable way ... No idea but it seemed like a nice optimization if we could do it. What can be done is to have the parent process open and listen() on the socket, then have each child do an accept() on the socket. That way you don't have to pass the socket. The function of the parent process would then be only to decide when to start new children. On some operating systems, only one child at a time can accept() on the socket. On these, you have to lock around the call to accept(). But how do you know the client wants the database you have forked? They could want a different one. -- Bruce Momjian| http://candle.pha.pa.us [EMAIL PROTECTED] | (610) 853-3000 + If your life is a hard drive, | 830 Blythe Avenue + Christ can be your backup.| Drexel Hill, Pennsylvania 19026 ---(end of broadcast)--- TIP 6: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] Pre-forking backend
Tom Lane wrote: This approach would only work as far as saving the fork() call itself, not the backend setup time. Not sure it's worth the trouble. I doubt that the fork itself is a huge component of our start time; it's setting up all the catalog caches and so forth that's expensive. On Unix, yeah, but on Windows, VMS, MPE/iX, possibly others, forking is expensive. Even on Unix, you're not losing anything by this architecture. The simple solution is to have wait on separate sockets and add a redirect capability to the protocol. The program would be: If the clients wants the database I have open, great, we're in business else if the client supports redirect, do redirect else if I can pass file descriptor on this OS, pass file descriptor to the right process else throw away what we've done and open the right database. Simple! It's just a small matter of programming. ---(end of broadcast)--- TIP 2: you can get off all lists at once with the unregister command (send unregister YourEmailAddressHere to [EMAIL PROTECTED])
