Re: [HACKERS] PostgreSQL pre-fork speedup

2004-05-12 Thread sdv mailer
I ran the new Pgpool-1.2.2 and it was a bit faster on the TCP but still slower than on UNIX socket. I used the same script as before. TCP Socket (Pgpool 1.2.0) -- 2.39 sec TCP Socket (Pgpool 1.2.2) -- 0.80 sec 0.80 sec 0.79 sec UNIX Socket (Pgpool 1.2.2) --- 0.026 sec

Re: [HACKERS] PostgreSQL pre-fork speedup

2004-05-06 Thread sdv mailer
] wrote: sdv mailer said: Forking is expensive on many systems. Linux is a bit better but still expensive compared to threads. On Windows, creating process is much more expensive than on Linux. Check this benchmark: http://cs.nmu.edu/~randy/Research/Papers/Scheduler

Re: [HACKERS] PostgreSQL pre-fork speedup

2004-05-06 Thread sdv mailer
Tatsuo, I did some benchmark on my Linux box (AMD 1.2Ghz, 256MB, Fedora Core 1 Linux 2.4.20-8) using Pgpool 1.2 and PostgreSQL 7.4. I ran the benchmark script repeatedly (10+ times each). I get 5x faster using Pgpool on UNIX socket, which is encouraging. This shows pre-fork does speed things up.

Re: [HACKERS] PostgreSQL pre-fork speedup

2004-05-06 Thread sdv mailer
Ok, I did some benchmark on my Linux box (AMD 1.2Ghz, 256MB, Fedora Core 1 Linux 2.4.20-8) using Pgpool 1.2 and PostgreSQL 7.4. I ran the benchmark script repeatedly (10+ times each). I get 5x faster using Pgpool on UNIX socket, which is encouraging. This shows pre-fork does speed things up even

Re: [HACKERS] PostgreSQL pre-fork speedup

2004-05-06 Thread sdv mailer
Pgpool connects to PostgreSQL on UNIX socket. I also ran on TCP socket but there is no significant difference if I recall correctly due to the inherent nature of connection pooling or pre-fork technology. ;-) --- Rod Taylor [EMAIL PROTECTED] wrote: However, when I tried TCP socket, Pgpool was

Re: [HACKERS] PostgreSQL pre-fork speedup

2004-05-06 Thread sdv mailer
Hi Bruce, Sorry for the confusion because Rod asked a question and I answered too quickly. This is what I mean. 15x Slower: --- Client --TCP-- PgPool --UNIX-- PostgreSQL Client --TCP-- PgPool --TCP-- PostgreSQL 5x Faster: -- Client --UNIX-- PgPool --UNIX-- PostgreSQL Client

Re: [HACKERS] PostgreSQL pre-fork speedup

2004-05-06 Thread sdv mailer
No SSL. No authentication either. Just friendly handshakes. __ Do you Yahoo!? Win a $20,000 Career Makeover at Yahoo! HotJobs http://hotjobs.sweepstakes.yahoo.com/careermakeover ---(end of

Re: [HACKERS] PostgreSQL pre-fork speedup

2004-05-06 Thread sdv mailer
I compared against both TCP and UNIX direct connections. No SSL, no authentication. See benchmark results posted below again: Direct -- 0.144 sec. Client --UNIX-- PG 0.152 sec. Client --TCP-- PG 5x Faster - 0.027 sec. Client --UNIX-- Pgpool --UNIX-- PG 0.028 sec. Client --UNIX--

Re: [HACKERS] PostgreSQL pre-fork speedup

2004-05-06 Thread sdv mailer
Nope. I commented out that block of code at 372 and no difference. __ Do you Yahoo!? Win a $20,000 Career Makeover at Yahoo! HotJobs http://hotjobs.sweepstakes.yahoo.com/careermakeover ---(end of

Re: [HACKERS] PostgreSQL pre-fork speedup

2004-05-06 Thread sdv mailer
Tom, You're correct about the test measuring a hot backend and not forking. How much exactly is the bulk of the startup done by cache initialization relative to the forking? What would be the impact on Win32 knowing that process creation is twice as slow than on Linux?

Re: [HACKERS] PostgreSQL pre-fork speedup

2004-05-05 Thread sdv mailer
it acts as a proxy there is 7% to 15% overhead according to his README file. http://www.mail-archive.com/[EMAIL PROTECTED]/msg44082.html --- Andrew Dunstan [EMAIL PROTECTED] wrote: sdv mailer wrote: [snip] Pre-fork will give MySQL one less argument to throw at PostgreSQL. I think

[HACKERS] PostgreSQL pre-fork speedup

2004-05-05 Thread sdv mailer
Hi, I know the issue of pre-fork PostgreSQL has been discussed previously. Someone mentionned pre-fork can be implemented when schemas become available in PostgreSQL because there will be less of the need to run multiple databases. I think Oracle 7 uses pre-forking and it helps speed up the

Re: [HACKERS] PostgreSQL pre-fork speedup

2004-05-05 Thread sdv mailer
I'm talking about connecting to multiple database servers on separate machines. Schemas don't apply here. How much work would it take to make a pre-fork smart enough to open different databases on incoming connection? How much of it can be modeled after Apache?

Re: [HACKERS] PostgreSQL pre-fork speedup

2004-05-05 Thread sdv mailer
Pre-fork does not equal to idle connections! Pre-fork scales with database load where as persistent connections scales with webserver load. A web server that is heavily loaded but not necessarily performing a lot of database activity will spawn hundreds of idle database connections using

Re: [HACKERS] PostgreSQL pre-fork speedup

2004-05-05 Thread sdv mailer
I've already tried pooling (SQLRelay) and persistent connection (PHP). They may work for other people but they do not work for me. I have already separated static from database driven codes but you can never balance web server load with database server load. Pre-fork scales with database load and

Re: [HACKERS] PostgreSQL pre-fork speedup

2004-05-05 Thread sdv mailer
as free thing. There are pros and cons. The general trend is going towards threads, but that's a different issue. --- scott.marlowe [EMAIL PROTECTED] wrote: On Wed, 5 May 2004, sdv mailer wrote: Forking is quite fast on Linux but creating a new process is still 10x more expensive than

Re: [HACKERS] PostgreSQL pre-fork speedup

2004-05-05 Thread sdv mailer
[EMAIL PROTECTED] wrote: On Wed, 5 May 2004, sdv mailer wrote: Forking is quite fast on Linux but creating a new process is still 10x more expensive than creating a thread and is even worse on Win32 platform. CPU load goes up because the OS needs to allocate/deallocate memory making

Re: [HACKERS] PostgreSQL pre-fork speedup

2004-05-04 Thread sdv mailer
We used to run persistent connection until the DB servers got maxed out because of too many idle connections sucking up all the memory. Web servers run different loads than database servers and persistent connections are notorious for crashing your DB. Connection pooling (eg. SQLRelay) didn't

Re: [HACKERS] PostgreSQL pre-fork speedup

2004-05-03 Thread sdv mailer
Forking consumes a large amount of CPU when you have many simultaneous connections and adds up to the latency. Particularly MySQL users may think PostgreSQL's connection time is much slower because these users tend to perform relatively simple queries. In my case, connection pooling and

[HACKERS] PostgreSQL pre-fork speedup

2004-05-02 Thread sdv mailer
I had lots of trouble posting so you may receive this more than once. My apologies.. -- Hi, I know the issue of pre-fork PostgreSQL has been discussed previously. Someone mentionned pre-fork can be implemented when schemas become available in PostgreSQL because there