Re: [HACKERS] updated qCache
On Wed, Apr 17, 2002 at 05:17:51PM -0400, Neil Conway wrote: Hi all, Here's an updated version of the experimental qCache patch I posted a couple days ago (which is a port of Karel Zak's 7.0 work to CVS HEAD). Changes: - fix segfault in EXECUTE under some circumstances (reported by Barry Lind) - fix some memory leaks (thanks to Karel Zak) - write more regression tests (make check still won't pass) - re-diff against CVS HEAD - more code cleanup, minor tweaks However, I've tentatively decided that I think the best way to go forward is to rewrite this code. IMHO the utility of plans cached in shared memory is fairly limited, but the code that implements this adds a lot of complex to the patch. I'm planning to re-implement PREPARE/EXECUTE with support only for locally-prepared plans, using the existing patch as a guide. The result should be a simpler patch -- once it's in CVS we can worry about more advanced plan caching techiques. Any complaints/comments on this plan? I agree too :-) I think remove the shared memory code from this patch is easy and local memory storage is there already done. Karel -- Karel Zak [EMAIL PROTECTED] http://home.zf.jcu.cz/~zakkr/ C, PostgreSQL, PHP, WWW, http://docs.linux.cz, http://mape.jcu.cz ---(end of broadcast)--- TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]
Re: [HACKERS] updated qCache
On Wed, Apr 17, 2002 at 06:05:59PM -0400, Neil Conway wrote: On Wed, 17 Apr 2002 14:34:45 -0700 I'm not saying it's a bad idea, I just think I'd like to concentrate on the locally-cached plans for now and see if there is a need to add shared plans later. Yes, later we can use shared memory buffer as pipe between backends: Backend A:Backend B: local-memory-query-plan -- shmem -- local-memory-query-plan In this idea is in the shared memory one query-plan only and backends use it for plan copying from A to B. It require persistent backends of course. Karel PS. it's idea only and nothing other, the original qcache was idea only too :-) -- Karel Zak [EMAIL PROTECTED] http://home.zf.jcu.cz/~zakkr/ C, PostgreSQL, PHP, WWW, http://docs.linux.cz, http://mape.jcu.cz ---(end of broadcast)--- TIP 3: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
[HACKERS] updated qCache
Hi all, Here's an updated version of the experimental qCache patch I posted a couple days ago (which is a port of Karel Zak's 7.0 work to CVS HEAD). Changes: - fix segfault in EXECUTE under some circumstances (reported by Barry Lind) - fix some memory leaks (thanks to Karel Zak) - write more regression tests (make check still won't pass) - re-diff against CVS HEAD - more code cleanup, minor tweaks However, I've tentatively decided that I think the best way to go forward is to rewrite this code. IMHO the utility of plans cached in shared memory is fairly limited, but the code that implements this adds a lot of complex to the patch. I'm planning to re-implement PREPARE/EXECUTE with support only for locally-prepared plans, using the existing patch as a guide. The result should be a simpler patch -- once it's in CVS we can worry about more advanced plan caching techiques. Any complaints/comments on this plan? Cheers, Neil -- Neil Conway [EMAIL PROTECTED] PGP Key ID: DB3C29FC qcache-2.patch.gz Description: Binary data ---(end of broadcast)--- TIP 5: Have you checked our extensive FAQ? http://www.postgresql.org/users-lounge/docs/faq.html
Re: [HACKERS] updated qCache
-Original Message- From: Neil Conway [mailto:[EMAIL PROTECTED]] Sent: Wednesday, April 17, 2002 2:18 PM To: PostgreSQL Hackers Subject: [HACKERS] updated qCache Hi all, Here's an updated version of the experimental qCache patch I posted a couple days ago (which is a port of Karel Zak's 7.0 work to CVS HEAD). Changes: - fix segfault in EXECUTE under some circumstances (reported by Barry Lind) - fix some memory leaks (thanks to Karel Zak) - write more regression tests (make check still won't pass) - re-diff against CVS HEAD - more code cleanup, minor tweaks However, I've tentatively decided that I think the best way to go forward is to rewrite this code. IMHO the utility of plans cached in shared memory is fairly limited, but the code that implements this adds a lot of complex to the patch. DC% Why do you imagine that the utility is limited? I'm planning to re-implement PREPARE/EXECUTE with support only for locally-prepared plans, using the existing patch as a guide. The result should be a simpler patch -- once it's in CVS we can worry about more advanced plan caching techiques. Any complaints/comments on this plan? DC% Why not allow both kinds and make it configurable... DC% local/shared/both. ---(end of broadcast)--- TIP 6: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] updated qCache
On Wed, 17 Apr 2002 14:34:45 -0700 Dann Corbit [EMAIL PROTECTED] wrote: However, I've tentatively decided that I think the best way to go forward is to rewrite this code. IMHO the utility of plans cached in shared memory is fairly limited, but the code that implements this adds a lot of complex to the patch. DC% Why do you imagine that the utility is limited? (1) It's difficult to tell whether a given plan has already been prepared: the server could have been restarted in the mean-time, for example. We could allow app developers to check if a given has already been prepared, but that's inconvenient, and the benefits seem fairly small. (2) Shared memory is a bad storage location for variable-sized data, like query plans. What happens if you're asked to cache a plan larger than the free space in the shared cache? You could perhaps free up some space by removing another entry, but that means that every invocation of EXECUTE needs to be prepared for the target plan to have been evicted from the cache -- which is irritating. (3) Managing concurrent access to the shared cache may (or may not) be a performance issue. I'm not saying it's a bad idea, I just think I'd like to concentrate on the locally-cached plans for now and see if there is a need to add shared plans later. I'm planning to re-implement PREPARE/EXECUTE with support only for locally-prepared plans, using the existing patch as a guide. The result should be a simpler patch -- once it's in CVS we can worry about more advanced plan caching techiques. Any complaints/comments on this plan? DC% Why not allow both kinds and make it configurable... DC% local/shared/both. That's what the current patch does. Cheers, Neil -- Neil Conway [EMAIL PROTECTED] PGP Key ID: DB3C29FC ---(end of broadcast)--- TIP 6: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] updated qCache
Neil Conway [EMAIL PROTECTED] writes: I'm planning to re-implement PREPARE/EXECUTE with support only for locally-prepared plans, using the existing patch as a guide. The result should be a simpler patch -- once it's in CVS we can worry about more advanced plan caching techiques. Any complaints/comments on this plan? That's what I wanted from day one ;-) regards, tom lane ---(end of broadcast)--- TIP 4: Don't 'kill -9' the postmaster
Re: [HACKERS] updated qCache
Neil Conway [EMAIL PROTECTED] writes: I'm planning to re-implement PREPARE/EXECUTE with support only for locally-prepared plans, using the existing patch as a guide. The result should be a simpler patch -- once it's in CVS we can worry about more advanced plan caching techiques. Any complaints/comments on this plan? That's what I wanted from day one ;-) So with this scheme, people just have to be careful to use a connection pool / persistent connections? Chris ---(end of broadcast)--- TIP 2: you can get off all lists at once with the unregister command (send unregister YourEmailAddressHere to [EMAIL PROTECTED])
Re: [HACKERS] updated qCache
Neil Conway [EMAIL PROTECTED] writes: I'm planning to re-implement PREPARE/EXECUTE with support only for locally-prepared plans, using the existing patch as a guide. The result should be a simpler patch -- once it's in CVS we can worry about more advanced plan caching techiques. Any complaints/comments on this plan? That's what I wanted from day one ;-) You know, if we had a threaded backend, we wouldn't have any of these problems :) Chris ---(end of broadcast)--- TIP 3: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
