On 2005.06.27, Bas Scheffers <[EMAIL PROTECTED]> wrote: > Dossy Shiobara said: > > This really ought to be a server-wide cache sitting in > > servPtr->adp.cache rather than itPtr->adp.cache ... > I would think so! :) The problem with that still is that it is not > persistent, which could mean that if you restart your server under high > load, all the cache generation that takes place could overwhelm it, while > under normal circumstances it's perfectly happy. (plus it can become a > memory hog!)
Yup, there is the stampeding herd problem at server startup. Persistent cache would be very cool. > Although that problem shouldn't be part of Jim's ADP caching, Agreed. > but the underlying ns_cache could be modified to support caching to > the file system, which you can specify wether you want it or not when > putting something in the cache. I disagree: I think whether to persist the cache to disk should be specified when the cache is created, not when an cache entry is inserted. This makes it possible to retrofit existing cache-using code with the new disk-backed cache. Basically, extend Ns_Cache so that when a cache is created, you can specify that the cache is disk-backed and specify a pathname to a directory that is the top of the cache -- perhaps the default directory is servers/$servername/caches/$cachename. We store the cached ADP output in a file named $cachedir/$adppath/$xx/[md5 $adpargs].dat, where $xx might be the first 2 characters of the [md5 $adpargs]. Things like Ns_CacheFlush() will need to be modified to go and unlink the file in the filesystem. Perhaps an Ns_CacheFlushEx() needs to be introduced to flush the in-memory cache without deleting the disk file, so that Ns_CacheDestroy() can clean up the cache metadata in memory without deleting the cache on disk. This allows us to persist the cache across server restarts, etc. Of course, things like Ns_CacheFindEntry() will need to be extended to look on disk for the cache entry if it's not found in the in-memory Tcl_Hashtable. But, mostly, adding a disk-backed cache should be do-able without breaking backwards compatibility in the API. > (with a, ehrm, cache of the cache in memory so popular content is very > fast, but with a limit on memory usage, so the server doesn't keep > growing) Could use a simple LRU alg. to keep some cached things in memory vs. on disk. Or, could just open the cached data files on disk and mmap() to them, but then you could run yourself out of open filehandles ... > > [ns_cache_flush] should work. > That can't do a cluster wide clear, can it? Another "someday" feature to > be added to the list! :) Executing commands across the cluster is a separate "problem" which can be solved pretty simply: today, you can use an Expect script to connect to each machine and connect to the nscp to issue a command, as the simplest implementation. -- Dossy -- Dossy Shiobara mail: [EMAIL PROTECTED] Panoptic Computer Network web: http://www.panoptic.com/ "He realized the fastest way to change is to laugh at your own folly -- then you can let go and quickly move on." (p. 70) -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> with the body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: field of your email blank.
