Hiya,
That's a very pertinent set of questions - I'll answer as best I can
below, the only caveat is that I won't proclaim myself a JCS expert,
given that I was really only introduced to it a few days ago ;)
Ok, fair enough! :)
...
Ok, I apologize that I just can't look into this myself right now, but let me try to state what I perceive to be the things we really need so we can narrow down whether this is the right direction to be going:
1) - we need a high-performance in-memory cache - we need that cache to be backed by some sort of persistence for two
reasons first, so that cached items aren't lost between server starts second so that items bumped off the memory cache can still be preserved as the assumption is that pulling a cached item off disk is still faster than regenerating the entire pipeline. for example, the most popular 100 items are served in memory, but the rest are on disk. (and at shutdown, the 100 in-memory ones go to disk).
** is JCS (properly configured) a good way to meet both of these requirements?
CM: Yes. A JCS store can be configured to use an MRU store (the test
config provided does just this.) This MRU store is in turn configurable
- number of objects catered for, what type of store (memory etc.) The
swapping to disk based store is handled in the same way as Cocoon
currently does - objects are dropped out of MRU store into diskbased
store. The only difference is that the "disk based" store can in fact be
any of the several stores supported by JCS (remote server, disk, JISP
etc.)
Ok.
At shutdown, things can be configured to store to disk (both store, and
index.) The only issue is that in the event of an abnormal shutdown the
index can be lost (this is only when the index is kept in memory of
course - other store types which use pure disk based index storage
wouldn't be effected by this.)
--
Ok, on second thought if this is limited to abnormal shutdown this should be OK at least for our use with cached pipelines. I don't really know firsthand the other uses of the Cache. If they are OK with no guarantee of persistence, then I think we're set.
2) currently we accomplish both of those transparently with a transient-store configured to use a persistent-store. we have some subsystems which use a transient-store configured _not_ to use persistence because of application requirements.
** is JCS (properly configured) a good way to meet this requirement?
CM: Yes. You could very easily configure a separate role for transient
store which would use a different group / region (JCS terms.) This
region would be configured to use _only_ memory store, with no support
for persistence. This could be accomplished as is, by changing roles
appropriately, and providing different config in the xconf for the
transient store.
--
Ok, sounds good too.
3) we have one subsystem which goes direct to the persistent-store right at shutdown which could just as easily go to the in-memory front-end as long as the persistence between restarts still happened.
** is JCS (properly configured) a good way to meet this requirement?
CM: Yes. Again, this could either be written to the default store,
which would go to MRU, and then disk at shutdown, _or_ a different group
/ region could be created and configured which would go straight to
disk, without an in-memory MRU involved.
--
I think this particular case works better going to the default store. It's a very minor use (one object, and only at shutdown) and so not worth a different cache entirely.
I am not following the details well here about the persistence issue. If we don't use the Disk Cache, what other persistent (on disk, survives
restarts) options are there?
CM: As I mentioned before, the only issue which needs more investigation
is abnormal shutdown with an in-memory index, however, this could
theoretically be up to the user to decide - if their application
requires an absolute guarantee of persistence at all times, they could
configure it to use an on-disk index at all times - this would require
no class change, just a change in the CCF.
Ok, I think this bears bringing this issue up in a new thread (so people notice it). The point I take out of this is that JCS makes it explicit that successful persistence is not currently guaranteed (at least in the default better-performing configuration). For all we know, this may have been the case with JISP too but just not explicit. I don't know if people have used our Cache as a persistence layer for application data. I would not have recommended it before, and definitely would not now if we go to JCS. I'm comfortable with this.
Geoff
