Re: Indexed Disk Cache safe copying of files

Fri, 19 Jun 2009 07:10:34 -0700 

Hi Aaron,
locking (at least, locking for writing) should be acceptable in our scenario. If a put finds the cache locked for writing, it can fail without creating much harm to the application. We probably could satisfy our needs this way, just flushing the keys to the disk when we want to perform the backup (and maybe, defragmenting ? is this needed for 
restoring purpose ?)
Do you see any other issue with this approach ?

Best regards,
 Silvano.


Aaron Smuts wrote:
You might be better off using a remote cache, perhaps backed by MySQL. 
As for the snapshot, there are a few ways it can be done.  I'd have to have to 
lock the disk for long.  The best way might be to take a copy of the items.  
Then get each and write it to a new data file, one at a time.  This would be 
more intensive than simply locking the datafile, copying it, and dumping the 
keys.  But the more intensive process requires little locking.

Aaron

--- On Thu, 6/18/09, Silvano <m...@ipsoft.it> wrote:


From: Silvano <m...@ipsoft.it>
Subject: Re: Indexed Disk Cache safe copying of files
To: "JCS Users List" <jcs-users@jakarta.apache.org>
Date: Thursday, June 18, 2009, 12:24 AM
Hi Aron, Thanks a lot for the
interest shown in supporting the issue.

This approach seems to be the most applicable to our case.
If you think you can work on it, please let we know and we
can cooperate and help with testing or supporting
implementation or whatever needed.

- createSnapshot would be first thing we need, at first
glance having some 'location' argument to control directory
where to write the consistent copy

- loadFromSnapshot might be a plus - however if a
consistent snapshot for a region can be moved at ease using
Unix commands, a restore from a snapshot could be done by
moving phisically the files to the desired location.

- timer ... again, this might be handled at the client, but
of course would be a plus to configure as a timer
We might need to have control over backing up different
regions either individually, or several together e.g. in a
short sequence so the backup is of diffferent regions is
roughly at the same time). We say 'roughly'
since the cache does not vary very often and/ or we could
limit writes to the cache at the application level for some
time if needed to help consistency of the snapshot.

Robustness of the snapshot creation would be the most
important requirement.
in general, we should consider that the files can be large
enough (we have so far 1.5 GB of cached data - reason for
caching a lot is that the backend is a CMS - rather slow to
generate web pages).

Silvano.


Aaron Smuts wrote:

The keys are not written until shutdown.  I could

create a method that could write them to disk and make a
copy of the datafile.  It could be called
"createSnapShot"  You could call it
programatically.  There are details to think
through.  Should we try to load from a snapshot on
startup? How many snapshots should we store? Could we create these via a timer? I guess this could 
all be configurable.

Aaron



----- Original Message ----
From: Silvano <m...@ipsoft.it>
To: jcs-users@jakarta.apache.org
Sent: Wednesday, June 17, 2009 4:23:52 AM
Subject: Indexed Disk Cache safe copying of files

Since we have a large cache that needs significant

time to be populated, we need to be able to copy the files
of the regions configured as Indexed Disk Cache between
different installations.

We believe this can be done safely if we turn off the

jvm, but we would need to copy safely the files while the
jvm is running. However it seems that the key files are
written only when you turn off the jvm (at least the size
changes when we turn off the jvm, so we suspect that they
are not synchronized to disk in normal operations).

We can make sure at application level that nobody

writes in the cache during the file copy process, but is
this sufficient?

Can anybody recommend a solution to this issue?

Best regards,
    Silvano.



---------------------------------------------------------------------

To unsubscribe, e-mail: jcs-users-unsubscr...@jakarta.apache.org
For additional commands, e-mail: jcs-users-h...@jakarta.apache.org



---------------------------------------------------------------------

To unsubscribe, e-mail: jcs-users-unsubscr...@jakarta.apache.org
For additional commands, e-mail: jcs-users-h...@jakarta.apache.org




---------------------------------------------------------------------
To unsubscribe, e-mail: jcs-users-unsubscr...@jakarta.apache.org
For additional commands, e-mail: jcs-users-h...@jakarta.apache.org




---------------------------------------------------------------------
To unsubscribe, e-mail: jcs-users-unsubscr...@jakarta.apache.org
For additional commands, e-mail: jcs-users-h...@jakarta.apache.org






---------------------------------------------------------------------
To unsubscribe, e-mail: jcs-users-unsubscr...@jakarta.apache.org
For additional commands, e-mail: jcs-users-h...@jakarta.apache.org

Reply via email to