The original problem was that the removal code was commented out in
the
src. Then I needed to change the code to use keys for removal (I believe
this
is already fixed in your latest version)
I feel like I missed the first part of this conversation.
So you have the updated code?
Well I uncommented out the code that was clearly mean't to be in there. The part where the remote server recieves a "put" and then iterates over all the clients and sends them remove commands. I can do a diff check if there is any other patches since I recieved the code.
> Can you paste your configuration files into an email.
Apart from uncommenting the callback ports they haven't changed much, on monday I will repost the remote and normal ccf files.
The data we are caching comes to about 10Meg maybe 20 at most and the remotecache was running with 350Meg memory!
What kind of stuff are you caching?
Binary data byte[]. Mainly this binary data is actually HTML pages probably varying in size between 4k and 10k but some of the binary data is image data (gifs). both are put in simply as byte[]'s
I noticed in your example you had given the remotecache 1.8gigs ram.
Is
this because you were storing more data, because JCS is very
inefficient
or due to a leak?
I gave the remote cache that much memory because I had it and it was connected to many machines.
Clearly, the situation you describe shouldn't be happening. Is the remote cache using the disk cache? Is the disk file growing?
It is using the disk cache, however we only cache around 3000-4000 items. I increased the memory cache size to 4000 as I thought the whole cache should fit easily into memory. Since I did this the disk cache remains empty. FYI when the memory cache was size 10 the size of the disk cache was about 10meg.
Now I have 12 machines connected to the remote cache does it require
12x
the memory?
If all 12 machines have unique data and you have the memory set to some huge number then it should be even bigger, since the cache itself requires some memory to run.
All the machines have the same items which only come to around 10meg in size. 2 machines are generating updated content and frequently putting new data into JCS to replace old (with the same keys of course). So around 60% of the data in the cache is overwritten with new data every 15minutes or so. This is because we process realtime-ish data and need to update the html in the cache to reflect this data.
Even if items were never expiring based on time then it shouldn't be a big problem as new items are rare.
Send the .ccf files.
Will do.
Thanks, Matt
Aaron
Wearily, Matt PS Up at 8.30am on a sunday restarting servers.
Travis Savo wrote:
Your cache.ccf looks fine.
If I understand you correctly, the problem is the original put works
fine,
but subsequent puts for that same region/key don't update the object remotely, correct?
I'll have to test this scenario locally and get back to you.
Writing unit tests for remote cache has been rapidly approaching the
top
of
my todo stack, and this may bump it up another notch, depending on
what
the
rest of my work load is like.
Things to try right off the bat: Turn on logging. See the puts
getting
done
to remote cache, and the removes getting broadcast from remote cache
and
received on the other clients? You should....
-Travis Savo
-----Original Message----- From: Matthew Cooke [mailto:[EMAIL PROTECTED] Sent: Monday, April 19, 2004 1:34 PM To: Turbine JCS Users List Subject: JCS remote cache problems
Travis (or anyone else!),
I'm having some problems with the remote cache implementation, it
seems
that items aren't getting updated on the servers after a put is performed on one machine. I'm not sure what the problem is, I've got a feeling only the first
put
is working and I didn't notice because the cached content doesn't
change
that fast in our system.
This could be a bug in my usage of JCS or in the config file it may
well
not be a bug in the improved JCS. What might cause items to not be deleted in the connected machines caches when a new put occurs?
Any ideas or suggestions much appreciated, I will be trying to
reproduce
this problem back in testing again tomorrow!
Kind Regards, Matt.
PS Below I attach first the ccf file used by most machines and below that the remote.ccf file used on the RemoteServer
# Normal config for MFN Production servers
############################################################## ################## DEFAULT CACHE REGION ##################### # sets the default aux value for any non configured caches jcs.default=DC,RC
jcs.default.cacheattributes=org.apache.jcs.engine.CompositeCacheAttribu
te
s
jcs.default.cacheattributes.MaxObjects=1000
jcs.default.cacheattributes.MemoryCacheName=org.apache.jcs.engine.memor
y.
lru
.LRUMemoryCache jcs.default.cacheattributes.UseMemoryShrinker=true jcs.default.cacheattributes.MaxMemoryIdleTimeSeconds=3600 jcs.default.cacheattributes.ShrinkerIntervalSeconds=60 jcs.default.cacheattributes.ShrinkerIntervalSeconds=60 jcs.default.elementattributes=org.apache.jcs.engine.ElementAttributes jcs.default.elementattributes.IsEternal=false jcs.default.elementattributes.MaxLifeSeconds=7 jcs.default.elementattributes.IdleTime=1800 jcs.default.elementattributes.IsSpool=true jcs.default.elementattributes.IsRemote=true jcs.default.elementattributes.IsLateral=true
# SYSTEM CACHE # should be defined for the storage of group attribute list jcs.system.groupIdCache=DC,RC
jcs.system.groupIdCache.cacheattributes=org.apache.jcs.engine.Composite
Ca
che
Attributes jcs.system.groupIdCache.cacheattributes.MaxObjects=10000
jcs.system.groupIdCache.cacheattributes.MemoryCacheName=org.apache.jcs.
en
gin
e.memory.lru.LRUMemoryCache
jcs.system.groupIdCache.elementattributes=org.apache.jcs.engine.Element
At
tri
butes jcs.system.groupIdCache.elementattributes.IsEternal=true jcs.system.groupIdCache.elementattributes.MaxLifeSeconds=3600 jcs.system.groupIdCache.elementattributes.IdleTime=1800 jcs.system.groupIdCache.elementattributes.IsSpool=true jcs.system.groupIdCache.elementattributes.IsRemote=true jcs.system.groupIdCache.elementattributes.IsLateral=true
############################################################## ################## CACHE REGIONS AVAILABLE ################### # Regions preconfirgured for caching
jcs.region.primaryCache=DC,RC
jcs.region.primaryCache.cacheattributes=org.apache.jcs.engine.Composite
Ca
che
Attributes jcs.region.primaryCache.cacheattributes.MaxObjects=4000
jcs.region.primaryCache.cacheattributes.MemoryCacheName=org.apache.jcs.
en
gin
e.memory.lru.LRUMemoryCache jcs.region.primaryCache.cacheattributes.UseMemoryShrinker=true jcs.region.primaryCache.cacheattributes.MaxMemoryIdleTimeSeconds=10 jcs.region.primaryCache.cacheattributes.ShrinkerIntervalSeconds=60
jcs.region.primaryCache.elementattributes=org.apache.jcs.engine.Element
At
tri
butes jcs.region.primaryCache.elementattributes.IsEternal=false jcs.region.primaryCache.elementattributes.MaxLifeSeconds=259200 jcs.region.primaryCache.elementattributes.IsSpool=true jcs.region.primaryCache.elementattributes.IsRemote=true jcs.region.primaryCache.elementattributes.IsLateral=true
############################################################## ################## AUXILIARY CACHES AVAILABLE ################
# Disk Cache used as an overflow cache when the main memory cache is
full.
jcs.auxiliary.DC=org.apache.jcs.auxiliary.disk.indexed.IndexedDiskCache
Fa
cto
ry
jcs.auxiliary.DC.attributes=org.apache.jcs.auxiliary.disk.indexed.Index
ed
Dis
kCacheAttributes jcs.auxiliary.DC.attributes.DiskPath=/usr/local/web/jcscache
# This is an RMI based remote cache. jcs.auxiliary.RC=org.apache.jcs.auxiliary.remote.RemoteCacheFactory
jcs.auxiliary.RC.attributes=org.apache.jcs.auxiliary.remote.RemoteCache
At
tri
butes jcs.auxiliary.RC.attributes.RemoteHost=builder jcs.auxiliary.RC.attributes.RemotePort=1099 jcs.auxiliary.RC.attributes.LocalPort=1103 jcs.auxiliary.RC.attributes.RemoveUponRemotePut=false #jcs.auxiliary.RC.attributes.RemoteServiceName=RemoteCache
-----------
# Config for MFN Central JCS server.
############################################################## ################## DEFAULT CACHE REGION ##################### # Registry used to register and provide the # IRemoteCacheService service. registry.host=localhost registry.port=1099
############################################################## ################## DEFAULT CACHE REGION ##################### # sets the default aux value for any non configured caches jcs.default=DC
jcs.default.cacheattributes=org.apache.jcs.engine.CompositeCacheAttribu
te
s
jcs.default.cacheattributes.MaxObjects=1000
jcs.default.cacheattributes.MemoryCacheName=org.apache.jcs.engine.memor
y.
lru
.LRUMemoryCache jcs.default.cacheattributes.UseMemoryShrinker=true jcs.default.cacheattributes.MaxMemoryIdleTimeSeconds=3600 jcs.default.cacheattributes.ShrinkerIntervalSeconds=60 jcs.default.cacheattributes.ShrinkerIntervalSeconds=60 jcs.default.elementattributes=org.apache.jcs.engine.ElementAttributes jcs.default.elementattributes.IsEternal=false jcs.default.elementattributes.MaxLifeSeconds=7 jcs.default.elementattributes.IdleTime=1800 jcs.default.elementattributes.IsSpool=true jcs.default.elementattributes.IsRemote=true jcs.default.elementattributes.IsLateral=true
# SYSTEM CACHE # should be defined for the storage of group attribute list jcs.system.groupIdCache=DC
jcs.system.groupIdCache.cacheattributes=org.apache.jcs.engine.Composite
Ca
che
Attributes jcs.system.groupIdCache.cacheattributes.MaxObjects=10000
jcs.system.groupIdCache.cacheattributes.MemoryCacheName=org.apache.jcs.
en
gin
e.memory.lru.LRUMemoryCache
jcs.system.groupIdCache.elementattributes=org.apache.jcs.engine.Element
At
tri
butes jcs.system.groupIdCache.elementattributes.IsEternal=true jcs.system.groupIdCache.elementattributes.MaxLifeSeconds=3600 jcs.system.groupIdCache.elementattributes.IdleTime=1800 jcs.system.groupIdCache.elementattributes.IsSpool=true jcs.system.groupIdCache.elementattributes.IsRemote=true jcs.system.groupIdCache.elementattributes.IsLateral=true
############################################################## ################## CACHE REGIONS AVAILABLE ################### # Regions preconfirgured for caching
jcs.region.primaryCache=DC
jcs.region.primaryCache.cacheattributes=org.apache.jcs.engine.Composite
Ca
che
Attributes jcs.region.primaryCache.cacheattributes.MaxObjects=10
jcs.region.primaryCache.cacheattributes.MemoryCacheName=org.apache.jcs.
en
gin
e.memory.lru.LRUMemoryCache jcs.region.primaryCache.cacheattributes.UseMemoryShrinker=true jcs.region.primaryCache.cacheattributes.MaxMemoryIdleTimeSeconds=10 jcs.region.primaryCache.cacheattributes.ShrinkerIntervalSeconds=60
jcs.region.primaryCache.elementattributes=org.apache.jcs.engine.Element
At
tri
butes jcs.region.primaryCache.elementattributes.IsEternal=false jcs.region.primaryCache.elementattributes.MaxLifeSeconds=60000 jcs.region.primaryCache.elementattributes.IsSpool=true jcs.region.primaryCache.elementattributes.IsRemote=true jcs.region.primaryCache.elementattributes.IsLateral=true
# Primary Disk Cache-- faster than the rest because of memory key
storage
jcs.auxiliary.DC=org.apache.jcs.auxiliary.disk.indexed.IndexedDiskCache
Fa
cto
ry
jcs.auxiliary.DC.attributes=org.apache.jcs.auxiliary.disk.indexed.Index
ed
Dis
kCacheAttributes jcs.auxiliary.DC.attributes.DiskPath=/usr/local/web/MasterJcsCache
--------------------------------------------------------------------- To unsubscribe, e-mail:
[EMAIL PROTECTED]
For additional commands, e-mail:
[EMAIL PROTECTED]
--------------------------------------------------------------------- To unsubscribe, e-mail:
[EMAIL PROTECTED]
For additional commands, e-mail:
[EMAIL PROTECTED]
--------------------------------------------------------------------- To unsubscribe, e-mail:
[EMAIL PROTECTED]
For additional commands, e-mail:
[EMAIL PROTECTED]
--------------------------------------------------------------------- To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
--------------------------------------------------------------------- To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
