Re: [appengine-java] Re: Howto return all values stored in memcache?
Yes. You need to do some research, there might be another solutions.
I don't think that returning all cached keys is that simple.
putIfUntouched is used to update your value atomically.
Why don't you update each single entity in a task queue after putting it
into the cache ?
Anyway, can't you update your entity in background with task queues right
after putting it into cache ? It would consume more resources 'cause your
not doing batch updates though.
On Mon, Oct 3, 2011 at 6:29 PM, mscwd01 mscw...@gmail.com wrote:
That's a shame, it's a real disadvantage not having that simple
functionality.
Thanks for your example, although I found it a little difficult to
understand it completely. Are you suggesting having an entry in the
cache with a key string allKeys which maps to a List of keys
belonging to all the other entities in the cache? Would this allow me
to iterate through the keys, saving the entity to the datastore and
then removing the entry from the cache?
Also what is the putIfUntouched statement for?
Thanks again
On Oct 3, 9:51 pm, Bruno Fuster brunofus...@gmail.com wrote:
I don't think so.
You could create a key with all entities ids that need to be updated.
Something like:
public void saveEntity() {
cache.put(key, entity);
queue.add(withUrl(/entitiesToUpdate?id=+entity.getId()));
}
and then
public void entitiesToUpdate(Long id) {
IdentifiableValue i = cache.getIdentifiable(allKeys);
ListLong keys = i.getValue();
keys.add(id);
boolean put = cache.putIfUntouched(allKeys, i, keys);
if (!put) {
//response 409 for conflict due to concurrency so appengine will
retry this later
}
}
On Mon, Oct 3, 2011 at 5:38 PM, mscwd01 mscw...@gmail.com wrote:
Hey
How would one return all the values stored within the memcache?
I have tried using entrySet() and values() methods on the cache but
both result in an java.lang.UnsupportedOperationException.
Ideally, I need to return all the entities stored within the cache and
bulk save them to the datastore - is this possible?
Thanks
--
You received this message because you are subscribed to the Google
Groups
Google App Engine for Java group.
To post to this group, send email to
google-appengine-java@googlegroups.com.
To unsubscribe from this group, send email to
google-appengine-java+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/google-appengine-java?hl=en.
--
Bruno Fuster
--
You received this message because you are subscribed to the Google Groups
Google App Engine for Java group.
To post to this group, send email to
google-appengine-java@googlegroups.com.
To unsubscribe from this group, send email to
google-appengine-java+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/google-appengine-java?hl=en.
--
Bruno Fuster
--
You received this message because you are subscribed to the Google Groups
Google App Engine for Java group.
To post to this group, send email to google-appengine-java@googlegroups.com.
To unsubscribe from this group, send email to
google-appengine-java+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/google-appengine-java?hl=en.
Re: [appengine-java] Re: Howto return all values stored in memcache?
The publicly available Memcache doesn't have this functionality either, and
it's really not as simple as you might think. What if there's a lot of data?
Do you need a cursor? Should you lock all Memcache shards? And so forth.
If you're interested in a project that does something similar, you should
check out Redis. It's a key/value store with data structures that exists all
in memory, where values are asynchronously flushed to disk.
--
Ikai Lan
Developer Programs Engineer, Google App Engine
plus.ikailan.com | twitter.com/ikai
On Mon, Oct 3, 2011 at 2:29 PM, mscwd01 mscw...@gmail.com wrote:
That's a shame, it's a real disadvantage not having that simple
functionality.
Thanks for your example, although I found it a little difficult to
understand it completely. Are you suggesting having an entry in the
cache with a key string allKeys which maps to a List of keys
belonging to all the other entities in the cache? Would this allow me
to iterate through the keys, saving the entity to the datastore and
then removing the entry from the cache?
Also what is the putIfUntouched statement for?
Thanks again
On Oct 3, 9:51 pm, Bruno Fuster brunofus...@gmail.com wrote:
I don't think so.
You could create a key with all entities ids that need to be updated.
Something like:
public void saveEntity() {
cache.put(key, entity);
queue.add(withUrl(/entitiesToUpdate?id=+entity.getId()));
}
and then
public void entitiesToUpdate(Long id) {
IdentifiableValue i = cache.getIdentifiable(allKeys);
ListLong keys = i.getValue();
keys.add(id);
boolean put = cache.putIfUntouched(allKeys, i, keys);
if (!put) {
//response 409 for conflict due to concurrency so appengine will
retry this later
}
}
On Mon, Oct 3, 2011 at 5:38 PM, mscwd01 mscw...@gmail.com wrote:
Hey
How would one return all the values stored within the memcache?
I have tried using entrySet() and values() methods on the cache but
both result in an java.lang.UnsupportedOperationException.
Ideally, I need to return all the entities stored within the cache and
bulk save them to the datastore - is this possible?
Thanks
--
You received this message because you are subscribed to the Google
Groups
Google App Engine for Java group.
To post to this group, send email to
google-appengine-java@googlegroups.com.
To unsubscribe from this group, send email to
google-appengine-java+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/google-appengine-java?hl=en.
--
Bruno Fuster
--
You received this message because you are subscribed to the Google Groups
Google App Engine for Java group.
To post to this group, send email to
google-appengine-java@googlegroups.com.
To unsubscribe from this group, send email to
google-appengine-java+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/google-appengine-java?hl=en.
--
You received this message because you are subscribed to the Google Groups
Google App Engine for Java group.
To post to this group, send email to google-appengine-java@googlegroups.com.
To unsubscribe from this group, send email to
google-appengine-java+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/google-appengine-java?hl=en.
Re: [appengine-java] Re: Howto return all values stored in memcache?
You wont be able to run Redis on AppEngine.
You can use AppScale though.
http://appscale.cs.ucsb.edu/datastores.html#redis
The only thing is that Redis only replicates master-slave by now, so you got
a single point of failure.
On Mon, Oct 3, 2011 at 7:11 PM, mscwd01 mscw...@gmail.com wrote:
Thanks Ikai, if I were to get Redis working on App Engine would it
work across multiple instances if it's all held in memory?
Regardless, people must be using the memcache to cache data before
persisting updates to the datastore, how is everyone else achieving
this?
On Oct 3, 10:56 pm, Ikai Lan (Google) ika...@google.com wrote:
The publicly available Memcache doesn't have this functionality either,
and
it's really not as simple as you might think. What if there's a lot of
data?
Do you need a cursor? Should you lock all Memcache shards? And so forth.
If you're interested in a project that does something similar, you should
check out Redis. It's a key/value store with data structures that exists
all
in memory, where values are asynchronously flushed to disk.
--
Ikai Lan
Developer Programs Engineer, Google App Engine
plus.ikailan.com | twitter.com/ikai
On Mon, Oct 3, 2011 at 2:29 PM, mscwd01 mscw...@gmail.com wrote:
That's a shame, it's a real disadvantage not having that simple
functionality.
Thanks for your example, although I found it a little difficult to
understand it completely. Are you suggesting having an entry in the
cache with a key string allKeys which maps to a List of keys
belonging to all the other entities in the cache? Would this allow me
to iterate through the keys, saving the entity to the datastore and
then removing the entry from the cache?
Also what is the putIfUntouched statement for?
Thanks again
On Oct 3, 9:51 pm, Bruno Fuster brunofus...@gmail.com wrote:
I don't think so.
You could create a key with all entities ids that need to be updated.
Something like:
public void saveEntity() {
cache.put(key, entity);
queue.add(withUrl(/entitiesToUpdate?id=+entity.getId()));
}
and then
public void entitiesToUpdate(Long id) {
IdentifiableValue i = cache.getIdentifiable(allKeys);
ListLong keys = i.getValue();
keys.add(id);
boolean put = cache.putIfUntouched(allKeys, i, keys);
if (!put) {
//response 409 for conflict due to concurrency so appengine
will
retry this later
}
}
On Mon, Oct 3, 2011 at 5:38 PM, mscwd01 mscw...@gmail.com wrote:
Hey
How would one return all the values stored within the memcache?
I have tried using entrySet() and values() methods on the cache but
both result in an java.lang.UnsupportedOperationException.
Ideally, I need to return all the entities stored within the cache
and
bulk save them to the datastore - is this possible?
Thanks
--
You received this message because you are subscribed to the Google
Groups
Google App Engine for Java group.
To post to this group, send email to
google-appengine-java@googlegroups.com.
To unsubscribe from this group, send email to
google-appengine-java+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/google-appengine-java?hl=en.
--
Bruno Fuster
--
You received this message because you are subscribed to the Google
Groups
Google App Engine for Java group.
To post to this group, send email to
google-appengine-java@googlegroups.com.
To unsubscribe from this group, send email to
google-appengine-java+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/google-appengine-java?hl=en.
--
You received this message because you are subscribed to the Google Groups
Google App Engine for Java group.
To post to this group, send email to
google-appengine-java@googlegroups.com.
To unsubscribe from this group, send email to
google-appengine-java+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/google-appengine-java?hl=en.
--
Bruno Fuster
--
You received this message because you are subscribed to the Google Groups
Google App Engine for Java group.
To post to this group, send email to google-appengine-java@googlegroups.com.
To unsubscribe from this group, send email to
google-appengine-java+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/google-appengine-java?hl=en.
Re: [appengine-java] Re: Howto return all values stored in memcache?
@mscwd01
You can't be sure when the cache will be evicted. You can define an
expiration, but appengine may flush your cache at any moment (its not that
common though).
So, IMHO, always write your entities into the datastore as they get updated.
Put them into the cache and only get back to the datastore if it gets
evicted. If you do not define an expiration time it will remain cached for a
long period.
If you want a better response time for your user just put into the cache and
dispatch a task to update the datastore in background (send the object as a
payload if you do not want to rely on the cache, which can get flushed).
On Mon, Oct 3, 2011 at 6:56 PM, Ikai Lan (Google) ika...@google.com wrote:
The publicly available Memcache doesn't have this functionality either, and
it's really not as simple as you might think. What if there's a lot of data?
Do you need a cursor? Should you lock all Memcache shards? And so forth.
If you're interested in a project that does something similar, you should
check out Redis. It's a key/value store with data structures that exists all
in memory, where values are asynchronously flushed to disk.
--
Ikai Lan
Developer Programs Engineer, Google App Engine
plus.ikailan.com | twitter.com/ikai
On Mon, Oct 3, 2011 at 2:29 PM, mscwd01 mscw...@gmail.com wrote:
That's a shame, it's a real disadvantage not having that simple
functionality.
Thanks for your example, although I found it a little difficult to
understand it completely. Are you suggesting having an entry in the
cache with a key string allKeys which maps to a List of keys
belonging to all the other entities in the cache? Would this allow me
to iterate through the keys, saving the entity to the datastore and
then removing the entry from the cache?
Also what is the putIfUntouched statement for?
Thanks again
On Oct 3, 9:51 pm, Bruno Fuster brunofus...@gmail.com wrote:
I don't think so.
You could create a key with all entities ids that need to be updated.
Something like:
public void saveEntity() {
cache.put(key, entity);
queue.add(withUrl(/entitiesToUpdate?id=+entity.getId()));
}
and then
public void entitiesToUpdate(Long id) {
IdentifiableValue i = cache.getIdentifiable(allKeys);
ListLong keys = i.getValue();
keys.add(id);
boolean put = cache.putIfUntouched(allKeys, i, keys);
if (!put) {
//response 409 for conflict due to concurrency so appengine will
retry this later
}
}
On Mon, Oct 3, 2011 at 5:38 PM, mscwd01 mscw...@gmail.com wrote:
Hey
How would one return all the values stored within the memcache?
I have tried using entrySet() and values() methods on the cache but
both result in an java.lang.UnsupportedOperationException.
Ideally, I need to return all the entities stored within the cache and
bulk save them to the datastore - is this possible?
Thanks
--
You received this message because you are subscribed to the Google
Groups
Google App Engine for Java group.
To post to this group, send email to
google-appengine-java@googlegroups.com.
To unsubscribe from this group, send email to
google-appengine-java+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/google-appengine-java?hl=en.
--
Bruno Fuster
--
You received this message because you are subscribed to the Google Groups
Google App Engine for Java group.
To post to this group, send email to
google-appengine-java@googlegroups.com.
To unsubscribe from this group, send email to
google-appengine-java+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/google-appengine-java?hl=en.
--
You received this message because you are subscribed to the Google Groups
Google App Engine for Java group.
To post to this group, send email to
google-appengine-java@googlegroups.com.
To unsubscribe from this group, send email to
google-appengine-java+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/google-appengine-java?hl=en.
--
Bruno Fuster
--
You received this message because you are subscribed to the Google Groups
Google App Engine for Java group.
To post to this group, send email to google-appengine-java@googlegroups.com.
To unsubscribe from this group, send email to
google-appengine-java+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/google-appengine-java?hl=en.
Re: [appengine-java] Re: Howto return all values stored in memcache?
Yeah, Redis won't work in App Engine. I was just citing it as an example of
something that was designed closer to the use you were looking for (as
opposed to memcache).
--
Ikai Lan
Developer Programs Engineer, Google App Engine
plus.ikailan.com | twitter.com/ikai
On Mon, Oct 3, 2011 at 3:18 PM, Bruno Fuster brunofus...@gmail.com wrote:
You wont be able to run Redis on AppEngine.
You can use AppScale though.
http://appscale.cs.ucsb.edu/datastores.html#redis
The only thing is that Redis only replicates master-slave by now, so you
got a single point of failure.
On Mon, Oct 3, 2011 at 7:11 PM, mscwd01 mscw...@gmail.com wrote:
Thanks Ikai, if I were to get Redis working on App Engine would it
work across multiple instances if it's all held in memory?
Regardless, people must be using the memcache to cache data before
persisting updates to the datastore, how is everyone else achieving
this?
On Oct 3, 10:56 pm, Ikai Lan (Google) ika...@google.com wrote:
The publicly available Memcache doesn't have this functionality either,
and
it's really not as simple as you might think. What if there's a lot of
data?
Do you need a cursor? Should you lock all Memcache shards? And so forth.
If you're interested in a project that does something similar, you
should
check out Redis. It's a key/value store with data structures that exists
all
in memory, where values are asynchronously flushed to disk.
--
Ikai Lan
Developer Programs Engineer, Google App Engine
plus.ikailan.com | twitter.com/ikai
On Mon, Oct 3, 2011 at 2:29 PM, mscwd01 mscw...@gmail.com wrote:
That's a shame, it's a real disadvantage not having that simple
functionality.
Thanks for your example, although I found it a little difficult to
understand it completely. Are you suggesting having an entry in the
cache with a key string allKeys which maps to a List of keys
belonging to all the other entities in the cache? Would this allow me
to iterate through the keys, saving the entity to the datastore and
then removing the entry from the cache?
Also what is the putIfUntouched statement for?
Thanks again
On Oct 3, 9:51 pm, Bruno Fuster brunofus...@gmail.com wrote:
I don't think so.
You could create a key with all entities ids that need to be
updated.
Something like:
public void saveEntity() {
cache.put(key, entity);
queue.add(withUrl(/entitiesToUpdate?id=+entity.getId()));
}
and then
public void entitiesToUpdate(Long id) {
IdentifiableValue i = cache.getIdentifiable(allKeys);
ListLong keys = i.getValue();
keys.add(id);
boolean put = cache.putIfUntouched(allKeys, i, keys);
if (!put) {
//response 409 for conflict due to concurrency so appengine
will
retry this later
}
}
On Mon, Oct 3, 2011 at 5:38 PM, mscwd01 mscw...@gmail.com wrote:
Hey
How would one return all the values stored within the memcache?
I have tried using entrySet() and values() methods on the cache
but
both result in an java.lang.UnsupportedOperationException.
Ideally, I need to return all the entities stored within the cache
and
bulk save them to the datastore - is this possible?
Thanks
--
You received this message because you are subscribed to the Google
Groups
Google App Engine for Java group.
To post to this group, send email to
google-appengine-java@googlegroups.com.
To unsubscribe from this group, send email to
google-appengine-java+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/google-appengine-java?hl=en.
--
Bruno Fuster
--
You received this message because you are subscribed to the Google
Groups
Google App Engine for Java group.
To post to this group, send email to
google-appengine-java@googlegroups.com.
To unsubscribe from this group, send email to
google-appengine-java+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/google-appengine-java?hl=en.
--
You received this message because you are subscribed to the Google Groups
Google App Engine for Java group.
To post to this group, send email to
google-appengine-java@googlegroups.com.
To unsubscribe from this group, send email to
google-appengine-java+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/google-appengine-java?hl=en.
--
Bruno Fuster
--
You received this message because you are subscribed to the Google Groups
Google App Engine for Java group.
To post to this group, send email to
google-appengine-java@googlegroups.com.
To unsubscribe from this group, send email to
google-appengine-java+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/google-appengine-java?hl=en.
