Hello,
I'm pretty new to appengine but I remembered reading Efficient model
memcaching [1] a while back and wanted to see it in action now that
I'm working on a project.
After some surprisingly slow results I decided to benchmark it
thoroughly. I run both picke, depickle and protobufy, pickle,
depickle, deprotobufy on a list of 100 entities.
model:
class WasteCategory(db.Model):
name = db.StringProperty(required=True)
icon = blobstore.BlobReferenceProperty()
listed = db.BooleanProperty(default=False)
benchmark code:
def time_it(self, name, func, n=100):
import logging
from google.appengine.api import quota
t = -quota.get_request_cpu_usage()
for i in xrange(0, n):
func()
t += quota.get_request_cpu_usage()
logging.debug("Running %s %d times took %d megacycles", name, n, t)
def benchmark(self):
import pickle
from google.appengine.ext import db
from tipfy import Response
def protobufy(data):
return [db.model_to_protobuf(x).Encode() for x in data]
def deprotobufy(data):
return [db.model_from_protobuf(x) for x in data]
data = []
for i in xrange(0, 100):
name = "category_%02d" % random.randint(1, 99)
wc = WasteCategory(name=name, listed=True)
data.append(wc)
self.time_it("pickle", lambda: pickle.dumps(data))
data = pickle.dumps(data)
self.time_it("depickle", lambda: pickle.loads(data))
data = pickle.loads(data)
self.time_it("protobufy", lambda: protobufy(data))
data = protobufy(data)
self.time_it("pickle_protobufed", lambda: pickle.dumps(data))
data = pickle.dumps(data)
self.time_it("depickle_protobufed", lambda: pickle.loads(data))
data = pickle.loads(data)
self.time_it("deprotobufy", lambda: deprotobufy(data))
data = deprotobufy(data)
return Response("")
Results:
12-07 04:56PM 36.559 Running pickle 100 times took 3221 megacycles
12-07 04:56PM 37.152 Running depickle 100 times took 1631 megacycles
12-07 04:56PM 39.778 Running protobufy 100 times took 7339 megacycles
12-07 04:56PM 40.424 Running pickle_protobufed 100 times took 1742 megacycles
12-07 04:56PM 41.446 Running depickle_protobufed 100 times took 2843 megacycles
12-07 04:56PM 43.823 Running deprotobufy 100 times took 6634 megacycles
Has pickling entities been optimized or am I missing something obvious here?
[1] - http://blog.notdot.net/2009/9/Efficient-model-memcaching
--
Best Regards
Piotr Jaroszyński
--
You received this message because you are subscribed to the Google Groups
"Google App Engine" group.
To post to this group, send email to [email protected].
To unsubscribe from this group, send email to
[email protected].
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en.
