Hello folks,
I recently stumbled upon twisted and was wondering if it could suit my
needs. On one hand, I want to use python but on another hand there are all
these scalability concerns with this language so, I though I would pick the
brains of the community. So.. a flask based app would look something like
this.
similar_types = ['foo', 'bar', 'baz']
def long_computation(rec_type):
# some long computation
return recs
@app.route('/fetch_similar_users/<user_id>'
def fetch_similar_users(user_id)
r = json.loads(requests.get('url/to/fetch/%s'%user_id).text)
recs = {}
for stype in similar_types:
recs[stype] = long_computation(rec_type)
return recs
Now, I tried to "twistify" but it failed.
@defer.inlinecallbacks
def long_computation(rec_type):
# some long computation
*defer.returnValue(recs)*
@defer.inlinecallbacks
def fetch_data(user_id):
r = yieldjson.loads(requests.get('url/to/fetch/%s'%user_id).text)
defer.returnValue(r)
@defer.inlinecallbacks
def fetch_recs(user_id):
data = yield fetch_data(user_id)
recs = {}
for stype in similar_types:
d = defer.ToThread(fetch_data, *(stype))
rec = yield d
recs[stype] = rec
defer.returnValue(recs)
I wrapped all the above in twisted render_Get method.. but then I did a
load test with locust (https://docs.locust.io/en/latest/what-is-locust.html)
framework.
It choked. As the time progressed, the response time increased.
I am guessing, things are still blocking.
Can you please help me look into the right place. Why exactly am I seeing
increase in response time as the time progresses. I am guessing things are
still working in "blocking" fashion but i thought the above should run
things in async.
Thanks
_______________________________________________
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
https://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
