First of all, by "architecture" I was referring to the server-side, not the
client-side.
Secondly, when I asked "how do you combine gevent and tornado", I was
referring to your statements:
"web2py app based on tornado long polling chat example."
The only requirement is to run it with gevent."
This implies that you "are" using both, so I wondered how exactly.
Furthermore:
I think you are conflating the concepts of "synchronous vs. asynchronous"
with the concepts of "blocking-IO vs. non-blocking IO".
I think you are also conflating the concepts of "threading" with the
concept of "concurrency".
"synchronous vs. asynchronous" - Is about code-structure.
"blocking-IO vs. non-blocking IO" - is about performance and
"semi-concurrency" capabilities.
It is impossible to make a non-blocking-IO kind of code, with just
structuring it "asynchronously".
It is not just a matter of getting the GIL to context-switch - you need
something extra for IO - you need a system-level callback-mechanism, that
would wake-up your co-routines, and you need that this call-back mechanism
would not block the co-routine that is using it, and finally you need a way
for both of these requirements to co-exist - a co-routine mechanism that
can use the non-blocking system-level callback.
Currently, the only way to achieve that in python, without any c-extension
that wrangles the stack, is to use threads.
It has nothing to do with concurrency, and nothing to do with thread-safety
(sharing data across threads, as you link denotes).
No. It is about having a way to relinquish the GIL to a system-callback, in
a non-blocking way.
The mechanism that you mentioned (epoll/kqueue/select, or IOCPs in windows),
are providing a system-level callback-mechanism for IO operations (a-la
"preemptive-multitasking"), but they do NOT supply integration of that with
a non-blocking co-routine in python (at least not to my knowledge). Again,
without a c-extension, you NEED a "thread" to do that - and it is exactly
what Tornado does (again, to the best of my knowledge from my research - I
can find the reference that says that, if you like). The link you added, is
not addressing this at all - it is simply denoting the shortcomings of
threads in general in python (lack of "true concurrency" due to the GIL),
and the problems of threading in-general (data-sharing, locking, etc.).
But BOTH of these issues do NOT relate to the "specific-usage of threads"
for "non-blocking-co-routines" - this is a separate issue.
You CAN (and that is what Tornado does) use threads only for THAT
specific requirement without expecting concurrency and without using
shared-data across threads.
Gevent solved this issue very differently, WITHOUT threads, and it is for
THAT reason that it requires a third-party c-extension. It is not just the
event-loop - it is the "non-blocking-co-routines" that such libraries as
libevent/libev
(or libuv of node.js) are solving. Gevent is not just an event-loop, it
uses the "greenlet" extension (which implements co-routines in a
"collaborative-multitasking"
model, a-la "stackless", which is something "threads" can-not do), as an
abstraction over these c-compiled libraries, to connect a non-blocking
event-loop to a collaboratively-multi-tasked-co-routine.
In short, only gevent can do this in c-python without threads.
All other pythonic-solutions who can do this, are NOT c-python
("stackless-python", PyPy, Iron-Python, Jython, etc.)
So unless Tornado is doing something that nobody is aware of, it IS using
threads to implement a non-blocking-co-routine-IO-event-loop.
It might not use threads for other things, and may not support using
threads to communicate with it's co-routines, but that's a separate issue.
Back to the issue - you said that your solution is using Tornado, and
"requires" gevent - how so?
--
---
You received this message because you are subscribed to the Google Groups
"web2py-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
For more options, visit https://groups.google.com/groups/opt_out.
