I have tried the real_thread approach as below, but it seems to get stuck
in a loop that calls stackless.schedule(). Isn't the webserver getting
scheduled (anymore, it runs at the start) ?
----------------------------------------------------------------
Where the program gets stuck (if i don't use this class, all runs well,
although pretty slow compared to without the real thread):
class Trigger(BaseTrigger): #part of flowmodel, see
below
def start(self): #called from
flowmodel.start() see MonkeyTasklets.run below
super(Trigger, self).start()
stackless.tasklet(self.run)()
def condition(self):
raise NotImplementedError()
def run(self):
while True: #gets stuck here
if self.condition(): #returns False
self.trigger()
else:
stackless.schedule()
where i create the real thread:
def run_server(self, config = None, threaded = False):
# in this case threaded = True
from flow.tasklets import MonkeyTasklets
clone = self.create_run_clone(config) #deepcopies
all models in which tasklets are created (except webserver tasklet)
clone.tasklets = MonkeyTasklets() #see below
if threaded:
import thread
def run_function(world):
clone.tasklets.run(world = world,
webserver = webserver,
maxcount =
world.config.RUN_LIMIT)
thread.real_thread.start_new_thread(run_function, (clone,))
#create real thread
else:
clone.tasklets.run(clone, webserver, maxcount =
clone.config.RUN_LIMIT)
return clone
#clone is returned to be
able to stop the thread via clone.tasklets.stop = True
This is where the main loop runs:
class MonkeyTasklets(object):
stacklesslib.app.install_stackless()
def __init__(self):
self.stop = None #used to stop the
loop and therefore the thread
def run(self, world, webserver = None, maxcount = None):
self.stop = False
world.flowmodel.start()
if webserver:
webserver.start_server(world_model = world)
#creates a tasklet for the webserver
counter = 0
while not self.stop or counter > maxcount:
counter += 1
try:
stacklesslib.main.mainloop.loop()
except Exception as e:
import asyncore
if isinstance(e, ReferenceError):
print("run:EXCEPTION", str(e), asyncore.socket_map)
else:
print("run:EXCEPTION", asyncore.socket_map)
traceback.print_exc()
sys.exc_clear()
if webserver:
webserver.stop()
world.flowmodel.stop()
monkeypatching takes place at the start of the program.
I would like to get this running (because i am into it right now), but it
isn't crucial.
Cheers, Lars
On Mon, Dec 2, 2013 at 9:52 AM, Kristján Valur Jónsson <
[email protected]> wrote:
> Well, that’s the thing, the way we “currently” do monkeypatching, it is
> an all-or-nothing approach.
>
> Either you replace your entire application’s “threading” usage with fake
> threads from stackless, or you don’t.
>
> Although, as a convention, I have put in the old modules as _*real*_*
> attributes inside the fake ones. This helps, if your code is aware of this
> trick.
>
>
>
> A few years ago, Jack Diedrich had a PyCon talk about using context
> managers to temporarily monkeypatch code.
>
> This would be useful, e.g. if you wanted to start up part of your
> application in such a way.
>
> Then you could do something like:
>
>
>
> with stackless.monkeypatch.patch_all():
>
> import flask
>
> import mywebserver
>
>
>
> import myGUI # using regular threads.
>
>
>
> I’m sure we could re-engineer stacklesslib.monkeypatch to make use of this
> pattern.
>
>
>
> K
>
>
>
> *From:* [email protected] [mailto:
> [email protected]] *On Behalf Of *lars van Gemerden
> *Sent:* 2. desember 2013 01:26
>
> *To:* The Stackless Python Mailing List
> *Subject:* Re: [Stackless] monkeypatching
>
>
>
> thank you for the help,
>
>
>
> With the fix it works now; i have one problem left, is that when i start
> the server from the gui, the server runs, but the (Qt/PySide) gui becomes
> unresponsive. Before, i started the web app and server in a separate thread
> (and had the scheduler run in that), but that probably wont work if
> threads are replaced with tasklets (if i understood the monkeypatching
> somewhat).
>
>
>
> Does anyone have any ideas how to solve that (to keep the gui responsive,
> also to be able to stop the server)? Before i tried multiprocessing, but
> that led to a lot of servers running :-)
>
>
>
> In another experiment I have overridden the server loop serve_forever() to
> call stackless.schedule() (or rather tasklets.schedule() in the code
> above). This seems to work apart from an occasional crash (weird attribute
> error for (my subclass of) WSGIServer).
>
>
>
> Also thanks a lot for putting up the 2.7.5 64 bit binaries!! It finally
> let me run the demo on my Windows 8 laptop (no idea why, something to do
> with PySide 32bit and win8 probably)
>
>
>
> Cheers, Lars
>
>
>
>
>
> On Mon, Dec 2, 2013 at 12:36 AM, Kristján Valur Jónsson <
> [email protected]> wrote:
>
> A bug!
>
> in threadpool.py, replace the final line with:
>
>
>
> return tasklet_call(wrapped, dispatcher=dispatcher, timeout=timeout,
> onOrphaned=onOrphaned)
>
> Needs more unittests :)
>
>
>
> K
> ------------------------------
>
> *From:* [email protected] [[email protected]]
> on behalf of lars van Gemerden [[email protected]]
> *Sent:* Sunday, December 01, 2013 1:58 PM
> *To:* The Stackless Python Mailing List
> *Subject:* Re: [Stackless] monkeypatching
>
> OK, thanks for the input,
>
>
>
> i am trying to get this to work, but i might have found a bug (which i
> find hard to debug):
>
>
>
> i have reimplemented the code above as:
>
> --------------------------------------------------------------
>
> import asyncore, traceback, sys, logging
>
>
>
> import stacklesslib.main
>
> import stacklesslib.app
>
>
>
> # the unittests use time.time() for various time tests.
>
> # Therefore, we must make sure that main uses this
>
> from time import time as elapsed_time
>
> stacklesslib.main.elapsed_time = elapsed_time
>
>
>
> class MonkeyTasklets(object):
>
>
>
> _stop = False
>
>
>
> stacklesslib.app.install_stackless()
>
>
>
> @classmethod
>
> def run(cls, world, webserver = None, maxcount = None): #param killer
> is ignored
>
> world.flowmodel.start()
>
> if webserver:
>
> cls.server = webserver.start_server(world_model = world)
> #starts a tasklet for running the webserver
>
> counter = 0
>
> while not cls._stop or counter > maxcount:
>
> counter += 1
>
> tick_time = elapsed_time()
>
> try:
>
> stacklesslib.main.mainloop.loop()
>
> except Exception as e:
>
> import asyncore
>
> if isinstance(e, ReferenceError):
>
> print("run:EXCEPTION", str(e), asyncore.socket_map)
>
> else:
>
> print("run:EXCEPTION", asyncore.socket_map)
>
> traceback.print_exc()
>
> sys.exc_clear()
>
> world.flowmodel.stop()
>
>
>
> @classmethod
>
> def schedule(cls):
>
> stackless.schedule()
>
>
>
> @classmethod
>
> def tasklet(cls, func):
>
> return stackless.tasklet(func)
>
>
>
> @classmethod
>
> def stop(cls):
>
> if hasattr(cls, "server"):
>
> cls.server.stop()
>
> cls._stop = True
>
>
>
>
>
> tasklets = MonkeyTasklets
>
>
>
> --------------------------------------------------------------
>
> and i get the exception:
>
>
>
> File *"d:\Documents\Code\python\floware\server\bottle.py"*, line 641, in
> *run*
> *run(self, **kwargs)*
> File *"d:\Documents\Code\python\floware\server\bottle.py"*, line 2720, in
> *run*
> *server.run(app)*
> File *"d:\Documents\Code\python\floware\server\webserver.py"*, line 329,
> in *run*
> *self.server.serve_forever()*
> File *"C:\Python27\lib\SocketServer.py"*, line 236, in *serve_forever*
> *poll_interval)*
> File *"C:\Python27\lib\SocketServer.py"*, line 155, in *_eintr_retry*
> *return func(*args)*
> File
> *"d:\Documents\Code\python\floware\stacklesslib\replacements\select.py"*,
> line 27, in *select*
> *return stacklesslib.threadpool.call_on_thread(real_select.select,
> args, kwargs)*
> File *"d:\Documents\Code\python\floware\stacklesslib\threadpool.py"*,
> line 123, in *call_on_thread*
> *return tasklet_call(wrapped, dispatcher, timeout=timeout,
> onOrphaned=onOrphaned)*
> File *"d:\Documents\Code\python\floware\stacklesslib\util.py"*, line 209,
> in *tasklet_call*
> *return channel_wait(chan, timeout)*
> File *"d:\Documents\Code\python\floware\stacklesslib\util.py"*, line 53,
> in *channel_wait*
> *return chan.receive()*
> File *"d:\Documents\Code\python\floware\stacklesslib\util.py"*, line 192,
> in *helper*
> *result = function(*args, **kwargs)*
>
> *TypeError: wrapped() argument after * must be a sequence, not function*
>
> --------------------------------------------------------------
>
> is this something i am doing wrong or indeed a buggy in stacklesslib and
> if so, how can it be fixed?
>
>
>
> Cheers, Lars
>
>
>
> On Sun, Dec 1, 2013 at 4:05 AM, Richard Tew <[email protected]>
> wrote:
>
> latest stacklesslib, i mean.
>
>
> On 12/1/13, Richard Tew <[email protected]> wrote:
> > Hi Lars,
> >
> > stacklessio is internal ccp stuff.
> >
> > main.py in stacklesslib is pretty much all you need to understand,
> > beyond calling patch_all(). It is a scheduler, and makes your Open
> > scheduler redundant.
> >
> > I suggest you get the latest stacklessio from:
> >
> > https://bitbucket.org/krisvale/stacklesslib
> >
> > And then you check out test / teststdlibunittests.py, which is a short
> > example of monkey patching and running the stacklesslib scheduler.
> >
> > Cheers,
> > Richard.
> >
> > On 12/1/13, lars van Gemerden <[email protected]> wrote:
> >> sorry,
> >>
> >> patch_all seemed to simple, question was a bit rethoric ;-) tried it 2
> >> mins
> >> after pressing send just in case and indeed no cigar.
> >>
> >> I am trying to get this demo running and all kinds of last minute ####
> is
> >> popping up.
> >>
> >> I tried reading the source but it doesn't click; i am not familiar with
> >> using sockets etc. at all.
> >>
> >> e.g.
> >>
> >> - do i need 'stacklessio'?
> >> - can i (essentially) just put a stackless.schedule() (or
> >> OpenTasklets.schedule() in the code below) in the serve_forever loop?
> >> - will this work at all with an adapted scheduler like this:
> >>
> >> class OpenTasklets(object):
> >>
> >> _stop = False
> >>
> >> @classmethod
> >> def run(cls, maxcount = None):
> >> cls.schedule_channel = stackless.channel()
> >> cls.schedule_channel.preference = 1
> >> counter = 0
> >> OpenTasklets._stop = False
> >> while stackless.getruncount() != 1:
> >> stackless.run()
> >> cls.reschedule(cls._stop or (maxcount and counter >
> >> maxcount))
> >> counter += 1
> >>
> >> @classmethod
> >> def schedule(cls):
> >> if cls.schedule_channel.receive():
> >> raise TaskletExit
> >>
> >> @classmethod
> >> def reschedule(cls, stop = False):
> >> while cls.schedule_channel.balance < 0:
> >> cls.schedule_channel.send(stop)
> >>
> >> Any help is appreciated ..
> >>
> >> Lars
> >>
> >>
> >>
> >>
> >> On Sat, Nov 30, 2013 at 8:53 PM, Richard Tew
> >> <[email protected]>wrote:
> >>
> >>> Lars :-)
> >>>
> >>> Yes, there is no documentation for stacklesslib yet. To use it, you
> >>> really need to be willing to read the source code.
> >>>
> >>> You should already know whether calling patch_all would be enough. It
> >>> would have been something you could have tried immediately, rather
> >>> than asking the list and waiting.
> >>>
> >>> Cheers,
> >>> Richard.
> >>>
> >>> On 11/30/13, lars van Gemerden <[email protected]> wrote:
> >>> > Hi all,
> >>> >
> >>> > I though i could avoid it to run my demo but it seems i need to let
> >>> > the
> >>> > webserver (simple server?, single thread, 'bottle' micro framework)
> >>> > yield
> >>> > to the scheduler i am using. I have downloaded stacklesslib 1.0.3,
> but
> >>> > can't find any documentation to help with monkeypatching.
> >>> >
> >>> > - Do i just run patch_all at the start of the program?
> >>> >
> >>> > Any help is very welcome ..
> >>> >
> >>> > Cheers, Lars
> >>> >
> >>> > --
> >>> > ====================================
> >>> > Lars van Gemerden
> >>> > [email protected]
> >>> > +31 6 26 88 55 39
> >>> > ====================================
> >>> >
> >>>
> >>> _______________________________________________
> >>> Stackless mailing list
> >>> [email protected]
> >>> http://www.stackless.com/mailman/listinfo/stackless
> >>>
> >>
> >>
> >>
> >> --
> >> ====================================
> >> Lars van Gemerden
> >> [email protected]
> >> +31 6 26 88 55 39
> >> ====================================
> >>
> >
>
> _______________________________________________
> Stackless mailing list
> [email protected]
> http://www.stackless.com/mailman/listinfo/stackless
>
>
>
>
>
> --
> ====================================
> Lars van Gemerden
> [email protected]
>
> +31 6 26 88 55 39
> ==================================== <#142b282cef7bdad7_142b0afb3d212348_>
>
>
> _______________________________________________
> Stackless mailing list
> *[email protected]*
> *http://www.stackless.com/mailman/listinfo/stackless*<#142b282cef7bdad7_142b0afb3d212348_>
>
>
>
> <#142b282cef7bdad7_142b0afb3d212348_>
>
> <#142b282cef7bdad7_142b0afb3d212348_>
>
> --
> ====================================
> Lars van Gemerden
> *[email protected]*
> +31 6 26 88 55 39
> ==================================== <#142b282cef7bdad7_142b0afb3d212348_>
>
> _______________________________________________
> Stackless mailing list
> [email protected]
> http://www.stackless.com/mailman/listinfo/stackless
>
--
====================================
Lars van Gemerden
[email protected]
+31 6 26 88 55 39
====================================
_______________________________________________
Stackless mailing list
[email protected]
http://www.stackless.com/mailman/listinfo/stackless
