Re: Synchronous and Asynchronous callbacks
Am Montag, 30. September 2019 11:46:43 UTC+2 schrieb Barry Scott:
> > On 29 Sep 2019, at 21:41, Eko palypse wrote:
> >
> > Am Sonntag, 29. September 2019 19:18:32 UTC+2 schrieb Barry Scott:
> >>> On 29 Sep 2019, at 14:14, Eko palypse wrote:
> >>>
> >>> Unfortunately, I can't make all callbacks synchronous or asynchronous
> >>> because this has negative effects on the application in one way or
> >>> another.
> >>
> >> Surely you can make all callbacks async?
> >> You do not have to have them wait, they can complete their work in one
> >> call.
> >>
> >> sync == async-that-does-not-await
> >>
> >> Barry
> >
> > Thank you for your answer but I'm sorry I don't get it.
> > What do you mean by I don't have to have them wait?
> >
> > Let me explain my understanding, with some pseudo code, how async callbacks
> > works and please keep in mind that I'm a novice when it comes to OO designs
> > and basic software design strategies/techniques, so I might be totally
> > wrong about what I'm doing.
>
> When you said async I thought you meant the specific programming model in
> Python
> that uses "async def", "await" etc. You are using threads.
>
> The problem you are facing is that the UI must not become unresponsive.
> Further a background thread cannot call any UI framework functions as the UI
> framework can only be called on the main thread.
>
> You can run fast event handers on the UI thread but slow running handlers
> must be run on another thread to allow the UI to be responsive.
>
> What I do is have the event handler for a long running task take the
> responsibility
> to queue the task onto a background thread. While the task is running it
> sends status
> to the UI thread so that UI elements are updated (progress bars, etc).
>
> If you are using PyQt5 you might find the code I used for SCM Workbench
> interesting.
>
> I can write event handlers that look like this:
>
> @thread_switcher
> def eventHandler( self, ... ):
> # starts in the foreground (UI thread)
> self.status.update('Working...')
>
> yield self.switchToBackground
> # now on the background thread
> self.doSlowTask()
>
> yield self.switchToForeground
> # now in the foreground
> self.status.update('Done')
>
> The code is in:
>
> https://github.com/barry-scott/scm-workbench/blob/master/Source/Common/wb_background_thread.py
>
> @thread_switcher adds an attribute to the function so the rest of the
> code knows this is an event handler that needs to use a background thread.
>
> When I add an event handler I call through a function (wrapWithThreadSwitcher)
> that wraps only functions that have been marked with @thread_switcher.
>
> The rest of the code is the machinery to handle moving between threads via
> the generator.
>
> Barry
>
> >
> > An SYNCHRONOUS callback is like this. Event/notification gets fired and
> > notification handler calls all registered methods one after the other.
> > Something like
> >
> > if notification.list_of_methods:
> >for method in list_of_methods:
> >method()
> >
> > whereas an ASYNCHRONOUS callback is more like this. Event/notification gets
> > fired and notification handler informs another thread that the event has
> > been fired and all registered async callback methods should get executed.
> > So something like this
> >
> > class EventThread(threading.Thread):
> >def __init__(...):
> >super()...
> >self.event = threading.Event()
> >self.kill = False
> >...
> >def run(self):
> >while True:
> >self.event.wait()
> >self.event.clear()
> >if not self.kill:
> >for method in self.list_of_methods:
> >method()
> >
> > et = EventThread()
> > if notification.list_of_methods:
> >et.event.set() # run all async methods
> >for method in list_of_methods: # run sync methods
> >method())
> >
> >
> > So if there is no synchronous callback for that notification then the
> > notification handler would just set the event and return.
> > The EventThread would then call one registered callback after the other.
> > Right?
> >
> > Using this approach does sound fine from UI point of view as there is
> > minimal impact and UI keeps responsive but from code execution point of view
> > it
Re: Synchronous and Asynchronous callbacks
> On 29 Sep 2019, at 21:41, Eko palypse wrote:
>
> Am Sonntag, 29. September 2019 19:18:32 UTC+2 schrieb Barry Scott:
>>> On 29 Sep 2019, at 14:14, Eko palypse wrote:
>>>
>>> Unfortunately, I can't make all callbacks synchronous or asynchronous
>>> because this has negative effects on the application in one way or another.
>>
>> Surely you can make all callbacks async?
>> You do not have to have them wait, they can complete their work in one call.
>>
>> sync == async-that-does-not-await
>>
>> Barry
>
> Thank you for your answer but I'm sorry I don't get it.
> What do you mean by I don't have to have them wait?
>
> Let me explain my understanding, with some pseudo code, how async callbacks
> works and please keep in mind that I'm a novice when it comes to OO designs
> and basic software design strategies/techniques, so I might be totally
> wrong about what I'm doing.
When you said async I thought you meant the specific programming model in Python
that uses "async def", "await" etc. You are using threads.
The problem you are facing is that the UI must not become unresponsive.
Further a background thread cannot call any UI framework functions as the UI
framework can only be called on the main thread.
You can run fast event handers on the UI thread but slow running handlers
must be run on another thread to allow the UI to be responsive.
What I do is have the event handler for a long running task take the
responsibility
to queue the task onto a background thread. While the task is running it sends
status
to the UI thread so that UI elements are updated (progress bars, etc).
If you are using PyQt5 you might find the code I used for SCM Workbench
interesting.
I can write event handlers that look like this:
@thread_switcher
def eventHandler( self, ... ):
# starts in the foreground (UI thread)
self.status.update('Working...')
yield self.switchToBackground
# now on the background thread
self.doSlowTask()
yield self.switchToForeground
# now in the foreground
self.status.update('Done')
The code is in:
https://github.com/barry-scott/scm-workbench/blob/master/Source/Common/wb_background_thread.py
@thread_switcher adds an attribute to the function so the rest of the
code knows this is an event handler that needs to use a background thread.
When I add an event handler I call through a function (wrapWithThreadSwitcher)
that wraps only functions that have been marked with @thread_switcher.
The rest of the code is the machinery to handle moving between threads via the
generator.
Barry
>
> An SYNCHRONOUS callback is like this. Event/notification gets fired and
> notification handler calls all registered methods one after the other.
> Something like
>
> if notification.list_of_methods:
>for method in list_of_methods:
>method()
>
> whereas an ASYNCHRONOUS callback is more like this. Event/notification gets
> fired and notification handler informs another thread that the event has
> been fired and all registered async callback methods should get executed.
> So something like this
>
> class EventThread(threading.Thread):
>def __init__(...):
>super()...
>self.event = threading.Event()
>self.kill = False
>...
>def run(self):
>while True:
>self.event.wait()
>self.event.clear()
>if not self.kill:
>for method in self.list_of_methods:
>method()
>
> et = EventThread()
> if notification.list_of_methods:
>et.event.set() # run all async methods
>for method in list_of_methods: # run sync methods
>method())
>
>
> So if there is no synchronous callback for that notification then the
> notification handler would just set the event and return.
> The EventThread would then call one registered callback after the other.
> Right?
>
> Using this approach does sound fine from UI point of view as there is
> minimal impact and UI keeps responsive but from code execution point of view
> it sound much more complicated. What if one async callback is slow or buggy?
> This could lead to unpredictable behavior where as in a synchronous execution
> you immediately know what is causing it.
> That was the reason I decided not to offer async callbacks until I
> discovered that other parts, like UI modifications aren't working properly.
> And there are notification which must be called in a synchronous way, like
> the modification event. If you want to intercept a modification safely then
> it can only be done within a synchronous callback.
>
> Does this makes sense to you? Or do I have wrong understanding or, again, an
> wrong assumption how this works?
>
> Thank you
> Eren
> --
> https://mail.python.org/mailman/listinfo/python-list
>
--
https://mail.python.org/mailman/listinfo/python-list
Re: Synchronous and Asynchronous callbacks
Am Sonntag, 29. September 2019 19:18:32 UTC+2 schrieb Barry Scott: > > On 29 Sep 2019, at 14:14, Eko palypse wrote: > > > > Unfortunately, I can't make all callbacks synchronous or asynchronous > > because this has negative effects on the application in one way or another. > > Surely you can make all callbacks async? > You do not have to have them wait, they can complete their work in one call. > > sync == async-that-does-not-await > > Barry Thank you for your answer but I'm sorry I don't get it. What do you mean by I don't have to have them wait? Let me explain my understanding, with some pseudo code, how async callbacks works and please keep in mind that I'm a novice when it comes to OO designs and basic software design strategies/techniques, so I might be totally wrong about what I'm doing. An SYNCHRONOUS callback is like this. Event/notification gets fired and notification handler calls all registered methods one after the other. Something like if notification.list_of_methods: for method in list_of_methods: method() whereas an ASYNCHRONOUS callback is more like this. Event/notification gets fired and notification handler informs another thread that the event has been fired and all registered async callback methods should get executed. So something like this class EventThread(threading.Thread): def __init__(...): super()... self.event = threading.Event() self.kill = False ... def run(self): while True: self.event.wait() self.event.clear() if not self.kill: for method in self.list_of_methods: method() et = EventThread() if notification.list_of_methods: et.event.set() # run all async methods for method in list_of_methods: # run sync methods method() So if there is no synchronous callback for that notification then the notification handler would just set the event and return. The EventThread would then call one registered callback after the other. Right? Using this approach does sound fine from UI point of view as there is minimal impact and UI keeps responsive but from code execution point of view it sound much more complicated. What if one async callback is slow or buggy? This could lead to unpredictable behavior where as in a synchronous execution you immediately know what is causing it. That was the reason I decided not to offer async callbacks until I discovered that other parts, like UI modifications aren't working properly. And there are notification which must be called in a synchronous way, like the modification event. If you want to intercept a modification safely then it can only be done within a synchronous callback. Does this makes sense to you? Or do I have wrong understanding or, again, an wrong assumption how this works? Thank you Eren -- https://mail.python.org/mailman/listinfo/python-list
Re: Synchronous and Asynchronous callbacks
> On 29 Sep 2019, at 14:14, Eko palypse wrote: > > Unfortunately, I can't make all callbacks synchronous or asynchronous because > this has negative effects on the application in one way or another. Surely you can make all callbacks async? You do not have to have them wait, they can complete their work in one call. sync == async-that-does-not-await Barry -- https://mail.python.org/mailman/listinfo/python-list
Re: Synchronous and Asynchronous callbacks
Am Sonntag, 29. September 2019 01:02:48 UTC+2 schrieb Paul Rubin: > Eko palypse writes: > > Two dicts, one for sync and the other for async registrations? > > Why not make all the callbacks asynchronous? Clients that don't want to > handle a callback immediately can keep it pending until they are ready, > perhaps in a queue. > > Whether this can work of course depends on how the application is put > together, but the pattern of a bunch of asynchronous tasks communicating > through fifos tends to work pretty well. I'm used to doing it with > threads, though those can be expensive if you use a lot of them. I > don't know whether async/await unavoidably complicates things. Thank you very much for your answer. Unfortunately, I can't make all callbacks synchronous or asynchronous because this has negative effects on the application in one way or another. I also came across the asyncio methods yesterday, Whoa, what can I say, I haven't really understood yet. My first impression is that either the whole library works asynchronously or not. Still too many Bohemian villages (not sure if this is also common in English - it should say there are too many things I don't understand). But as I said, thank you very much for the time and your thoughts you have given about my problem, very much appreciated. Eren -- https://mail.python.org/mailman/listinfo/python-list
Synchronous and Asynchronous callbacks
Hello, I'm looking for a solution to a problem which is described best as an observer pattern issue I guess. The scenario is the following, client code can register a callback for one or a list of notifications. This is currently solved by having a dictionary with the possible notifications as the keys and lists of methods being the values. Notification sent, triggers the list of method to be called. Nothing fancy. Now, where it gets trickier is that the client should be able to register synchronous and asynchronous callbacks because some of the actions can't be done in a synchronous callback like updating the statusbar of the UI as the main app would overwrite it. I've read that this isn't a good idea of having synchronous and asynchronous callbacks but to be honest, I don't see how this could be solved otherwise, hence the question what to do in such a case? And as I currently think that there is no other solution then to provide sync and async callback registration possibility how would a solution look like? Two dicts, one for sync and the other for async registrations? That would mean that each notification handler needs to execute one dict after the other. In an ideal world, the notification handler wouldn't know about the difference at all and would call its list of methods(objects?) Any tips for me? Just to be sure as I don't know whether this is relevant, each notification handler provides a different set of arguments to the registered callback functions, hence the multiple notification handlers. Thank you Eren -- https://mail.python.org/mailman/listinfo/python-list
