Re: [pygtk] idle_add vs. threads_enter/threads_leave

[John Stowers]

On Wed, 2010-07-07 at 09:26 +0800, Jason Heeris wrote:
 Tim Evans wrote:
  GTK+ 2.14.4
  PyGObject 2.14.2
  PyGTK 2.12.1
 
 Mine is
 
 GTK+ 2.20
 PyGObject 2.21.2
 PyGTK 2.17.1
 
 A few things about your changes confused me -
 
 1. You call glib.idle_add, but never called glib.threads_init - won't
 this break on Linux?
 2. You use BOTH gtk.gdk.lock and glib.idle_add - but surely one of
 these is superfluous, since glib.idle_add executes the function in the
 main loop, where the lock is already acquired?
 
 Anyway, I came up with a different, simpler way, but I don't know
 enough to say if it will never fail. I can't push to github from here,
 but basically it's the same as my first example[1] but with this main
 block for controller.py:
 
 
 if __name__ == __main__:
 
 import glib
 import gtk.gdk
 gtk.gdk.threads_init()
 glib.threads_init()
 
 view = MyView()
 model = MyModel()
 controller = MyController(model, view)
 
 with gtk.gdk.lock:
 LaunchUI(view)
 
 
 Works on both win32 and linux, using glib.idle_add throughout. I was a
 bit worried that trying to acquire the gtk.gdk.lock from the main
 thread would cause problems under linux as per [2], but it works fine.

This is more or less my approach to. On both windows and linux I
successfully use this idiom as follows

glib.threads_init()
gtk.gdk.threads_init()

class App:
  .. do ui stuff, start worker thread/s ..
  def main()
if is_windows: gtk.gdk.threads_enter()
gtk.main()
if is_windows: gtk.gdk.threads_leave()
  def frobnicate(self, foo):
return False

class ThreadWorker(threading.Thread)
  def run()
while self.running:
  if something_interesting():
 gobject.idle_add(self.app.frobnicate, foo,
priority=gobject.PRIORITY_HIGHEST)

or I use the idiom described here;
http://www.johnstowers.co.nz/blog/index.php/2007/03/12/threading-and-pygtk/

Which is basically the same, except ThreadWorker is also a GObject and
signal emission is done in idle_add, hence all connected signal callbacs
get called in the gtk.main thread

John
  
 
 Lessons learnt:
 
 1. Call gtk.gdk.threads_init() BEFORE glib.threads_init() (otherwise
 glib.idle_add won't work)
 2. Call them both before you do anything else
 3. Start the main loop in the gtk.gdk.lock context manager or
 surrounded by gtk.gdk.threads_enter/leave()
 
 Thanks for everyone's patience - I rarely have to develop under win32,
 so I'm not really aware of these gotchas.
 
 - Jason
 
 [1] http://github.com/detly/gtk-async-test
 [2] http://faq.pygtk.org/index.py?req=showfile=faq20.015.htp
 ___
 pygtk mailing list   pygtk@daa.com.au
 http://www.daa.com.au/mailman/listinfo/pygtk
 Read the PyGTK FAQ: http://faq.pygtk.org/


___
pygtk mailing list   pygtk@daa.com.au
http://www.daa.com.au/mailman/listinfo/pygtk
Read the PyGTK FAQ: http://faq.pygtk.org/

Re: [pygtk] idle_add vs. threads_enter/threads_leave

[John Stowers]

On Wed, 2010-07-07 at 08:00 +0800, Jason Heeris wrote:
 Antoine Martin wrote:
 
  It means that most of your code is not using threads at all, only the
  bits that are *slow*
 
 Those are the only bits that use threads anyway.
 
  I've lost track of your particular issue though, so maybe this is not
  suitable for your use-case? How much slow work do you do compared to the
  rest? (how many code paths, rather than raw amount)
 
 I have two kinds of long-blocking work: reading a large file and 
 processing it to get a single result, and communication over a serial 
 port (send command, await response, times about a hundred).

In the serial port case, I use glib.io_add_watch on the file descriptor.
Check out my UAV ground station software which does a *lot* of serial
comms, and works perfectly in windows and linux [1]

I also have a little helper lib that makes working with gobject and
python-libserial a little nicer [2]

John

[1] http://github.com/nzjrs/wasp
[2] http://github.com/nzjrs/libserial

 
 — Jason
 ___
 pygtk mailing list   pygtk@daa.com.au
 http://www.daa.com.au/mailman/listinfo/pygtk
 Read the PyGTK FAQ: http://faq.pygtk.org/


___
pygtk mailing list   pygtk@daa.com.au
http://www.daa.com.au/mailman/listinfo/pygtk
Read the PyGTK FAQ: http://faq.pygtk.org/

[pygtk] idle_add vs. threads_enter/threads_leave

[Jason Heeris]

Tim Evans wrote:
 Something worth noting is that if you're targeting Windows then 2.a. is
 your *only* option.

In fact, gtk.gdk.threads_init() will freeze straight off, you don't
need to wait for threads to start.

 I would also point out that whenever it next feels like it is almost
 always going to be the same as right away unless you're doing
 something yourself that still blocks the main thread.

Unfortunately, see my other email for my problems with
glib.idle_add() - basically, it doesn't work on Windows either.

There's a quick demo prog at:

http://github.com/detly/gtk-async-test

If you have access to a win32 box, run controller.py to see the
problem (or, in fact, fail to see it - you should click the start
button and see the progress bar go, if you don't, that's the problem).
It works fine under Debian.

So, in short, PyGTK + Windows + async = no :/

Cheers,
Jason
___
pygtk mailing list   pygtk@daa.com.au
http://www.daa.com.au/mailman/listinfo/pygtk
Read the PyGTK FAQ: http://faq.pygtk.org/

[pygtk] idle_add vs. threads_enter/threads_leave

[Jason Heeris]

Antoine Martin wrote:
 I don't know of any examples unfortunately and I must admit that I spent
 quite a bit of time getting to grips with it, but in the end it is
 remarkably simple (much more simple than I first thought too - so don't
 let that put you off):
 import gtk.gdk
 gtk.gdk.threads_init()

This is not a good idea under Windows, sometimes it'll freeze here,
other times... later.

 if you need to interact with the GUI from that thread, do it via 
 gobject.idle_add()

See, at this point I don't understand why Twisted gives me an
advantage. I'm still using threads, I'm still using the problematic
idle_add... what's different?

- Jason
___
pygtk mailing list   pygtk@daa.com.au
http://www.daa.com.au/mailman/listinfo/pygtk
Read the PyGTK FAQ: http://faq.pygtk.org/

Re: [pygtk] idle_add vs. threads_enter/threads_leave

[Antoine Martin]

On 07/06/2010 03:53 PM, Jason Heeris wrote:
 Antoine Martin wrote:
 I don't know of any examples unfortunately and I must admit that I spent
 quite a bit of time getting to grips with it, but in the end it is
 remarkably simple (much more simple than I first thought too - so don't
 let that put you off):
 import gtk.gdk
 gtk.gdk.threads_init()
 
 This is not a good idea under Windows, sometimes it'll freeze here,
 other times... later.

Yes, sorry about that, forgot to mention that I do this in my code:
if not sys.platform.startswith(win):
import gtk.gdk
gtk.gdk.threads_init()

 if you need to interact with the GUI from that thread, do it via 
 gobject.idle_add()
 
 See, at this point I don't understand why Twisted gives me an
 advantage. I'm still using threads, I'm still using the problematic
 idle_add... what's different?
It means that most of your code is not using threads at all, only the
bits that are *slow* (and in my case these are fairly easy to spot:
execing and file-io) so you can call the gtk UI code from your network
callbacks or your network code from your UI callbacks without needing
any trickery.

I've lost track of your particular issue though, so maybe this is not
suitable for your use-case? How much slow work do you do compared to the
rest? (how many code paths, rather than raw amount)

Antoine



 
 - Jason
 ___
 pygtk mailing list   pygtk@daa.com.au
 http://www.daa.com.au/mailman/listinfo/pygtk
 Read the PyGTK FAQ: http://faq.pygtk.org/

___
pygtk mailing list   pygtk@daa.com.au
http://www.daa.com.au/mailman/listinfo/pygtk
Read the PyGTK FAQ: http://faq.pygtk.org/

[pygtk] idle_add vs. threads_enter/threads_leave

[Jason Heeris]

Antoine Martin wrote:

 It means that most of your code is not using threads at all, only the
 bits that are *slow*

Those are the only bits that use threads anyway.

 I've lost track of your particular issue though, so maybe this is not
 suitable for your use-case? How much slow work do you do compared to the
 rest? (how many code paths, rather than raw amount)

I have two kinds of long-blocking work: reading a large file and 
processing it to get a single result, and communication over a serial 
port (send command, await response, times about a hundred).

— Jason
___
pygtk mailing list   pygtk@daa.com.au
http://www.daa.com.au/mailman/listinfo/pygtk
Read the PyGTK FAQ: http://faq.pygtk.org/

[pygtk] idle_add vs. threads_enter/threads_leave

[Jason Heeris]

Tim Evans wrote:
 GTK+ 2.14.4
 PyGObject 2.14.2
 PyGTK 2.12.1

Mine is

GTK+ 2.20
PyGObject 2.21.2
PyGTK 2.17.1

A few things about your changes confused me -

1. You call glib.idle_add, but never called glib.threads_init - won't
this break on Linux?
2. You use BOTH gtk.gdk.lock and glib.idle_add - but surely one of
these is superfluous, since glib.idle_add executes the function in the
main loop, where the lock is already acquired?

Anyway, I came up with a different, simpler way, but I don't know
enough to say if it will never fail. I can't push to github from here,
but basically it's the same as my first example[1] but with this main
block for controller.py:


if __name__ == __main__:

import glib
import gtk.gdk
gtk.gdk.threads_init()
glib.threads_init()

view = MyView()
model = MyModel()
controller = MyController(model, view)

with gtk.gdk.lock:
LaunchUI(view)


Works on both win32 and linux, using glib.idle_add throughout. I was a
bit worried that trying to acquire the gtk.gdk.lock from the main
thread would cause problems under linux as per [2], but it works fine.

Lessons learnt:

1. Call gtk.gdk.threads_init() BEFORE glib.threads_init() (otherwise
glib.idle_add won't work)
2. Call them both before you do anything else
3. Start the main loop in the gtk.gdk.lock context manager or
surrounded by gtk.gdk.threads_enter/leave()

Thanks for everyone's patience - I rarely have to develop under win32,
so I'm not really aware of these gotchas.

- Jason

[1] http://github.com/detly/gtk-async-test
[2] http://faq.pygtk.org/index.py?req=showfile=faq20.015.htp
___
pygtk mailing list   pygtk@daa.com.au
http://www.daa.com.au/mailman/listinfo/pygtk
Read the PyGTK FAQ: http://faq.pygtk.org/

Re: [pygtk] idle_add vs. threads_enter/threads_leave

[Tim Evans]

On 2010-07-07 13:26, Jason Heeris wrote:
 Tim Evans wrote:
 GTK+ 2.14.4
 PyGObject 2.14.2
 PyGTK 2.12.1

 Mine is

 GTK+ 2.20
 PyGObject 2.21.2
 PyGTK 2.17.1

 A few things about your changes confused me -

 1. You call glib.idle_add, but never called glib.threads_init - won't
 this break on Linux?
 2. You use BOTH gtk.gdk.lock and glib.idle_add - but surely one of
 these is superfluous, since glib.idle_add executes the function in the
 main loop, where the lock is already acquired?

 Anyway, I came up with a different, simpler way, but I don't know
 enough to say if it will never fail. I can't push to github from here,
 but basically it's the same as my first example[1] but with this main
 block for controller.py:

My older version of PyGObject doesn't have glib.threads_init(), or even 
a separate glib module, that was one of the new features. With the older 
version you didn't have the option of turning on threads in pygobject 
without turning on threads in gdk. Perhaps you still don't if you want 
things to work, I'm not sure.

Idle and timeout callbacks are called without the gdk thread lock. You 
need to acquire it in each callback. In practise I always use a custom 
idle_add_lock that does that for me. Again, this may be specific to my 
older PyGObject.

   def idle_add_lock(func, *args):
   def idle():
   with gtk.gdk.lock:
   return func(*args)
   return gobject.idle_add(idle)

-- 
Tim Evans
Applied Research Associates NZ
http://www.aranz.com/
___
pygtk mailing list   pygtk@daa.com.au
http://www.daa.com.au/mailman/listinfo/pygtk
Read the PyGTK FAQ: http://faq.pygtk.org/

[pygtk] idle_add vs. threads_enter/threads_leave

[Jason Heeris]

Hi,

I've been reading some conflicting advice on PyGTK and threading
recently, probably because the API went through some fairly rapid
changes recently.

I'm a bit confused about the following points (consider them in
context of PyGTK/PyGObject unstable, ie. 2.17/2.21 respectively):

  1. If I want to do async activity in a new Python thread spawned
from the main loop, I need to have called glib.threads_init() *before*
starting the main loop, right? But what about gtk.gdk.threads_init()?
  2. Any GTK interaction, such as emitting a signal from this new
thread, must be either:
  a. done via glib.idle_add (if I'm happy to let GTK do it
whenever it next feels like it), -OR-
  b. wrapped in gtk.gdk.threads_enter()/...leave(), -OR-
  c. in a with: gtk.gdk.lock: block (equivalent to 2.b)
  3. I don't need to do the threads_enter/leave (or use the context
manager) if I only use glib.idle_add (or timeout_add, etc)

Cheers,
Jason
___
pygtk mailing list   pygtk@daa.com.au
http://www.daa.com.au/mailman/listinfo/pygtk
Read the PyGTK FAQ: http://faq.pygtk.org/

Re: [pygtk] idle_add vs. threads_enter/threads_leave

[Jason Heeris]

On 5 July 2010 17:36, A.T.Hofkamp a.t.hofk...@tue.nl wrote:
 Or you could drop threads entirely, and do your async activities using the
 Twisted framework, designed for making asynchronous programs (where GTK
 event handling is just one of the asynchronous sources).

Please don't take this as a flame, but I've had this advice before
(usually on #python), and I usually go through this process:

1. Look at Twisted website
2. Only read things about twisted being a networking framework
3. Someone corrects me, saying Twisted is so much more! (That may be
the case, but the fact that the website goes on and on without
mentioning this does not inspire confidence.)
4. I search the Twisted website for documentation on/examples of GTK
applications
5. I give up and search the Twisted website for documentation
on/examples of *any* non-network based applications
6. I give up and search the greater web for documentation on/examples
of Twisted GTK applications
7. I give up on Twisted and find another way.

Really, I would love to get a grip on Twisted. It seems really useful.
But I've still not found any kind of stepping stone into it. Please,
please, please if you know of one, post it so I can add another tool
to my belt. I will rescind everything I've said here and then some.

(Since the Twisted website is currently down, I can't tell if things
have changed and I should get over it and have another try.)

Cheers,
Jason
___
pygtk mailing list   pygtk@daa.com.au
http://www.daa.com.au/mailman/listinfo/pygtk
Read the PyGTK FAQ: http://faq.pygtk.org/

Re: [pygtk] idle_add vs. threads_enter/threads_leave

[Pietro Battiston]

Il giorno lun, 05/07/2010 alle 16.32 +0800, Jason Heeris ha scritto:
 Hi,
 
 I've been reading some conflicting advice on PyGTK and threading
 recently, probably because the API went through some fairly rapid
 changes recently.
 
 I'm a bit confused about the following points (consider them in
 context of PyGTK/PyGObject unstable, ie. 2.17/2.21 respectively):
 
   1. If I want to do async activity in a new Python thread spawned
 from the main loop, I need to have called glib.threads_init() *before*
 starting the main loop, right? But what about gtk.gdk.threads_init()?
   2. Any GTK interaction, such as emitting a signal from this new
 thread, must be either:
   a. done via glib.idle_add (if I'm happy to let GTK do it
 whenever it next feels like it), -OR-
   b. wrapped in gtk.gdk.threads_enter()/...leave(), -OR-
   c. in a with: gtk.gdk.lock: block (equivalent to 2.b)

Apart from 2.c, I guess
http://faq.pygtk.org/index.py?file=faq20.006.htpreq=show
is clear enough?

   3. I don't need to do the threads_enter/leave (or use the context
 manager) if I only use glib.idle_add (or timeout_add, etc)

No, you don't. If you don't really need threads, life is much simpler.

Cheers,

Pietro

___
pygtk mailing list   pygtk@daa.com.au
http://www.daa.com.au/mailman/listinfo/pygtk
Read the PyGTK FAQ: http://faq.pygtk.org/

Re: [pygtk] idle_add vs. threads_enter/threads_leave

[Neil Benn]

Hello,

I agree with that wholeheartedly.  I typical code in 3 or 4
languages and the other languages I use (C#, Java, etc) just have threads
which are easy to use.  When I was told to use twisted I looked at the
website and though 'phew that is a lot to learn'.  I was then about to drop
Python because I was worried that Python threading didn;t work properly
(because I was told to not use threads in Python as it is too difficult).  I
persevered however and threads work fine in Python - OK the GIL can make
things a little more complicated but threading in Python is not much harder
to use then in Java or C# (in fact because of the GIL and single processor I
don't have to worry about volatile variables).  Twisted has clear and
definite advantages but it is another framework I have to learn when I just
want to pop up a GUI and have it responsive while a db call is running in
the background.  My needs are not as great as the twisted
framework's learning hump.

Cheers,

Neil

On 5 July 2010 10:57, Jason Heeris jason.hee...@gmail.com wrote:

 On 5 July 2010 17:36, A.T.Hofkamp a.t.hofk...@tue.nl wrote:
  Or you could drop threads entirely, and do your async activities using
 the
  Twisted framework, designed for making asynchronous programs (where GTK
  event handling is just one of the asynchronous sources).

 Please don't take this as a flame, but I've had this advice before
 (usually on #python), and I usually go through this process:

 1. Look at Twisted website
 2. Only read things about twisted being a networking framework
 3. Someone corrects me, saying Twisted is so much more! (That may be
 the case, but the fact that the website goes on and on without
 mentioning this does not inspire confidence.)
 4. I search the Twisted website for documentation on/examples of GTK
 applications
 5. I give up and search the Twisted website for documentation
 on/examples of *any* non-network based applications
 6. I give up and search the greater web for documentation on/examples
 of Twisted GTK applications
 7. I give up on Twisted and find another way.

 Really, I would love to get a grip on Twisted. It seems really useful.
 But I've still not found any kind of stepping stone into it. Please,
 please, please if you know of one, post it so I can add another tool
 to my belt. I will rescind everything I've said here and then some.

 (Since the Twisted website is currently down, I can't tell if things
 have changed and I should get over it and have another try.)

 Cheers,
 Jason
 ___
 pygtk mailing list   pygtk@daa.com.au
 http://www.daa.com.au/mailman/listinfo/pygtk
 Read the PyGTK FAQ: http://faq.pygtk.org/




-- 
-- 

Neil Benn Msc
Director
Ziath Ltd
Phone :+44 (0)7508 107942
Website - http://www.ziath.com

IMPORTANT NOTICE:  This message, including any attached documents, is
intended only for the use of the individual or entity to which it is
addressed, and may contain information that is privileged, confidential and
exempt from disclosure under applicable law.  If the reader of this message
is not the intended recipient, or the employee or agent responsible for
delivering the message to the intended recipient, you are hereby notified
that any dissemination, distribution or copying of this communication is
strictly prohibited. If you have received this communication in error,
please notify Ziath Ltd immediately by email at i...@ziath.com. Thank you.
___
pygtk mailing list   pygtk@daa.com.au
http://www.daa.com.au/mailman/listinfo/pygtk
Read the PyGTK FAQ: http://faq.pygtk.org/

Re: [pygtk] idle_add vs. threads_enter/threads_leave

[Jason Heeris]

On 5 July 2010 18:05, Neil Benn neil.b...@ziath.com wrote:
 I persevered however and threads work fine in Python - OK the GIL can make
 things a little more complicated but threading in Python is not much harder
 to use then in Java or C# (in fact because of the GIL and single processor I
 don't have to worry about volatile variables).

Incidentally, typically I'll use Python's multiprocessing module, or
the Python-DBUS bindings (particulary the asynchronous features of
DBUS) if there's some really intensive work to be done. So it might be
worth a look for you too. Keep in mind, though, DBUS is emphatically
NOT optimised for throughput - it's not a good idea for pushing around
a lot of data (instead of, say, reading a lot of data and passing back
a summary). This is (presumably) where Twisted shines, because of its
origins as a networking framework.

- Jason
___
pygtk mailing list   pygtk@daa.com.au
http://www.daa.com.au/mailman/listinfo/pygtk
Read the PyGTK FAQ: http://faq.pygtk.org/

Re: [pygtk] idle_add vs. threads_enter/threads_leave

[Pietro Battiston]

Il giorno lun, 05/07/2010 alle 18.09 +0800, Jason Heeris ha scritto:
 On 5 July 2010 17:53, Pietro Battiston m...@pietrobattiston.it wrote:
  Apart from 2.c, I guess
  http://faq.pygtk.org/index.py?file=faq20.006.htpreq=show
  is clear enough?
 
 It's clear, except I can't tell if it applies to PyGObject 2.20 or
 2.21 (the idle_add function moves between glib and gobject depending
 on the version, but I have no idea at what version it first changed).
 

idle_add, timeout_add_defaults... moved to glib but you can still find
it in gobject in 2.21, I guess simply for backward compatibility.

And anyway, that doesn't concern threads: I didn't experience any break
in applications using threading that I wrote one year ago.

3. I don't need to do the threads_enter/leave (or use the context
  manager) if I only use glib.idle_add (or timeout_add, etc)
 
  No, you don't. If you don't really need threads, life is much simpler.
 
 Indeed. But a time consuming activity still locks the UI. As a trivial 
 example:
 
 
 def do_processing(self):
 import time
 time.sleep(10)
 self.emit('processing-done')
 
 def on_button_clicked(self, widget):
 glib.idle_add(self.do_processing)
 

Sure... in that case you _do_ really need threads... I simply don't know
what's your case. You wouldn't need threading if you could do:


def do_processing(self):
if self.counter  1000:
import time
time.sleep(.01)
return True
self.counter += 1
self.emit('processing-done')

def on_button_clicked(self, widget):
self.counter = 0
glib.idle_add(self.do_processing)


Notice the final effect is still sleeping 10 seconds.


 
 ...will still cause a 10 second hang when the GTK main loop gets
 around to running do_processing.
 
 Having said that, I'm trying to write something that runs on Linux and
 Windows. Since GTK threads under win32 are mysterious and feared,
 threads aren't really the answer *anyway*.
 

I don't know what are your needs: a nice solution could be the
multiprocess module.

But anyway, you find yourself really needing threading, I seem to
understand that using them but not making them directly interact with
the GUI is considered win-safe (though I never tested personally on
windows, that is the way I always used threads in pygtk).

Pietro

___
pygtk mailing list   pygtk@daa.com.au
http://www.daa.com.au/mailman/listinfo/pygtk
Read the PyGTK FAQ: http://faq.pygtk.org/

Re: [pygtk] idle_add vs. threads_enter/threads_leave

[Antoine Martin]

On 07/05/2010 04:57 PM, Jason Heeris wrote:
 On 5 July 2010 17:36, A.T.Hofkamp a.t.hofk...@tue.nl wrote:
 Or you could drop threads entirely, and do your async activities using the
 Twisted framework, designed for making asynchronous programs (where GTK
 event handling is just one of the asynchronous sources).
 
 Please don't take this as a flame, but I've had this advice before
 (usually on #python), and I usually go through this process:
 
 1. Look at Twisted website
 2. Only read things about twisted being a networking framework
 3. Someone corrects me, saying Twisted is so much more! (That may be
 the case, but the fact that the website goes on and on without
 mentioning this does not inspire confidence.)
 4. I search the Twisted website for documentation on/examples of GTK
 applications
 5. I give up and search the Twisted website for documentation
 on/examples of *any* non-network based applications
 6. I give up and search the greater web for documentation on/examples
 of Twisted GTK applications
 7. I give up on Twisted and find another way.
 
 Really, I would love to get a grip on Twisted. It seems really useful.
 But I've still not found any kind of stepping stone into it. Please,
 please, please if you know of one, post it so I can add another tool
 to my belt. I will rescind everything I've said here and then some.
I don't know of any examples unfortunately and I must admit that I spent
quite a bit of time getting to grips with it, but in the end it is
remarkably simple (much more simple than I first thought too - so don't
let that put you off):
import gtk.gdk
gtk.gdk.threads_init()
from twisted.internet import gtk2reactor, reactor
gtk2reactor.install(useGtk=useGtk)
reactor.run()

You can just use pygtk and twisted as you normally would: network events
are handled just like gui events, there is just one main loop waiting,
events get dispatched to whatever method you registered. (ie: via
connect for gtk, or reactor.XXX for twisted)

There are pretty good examples for using Twisted on their website (when
it is up). As for pygtk, finding examples shouldn't be a problem.

Now, for IO operations that can block (ie: db query, file operations),
just run those in a separate thread (thread.start_new_thread) and if you
need to interact with the GUI from that thread, do it via gobject.idle_add()

For someone who was used to Java/C#/C threading models, this is a
totally different approach and I must say that I really like the
simplicity of it all.

Hope this helps.

Antoine

 
 (Since the Twisted website is currently down, I can't tell if things
 have changed and I should get over it and have another try.)
 
 Cheers,
 Jason
 ___
 pygtk mailing list   pygtk@daa.com.au
 http://www.daa.com.au/mailman/listinfo/pygtk
 Read the PyGTK FAQ: http://faq.pygtk.org/

___
pygtk mailing list   pygtk@daa.com.au
http://www.daa.com.au/mailman/listinfo/pygtk
Read the PyGTK FAQ: http://faq.pygtk.org/

Re: [pygtk] idle_add vs. threads_enter/threads_leave

[Pietro Battiston]

Il giorno lun, 05/07/2010 alle 18.55 +0800, Jason Heeris ha scritto:
 On 5 July 2010 18:32, Pietro Battiston m...@pietrobattiston.it wrote:
  You wouldn't need threading if you could do:
 
  def do_processing(self):
 if self.counter  1000:
 import time
 time.sleep(.01)
 return True
 self.counter += 1
 self.emit('processing-done')
 
 Well... this in itself wouldn't work, since anything after the
 return would be skipped.

Uhm sure.

 But I hadn't thought of this before — you
 could make this into a generator (replace return with yield), and
 somehow wrap it in a for loop that did what idle_add expects. But
 now we're getting kind of close to rolling our own threading
 implementation, which seems a bit hacky.

Personally, I find that when the lenghty operation can be fragmented in
small enough parts, this is not hacky, this is perfectly natural (much
more than having threads or even other event loops misteriously flowing
around).

When the operation _can't_ be fragmented, again it's not hacky, it's
just impossible.


If you want an example, my gvb app, which you can find in
Debian/Ubuntu, doesn't use threading at all (and multiprocess neither).
That's a problem in the initializations, which take a lot of time and
block the UI, but for drawing a frame at a time it's perfect - and
notice frames of the simulation _don't necessarily correspond_ to
frames rendered to screen.

Pietro

___
pygtk mailing list   pygtk@daa.com.au
http://www.daa.com.au/mailman/listinfo/pygtk
Read the PyGTK FAQ: http://faq.pygtk.org/

Re: [pygtk] idle_add vs. threads_enter/threads_leave

[A.T.Hofkamp]

Jason Heeris wrote:
 On 5 July 2010 17:36, A.T.Hofkamp a.t.hofk...@tue.nl wrote:
 Or you could drop threads entirely, and do your async activities using the
 Twisted framework, designed for making asynchronous programs (where GTK
 event handling is just one of the asynchronous sources).
 
 Please don't take this as a flame, but I've had this advice before
 (usually on #python), and I usually go through this process:

I was mostly triggered by the fact that you are doing asynchronous activities 
next to the GTK 
application, which are apparently complicated enough to use threads for.


 1. Look at Twisted website
 2. Only read things about twisted being a networking framework

Network communication is *by* *far* the most complicated form of asynchronous 
communication 
(anything can go wrong at any moment). Obviously, a framework for asynchronous 
communication should 
put the focus there.

About not mentioning guis:
If you write a spread sheet program, and explain its features, you don't 
explain that you can add 
two numbers, do you?

 3. Someone corrects me, saying Twisted is so much more! (That may be
 the case, but the fact that the website goes on and on without
 mentioning this does not inspire confidence.)

Twisted is technically very good, but their PR is not, let's say, optimal.

Combined with the (at first) twisted way of doing things (namely event-based 
programming in the 
extreme), you are looking at a very steep learning curve.

I completely agree with you that this is a weak point.
I found the best source of information to be their howtos and 
tutorials/examples, to get the global 
picture, and the API documentation (for the protocols, mostly).

 4. I search the Twisted website for documentation on/examples of GTK
 applications

Basically, asynchronous communication means you are doing event-based 
programming everywhere.

You are constantly waiting for something of interest to happen (the Deferreds 
in Twisted-speak). 
When the event does happen, the handler is triggered, which handles the event, 
and sets up new 
Deferreds (new handlers) for new events that may happen.

 From this point of view, it is totally irrelevant what event you get (a 
network comm event, a GUI 
event, or something from another source), only the actual handler of the event 
needs to know details 
of the event.
The Twisted main-loop (reactor) only needs to know that it should watch for 
GTK+ events too, so you 
need a gtk2reactor instead of a normal one.

See also

http://twistedmatrix.com/documents/current/core/howto/choosing-reactor.html#auto11

That page also refers to an example doc/core/examples/pbgtk2.py


 From a complexity point of view, there is very little that can go wrong with 
event handling in a 
GTK gui, so little need to discuss things at length.


 5. I give up and search the Twisted website for documentation
 on/examples of *any* non-network based applications

As I tried to explain above, to Twisted *everything* is an event that is waited 
for, and when it 
happens, it should be handled.

There is no difference between network-based events and non-network-based 
events. They are all done 
using Deferreds that fire when the event happens.

In the same way, Twisted does not care about the comm protocol use to deliver 
the event. Whether a 
user pressed a button at a web-form, or sent a IRC message does not matter for 
computing the response.

 6. I give up and search the greater web for documentation on/examples
 of Twisted GTK applications
 7. I give up on Twisted and find another way.
 
 Really, I would love to get a grip on Twisted. It seems really useful.
 But I've still not found any kind of stepping stone into it. Please,
 please, please if you know of one, post it so I can add another tool
 to my belt. I will rescind everything I've said here and then some.

I'd recommend to get to grips with the Deferreds using a communication protocol 
example, as those 
are around. Once you get how events are done, apply it to gtk events.

 (Since the Twisted website is currently down, I can't tell if things
 have changed and I should get over it and have another try.)

It's up again (or was, at least). It has not changed much, at first sight.

I am not sure why there is so little nice introductory information. I think it 
is because the gap 
between both worlds is so big. Either you don't get it, and then you cannot 
write about it due to 
lack of understanding, or you do get it, and then you cannot imagine how to 
explain it to somebody 
that doesn't.

(Like programming, can you explain the essence of what programming is, to 
somebody that has no clue 
about computers?)

Albert
___
pygtk mailing list   pygtk@daa.com.au
http://www.daa.com.au/mailman/listinfo/pygtk
Read the PyGTK FAQ: http://faq.pygtk.org/

Re: [pygtk] idle_add vs. threads_enter/threads_leave

[Jason Heeris]

On 5 July 2010 21:48, A.T.Hofkamp a.t.hofk...@tue.nl wrote:
 I was mostly triggered by the fact that you are doing asynchronous
 activities next to the GTK application, which are apparently complicated
 enough to use threads for.

It's not complicated, there's simply a time consuming operation involved.

 About not mentioning guis:
 If you write a spread sheet program, and explain its features, you don't
 explain that you can add two numbers, do you?

Yes, but if it can do arbitrary precision arithmetic, you might
mention that feature, even though it's *obviously* a *trivial*
extension of the ability to do batch computation :P

It's somewhat counter-intuitive to have a module named
twisted.internet that can be used for things that have nothing to do
with the internet. It makes a potential programmer think, I don't
really want a full blown internet stack just for a quick and simple
desktop app. Just like I don't open up a spreadsheet just to add two
numbers, even though it's technically capable of that.

But fine, it's just a name, I can ignore it.

 I completely agree with you that this is a weak point.
 I found the best source of information to be their howtos and
 tutorials/examples, to get the global picture, and the API documentation
 (for the protocols, mostly).

I'll poke around, thanks.

 Basically, asynchronous communication means you are doing event-based
 programming everywhere.

I like that idea. I understand the principle. It's mainly how I use
GTK, where possible.

 From this point of view, it is totally irrelevant what event you get (a
 network comm event, a GUI event, or something from another source), only the
 actual handler of the event needs to know details of the event.

Yes, but the *details* are what I need to know. How do I write a file
processor model: something that takes a filename, returns a initial
status immediately, and then X minutes later returns the result of
processing the file? In fact, ignore that, how do I write a class that
does time.sleep(100) and then sends a signal saying that it's done?

 http://twistedmatrix.com/documents/current/core/howto/choosing-reactor.html#auto11

 That page also refers to an example doc/core/examples/pbgtk2.py

...which still seems to involve some sort of network communication,
which is extra noise that just gets in the way of understanding the
GTK interaction.

 In the same way, Twisted does not care about the comm protocol use to
 deliver the event. Whether a user pressed a button at a web-form, or sent a
 IRC message does not matter for computing the response.

If Twisted doesn't care about the protocol, why can't I find a single
example that does not involve a network-based protocol?

 I am not sure why there is so little nice introductory information. I think
 it is because the gap between both worlds is so big. Either you don't get
 it, and then you cannot write about it due to lack of understanding, or you
 do get it, and then you cannot imagine how to explain it to somebody that
 doesn't.

It's not about getting event based programming. Take this very
email, for example. You go on at length about the architectural
principles of Twisted. But in a tenth of the space I could have
written an illustrative example of using GTK+ with Python threads to
do asynchronous activity (now). Why can't anyone just do that with
Twisted instead of writing essays?

What letters do I type to get anything done? Nothing I have read so
far has given me any hint about this.

 (Like programming, can you explain the essence of what programming is, to
 somebody that has no clue about computers?)

You could actually show them a computer, and show them the things it
can do, and then show them how to do the things they want to do with
it. That's how I would start.

Thanks for the pointers though. If I ever find the time, I'll try it out.

— Jason
___
pygtk mailing list   pygtk@daa.com.au
http://www.daa.com.au/mailman/listinfo/pygtk
Read the PyGTK FAQ: http://faq.pygtk.org/

Re: [pygtk] idle_add vs. threads_enter/threads_leave

[Tim Evans]

On 2010-07-05 20:32, Jason Heeris wrote:
2. Any GTK interaction, such as emitting a signal from this new
 thread, must be either:
a. done via glib.idle_add (if I'm happy to let GTK do it
 whenever it next feels like it), -OR-
b. wrapped in gtk.gdk.threads_enter()/...leave(), -OR-
c. in a with: gtk.gdk.lock: block (equivalent to 2.b)

Something worth noting is that if you're targeting Windows then 2.a. is 
your *only* option. Calling GTK+ functions from any thread other than 
the main thread will lock up on Windows. The win32 api handles threads 
quite differently from X11 and GTK's abstraction doesn't work there.

I would also point out that whenever it next feels like it is almost 
always going to be the same as right away unless you're doing 
something yourself that still blocks the main thread. If you don't want 
your processing thread to continue running while the idle handler gets 
queued and processes it's pretty easy to wait for it. Untested code:

def idle_add_and_wait(func, *args):
 event = threading.Event()
 def idle():
 try:
 func(*args)
 return False
 finally:
 event.set()
 gobject.idle_add(idle)
 event.wait()

-- 
Tim Evans
Applied Research Associates NZ
http://www.aranz.com/
___
pygtk mailing list   pygtk@daa.com.au
http://www.daa.com.au/mailman/listinfo/pygtk
Read the PyGTK FAQ: http://faq.pygtk.org/

Re: [pygtk] idle_add vs. threads_enter/threads_leave

[Jason Heeris]

On 5 July 2010 17:53, Pietro Battiston m...@pietrobattiston.it wrote:
 Il giorno lun, 05/07/2010 alle 16.32 +0800, Jason Heeris ha scritto:
   3. I don't need to do the threads_enter/leave (or use the context
 manager) if I only use glib.idle_add (or timeout_add, etc)

 No, you don't. If you don't really need threads, life is much simpler.

Aaaah, I've just discovered that using only glib.idle_add on Windows
does *not* mean that things get updated at the next available
oppurtunity. They only get updated when the widget gets redrawn (I
think... or maybe when the user interacts with the UI). Maybe this is
a known issue.

— Jason
___
pygtk mailing list   pygtk@daa.com.au
http://www.daa.com.au/mailman/listinfo/pygtk
Read the PyGTK FAQ: http://faq.pygtk.org/