Re: How to use two threads (GUI and backend)

[D'Arcy Cain]

On 2016-10-27 07:33 AM, jmp wrote:

On 10/27/2016 12:22 PM, pozz wrote:

(blocking) thread. The blocking function read returns *immediately* when
all the bytes are received.  And I think during blocking time, the
thread isn't consuming CPU clocks.


Threads do consume CPU clocks.


Sometimes they do but read what pozz wrote.  "during blocking time, the 
thread isn't consuming CPU clocks".  That means that if you have two 
threads with one computing π and the other blocked waiting for user 
input then the first one is using CPU cycles - as many as it can get - 
and the other is using none until the user enters something.  This is 
kind of the point of threads - only things that need cycles get cycles.



An operation within a thread will not consume less CPU clocks, however,


It will if it blocks for something.  A simple sleep will also give up 
the processor.



From what I understand of your context, you don't want you GUI to
"freeze" when waiting for the remote application. That's a valid concern.


And that's why you use threads.  You can also use select in a state 
machine loop.  Depends on your model and somewhat on your preferences.



What you should not do is focus on gaining "CPU clocks". You just don't
care. It's probably not an issue. If it is, drop python and implement
your app in C.


In fact, going to C often is less of a win than you may think.  Python 
is pretty damn efficient if you write good code.


--
D'Arcy J.M. Cain
System Administrator, Vex.Net
http://www.Vex.Net/ IM:da...@vex.net
VoIP: sip:da...@vex.net
--
https://mail.python.org/mailman/listinfo/python-list

Re: How to use two threads (GUI and backend)

[jmp]

On 10/27/2016 02:55 PM, Chris Angelico wrote:

On Thu, Oct 27, 2016 at 11:33 PM, jmp  wrote:

On 10/27/2016 01:43 PM, Chris Angelico wrote:


Blocked threads don't consume CPU time. Why would they?

ChrisA



Agreed. My point being that a blocked thread achieve nothing, except
parallelism, i.e. other threads can be processed.

To be more specific, if you compute factorial(51354) in a thread, it will
still require approx. the same amount of CPU clocks than in a main thread
(probably slightly more due to the scheduler overhead).

jm


Of course. But the OP wants to do blocking calls, which don't cost you
like that. So it's fine.

ChrisA


Sure but the OP is very focus on performance(that's a mistake imo).

"Because I don't want to drop python, I want to learn the best technique 
to use to have the best performance. "


I just wanted to point that using thread implements parallelism, not 
performance. And that's probably what its gui needs. And that's probably 
why using a higher level API would have been acceptable.


JM

Best performance is achieved by sacrificing a lot in python. A better 
technique than polling threads would be sleeping thread where the thread 
is put to hold until a hardware interrupt wakes up the thread.







--
https://mail.python.org/mailman/listinfo/python-list

Re: How to use two threads (GUI and backend)

[Chris Angelico]

On Thu, Oct 27, 2016 at 11:33 PM, jmp  wrote:
> On 10/27/2016 01:43 PM, Chris Angelico wrote:
>>
>> Blocked threads don't consume CPU time. Why would they?
>>
>> ChrisA
>>
>
> Agreed. My point being that a blocked thread achieve nothing, except
> parallelism, i.e. other threads can be processed.
>
> To be more specific, if you compute factorial(51354) in a thread, it will
> still require approx. the same amount of CPU clocks than in a main thread
> (probably slightly more due to the scheduler overhead).
>
> jm

Of course. But the OP wants to do blocking calls, which don't cost you
like that. So it's fine.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: How to use two threads (GUI and backend)

[jmp]

On 10/27/2016 01:43 PM, Chris Angelico wrote:

Blocked threads don't consume CPU time. Why would they?

ChrisA



Agreed. My point being that a blocked thread achieve nothing, except 
parallelism, i.e. other threads can be processed.


To be more specific, if you compute factorial(51354) in a thread, it 
will still require approx. the same amount of CPU clocks than in a main 
thread (probably slightly more due to the scheduler overhead).


jm


--
https://mail.python.org/mailman/listinfo/python-list

Re: How to use two threads (GUI and backend)

[Chris Angelico]

On Thu, Oct 27, 2016 at 10:56 PM, pozz  wrote:
> Yes of course, but when the backend thread calls the *blocking* function
> pyserial.read(), it *doesn't* consume CPU clocks (at least, I hope).
> The low-level implementation of pyserial.read() should move the thread in a
> "suspend" or "waiting" state, so the thread scheduler shouldn't activate it.
> The suspend state is exited (automatically from OS, I think) when one or
> more bytes are ready in the input buffer.

Exactly.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: How to use two threads (GUI and backend)

[pozz]

Il 27/10/2016 13:33, jmp ha scritto:

On 10/27/2016 12:22 PM, pozz wrote:

Anyway I don't like this approach, because the main (and single) thread
should check in_waiting every X milliseconds.
If X is too high, I could wait for the answer even if it is already
ready in the input buffer.
If X is too low, the application consumes a lot of clocks to check
in_waiting.

I would prefer to have a callback automatically called when the read
operation is complete.  And I think the only method is using another
(blocking) thread. The blocking function read returns *immediately* when
all the bytes are received.  And I think during blocking time, the
thread isn't consuming CPU clocks.


Threads do consume CPU clocks.
An operation within a thread will not consume less CPU clocks, however,
the scheduler will interrupt the thread and give other
threads/operations a chance to process as well.
Threads implement paralellism, not performances.


Yes of course, but when the backend thread calls the *blocking* function 
pyserial.read(), it *doesn't* consume CPU clocks (at least, I hope).
The low-level implementation of pyserial.read() should move the thread 
in a "suspend" or "waiting" state, so the thread scheduler shouldn't 
activate it. The suspend state is exited (automatically from OS, I 
think) when one or more bytes are ready in the input buffer.




From what I understand of your context, you don't want you GUI to
"freeze" when waiting for the remote application. That's a valid concern.

You can use threads to fix that(or you can use already written working
python libraries that would mask this low level programing, it's up to
you).

What you should not do is focus on gaining "CPU clocks". You just don't
care. It's probably not an issue. If it is, drop python and implement
your app in C.


Because I don't want to drop python, I want to learn the best tecnique 
to use to have the best performance.


--
https://mail.python.org/mailman/listinfo/python-list

Re: How to use two threads (GUI and backend)

[Chris Angelico]

On Thu, Oct 27, 2016 at 10:33 PM, jmp  wrote:
> On 10/27/2016 12:22 PM, pozz wrote:
>>
>> Anyway I don't like this approach, because the main (and single) thread
>> should check in_waiting every X milliseconds.
>> If X is too high, I could wait for the answer even if it is already
>> ready in the input buffer.
>> If X is too low, the application consumes a lot of clocks to check
>> in_waiting.
>>
>> I would prefer to have a callback automatically called when the read
>> operation is complete.  And I think the only method is using another
>> (blocking) thread. The blocking function read returns *immediately* when
>> all the bytes are received.  And I think during blocking time, the
>> thread isn't consuming CPU clocks.
>
>
> Threads do consume CPU clocks.
> An operation within a thread will not consume less CPU clocks, however, the
> scheduler will interrupt the thread and give other threads/operations a
> chance to process as well.
> Threads implement paralellism, not performances.

Blocked threads don't consume CPU time. Why would they?

Remember, folks, *EVERY* program has at least one thread. Threads
aren't some weird and magical thing that you have to be scared of.
They're things you use *all the time*, and they are exactly what
you're used to.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: How to use two threads (GUI and backend)

[jmp]

On 10/27/2016 12:22 PM, pozz wrote:

Anyway I don't like this approach, because the main (and single) thread
should check in_waiting every X milliseconds.
If X is too high, I could wait for the answer even if it is already
ready in the input buffer.
If X is too low, the application consumes a lot of clocks to check
in_waiting.

I would prefer to have a callback automatically called when the read
operation is complete.  And I think the only method is using another
(blocking) thread. The blocking function read returns *immediately* when
all the bytes are received.  And I think during blocking time, the
thread isn't consuming CPU clocks.


Threads do consume CPU clocks.
An operation within a thread will not consume less CPU clocks, however, 
the scheduler will interrupt the thread and give other 
threads/operations a chance to process as well.

Threads implement paralellism, not performances.

From what I understand of your context, you don't want you GUI to 
"freeze" when waiting for the remote application. That's a valid concern.


You can use threads to fix that(or you can use already written working 
python libraries that would mask this low level programing, it's up to you).


What you should not do is focus on gaining "CPU clocks". You just don't 
care. It's probably not an issue. If it is, drop python and implement 
your app in C.


JM


--
https://mail.python.org/mailman/listinfo/python-list

Re: How to use two threads (GUI and backend)

[Demosthenes Koptsis]

Here is an example about threads and PyQT

https://www.youtube.com/watch?v=ivcxZSHL7jM=2



On 10/27/2016 01:22 PM, pozz wrote:

Il 26/10/2016 16:18, jmp ha scritto:

On 10/26/2016 02:45 PM, pozz wrote:

Il 26/10/2016 13:16, jmp ha scritto:

[...]
I suggest you write a GUI that make synchronouscalls to a remote
application, if possible. If the remote app is in python, you have
access to remote protocols already written for you, Pyro is one of 
them,

you can skip the low level communication part.


I'm not sure Pyro (or similar alternatives) helps in my case.

The real problem is that retrieving status from remote device is a slow
operation.  If the GUI thread blocks waiting for the answer, the GUI
blocks and the user complains.

 From Pyro documentation:
---
Normal method calls always block until the response is returned. This
can be any normal return value, None, or an error in the form of a
raised exception. The client code execution is suspended until the
method call has finished and produced its result.
---

So, even with Pyro, I need to have another thread that manages Pyro
communication (instead of serial communication)... additional problems.



Also from the Pyro doc:

You can execute a remote method call and tell Pyro: “hey, I don’t need
the results right now. Go ahead and compute them, I’ll come back later
once I need them”. The call will be processed in the background and you
can collect the results at a later time.

[...]

It is possible to define one or more callables (the “call chain”) that
should be invoked automatically by Pyro as soon as the result value
becomes available.


I already read that, it is the feature "Asynchronous ('future') remote 
calls & call chains".


This approach can be taken also without pyro at all, just using 
pyserial module (and I think all the communication libraries).

With pyserial, I can set a read timeout value of zero:

 timeout = 0: non-blocking mode, return immediately in
 any case, returning zero or more, up to the requested
 number of bytes

In this way, I can implement exactly the same mechanism of pyro in 
asyncronous mode.  With pyserial I could avoid setting timeout=0, 
using in_waiting property ("number of bytes in the input buffer").


Anyway I don't like this approach, because the main (and single) 
thread should check in_waiting every X milliseconds.
If X is too high, I could wait for the answer even if it is already 
ready in the input buffer.
If X is too low, the application consumes a lot of clocks to check 
in_waiting.


I would prefer to have a callback automatically called when the read 
operation is complete.  And I think the only method is using another 
(blocking) thread. The blocking function read returns *immediately* 
when all the bytes are received.  And I think during blocking time, 
the thread isn't consuming CPU clocks.


I could try with asyncio feature of pyserial, but it is classified as 
"experimental".





--
https://mail.python.org/mailman/listinfo/python-list

Re: How to use two threads (GUI and backend)

[pozz]

Il 26/10/2016 16:18, jmp ha scritto:

On 10/26/2016 02:45 PM, pozz wrote:

Il 26/10/2016 13:16, jmp ha scritto:

[...]
I suggest you write a GUI that make synchronouscalls to a remote
application, if possible. If the remote app is in python, you have
access to remote protocols already written for you, Pyro is one of them,
you can skip the low level communication part.


I'm not sure Pyro (or similar alternatives) helps in my case.

The real problem is that retrieving status from remote device is a slow
operation.  If the GUI thread blocks waiting for the answer, the GUI
blocks and the user complains.

 From Pyro documentation:
---
Normal method calls always block until the response is returned. This
can be any normal return value, None, or an error in the form of a
raised exception. The client code execution is suspended until the
method call has finished and produced its result.
---

So, even with Pyro, I need to have another thread that manages Pyro
communication (instead of serial communication)... additional problems.



Also from the Pyro doc:

You can execute a remote method call and tell Pyro: “hey, I don’t need
the results right now. Go ahead and compute them, I’ll come back later
once I need them”. The call will be processed in the background and you
can collect the results at a later time.

[...]

It is possible to define one or more callables (the “call chain”) that
should be invoked automatically by Pyro as soon as the result value
becomes available.


I already read that, it is the feature "Asynchronous ('future') remote 
calls & call chains".


This approach can be taken also without pyro at all, just using pyserial 
module (and I think all the communication libraries).

With pyserial, I can set a read timeout value of zero:

 timeout = 0: non-blocking mode, return immediately in
 any case, returning zero or more, up to the requested
 number of bytes

In this way, I can implement exactly the same mechanism of pyro in 
asyncronous mode.  With pyserial I could avoid setting timeout=0, using 
in_waiting property ("number of bytes in the input buffer").


Anyway I don't like this approach, because the main (and single) thread 
should check in_waiting every X milliseconds.
If X is too high, I could wait for the answer even if it is already 
ready in the input buffer.
If X is too low, the application consumes a lot of clocks to check 
in_waiting.


I would prefer to have a callback automatically called when the read 
operation is complete.  And I think the only method is using another 
(blocking) thread. The blocking function read returns *immediately* when 
all the bytes are received.  And I think during blocking time, the 
thread isn't consuming CPU clocks.


I could try with asyncio feature of pyserial, but it is classified as 
"experimental".



--
https://mail.python.org/mailman/listinfo/python-list

Re: How to use two threads (GUI and backend)

[Marko Rauhamaa]

Chris Angelico :
> Python-the-language doesn't permit those kinds of rewrites.

[Citation needed]

Is there something here, perhaps?

   https://docs.python.org/3/library/concurrency.html>


Marko
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: How to use two threads (GUI and backend)

[Chris Angelico]

On Thu, Oct 27, 2016 at 1:42 AM, Marko Rauhamaa  wrote:
> Chris Angelico :
>> And since Python doesn't rewrite the code, you don't have a problem.
>
> Do you mean Python or CPython?
>
> And how do you know?

Both, and I know because Python-the-language doesn't permit those
kinds of rewrites. PyPy does do a whole lot of optimization, but it
very carefully ensures that semantics are retained with reference to
object identities and execution order and so on.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: How to use two threads (GUI and backend)

[Marko Rauhamaa]

Chris Angelico :
> And since Python doesn't rewrite the code, you don't have a problem.

Do you mean Python or CPython?

And how do you know?


Marko
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: How to use two threads (GUI and backend)

[Chris Angelico]

On Thu, Oct 27, 2016 at 1:21 AM, Marko Rauhamaa  wrote:
> Analogous code in C or Java would not be guaranteed to finish if func1()
> and func2() were in different execution contexts. In fact, it would be
> almost guaranteed to hang.
>
> That is because the compiler can see that "active" cannot change within
> T1.run() and would rewrite the code...

And since Python doesn't rewrite the code, you don't have a problem.
Optimization has costs.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: How to use two threads (GUI and backend)

[Marko Rauhamaa]

Chris Angelico :

> On Thu, Oct 27, 2016 at 12:37 AM, Marko Rauhamaa  wrote:
>> I don't know what "Global state is shared across all threads" means
>> in this context. It sounds like something that would be true for,
>> say, Java and C as well. However, those languages don't promise to
>> propagate improperly synchronized changes between threads.
>>
>> Now I would like to ask for some documentation.
>
> Here you have two functions and a global:
>
> active = True
>
> def func1():
> while active:
> # do work
> time.sleep(1)
> func2()
>
> def func2():
> global active
> if random.random() < 0.1:
> active = False

The thread version:


   active = True

   class T1(threading.Thread):
   def run(self):
   while active:
   # do work
   time.sleep(1)

   class T2(threading.Thread):
   def run(self):
   global active
   if random.random() < 0.1:
   active = False

   t1, t2 = T1(), T2()
   t1.start()
   t2.start()
   t1.join()
   t2.join


Analogous code in C or Java would not be guaranteed to finish if func1()
and func2() were in different execution contexts. In fact, it would be
almost guaranteed to hang.

That is because the compiler can see that "active" cannot change within
T1.run() and would rewrite the code as:


   class T1(threading.Thread):
   def run(self):
   if active:
   while True:
   # do work
   time.sleep(1)


Similarly, setting a flag in a signal handler might not be noticed by
the main program if it were written in C. (You need to mark the variable
as volatile.)

> I'm sure you understand that these functions share the global state of
> the 'active' flag. One changes it, the other sees the change. So far,
> nothing controversial or difficult.
>
> It's exactly the same with threads.

Extraordinary claims require extraordinary evidence.

   http://stackoverflow.com/questions/3549833/python-threading-memo
   ry-model-and-visibility>

Now, Jython developers guarantee the volatility of all memory access.
They even state this as normative for Python:

   The fundamental thing to know about Python, and what we have
   implemented in Jython, is that setting any attribute in Python is a
   volatile write; and getting any attribute is a volatile read.

   [...]

   This means that safe publication is pretty much trivial in Python,
   when compared to Java. Safe publication means the thread safe
   association of an object with a name. [...] this is always a
   memory-fenced operation in Python

   http://www.jython.org/jythonbook/en/1.0/Concurrency.html>


Marko
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: How to use two threads (GUI and backend)

[jmp]

On 10/26/2016 02:45 PM, pozz wrote:

Il 26/10/2016 13:16, jmp ha scritto:

[...]
I suggest you write a GUI that make synchronouscalls to a remote
application, if possible. If the remote app is in python, you have
access to remote protocols already written for you, Pyro is one of them,
you can skip the low level communication part.


I'm not sure Pyro (or similar alternatives) helps in my case.

The real problem is that retrieving status from remote device is a slow
operation.  If the GUI thread blocks waiting for the answer, the GUI
blocks and the user complains.

 From Pyro documentation:
---
Normal method calls always block until the response is returned. This
can be any normal return value, None, or an error in the form of a
raised exception. The client code execution is suspended until the
method call has finished and produced its result.
---

So, even with Pyro, I need to have another thread that manages Pyro
communication (instead of serial communication)... additional problems.



Also from the Pyro doc:

You can execute a remote method call and tell Pyro: “hey, I don’t need 
the results right now. Go ahead and compute them, I’ll come back later 
once I need them”. The call will be processed in the background and you 
can collect the results at a later time.


[...]

It is possible to define one or more callables (the “call chain”) that 
should be invoked automatically by Pyro as soon as the result value 
becomes available.


jm

--
https://mail.python.org/mailman/listinfo/python-list

Re: How to use two threads (GUI and backend)

[Chris Angelico]

On Thu, Oct 27, 2016 at 12:37 AM, Marko Rauhamaa  wrote:
> Chris Angelico :
>
>> On Wed, Oct 26, 2016 at 11:58 PM, Marko Rauhamaa  wrote:
>>> I can't think of a valid program that could take advantage of this
>>> primitive guarantee of Python's. For example, there is no "volatile"
>>> in Python so you can't coordinate Python threads safely without
>>> proper synchronization. If you set a variable in one thread and read
>>> it in another thread, the latter might never see the change.
>>
>> Incorrect. If you set something in one thread and read it in another,
>> it WILL see it, just as it would with any other way of running two
>> functions. (Obviously function locals won't be seen, because they
>> never will.) Global state is shared across all threads.
>
> I don't know what "Global state is shared across all threads" means in
> this context. It sounds like something that would be true for, say, Java
> and C as well. However, those languages don't promise to propagate
> improperly synchronized changes between threads.
>
> Now I would like to ask for some documentation.

Here you have two functions and a global:


active = True

def func1():
while active:
# do work
time.sleep(1)
func2()

def func2():
global active
if random.random() < 0.1:
active = False


I'm sure you understand that these functions share the global state of
the 'active' flag. One changes it, the other sees the change. So far,
nothing controversial or difficult.

It's exactly the same with threads. If you remove the func2() call
from func1 and have it operate on a separate thread, and then call
func2 from the main thread (or another secondary thread), func1 will
notice the change the very next time it gets to the top of the loop.
The two functions are executing in the same process, the same module,
the same everything except their call stack (ie locals).

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: How to use two threads (GUI and backend)

[Marko Rauhamaa]

Chris Angelico :

> On Wed, Oct 26, 2016 at 11:58 PM, Marko Rauhamaa  wrote:
>> I can't think of a valid program that could take advantage of this
>> primitive guarantee of Python's. For example, there is no "volatile"
>> in Python so you can't coordinate Python threads safely without
>> proper synchronization. If you set a variable in one thread and read
>> it in another thread, the latter might never see the change.
>
> Incorrect. If you set something in one thread and read it in another,
> it WILL see it, just as it would with any other way of running two
> functions. (Obviously function locals won't be seen, because they
> never will.) Global state is shared across all threads.

I don't know what "Global state is shared across all threads" means in
this context. It sounds like something that would be true for, say, Java
and C as well. However, those languages don't promise to propagate
improperly synchronized changes between threads.

Now I would like to ask for some documentation.


Marko
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: How to use two threads (GUI and backend)

[Chris Angelico]

On Thu, Oct 27, 2016 at 12:02 AM, Marko Rauhamaa  wrote:
> pozz :
>
>> The real problem is that retrieving status from remote device is a
>> slow operation. If the GUI thread blocks waiting for the answer, the
>> GUI blocks and the user complains.
>
> Correct. Obnoxious, blocking APIs abound.
>
> However, I have usually used processes (instead of threads) to
> encapsulate blocking APIs. Processes have neater resource isolation and
> a better-behaving life cycle. For example, you can actually kill a
> process while you can't kill a thread.

Why is there so much FUD against threads? Processes involve a lot more
overhead, and are completely unnecessary for this task. Threads will
work just fine.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: How to use two threads (GUI and backend)

[Marko Rauhamaa]

pozz :

> The real problem is that retrieving status from remote device is a
> slow operation. If the GUI thread blocks waiting for the answer, the
> GUI blocks and the user complains.

Correct. Obnoxious, blocking APIs abound.

However, I have usually used processes (instead of threads) to
encapsulate blocking APIs. Processes have neater resource isolation and
a better-behaving life cycle. For example, you can actually kill a
process while you can't kill a thread.


Marko
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: How to use two threads (GUI and backend)

[Chris Angelico]

On Wed, Oct 26, 2016 at 11:58 PM, Marko Rauhamaa  wrote:
> In practice, this coherency has been implemented in CPython with a
> global lock (GIL). CPython programs are effectively single-threaded.
> They only let go of the lock when they are performing a system call.
>
> I can't think of a valid program that could take advantage of this
> primitive guarantee of Python's. For example, there is no "volatile" in
> Python so you can't coordinate Python threads safely without proper
> synchronization. If you set a variable in one thread and read it in
> another thread, the latter might never see the change.

Incorrect. If you set something in one thread and read it in another,
it WILL see it, just as it would with any other way of running two
functions. (Obviously function locals won't be seen, because they
never will.) Global state is shared across all threads.

pozz, a classic worker model like you're suggesting will be fine IMO. Go for it.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: How to use two threads (GUI and backend)

[Marko Rauhamaa]

pozz :

> Il 26/10/2016 13:27, Antoon Pardon ha scritto:
>> Op 26-10-16 om 12:22 schreef pozz:
>>> Is it safe to access this variable from two different threads?
>>> Should I implement a safer and more complex mechanism? If yes, what
>>> mechanism?
>>
>> Accessing from multiple thread shouldn't be a problem. As long as you
>> only change it in one thread.
>
> I don't want to doubt what you have written, but... are you
> definitevely sure? I tried to search for some authoritative
> documentation about this topic, but I couldn't find any.

I didn't check but I would guess you are right. That kind of
authoritative statement is not made explicitly. Java, on the other hand,
has been documented nicely:

  https://docs.oracle.com/javase/specs/jls/se7/html/jls-17.html#jl
  s-17.4.5>

> If the main loop is updating the variable from 0x01020304 to
> 0xA1A2A3A4 and the change happens on a byte basis, the ISR could
> access a completely wrong value, for example 0x0102A3A4.
>
> So the main question here is: does python *specification/standard*
> guarantees atomic operations? If yes, what are they?

Python guarantees that even a pathological Python application program
that only employs ordinary, safe operations cannot crash Python. It
follows (de facto) that "pointers" must be protected against race
conditions and other artifacts. Similarly, you can't render lists, dicts
and other complex data structures incoherent with any ordinary means
regardless of race conditions.

Since Python's integers are (really or conceptually) objects behind
pointers, any Python implementation would be considered out of
compliance if it didn't guarantee either 0x01020304 or 0xa1a2a3a4 in
your example.

In practice, this coherency has been implemented in CPython with a
global lock (GIL). CPython programs are effectively single-threaded.
They only let go of the lock when they are performing a system call.

I can't think of a valid program that could take advantage of this
primitive guarantee of Python's. For example, there is no "volatile" in
Python so you can't coordinate Python threads safely without proper
synchronization. If you set a variable in one thread and read it in
another thread, the latter might never see the change.


Marko
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: How to use two threads (GUI and backend)

[pozz]

Il 26/10/2016 13:16, jmp ha scritto:

[...]
I suggest you write a GUI that make synchronouscalls to a remote
application, if possible. If the remote app is in python, you have
access to remote protocols already written for you, Pyro is one of them,
you can skip the low level communication part.


I'm not sure Pyro (or similar alternatives) helps in my case.

The real problem is that retrieving status from remote device is a slow 
operation.  If the GUI thread blocks waiting for the answer, the GUI 
blocks and the user complains.


From Pyro documentation:
---
Normal method calls always block until the response is returned. This 
can be any normal return value, None, or an error in the form of a 
raised exception. The client code execution is suspended until the 
method call has finished and produced its result.

---

So, even with Pyro, I need to have another thread that manages Pyro 
communication (instead of serial communication)... additional problems.


--
https://mail.python.org/mailman/listinfo/python-list

Re: How to use two threads (GUI and backend)

[pozz]

Il 26/10/2016 13:27, Antoon Pardon ha scritto:

Op 26-10-16 om 12:22 schreef pozz:

Il 26/10/2016 09:13, pozz ha scritto:

[...]
When the user press Start button (the pressed handler is in the GUI
class):

  self.comm_active = True
  threading.Thread(target=self.comm_thread).start()

The backend thread is:

  def comm_thread(self):
while self.comm_active:
  self.device.get_status()
  GLib.idle_add(self.polling_received)
  time.sleep(1)
self.m.close()
[...]


Now I have some concerns even in using self.comm_active.  It is a boolean 
variable
accessed by the GUI thread (inside Start/Stop buttons handler) and backend 
thread
(in the "while self.comm_active" instruction).



Is it safe to access this variable from two different threads?
Should I implement a safer and more complex mechanism?  If yes, what mechanism?


Accessing from multiple thread shouldn't be a problem. As long as you only 
change
it in one thread.


I don't want to doubt what you have written, but... are you definitevely 
sure?  I tried to search for some authoritative documentation about this 
topic, but I couldn't find any.


I have many years of experiece in embedded firmware written in C for 
small microcontrollers, so I know the problems that could occur when a 
variable is read in one ISR (interrupt service routine) and written in 
the main loop (or viceversa).
ISR and main loop can be considered two threads. If the variable is 
32-bits and the microcontroller can't write atomically (without any 
interruption) a 32-bit variable, bad things could occur.


If the main loop is updating the variable from 0x01020304 to 0xA1A2A3A4 
and the change happens on a byte basis, the ISR could access a 
completely wrong value, for example 0x0102A3A4.


So the main question here is: does python *specification/standard* 
guarantees atomic operations? If yes, what are they?


--
https://mail.python.org/mailman/listinfo/python-list

Re: How to use two threads (GUI and backend)

[Antoon Pardon]

Op 26-10-16 om 12:22 schreef pozz:
> Il 26/10/2016 09:13, pozz ha scritto:
> > [...]
>> When the user press Start button (the pressed handler is in the GUI
>> class):
>>
>>   self.comm_active = True
>>   threading.Thread(target=self.comm_thread).start()
>>
>> The backend thread is:
>>
>>   def comm_thread(self):
>> while self.comm_active:
>>   self.device.get_status()
>>   GLib.idle_add(self.polling_received)
>>   time.sleep(1)
>> self.m.close()
> > [...]
>
> Now I have some concerns even in using self.comm_active.  It is a boolean 
> variable
> accessed by the GUI thread (inside Start/Stop buttons handler) and backend 
> thread
> (in the "while self.comm_active" instruction).

> Is it safe to access this variable from two different threads?
> Should I implement a safer and more complex mechanism?  If yes, what 
> mechanism?

Accessing from multiple thread shouldn't be a problem. As long as you only 
change
it in one thread.


-- 
https://mail.python.org/mailman/listinfo/python-list

Re: How to use two threads (GUI and backend)

[jmp]

On 10/26/2016 12:22 PM, pozz wrote:

Il 26/10/2016 09:13, pozz ha scritto:
 > [...]

When the user press Start button (the pressed handler is in the GUI
class):

  self.comm_active = True
  threading.Thread(target=self.comm_thread).start()

The backend thread is:

  def comm_thread(self):
while self.comm_active:
  self.device.get_status()
  GLib.idle_add(self.polling_received)
  time.sleep(1)
self.m.close()

 > [...]


Now I have some concerns even in using self.comm_active.  It is a
boolean variable accessed by the GUI thread (inside Start/Stop buttons
handler) and backend thread (in the "while self.comm_active" instruction).

Is it safe to access this variable from two different threads?  Should I
implement a safer and more complex mechanism?  If yes, what mechanism?




from http://nedbatchelder.com/blog/201204/two_problems.html

Some people, when confronted with a problem, think, "I know, I'll use 
threads," and then two they hav erpoblesms.



I suggest you write a GUI that make synchronous	calls to a remote 
application, if possible. If the remote app is in python, you have 
access to remote protocols already written for you, Pyro is one of them, 
you can skip the low level communication part.


jm

--
https://mail.python.org/mailman/listinfo/python-list

Re: How to use two threads (GUI and backend)

[pozz]

Il 26/10/2016 09:13, pozz ha scritto:
> [...]

When the user press Start button (the pressed handler is in the GUI class):

  self.comm_active = True
  threading.Thread(target=self.comm_thread).start()

The backend thread is:

  def comm_thread(self):
while self.comm_active:
  self.device.get_status()
  GLib.idle_add(self.polling_received)
  time.sleep(1)
self.m.close()

> [...]


Now I have some concerns even in using self.comm_active.  It is a 
boolean variable accessed by the GUI thread (inside Start/Stop buttons 
handler) and backend thread (in the "while self.comm_active" instruction).


Is it safe to access this variable from two different threads?  Should I 
implement a safer and more complex mechanism?  If yes, what mechanism?


--
https://mail.python.org/mailman/listinfo/python-list

Re: How to use two threads (GUI and backend)

[pozz]

Il 26/10/2016 09:13, pozz ha scritto:
> [...]

What is the best approach to use in my scenario (GUI and backend
communication)?


I just found this[1] page, where the thread approach is explained with 
the following code:


---
  import threading
  import time

  from gi.repository import GLib, Gtk, GObject


  def app_main():
win = Gtk.Window(default_height=50, default_width=300)
win.connect("delete-event", Gtk.main_quit)

progress = Gtk.ProgressBar(show_text=True)
win.add(progress)

def update_progess(i):
  progress.pulse()
  progress.set_text(str(i))
  return False

def example_target():
  for i in range(50):
GLib.idle_add(update_progess, i)
time.sleep(0.2)

win.show_all()

thread = threading.Thread(target=example_target)
thread.daemon = True
thread.start()


  if __name__ == "__main__":
# Calling GObject.threads_init() is not needed for PyGObject 3.10.2+
GObject.threads_init()

app_main()
---

This is similar to my approach, with a main difference: the callback 
update_progress() added to the GLib idle loop (so executed in the main 
GUI thread) receives all the data as arguments (the value i to write as 
text in the progress widget).


In my case, I have many many properties of the remote device. So my 
first idea is to get directly the value by accessing variables changed 
during backend thread... I think this is wrong.






[1] https://wiki.gnome.org/Projects/PyGObject/Threading
--
https://mail.python.org/mailman/listinfo/python-list

How to use two threads (GUI and backend)

[pozz]

I'm designing a GUI application in Python (with pyGObject, so GTK).  The 
application communicates with a remote device (connected through RS232, 
but it could be on Internet) to retrieve its status and set/get its 
configuration.


When the user press "Start" button, the application starts sending "GET 
STATUS" requests to the remote device, waiting its response.  When the 
response arrives, the GUI widgets are refreshed with the new status. The 
"GET STATUS" requests are send at a regular frequency (polling mode).


I thought two split the application in two threads: the GUI main thread 
that manages graphical widgets and user interaction; the backend thread 
that manages low-level communication.


When the user press Start button (the pressed handler is in the GUI class):

  self.comm_active = True
  threading.Thread(target=self.comm_thread).start()

The backend thread is:

  def comm_thread(self):
while self.comm_active:
  self.device.get_status()
  GLib.idle_add(self.polling_received)
  time.sleep(1)
self.m.close()

self.device.get_status() is blocking. It is executed in backend thread, 
so the GUI isn't blocked. self.polling_received() function will be 
executed in main thread (thanks to GLib.idle_add), because it will 
change widgets properties.


Now the get_stats() of self.comm object:

  def get_status(self):



self.property1 = 
self.property2 = 
return

And self.polling_received() of GUI class:

  def polling_received(self):
txtEntry1.set_text(self.comm.property1)
txtEntry2.set_text(self.comm.property2)



I didn't fully tested this, but it seems it works well. However I have 
some concerns, mainly for thread syncronizations.


self.polling_received() is executed in GUI thread and reads properties 
(self.comm.property1, ...) that are changed during parsing of responses 
in self.comm.get_status() function that is executed in the backend thread.
So the two threads use the same variables/objects without 
synchronization. Is this a problem?


What is the best approach to use in my scenario (GUI and backend 
communication)?

--
https://mail.python.org/mailman/listinfo/python-list