Re: [Kicad-developers] KiCad Coroutines

[Tomasz Wlostowski]

On 21.01.2016 13:06, "Torsten Hüter" wrote:
> Hi,
>  
> Lorenzo:
>  
> Sorry, perhaps I've expressed it not correctly in my latest mail, I've just 
> written down my subjective impressions, while working on the code, that are 
> mostly rhetoric questions.
>  
>>> Stackless coroutines:
>>> -
>>>
>>> * Relative easy to handle, because based on switch()/case().
>>> * Safer, a wrong state causes only the exit of the coroutine but not a 
>>> crash.
>>> * Simple implementation
>>> Simple?!? Well, maybe implementing but maintaining is another thing...

Hi Torsten,

Many thanks for your work. I'm not particularly thrilled about stackless
coroutines for the reasons below:
- They would require a major rewrite of the tool code. If we were to
rewrite it, I'd rather drop coroutines completely.
- Performance is not an issue. The tools just handle UI events.
protothreads library was invented for asynchronous I/O, where
performance is critical. In Kicad, coroutines overhead is minimal.
- They are based on macros, which not only means strange-looking (not to
say ugly) code, but is also hard to debug.
- I can't agree that if something goes wrong, the stackless coroutine
will just exit, but a stackful one will crash. If there's a segfault,
you'll get a crash independently of the coroutine implementation.

Back to the original issue: Boost::context stopped working on MSYS2. The
reason is Boost developers don't want to use GNU assembler on Windows,
and the GNU asm port of the context switching functions is not the same
as the MASM one (since the authors have changed the context ABI at some
point). We shouldn't play cat and mouse with Boost devs by maintaining
MSYS patches that won't get accepted into official Boost tree.

I've been working recently on a library [1] derived from Boost::context,
but which can be easily added to any C++ project, as it consists only of
2 files (.h and .cpp), just like ClipperLib, which we successfully used
for processing polygons.

Libcontext contains all Boost::context switching functions converted to
inline assembly, with compiler and ABI detection. So far,
it has been tested on: i386/x86_64 Linux, Win32/Win64 (GCC) and OSX
(both 32 and 64-bit) - so all the platforms Kicad runs on are already
supported (I'll also enable ARM support - as soon as I have a working
test program a raspi). I think this should be sufficient for the
foreseeable future.

Best,
Tom

[1] https://github.com/twlostow/libcontext



___
Mailing list: https://launchpad.net/~kicad-developers
Post to : kicad-developers@lists.launchpad.net
Unsubscribe : https://launchpad.net/~kicad-developers
More help   : https://help.launchpad.net/ListHelp

Re: [Kicad-developers] KiCad Coroutines

[Torsten Hüter]

Hi,
 
Lorenzo:
 
Sorry, perhaps I've expressed it not correctly in my latest mail, I've just 
written down my subjective impressions, while working on the code, that are 
mostly rhetoric questions.
 
>> Stackless coroutines:
>> -
>>
>> * Relative easy to handle, because based on switch()/case().
>> * Safer, a wrong state causes only the exit of the coroutine but not a crash.
>> * Simple implementation
>>Simple?!? Well, maybe implementing but maintaining is another thing...
 
I don't understand this comment. Maintaining of this code is easier, because it 
depends only on the C++ standard library, works on every platform - as long as 
the language specification is not changed. If you mean the functions itself, 
yes - more work but still not very hard to maintain - as long as you're knowing 
the restrictions of this approach.
 
>>In my experience once you do the stack initialization correctly the
>>setjmp/longjmp is the 'best' coroutine you can do; the non-portable code
>>is relegated to the coroutine creation code so it's easily locked away
>>in some untouchable source file :P
>>
>>OTOH I never did that on Win32/64 so maybe it's totally undependable to work 
>>:D
>>Isn't there some MSDN article on doing that?
 
But this is the whole point :) - an robust implementation that works on every 
platform, with minimal side effects and ideally easy to maintain, dropping the 
dependency of the Boost library. I've not the impression that 
setjmp()/longjmp() is the easiest/best solution - please also read the first 
mail about this subject. In the "Protothreads" link you can find also a 
discussion about setjmp()/longjmp().
 
--
 
There are multiple ways to solve these issues and there is no requirement to 
use coroutines at all. They are just used by the tool framework code to avoid 
writing explicit state machines, also the coroutines itself manage there the 
events. So the most radical way would be to completely drop coroutines. I've 
shown with my examples some alternatives.
 
Thanks,
Torsten

___
Mailing list: https://launchpad.net/~kicad-developers
Post to : kicad-developers@lists.launchpad.net
Unsubscribe : https://launchpad.net/~kicad-developers
More help   : https://help.launchpad.net/ListHelp

Re: [Kicad-developers] KiCad Coroutines

[Kristian Nielsen]

Lorenzo Marcantonio  writes:

> OTOH I never did that on Win32/64 so maybe it's totally undependable to work 
> :D
> Isn't there some MSDN article on doing that?

On windows, fibers can be used, and seem to work well. On most other
platforms, ucontext can be used (only exception I am aware of is that it was
removed on (some of the?) BSDs).

In the MariaDB non-blocking client library, ucontext and windows fibers are
used (and some custom assembly for performance, but that should be
unnecessary for KiCAD, I think). It seems to cover most things.

I would be happy to help with Co-routine infrastructure in KiCAD. Still, it
would be preferable if some existing layer on top (like Boost::Context or
similar) can be used, to not have to maintain KiCAD's own implementation.

> Simple?!? Well, maybe implementing but maintaining is another thing...

I agree. The main point of using co-routines in the first place is to
maintain the state of the stack across waiting operations.

 - Kristian.

___
Mailing list: https://launchpad.net/~kicad-developers
Post to : kicad-developers@lists.launchpad.net
Unsubscribe : https://launchpad.net/~kicad-developers
More help   : https://help.launchpad.net/ListHelp

Re: [Kicad-developers] KiCad Coroutines

[Lorenzo Marcantonio]

On Wed, Jan 20, 2016 at 11:58:56PM +0100, "Torsten Hüter" wrote:

> * It's very tricky to get stackful implementations to run stable. Any small 
> fault results in a crash or deadlock.

What's strange in that? When doing pointers even small faults give you a crash 
(if you are lucky :D).

> * The most critical part is stack handling. For Windows Fibers / Pthreads the 
> library does the job. There are also 
> some hacks to get Setjmp()/Longjmp() working with multiple stacks, as 
> Jean-Pauls link has shown, I've tried it but that was crashing 
> (Setjmp()/Longjmp() seems to be broken too on MinGW64).

longjmp is not guaranteed between 'contexts', so that would be expected too...

> * Side effects / show stoppers: It's not recommened to use threads for the 
> wxWidgets GUI, OpenGL requires a context switch per thread.

Not a problem, we already declared that all the UI work is done in event 
handlers (as it should be)

> * As already known, Boost::Context is broken on Msys2 - makes it sense that 
> such a low level library has to be maintained by KiCad developers (?)

Otherwise just hope that boost people fix that!

> * Is context switching always safe (side effects with 3rd party libraries / 
> Python etc.)? 

Context as in thread usually yes, if you are not running concurrently... thread 
safety *usually* means proper locking of shared data structures

> Stackless coroutines:
> -
>
> * Relative easy to handle, because based on switch()/case().
> * Safer, a wrong state causes only the exit of the coroutine but not a crash.
> * Simple implementation

Simple?!? Well, maybe implementing but maintaining is another thing...

In my experience once you do the stack initialization correctly the
setjmp/longjmp is the 'best' coroutine you can do; the non-portable code
is relegated to the coroutine creation code so it's easily locked away
in some untouchable source file :P

OTOH I never did that on Win32/64 so maybe it's totally undependable to work :D
Isn't there some MSDN article on doing that?

-- 
Lorenzo Marcantonio
CZ Srl - Parma


signature.asc
Description: PGP signature
___
Mailing list: https://launchpad.net/~kicad-developers
Post to : kicad-developers@lists.launchpad.net
Unsubscribe : https://launchpad.net/~kicad-developers
More help   : https://help.launchpad.net/ListHelp

Re: [Kicad-developers] KiCad Coroutines

[Torsten Hüter]

Hi Tom,
 
thanks for your point of view - I'm guessing I've missed that you're working 
already on your variant of boost::context. This is of course a feasible way. 
I'm not the biggest fan of that either, because the (low level) assembler code 
has to be maintained, tested on all platforms, but as long as you can guarantee 
that it works always, it should be good.
 
I've reviewed also the tool code and I've found there not such a huge demand to 
have coroutines, they're "nice to have".
 
--
 
I've used protothreads for some embedded projects, you're right they look 
"uglier" and are sometimes a bit tricky - but easier than state machines to 
write. The Contiki OS is based on them. The comment about the stability was 
related to the implementation of the coroutine itself and is a *subjective 
impression*, it's of course true that it can't prevent a segfault somewhere 
else.
 
--
 
I can of course test your code, I'll use the "stress test" for that :):)
 
Thanks,
Torsten
 

Many thanks for your work. I'm not particularly thrilled about stackless
coroutines for the reasons below:
- They would require a major rewrite of the tool code. If we were to
rewrite it, I'd rather drop coroutines completely.
- Performance is not an issue. The tools just handle UI events.
protothreads library was invented for asynchronous I/O, where
performance is critical. In Kicad, coroutines overhead is minimal.
- They are based on macros, which not only means strange-looking (not to
say ugly) code, but is also hard to debug.
- I can't agree that if something goes wrong, the stackless coroutine
will just exit, but a stackful one will crash. If there's a segfault,
you'll get a crash independently of the coroutine implementation.

Back to the original issue: Boost::context stopped working on MSYS2. The
reason is Boost developers don't want to use GNU assembler on Windows,
and the GNU asm port of the context switching functions is not the same
as the MASM one (since the authors have changed the context ABI at some
point). We shouldn't play cat and mouse with Boost devs by maintaining
MSYS patches that won't get accepted into official Boost tree.

I've been working recently on a library [1] derived from Boost::context,
but which can be easily added to any C++ project, as it consists only of
2 files (.h and .cpp), just like ClipperLib, which we successfully used
for processing polygons.

Libcontext contains all Boost::context switching functions converted to
inline assembly, with compiler and ABI detection. So far,
it has been tested on: i386/x86_64 Linux, Win32/Win64 (GCC) and OSX
(both 32 and 64-bit) - so all the platforms Kicad runs on are already
supported (I'll also enable ARM support - as soon as I have a working
test program a raspi). I think this should be sufficient for the
foreseeable future.

Best,
Tom

[1] https://github.com/twlostow/libcontext

 

___
Mailing list: https://launchpad.net/~kicad-developers
Post to : kicad-developers@lists.launchpad.net
Unsubscribe : https://launchpad.net/~kicad-developers
More help   : https://help.launchpad.net/ListHelp

Re: [Kicad-developers] KiCad Coroutines

[Torsten Hüter]

Hi Tom,

I've uploaded now a few different implementations on my personal branch:

https://code.launchpad.net/~torstenhtr/+junk/coroutines

I've tried, as I've written in my first mail, Windows Fibers and Pthreads. I've 
implemented a "stress test" to compare the implementations and also "stackless" 
coroutines. 

My impressions are so far:

Stackful coroutines:


* It's very tricky to get stackful implementations to run stable. Any small 
fault results in a crash or deadlock.

* The most critical part is stack handling. For Windows Fibers / Pthreads the 
library does the job. There are also 
some hacks to get Setjmp()/Longjmp() working with multiple stacks, as 
Jean-Pauls link has shown, I've tried it but that was crashing 
(Setjmp()/Longjmp() seems to be broken too on MinGW64).

* Side effects / show stoppers: It's not recommened to use threads for the 
wxWidgets GUI, OpenGL requires a context switch per thread.

* As already known, Boost::Context is broken on Msys2 - makes it sense that 
such a low level library has to be maintained by KiCad developers (?)

* Is context switching always safe (side effects with 3rd party libraries / 
Python etc.)? 

Stackless coroutines:
-

* Relative easy to handle, because based on switch()/case().
* Safer, a wrong state causes only the exit of the coroutine but not a crash.
* Simple implementation

* More verbose, you need to use macros like CR_BEGIN(..) CR_END(..)
* Syntax restrictions
* Special care of local variables is required

You can find a paper about this approach here:
http://dunkels.com/adam/dunkels06protothreads.pdf

I've used your coroutine class and have adapted it to be similar to 
protothreads. The actual state is stored in the instance of this class. Compare 
also the "stress test" example. All implementations have the same output as 
result of this test program.

The stackless implementation is my favorite. There are less side effects, based 
on the C++ standard library and the coroutines consume less resources. This 
outweights in my opinion the disadvantage, that the coroutine functions are 
more time-consuming to write.

Thanks,
Torsten

>> void my_tool()
>> {
>> wait(event1);
>> do_something();
>> wait(event2);
>> do_something_else();
>> }
>> 
>> All of the stack switching "magic" is hidden. You can find an example in
>> include/tool/examples/coroutine_example.cpp
>> 
>> 
>> Concerning boost::context, I see two options:
>> - get rid of the coroutines (i.e. rewrite all event loops/calls in the
>> GAL tools). Painful but doable, I would prefer to spend my time doing
>> something more productive (e.g. porting the remaining tools to GAL).
>> - turn boost::context into a single .cpp file library that one can
>> just add to the project and forget about it (think of ClipperLib as an
>> example).
>> 
>> As a side comment, I'm not surprised that boost devs insist on using
>> MASM under Windows - the syntax of the GNU assembler on x86/x86_64
>> platforms is just disgusting.
>> 
>> Tom

___
Mailing list: https://launchpad.net/~kicad-developers
Post to : kicad-developers@lists.launchpad.net
Unsubscribe : https://launchpad.net/~kicad-developers
More help   : https://help.launchpad.net/ListHelp

Re: [Kicad-developers] KiCad Coroutines

[Torsten Hüter]

Hi Tom,

 

I'm working on some alternatives, that I'm getting a better impression. I'll upload the example code this weekend, then we can discuss. One alternative are for instance stackless coroutines. I've also checked how you're using them in your code and have some ideas to modify that.

 

Thanks,

Torsten

 



@Edwin: Coroutines are not a replacement for threads. They *aren't*
there for improving performance, just to simplify event handling in
complex state machines. For example:

case 1) - without coroutines - we need to code a (possibly large) FSM
by hand. This doesn't scale well in terms of code complexity,
especially if there are nested state machines .

void handle_event(event)
{
switch(state)
{
case state1:
if (event == event1) {
do something();
state = state2;
}
case state2:
if (event == event2) {
do_something();
state = state3;
}
case state3:
/// and so on
}
}

case 2) - with coroutines - we can execute our FSM in sequential code
and receive events using a wait() function, which yields the current
coroutine (i.e. transfers the control to the master event loop in case
of pcbnew) and wakes up the execution of the tool that called wait()
when a matching event arrives:

void my_tool()
{
wait(event1);
do_something();
wait(event2);
do_something_else();
}

All of the stack switching "magic" is hidden. You can find an example in
include/tool/examples/coroutine_example.cpp


Concerning boost::context, I see two options:
- get rid of the coroutines (i.e. rewrite all event loops/calls in the
GAL tools). Painful but doable, I would prefer to spend my time doing
something more productive (e.g. porting the remaining tools to GAL).
- turn boost::context into a single .cpp file library that one can
just add to the project and forget about it (think of ClipperLib as an
example).

As a side comment, I'm not surprised that boost devs insist on using
MASM under Windows - the syntax of the GNU assembler on x86/x86_64
platforms is just disgusting.

Tom


___
Mailing list: https://launchpad.net/~kicad-developers
Post to : kicad-developers@lists.launchpad.net
Unsubscribe : https://launchpad.net/~kicad-developers
More help : https://help.launchpad.net/ListHelp




___
Mailing list: https://launchpad.net/~kicad-developers
Post to : kicad-developers@lists.launchpad.net
Unsubscribe : https://launchpad.net/~kicad-developers
More help   : https://help.launchpad.net/ListHelp

Re: [Kicad-developers] KiCad Coroutines

[Maciej Sumiński]

On 01/07/2016 11:39 AM, "Torsten Hüter" wrote:
> Hi Wayne,
> 
> this is of course right, but the idea would be to emulate cooperative 
> multitasking with threads. This is done by synchronizing the threads with 
> condition variables and mutexes. The principle can be found in the articles 
> [1] or [2].
> 
> The idea is good, BUT after further research I'd say that there are two show 
> stoppers:
> 
> (a) it's not recommended to use secondary threads to call GUI functions
> 
> http://docs.wxwidgets.org/trunk/overview_thread.html
> 
> (b) the OpenGL context is only valid for the current thread, it's required to 
> change the context for every thread
> 
> https://www.opengl.org/wiki/OpenGL_and_multithreading
> 
> Ideally rendering and gui handling should happen only in the main thread; but 
> that would be to restrictive for our purposes. 
> 
> This makes threads in my opinion unattractive for coroutines.

Tools in the Tool Framework never directly interact with OpenGL context.
Even if they request some redrawing/refreshing, it happens in OnPaint
(or similar) event handler.

Regards,
Orson

> --
> 
> Yes, state machines or stackless coroutines are possible too, but then the 
> tool-framework code has to be changed a lot. I'll evaluate the other options 
> first. Setjmp/longjmp could work also, but here is stack handling the major 
> issue.
> 
> Thanks,
> Torsten




signature.asc
Description: OpenPGP digital signature
___
Mailing list: https://launchpad.net/~kicad-developers
Post to : kicad-developers@lists.launchpad.net
Unsubscribe : https://launchpad.net/~kicad-developers
More help   : https://help.launchpad.net/ListHelp

Re: [Kicad-developers] KiCad Coroutines

[Lorenzo Marcantonio]

On Thu, Jan 07, 2016 at 12:29:17PM +0100, Kristian Nielsen wrote:
> In my experience, it is important not to underestimate the value of
> co-routines for larger event-driven systems / state machines. Small systems

The Knuth book has one *huge* chapter for coroutines. In assembly. The
target has no stack so there is no problem in context switching :P

If you don't need to actively multiprocess they have the *huge*
advantage of not needing synchronization. Alternatives are of course
state machines or continuation passing, for some problem areas are even
better than linear flow

> Unfortunately co-routines are not familiar to many developers, and have poor
> cross-platform support, which makes them harder to use.

Mostly because none of the mainstream languages uses them... the most
similar thing to them AFAIK are scheme continuation and LUA threads; and
of course Haskell monads (more or less...) in a horribly different way :D

And, of course, *every* embedded system with a cooperative scheduler at
a yield point!

-- 
Lorenzo Marcantonio
CZ Srl - Parma


signature.asc
Description: PGP signature
___
Mailing list: https://launchpad.net/~kicad-developers
Post to : kicad-developers@lists.launchpad.net
Unsubscribe : https://launchpad.net/~kicad-developers
More help   : https://help.launchpad.net/ListHelp

Re: [Kicad-developers] KiCad Coroutines

[Mário Luzeiro]

Hi all,

Sorry I don't know nothing about co-routines.
Just curious, Why and where kicad is using it? And why and where kicad need 
multi-threading?

Mario
___
Mailing list: https://launchpad.net/~kicad-developers
Post to : kicad-developers@lists.launchpad.net
Unsubscribe : https://launchpad.net/~kicad-developers
More help   : https://help.launchpad.net/ListHelp

Re: [Kicad-developers] KiCad Coroutines

[Lorenzo Marcantonio]

On Thu, Jan 07, 2016 at 11:39:24AM +0100, "Torsten Hüter" wrote:

> Ideally rendering and gui handling should happen only in the main
> thread; but that would be to restrictive for our purposes.

Are we sure doing GUI calls in event handling is a good idea? Since win
3.1 the recommended way was to schedule an update and handle it in the
'preferred' event...

Also how do you do an 'incremental' opengl update? An FBO for the
current screen and then a paste over with the current modifications?

> This makes threads in my opinion unattractive for coroutines.

Threads are not free, either... I personally would be doing this with
coroutines but only scheduling the updates, this also help detach the
drawing code from the tool process/state machine (like in MVC, ideally,
the coroutine is the controller part and the paint code is the view).
Yes, I know that doing a 'pure' MVC is neither easy nor fun but in the
long time I think it would be better than interspersing drawing with the
controller.

--
Lorenzo Marcantonio
CZ Srl - Parma

___
Mailing list: https://launchpad.net/~kicad-developers
Post to : kicad-developers@lists.launchpad.net
Unsubscribe : https://launchpad.net/~kicad-developers
More help   : https://help.launchpad.net/ListHelp

Re: [Kicad-developers] KiCad Coroutines

[Lorenzo Marcantonio]

On Thu, Jan 07, 2016 at 12:06:08PM +0100, Maciej Sumiński wrote:
> Tools in the Tool Framework never directly interact with OpenGL context.
> Even if they request some redrawing/refreshing, it happens in OnPaint
> (or similar) event handler.

OK that's what I was worried about :D 100% agree with the design choice
(otherwise it would be hell to manage the OpenGL state...)

-- 
Lorenzo Marcantonio
CZ Srl - Parma


signature.asc
Description: PGP signature
___
Mailing list: https://launchpad.net/~kicad-developers
Post to : kicad-developers@lists.launchpad.net
Unsubscribe : https://launchpad.net/~kicad-developers
More help   : https://help.launchpad.net/ListHelp

Re: [Kicad-developers] KiCad Coroutines

[Torsten Hüter]

Hi Wayne,

this is of course right, but the idea would be to emulate cooperative 
multitasking with threads. This is done by synchronizing the threads with 
condition variables and mutexes. The principle can be found in the articles [1] 
or [2].

The idea is good, BUT after further research I'd say that there are two show 
stoppers:

(a) it's not recommended to use secondary threads to call GUI functions

http://docs.wxwidgets.org/trunk/overview_thread.html

(b) the OpenGL context is only valid for the current thread, it's required to 
change the context for every thread

https://www.opengl.org/wiki/OpenGL_and_multithreading

Ideally rendering and gui handling should happen only in the main thread; but 
that would be to restrictive for our purposes. 

This makes threads in my opinion unattractive for coroutines.

--

Yes, state machines or stackless coroutines are possible too, but then the 
tool-framework code has to be changed a lot. I'll evaluate the other options 
first. Setjmp/longjmp could work also, but here is stack handling the major 
issue.

Thanks,
Torsten


[1] Coroutines in C/C++: beyond Duff's Device, 
http://sjansen.blogspot.de/2008/04/coroutines-in-cc-beyond-duffs-device.html 
[2] Cross-Platform Coroutines in C++, 
http://www.drdobbs.com/cpp/cross-platform-coroutines-in-c/184404529 
 

> Torsten,
>
> I've been thinking about your proposed changes and I'm not sure I'm
> comfortable making KiCad multi-threaded using a library such as pthreads
> or any other threading implementation. Context switching (cooperative
> multi-tasking) is bad enough but at least everything is happening in the
> same thread so no locking is required. Pre-emptive multi-tasking is
> much more difficult to deal with. Access to everything being touched by
> multiple threads has to be protected with locks which can make for some
> ugly code and debugging threaded applications can be difficult. What
> about using a state machine or some clever way to use events?

Wayne

On 1/6/2016 10:48 AM, Tomasz Wlostowski wrote:
> On 04.01.2016 15:07, Mark Roszko wrote:
>>> My offer would be to test an alternative implementation, to drop another 
>>> Boost dependency and build upon a standard library foundation.
>>> pthreads is not standard library <.<
>>
>
> Hi Mark & Torsten,
>
> Feel free to test any implementation you like (personally I would try
> with setjmp() first). I don't have much time to work on this issue (I'm
> busy these days adding a reasonably fast wxDC support to the GAL canvas...).
>
> Cheers,
> Tom
>

___
Mailing list: https://launchpad.net/~kicad-developers
Post to : kicad-developers@lists.launchpad.net
Unsubscribe : https://launchpad.net/~kicad-developers
More help   : https://help.launchpad.net/ListHelp

Re: [Kicad-developers] KiCad Coroutines

[Lorenzo Marcantonio]

On Thu, Jan 07, 2016 at 12:58:11PM +0100, jp charras wrote:
> However I am not sure it is "thread based".
> Threads are used to manage the stack using C++ language only.
> I am not sure they are used to manage coroutines themselves.

My fault, didn't read all the article.

Actually it only use the pthread to allocate a context (or more
correctly, the stack!) then use setjmp/longjmp as a switching primitive.
Then the thread dies... so in fact execution only remains in the main
thread (no GUI issues).

After that is the classic setjmp dispatcher without the tricks for
creating *correctly* the stack for the new context (embedded OSs usually
give you an helper function for that...), so it suffers from the same
issues. Be careful with local variables, registers are not preserved,
volatile declarations will abound :D

Also the pthread_attr_setstack seems quite esoteric, will need testing
to check its behaviour on Win32. If the function works as advertised
(i.e. the stack is correctly set) it would be the most portable thing I
could think of.

Another thing potentially problematic: if, as probably is, a pthread is
mapped to a Win32 thread, it takes a setjmp from a thread context (which
will die just after that) and then longjmp to it from *another* thread
(the main thread). That's potentially asking for trouble, depending on
what is in the saved context (which is an opaque array...).

Here (http://en.cppreference.com/w/c/program/longjmp) says:

Jumping across threads (if the function that called setjmp was
executed by another thread) is also undefined behavior.

'Undefined' is usually a bad thing (i.e. will break on the next
compiler/OS upgrade)...

Another thing I see is a probable portability flaw in the stack
allocation code, the man page says:

The address specified in stackaddr should be suitably aligned: for
full portability, align it on a page boundary
(sysconf(_SC_PAGESIZE)). posix_memalign(3) may be useful for
allocation. Probably, stacksize should also be a multiple of the
system page size.

I would suppose a sane pthread implementation would align internally to
the correct size, but who knows!

So far this is the 'best' coroutine approach suggested, *if* it works
reliably on Win32. Another idea worth to check: is Pth available on Windows?

-- 
Lorenzo Marcantonio
CZ Srl - Parma


signature.asc
Description: PGP signature
___
Mailing list: https://launchpad.net/~kicad-developers
Post to : kicad-developers@lists.launchpad.net
Unsubscribe : https://launchpad.net/~kicad-developers
More help   : https://help.launchpad.net/ListHelp

Re: [Kicad-developers] KiCad Coroutines

[Lorenzo Marcantonio]

On Thu, Jan 07, 2016 at 01:24:18PM +0100, Edwin van den Oetelaar wrote:
> I have done some embedded work in which these cooperative concepts happened.

> by using co-routines this way you gain perceived concurrency but no real
> increased power.

It's not done for concurrency, only for flow control. Otherwise it would
be a thread, not a coroutine :D

When a coroutine stops only another one resumes, i.e. you manually and
explicitly jump between threads of execution. Moreover you have full
control of *which* coroutine pass to the CPU; it's a fundamentally
different thing.

- Threads: many control flows, everyone run in parallel. If infinite
  cores are not available, scheduling is done by the OS in marginally
  controllable way (except with synchronization)

- Coroutine: many control flows, only one runs at a time and then
  suspend to pass the control to the desired one; no synchronization
  needed since only one is in control at any time

-- 
Lorenzo Marcantonio
CZ Srl - Parma


signature.asc
Description: PGP signature
___
Mailing list: https://launchpad.net/~kicad-developers
Post to : kicad-developers@lists.launchpad.net
Unsubscribe : https://launchpad.net/~kicad-developers
More help   : https://help.launchpad.net/ListHelp

Re: [Kicad-developers] KiCad Coroutines

[Kristian Nielsen]

Lorenzo Marcantonio  writes:

> On Thu, Jan 07, 2016 at 12:08:22PM +, Mário Luzeiro wrote:

> Some thing like this:
> - Select a tool (suspend for a selection)
> - Other things happens, then user select a thing
> - (restart the tool coroutine) Check thing selected and so on

So to me, this sounds like an excellent use-case for co-routines.

I suppose the idea is to have code something like this (random pseudocode)?

  a = some_prepare_tool_stuff();
  b = other_stuff();
  select = get_selection();
  if (select.aborted) {
cleanup(a, b);
return;
  }
  more_tool_stuff(a, b, select.data);

The great thing about co-routines is that it allows to abstract things in
helper functions, like get_selection() above. You don't need to know exactly
how co-routine scheduling will happen inside get_selection, all the details
can be handled inside that function.

In contrast, without co-routines, every such usage will have to be split up
in multiple event handler / callback functions, at each point where event
handling can happen. This breaks the code flow, breaks the data flow, and
makes things a lot harder to read and understand.

Just my two cents,

 - Kristian.

___
Mailing list: https://launchpad.net/~kicad-developers
Post to : kicad-developers@lists.launchpad.net
Unsubscribe : https://launchpad.net/~kicad-developers
More help   : https://help.launchpad.net/ListHelp

Re: [Kicad-developers] KiCad Coroutines

[Edwin van den Oetelaar]

Hello all,

In that case, take a look at http://dunkels.com/adam/pt/examples.html for
inspiration. (proto-threads)

Greetings,
Edwin van den Oetelaar

On Thu, Jan 7, 2016 at 1:39 PM, Lorenzo Marcantonio <
l.marcanto...@cz-dynamic.it> wrote:

> On Thu, Jan 07, 2016 at 01:24:18PM +0100, Edwin van den Oetelaar wrote:
> > I have done some embedded work in which these cooperative concepts
> happened.
>
> > by using co-routines this way you gain perceived concurrency but no real
> > increased power.
>
> It's not done for concurrency, only for flow control. Otherwise it would
> be a thread, not a coroutine :D
>
> When a coroutine stops only another one resumes, i.e. you manually and
> explicitly jump between threads of execution. Moreover you have full
> control of *which* coroutine pass to the CPU; it's a fundamentally
> different thing.
>
> - Threads: many control flows, everyone run in parallel. If infinite
>   cores are not available, scheduling is done by the OS in marginally
>   controllable way (except with synchronization)
>
> - Coroutine: many control flows, only one runs at a time and then
>   suspend to pass the control to the desired one; no synchronization
>   needed since only one is in control at any time
>
> --
> Lorenzo Marcantonio
> CZ Srl - Parma
>
> ___
> Mailing list: https://launchpad.net/~kicad-developers
> Post to : kicad-developers@lists.launchpad.net
> Unsubscribe : https://launchpad.net/~kicad-developers
> More help   : https://help.launchpad.net/ListHelp
>
>
___
Mailing list: https://launchpad.net/~kicad-developers
Post to : kicad-developers@lists.launchpad.net
Unsubscribe : https://launchpad.net/~kicad-developers
More help   : https://help.launchpad.net/ListHelp

Re: [Kicad-developers] KiCad Coroutines

[Wayne Stambaugh]

On 1/7/2016 7:53 AM, Tomasz Wlostowski wrote:
> On 07.01.2016 13:34, Lorenzo Marcantonio wrote:
>> On Thu, Jan 07, 2016 at 12:08:22PM +, Mário Luzeiro wrote:
>>> Hi all,
>>>
>>> Sorry I don't know nothing about co-routines. Just curious, Why
>>> and where kicad is using it? And why and where kicad need
>>> multi-threading?
>>
>> It's not multithreading, it's suspending a thread of work in the
>> new tool code.
>>
>> Some thing like this: - Select a tool (suspend for a selection) -
>> Other things happens, then user select a thing - (restart the tool
>> coroutine) Check thing selected and so on
>>
>> The goal is *not* to use multiple cores or doing things in
>> background; the idea is to stop inside a function and then resume
>> from there when something desiderable happens.
>>
> 
> I agree, the whole idea was to simplify event handling in the tool, so
> instead of writing a complex state machine, you could just suspend the
> execution of your tool code to wait for a matching event:
> 
> @Edwin: Coroutines are not a replacement for threads. They *aren't*
> there for improving performance, just to simplify event handling in
> complex state machines. For example:
> 
> case 1) - without coroutines - we need to code a (possibly large) FSM
> by hand. This doesn't scale well in terms of code complexity,
> especially if there are nested state machines .
> 
> void handle_event(event)
> {
> switch(state)
> {
> case state1:
> if (event == event1) {
> do something();
> state = state2;
> }
> case state2:
> if (event == event2) {
> do_something();
> state = state3;
> }
> case state3:
> /// and so on
> }
> }
> 
> case 2) - with coroutines - we can execute our FSM in sequential code
> and receive events using a wait() function, which yields the current
> coroutine (i.e. transfers the control to the master event loop in case
> of pcbnew) and wakes up the execution of the tool that called wait()
> when a matching event arrives:
> 
> void my_tool()
> {
>   wait(event1);
>   do_something();
>   wait(event2);
>   do_something_else();
> }
> 
> All of the stack switching "magic" is hidden. You can find an example in
> include/tool/examples/coroutine_example.cpp
> 
> 
> Concerning boost::context, I see two options:
> - get rid of the coroutines (i.e. rewrite all event loops/calls in the
> GAL tools). Painful but doable, I would prefer to spend my time doing
> something more productive (e.g. porting the remaining tools to GAL).
> - turn boost::context into a single .cpp file library that one can
> just add to the project and forget about it (think of ClipperLib as an
> example).

After following this thread, I'm leaning towards Tom's opinion and
thinking that sticking with boost::context may be the least painful path
forward.  As of right now, the only place boost context is giving us
trouble is 64 bit windows builds.  I have already documented the work
around for msys2/mingw64 in the compiling KiCad developer doc.  Before
we dump boost context into the KiCad source, we should try to fix the 64
bit windows version current boost context switching library.  There are
now assembly files for building the boost context library on windows
using the gnu assembler so the boost devs appear to have changed their
minds about only using masm on windows.  Perhaps we can send a patch to
the msys2 folks who are very responsive to bug fix patches and push the
changes upstream to boost.  I'm not opposed to adding boost context to
our source repo but this always seems to me to be against one of the
major advantages of using open source software which is everyone
benefiting from bug fixes and improvements.  If we make local changes to
boost context to fix our problem and do not provide a patch to resolve
the problem to the boost project, then we are not helping out the
community at large.  Given that I am writing this email using
Thunderbird, using emacs as my editor, gcc as my compiler, bash as my
shell, etc., I think we should make our best effort at pushing fixes
upstream even though is a bit more work.

Wayne

> 
> As a side comment, I'm not surprised that boost devs insist on using
> MASM under Windows - the syntax of the GNU assembler on x86/x86_64
> platforms is just disgusting.
> 
> Tom
> 
> 
> ___
> Mailing list: https://launchpad.net/~kicad-developers
> Post to : kicad-developers@lists.launchpad.net
> Unsubscribe : https://launchpad.net/~kicad-developers
> More help   : https://help.launchpad.net/ListHelp
> 

___
Mailing list: https://launchpad.net/~kicad-developers
Post to : kicad-developers@lists.launchpad.net
Unsubscribe : https://launchpad.net/~kicad-developers
More help   : https://help.launchpad.net/ListHelp

Re: [Kicad-developers] KiCad Coroutines

[Tomasz Wlostowski]

On 04.01.2016 15:07, Mark Roszko wrote:
>> My offer would be to test an alternative implementation, to drop another 
>> Boost dependency and build upon a standard library foundation.
>> pthreads is not standard library <.<
> 

Hi Mark & Torsten,

Feel free to test any implementation you like (personally I would try
with setjmp() first). I don't have much time to work on this issue (I'm
busy these days adding a reasonably fast wxDC support to the GAL canvas...).

Cheers,
Tom

___
Mailing list: https://launchpad.net/~kicad-developers
Post to : kicad-developers@lists.launchpad.net
Unsubscribe : https://launchpad.net/~kicad-developers
More help   : https://help.launchpad.net/ListHelp

Re: [Kicad-developers] KiCad Coroutines

[Wayne Stambaugh]

Torsten,

I've been thinking about your proposed changes and I'm not sure I'm
comfortable making KiCad multi-threaded using a library such as pthreads
or any other threading implementation.  Context switching (cooperative
multi-tasking) is bad enough but at least everything is happening in the
same thread so no locking is required.  Pre-emptive multi-tasking is
much more difficult to deal with.  Access to everything being touched by
multiple threads has to be protected with locks which can make for some
ugly code and debugging threaded applications can be difficult.  What
about using a state machine or some clever way to use events?

Wayne

On 1/6/2016 10:48 AM, Tomasz Wlostowski wrote:
> On 04.01.2016 15:07, Mark Roszko wrote:
>>> My offer would be to test an alternative implementation, to drop another 
>>> Boost dependency and build upon a standard library foundation.
>>> pthreads is not standard library <.<
>>
> 
> Hi Mark & Torsten,
> 
> Feel free to test any implementation you like (personally I would try
> with setjmp() first). I don't have much time to work on this issue (I'm
> busy these days adding a reasonably fast wxDC support to the GAL canvas...).
> 
> Cheers,
> Tom
> 
> ___
> Mailing list: https://launchpad.net/~kicad-developers
> Post to : kicad-developers@lists.launchpad.net
> Unsubscribe : https://launchpad.net/~kicad-developers
> More help   : https://help.launchpad.net/ListHelp
> 

___
Mailing list: https://launchpad.net/~kicad-developers
Post to : kicad-developers@lists.launchpad.net
Unsubscribe : https://launchpad.net/~kicad-developers
More help   : https://help.launchpad.net/ListHelp

Re: [Kicad-developers] KiCad Coroutines

[Torsten Hüter]

Hi Jean-Pierre,

this is a good argument, if Boost does not have to be compiled anymore. I'll 
see what I can do and give feedback, at least an alternative was then tried.

@Mark:
Pthreads are standardised, because they're part of the POSIX family of 
standards (https://standards.ieee.org/findstds/standard/1003.1-2008.html) 
They're not of course part of the C++ standard library. I'll compare them to 
C++11 threads, but I'd say the implementation would be very similar (that's a 
later topic, KiCad is not yet ready for C++11). First I have to check, if the 
concept works.

Thanks,
Torsten

>AFAIK, boost libraries are mainly used in Avhttp and coroutines.
>Bye Avhttp (thanks, Mark).
>boost::coroutine still use them, and it was always a nightmare, since
>the beginning.

>So, if an alternative implementation to boost::context exists, I will be
>*very* happy.

___
Mailing list: https://launchpad.net/~kicad-developers
Post to : kicad-developers@lists.launchpad.net
Unsubscribe : https://launchpad.net/~kicad-developers
More help   : https://help.launchpad.net/ListHelp

Re: [Kicad-developers] KiCad Coroutines

[Mark Roszko]

>Pthreads are standardised

I meant it in a cross platform availability sense.

___
Mailing list: https://launchpad.net/~kicad-developers
Post to : kicad-developers@lists.launchpad.net
Unsubscribe : https://launchpad.net/~kicad-developers
More help   : https://help.launchpad.net/ListHelp

Re: [Kicad-developers] KiCad Coroutines

[Torsten Hüter]

Mark,

 

You're right Pthreads should eventually work on Windows/MinGW too.

 

I don't know if their policy has changed, but the official documentation lists still MASM as requirement for Windows. I did no further research about the reasons.

 

Please read the (old) statement of our project leader here:

 

http://mingw-users.1079350.n2.nabble.com/Boost-Context-MinGW-EXPORT-keyword-td7582532.html

 

Quote from Wayne:

 

"[..] Good luck!  The KiCad project had the same issue.  We even submitted our
patch but they refused it.  The answer was always the same.  Use MASM
with no reasonable explanation as to why the gnu assembler was
unacceptable.

[..]

I'm beginning to regret using Boost in KiCad.

[..]"

 

I don't know if this opinion is still valid.

 

--

 

My offer would be to test an alternative implementation, to drop another Boost dependency and build upon a standard library foundation.

 

Thanks,

Torsten

 

 

Gesendet: Montag, 04. Januar 2016 um 06:07 Uhr
Von: "Mark Roszko" <mark.ros...@gmail.com>
An: "Torsten Hüter" <torsten...@gmx.de>
Cc: "KiCad Mailinglist" <kicad-developers@lists.launchpad.net>
Betreff: Re: [Kicad-developers] KiCad Coroutines

>Summary: Fibers (Windows) and Pthreads (all POSIX-compliant operating systems) seem to be the most promising alternatives for context switching

Since msys2 uses mingw, it used to have a pthreads implementation when
I used it years ago. Not sure if the latest has a package for it.





>I did some research and this thing seems to cause constantly trouble, because some assembly "hackery" is required and only MASM is officially supported

Not true at all.
MINGW64 is supported, someone just broke it in boost context
1.59 on our use case.
Works fine in 1.57. The problem is finding
someone brave enough to debug assembly <.<




___
Mailing list: https://launchpad.net/~kicad-developers
Post to : kicad-developers@lists.launchpad.net
Unsubscribe : https://launchpad.net/~kicad-developers
More help   : https://help.launchpad.net/ListHelp

Re: [Kicad-developers] KiCad Coroutines

[jp charras]

Le 04/01/2016 10:59, "Torsten Hüter" a écrit :
> Mark,
> You're right Pthreads should eventually work on Windows/MinGW too.
> I don't know if their policy has changed, but the official documentation 
> lists 
> still MASM as requirement for Windows. I did no further research about the 
> reasons.
> Please read the (old) statement of our project leader here:
> http://mingw-users.1079350.n2.nabble.com/Boost-Context-MinGW-EXPORT-keyword-td7582532.html
> Quote from Wayne:
> "[..] Good luck!  The KiCad project had the same issue.  We even submitted our
> patch but they refused it.  The answer was always the same.  Use MASM
> with no reasonable explanation as to why the gnu assembler was
> unacceptable.
> [..]
> I'm beginning to regret using Boost in KiCad.
> [..]"
> I don't know if this opinion is still valid.

Torsten,

Here is my opinion:

Building Boost libraries on Windows/MinGW is really a pain, due to the
fact it is not officially supported.
The last info I read about Boost mingw support is:
"it works or perhaps it does not work",
and experience show me each version of boost needed different
workarounds to build them.

When we used the boost headers only, we did not have any issue (just
sometimes bugs, like in any tool)
The nightmare started when using boost libraries.
The msys2 guys made a great work when supporting boost libraries. Thanks
to them.
But this is very annoying to be fully dependent of them.

AFAIK, boost libraries are mainly used in Avhttp and coroutines.
Bye Avhttp (thanks, Mark).
boost::coroutine still use them, and it was always a nightmare, since
the beginning.

So, if an alternative implementation to boost::context exists, I will be
*very* happy.

(Note: *very happy* is an understatement)

> --
> My offer would be to test an alternative implementation, to drop another 
> Boost 
> dependency and build upon a standard library foundation.
> Thanks,
> Torsten
> *Gesendet:* Montag, 04. Januar 2016 um 06:07 Uhr
> *Von:* "Mark Roszko" <mark.ros...@gmail.com>
> *An:* "Torsten Hüter" <torsten...@gmx.de>
> *Cc:* "KiCad Mailinglist" <kicad-developers@lists.launchpad.net>
> *Betreff:* Re: [Kicad-developers] KiCad Coroutines
>  >Summary: Fibers (Windows) and Pthreads (all POSIX-compliant operating 
> systems) 
> seem to be the most promising alternatives for context switching
> 
> Since msys2 uses mingw, it used to have a pthreads implementation when
> I used it years ago. Not sure if the latest has a package for it.


-- 
Jean-Pierre CHARRAS

___
Mailing list: https://launchpad.net/~kicad-developers
Post to : kicad-developers@lists.launchpad.net
Unsubscribe : https://launchpad.net/~kicad-developers
More help   : https://help.launchpad.net/ListHelp

Re: [Kicad-developers] KiCad Coroutines

[Wayne Stambaugh]

On 1/4/2016 7:08 AM, jp charras wrote:
> Le 04/01/2016 10:59, "Torsten Hüter" a écrit :
>> Mark,
>> You're right Pthreads should eventually work on Windows/MinGW too.
>> I don't know if their policy has changed, but the official documentation 
>> lists 
>> still MASM as requirement for Windows. I did no further research about the 
>> reasons.
>> Please read the (old) statement of our project leader here:
>> http://mingw-users.1079350.n2.nabble.com/Boost-Context-MinGW-EXPORT-keyword-td7582532.html
>> Quote from Wayne:
>> "[..] Good luck!  The KiCad project had the same issue.  We even submitted 
>> our
>> patch but they refused it.  The answer was always the same.  Use MASM
>> with no reasonable explanation as to why the gnu assembler was
>> unacceptable.
>> [..]
>> I'm beginning to regret using Boost in KiCad.
>> [..]"
>> I don't know if this opinion is still valid.

It is still valid.  However, the Boost project as of 1.59 is now
providing GNU assembly files for the context library on windows so
building them is not quite as bad as it used to be.  Unfortunately, the
assembly files they have provided are broken on x86_64 builds.  Why they
did not use the patches that we provided them I cannot say for sure but
at least they are addressing the issue.  I believe there is a new
maintainer for the Boost context library who seems to be willing to work
with users which is a big improvement over the previous situation.

> 
> Torsten,
> 
> Here is my opinion:
> 
> Building Boost libraries on Windows/MinGW is really a pain, due to the
> fact it is not officially supported.
> The last info I read about Boost mingw support is:
> "it works or perhaps it does not work",
> and experience show me each version of boost needed different
> workarounds to build them.
> 
> When we used the boost headers only, we did not have any issue (just
> sometimes bugs, like in any tool)
> The nightmare started when using boost libraries.
> The msys2 guys made a great work when supporting boost libraries. Thanks
> to them.
> But this is very annoying to be fully dependent of them.
> 
> AFAIK, boost libraries are mainly used in Avhttp and coroutines.
> Bye Avhttp (thanks, Mark).
> boost::coroutine still use them, and it was always a nightmare, since
> the beginning.
> 
> So, if an alternative implementation to boost::context exists, I will be
> *very* happy.
> 
> (Note: *very happy* is an understatement)
> 
>> --
>> My offer would be to test an alternative implementation, to drop another 
>> Boost 
>> dependency and build upon a standard library foundation.

I'm fine with this as long as @Tom is OK with it and it's not too
disruptive.

Cheers,

Wayne

>> Thanks,
>> Torsten
>> *Gesendet:* Montag, 04. Januar 2016 um 06:07 Uhr
>> *Von:* "Mark Roszko" <mark.ros...@gmail.com>
>> *An:* "Torsten Hüter" <torsten...@gmx.de>
>> *Cc:* "KiCad Mailinglist" <kicad-developers@lists.launchpad.net>
>> *Betreff:* Re: [Kicad-developers] KiCad Coroutines
>>  >Summary: Fibers (Windows) and Pthreads (all POSIX-compliant operating 
>> systems) 
>> seem to be the most promising alternatives for context switching
>>
>> Since msys2 uses mingw, it used to have a pthreads implementation when
>> I used it years ago. Not sure if the latest has a package for it.
> 
> 

___
Mailing list: https://launchpad.net/~kicad-developers
Post to : kicad-developers@lists.launchpad.net
Unsubscribe : https://launchpad.net/~kicad-developers
More help   : https://help.launchpad.net/ListHelp

Re: [Kicad-developers] KiCad Coroutines

[Mark Roszko]

>My offer would be to test an alternative implementation, to drop another Boost 
>dependency and build upon a standard library foundation.
>pthreads is not standard library <.<

C+11 threads are standard library not pthreads ^^ Ideal implementation
is in a way that can be swapped for the C++11 variant later on once
kicad switches over

___
Mailing list: https://launchpad.net/~kicad-developers
Post to : kicad-developers@lists.launchpad.net
Unsubscribe : https://launchpad.net/~kicad-developers
More help   : https://help.launchpad.net/ListHelp

Re: [Kicad-developers] KiCad Coroutines

[Torsten Hüter]

Hi Wayne,
 
> What version(s) of Windows support fibers? I didn't see any indication
> on the links above.
 
See this link:

https://support.microsoft.com/en-us/kb/128531

Fibers were added to the WIN32-API with the Service Pack 5 for Windows NT 3.51.

Quote from the above link:

"[..] 
Service Pack 5 contains 3 additions to the Win32 API:

Fibers
--
A fiber is a lightweight thread that is manually scheduled.  Fibers do not
provide advantages over a well-designed multithreaded application.
However, fibers can make it easier to port applications that were designed
to schedule their own threads.
[..]"

So any actual desktop Windows version (XP and later) should support these 
functions.

I've tried the provided example from Microsoft on MSYS2 / Windows 10 (64 Bit) 
and it was running well.

> I don't have any strong opinions either way. However, I did not write
> the code. Tom may have a stronger opinion about this than I do since he
> wrote the code. Personally I'm not the biggest fan of coroutines. I
> know the make the code easier to read but I'm not convinced they are
> necessary in an event driven design.

This is true, but as you have written - the code is easier to read. Another 
alternatives would be to use state machines (with switch/case etc.) or 
stackless coroutines (Protothreads are an example, 
https://en.wikipedia.org/wiki/Protothreads). In Python are Coroutines a 
language concept, see also https://wiki.python.org/moin/Generators .

An example use case would be a long running computation, with a coroutine you 
could yield to the main thread and update a status bar or similar. 
 
Thanks,
Torsten

___
Mailing list: https://launchpad.net/~kicad-developers
Post to : kicad-developers@lists.launchpad.net
Unsubscribe : https://launchpad.net/~kicad-developers
More help   : https://help.launchpad.net/ListHelp

Re: [Kicad-developers] KiCad Coroutines

[Mark Roszko]

>Summary: Fibers (Windows) and Pthreads (all POSIX-compliant operating systems) 
>seem to be the most promising alternatives for context switching

Since msys2 uses mingw, it used to have a pthreads implementation when
I used it years ago. Not sure if the latest has a package for it.





>I did some research and this thing seems to cause constantly trouble, because 
>some assembly "hackery" is required and only MASM is officially supported

Not true at all.
MINGW64 is supported, someone just broke it in boost context
1.59 on our use case.
Works fine in 1.57. The problem is finding
someone brave enough to debug assembly <.<

___
Mailing list: https://launchpad.net/~kicad-developers
Post to : kicad-developers@lists.launchpad.net
Unsubscribe : https://launchpad.net/~kicad-developers
More help   : https://help.launchpad.net/ListHelp

Re: [Kicad-developers] KiCad Coroutines

[Wayne Stambaugh]

On 1/3/2016 1:18 PM, "Torsten Hüter" wrote:
> Hi Wayne,
> 
> I've recently tried to compile KiCad on Windows (64Bit) and I've noticed that 
> PcbNew is crashing at context switching - but I had no problems early 2015 
> with that. I've seen that you have already filed a bug report [1]. I did some 
> research and this thing seems to cause constantly trouble, because some 
> assembly "hackery" is required and only MASM is officially supported [2].
> 
> Rather than trying to fix Boost, makes it sense to consider an alternative 
> way or library for the same purpose?
> 
> I've found so far these alternatives as back-end for "stackful" coroutines:
> 
> (1) Fibers on Windows
> 
> https://msdn.microsoft.com/de-de/library/windows/desktop/ms682661%28v=vs.85%29.aspx
> https://en.wikipedia.org/wiki/Fiber_%28computer_science%29

What version(s) of Windows support fibers?  I didn't see any indication
on the links above.

> 
> This seems to be the safest alternative for Windows, because Fibers are fully 
> supported by the OS and are thread-safe.
> 
> (2) POSIX ucontext.h - setcontext, getcontext, makecontext and swapcontext
> 
> Very similar compared to Windows Fibers, but deprecated since POSIX.1-2008, 
> quote from https://en.wikipedia.org/wiki/Setcontext :
> 
> "[..] POSIX.1-2004 obsoleted these functions, and in POSIX.1-2008 they were 
> removed [..]"
> 
> It is still possible to use these functions on Linux, but these are not 
> future-proof and there are no replacement functions.
> 
> An interesting talk about this topic can be found here:
> http://stackoverflow.com/questions/4298986/is-there-something-to-replace-the-ucontext-h-functions
> 
> (3) POSIX Threads
> 
> https://computing.llnl.gov/tutorials/pthreads/
> 
> It is possible to emulate Fibers with pthreads, here is a good article about 
> this subject:
> 
> http://sjansen.blogspot.de/2008/04/coroutines-in-cc-beyond-duffs-device.html
> 
> The downside is the larger overhead, but this shouldn't be a big issue for 
> KiCad, I'm sure that not more than a few coroutines are created for the tool 
> framework. Quote from the above article:
> 
> "[..] As an interesting sidenote, I'm not sure how debuggers such as gdb and 
> valgrind handle applications that switch their stacks. It might be that using 
> threads actually makes things easier to debug in this case (as the tools 
> already have support for threads). [..]"
> 
> Pthreads offer also some mechanisms for stack overflow detection.
> 
> (4) setjmp / longjmp
> 
> https://en.wikipedia.org/wiki/Setjmp.h
> 
> Context switching is also possible using these functions and some libraries 
> like "picoro" use that for Coroutines. However stack handling is difficult, 
> especially stack overflow protection is not easy to implement.
> 
> http://dotat.at/cgi/git/picoro.git
> 
> --
> 
> Summary: Fibers (Windows) and Pthreads (all POSIX-compliant operating 
> systems) seem to be the most promising alternatives for context switching. 
> This way it would be possible to get rid of the Boost.Context dependency 
> while using standard libraries.
> 
> What do you think, should I invest more time into this topic (refactoring of 
> coroutine.h using these alternatives and write a stress test application)?

I don't have any strong opinions either way.  However, I did not write
the code.  Tom may have a stronger opinion about this than I do since he
wrote the code.  Personally I'm not the biggest fan of coroutines.  I
know the make the code easier to read but I'm not convinced they are
necessary in an event driven design.

Thanks for the links.  They are interesting.

Cheers,

Wayne


> 
> Thanks, 
> Torsten
> 
> [1] KiCad issues with Boost 1.59, bug report, 
> https://github.com/Alexpux/MINGW-packages/issues/836
> [2] Boost context requirements, 
> http://www.boost.org/doc/libs/1_60_0/libs/context/doc/html/context/requirements.html
> 
> ___
> Mailing list: https://launchpad.net/~kicad-developers
> Post to : kicad-developers@lists.launchpad.net
> Unsubscribe : https://launchpad.net/~kicad-developers
> More help   : https://help.launchpad.net/ListHelp
> 


___
Mailing list: https://launchpad.net/~kicad-developers
Post to : kicad-developers@lists.launchpad.net
Unsubscribe : https://launchpad.net/~kicad-developers
More help   : https://help.launchpad.net/ListHelp