Re: [Kicad-developers] [PATCH] Fix use-after-release in SYMBOL_LIB_TABLE::Parse()

[Kristian Nielsen]

Or maybe something like this patch instead, to not leak the row (sorry, I
mixed up LIB_TABLE::InsertRow() in common/lib_table_base.cpp with the
function of the same name in new/sch_lib_table.cpp; the latter uses an
auto_ptr&, the former does not).

Or maybe this code is still in flux, and the ownership will be changed/fixed
in other ways - in any case, just wanted to point out this, now that I saw
it. The main point is to not use row->GetNickName() after giving up
ownership of the pointer to InsertRow().

 - Kristian.

Kristian Nielsen <kniel...@knielsen-hq.org> writes:

> I stumbled upon this in SYMBOL_LIB_TABLE::Parse():
>
> if( !InsertRow( row.release() ) )
> {
> wxString msg = wxString::Format(
> _( "'%s' is a duplicate symbol library nickname" 
> ),
> GetChars( row->GetNickName() ) );
>
> I got a segfault from this, because the error message accesses row after
> row.release() has given up ownership.

diff --git a/eeschema/symbol_lib_table.cpp b/eeschema/symbol_lib_table.cpp
index 5ed5565..3cf8e7c 100644
--- a/eeschema/symbol_lib_table.cpp
+++ b/eeschema/symbol_lib_table.cpp
@@ -185,11 +185,13 @@ void SYMBOL_LIB_TABLE::Parse( LIB_TABLE_LEXER* in )
 // use doReplace in InsertRow().  (However a fallBack table can have a
 // conflicting nickName and ours will supercede that one since in
 // FindLib() we search this table before any fall back.)
-if( !InsertRow( row.release() ) )
+LIB_TABLE_ROW* row_to_insert = row.release();
+if( !InsertRow( row_to_insert ) )
 {
 wxString msg = wxString::Format(
 _( "'%s' is a duplicate symbol library nickname" ),
-GetChars( row->GetNickName() ) );
+GetChars( row_to_insert->GetNickName() ) );
+delete row_to_insert;
 THROW_PARSE_ERROR( msg, in->CurSource(), in->CurLine(), lineNum, offset );
 }
 }
___
Mailing list: https://launchpad.net/~kicad-developers
Post to : kicad-developers@lists.launchpad.net
Unsubscribe : https://launchpad.net/~kicad-developers
More help   : https://help.launchpad.net/ListHelp

[Kicad-developers] [PATCH] Fix use-after-release in SYMBOL_LIB_TABLE::Parse()

[Kristian Nielsen]

I stumbled upon this in SYMBOL_LIB_TABLE::Parse():

if( !InsertRow( row.release() ) )
{
wxString msg = wxString::Format(
_( "'%s' is a duplicate symbol library nickname" ),
GetChars( row->GetNickName() ) );

I got a segfault from this, because the error message accesses row after
row.release() has given up ownership.

I'm not sure what the status of the new symbol library table code is at this
point, but I thought I might as well mention it. Attached patch fixes the
segfault.

Hope this helps,

 - Kristian.

diff --git a/eeschema/symbol_lib_table.cpp b/eeschema/symbol_lib_table.cpp
index 5ed5565..556abf1 100644
--- a/eeschema/symbol_lib_table.cpp
+++ b/eeschema/symbol_lib_table.cpp
@@ -185,13 +185,13 @@ void SYMBOL_LIB_TABLE::Parse( LIB_TABLE_LEXER* in )
 // use doReplace in InsertRow().  (However a fallBack table can have a
 // conflicting nickName and ours will supercede that one since in
 // FindLib() we search this table before any fall back.)
+// Note that we need to format the error message in advance, as the
+// ownership of the row pointer is passed to InsertRow().
+wxString msg = wxString::Format(
+_( "'%s' is a duplicate symbol library nickname" ),
+GetChars( row->GetNickName() ) );
 if( !InsertRow( row.release() ) )
-{
-wxString msg = wxString::Format(
-_( "'%s' is a duplicate symbol library nickname" ),
-GetChars( row->GetNickName() ) );
 THROW_PARSE_ERROR( msg, in->CurSource(), in->CurLine(), lineNum, offset );
-}
 }
 }
 
___
Mailing list: https://launchpad.net/~kicad-developers
Post to : kicad-developers@lists.launchpad.net
Unsubscribe : https://launchpad.net/~kicad-developers
More help   : https://help.launchpad.net/ListHelp

Re: [Kicad-developers] KiCad Coroutines

[Kristian Nielsen]

Lorenzo Marcantonio  writes:

> OTOH I never did that on Win32/64 so maybe it's totally undependable to work 
> :D
> Isn't there some MSDN article on doing that?

On windows, fibers can be used, and seem to work well. On most other
platforms, ucontext can be used (only exception I am aware of is that it was
removed on (some of the?) BSDs).

In the MariaDB non-blocking client library, ucontext and windows fibers are
used (and some custom assembly for performance, but that should be
unnecessary for KiCAD, I think). It seems to cover most things.

I would be happy to help with Co-routine infrastructure in KiCAD. Still, it
would be preferable if some existing layer on top (like Boost::Context or
similar) can be used, to not have to maintain KiCAD's own implementation.

> Simple?!? Well, maybe implementing but maintaining is another thing...

I agree. The main point of using co-routines in the first place is to
maintain the state of the stack across waiting operations.

 - Kristian.

___
Mailing list: https://launchpad.net/~kicad-developers
Post to : kicad-developers@lists.launchpad.net
Unsubscribe : https://launchpad.net/~kicad-developers
More help   : https://help.launchpad.net/ListHelp

Re: [Kicad-developers] KiCad Coroutines

[Kristian Nielsen]

Lorenzo Marcantonio  writes:

> On Thu, Jan 07, 2016 at 12:08:22PM +, Mário Luzeiro wrote:

> Some thing like this:
> - Select a tool (suspend for a selection)
> - Other things happens, then user select a thing
> - (restart the tool coroutine) Check thing selected and so on

So to me, this sounds like an excellent use-case for co-routines.

I suppose the idea is to have code something like this (random pseudocode)?

  a = some_prepare_tool_stuff();
  b = other_stuff();
  select = get_selection();
  if (select.aborted) {
cleanup(a, b);
return;
  }
  more_tool_stuff(a, b, select.data);

The great thing about co-routines is that it allows to abstract things in
helper functions, like get_selection() above. You don't need to know exactly
how co-routine scheduling will happen inside get_selection, all the details
can be handled inside that function.

In contrast, without co-routines, every such usage will have to be split up
in multiple event handler / callback functions, at each point where event
handling can happen. This breaks the code flow, breaks the data flow, and
makes things a lot harder to read and understand.

Just my two cents,

 - Kristian.

___
Mailing list: https://launchpad.net/~kicad-developers
Post to : kicad-developers@lists.launchpad.net
Unsubscribe : https://launchpad.net/~kicad-developers
More help   : https://help.launchpad.net/ListHelp

Re: [Kicad-developers] Removal of download_boost.cmake.

[Kristian Nielsen]

Wayne Stambaugh  writes:

> The only boost patch that is not mingw or osx specific is
> boost_cstdint.patch and I'm not sure this patch makes any difference.
> There is one way to find out.  Build kicad with stock boost on 14.04 and
> see there are any issues.  Would someone please test this since there
> seems to be a lot of ubuntu 14.04 users out there?

I did a test build on a fresh Ubuntu 14.04 (amd64) that I had lying
around. It seems to work ok:

  cmake -DKICAD_SKIP_BOOST=YES -DCMAKE_INSTALL_PREFIX=/home/knielsen/kicad ../

This was from newest Git (Bzr 6372, Git 5d429ed).

The build succeeds, and KiCAD seems to run ok (schematics, pcbnew,
3d-viewer, ...). So removing bundled boost seems ok from Ubuntu 14.04
perspective.

To get it to build, I had to install libwx 3.0.2 and GLM 0.9.6.3 from
source, since Ubuntu had too old versions. I used this to install remaining
dependencies:

  sudo apt-get install git build-essential cmake checkinstall doxygen 
zlib1g-dev libwebkitgtk-dev libglew-dev libcairo2-dev libbz2-dev libssl-dev 
libgles2-mesa-dev libgl1-mesa-dev libglu1-mesa-dev libopenvg1-mesa-dev 
libosmesa6-dev libwebkitgtk-dev libboost-all-dev

I did have one issue with GLM, which should be unrelated to boost. I
installed GLM in CMAKE_INSTALL_PREFIX, and cmake finds it:

  -- Found GLM: /home/knielsen/kicad/include (found suitable version "0.9.6.3", 
minimum required is "0.9.5.4") 

But the build still fails, it looks like cmake did not add the required -I
option for the path it found:

cd /home/knielsen/kicad/kicad-source-mirror/build/3d-viewer && /usr/bin/c++   
-DHAVE_STDINT_H -DKICAD_KEEPCASE -DPCBNEW -DUSE_OPENMP -DWXUSINGDLL 
-DWX_COMPATIBILITY -D_FILE_OFFSET_BITS=64 -D__WXGTK__ -Wall  
-fvisibility=hidden -fvisibility-inlines-hidden -Wno-unused-local-typedefs 
-Wno-strict-aliasing -fopenmp -pthread -O3 -DNDEBUG -DNDEBUG -fPIC 
-I/home/knielsen/kicad/kicad-source-mirror/include 
-I/home/knielsen/kicad/kicad-source-mirror/3d-viewer/. -isystem 
/home/knielsen/kicad/lib/wx/include/gtk2-unicode-3.0 -isystem 
/home/knielsen/kicad/include/wx-3.0 
-I/home/knielsen/kicad/kicad-source-mirror/3d-viewer/textures 
-I/home/knielsen/kicad/kicad-source-mirror/3d-viewer/../pcbnew 
-I/home/knielsen/kicad/kicad-source-mirror/3d-viewer/../polygon 
-I/home/knielsen/kicad/kicad-source-mirror/build-o 
CMakeFiles/3d-viewer.dir/dialogs/dialog_3D_view_option.cpp.o -c 
/home/knielsen/kicad/kicad-source-mirror/3d-viewer/dialogs/dialog_3D_view_option.cpp

In file included from 
/home/knielsen/kicad/kicad-source-mirror/3d-viewer/./3d_struct.h:36:0,
 from 
/home/knielsen/kicad/kicad-source-mirror/3d-viewer/./3d_viewer.h:40,
 from 
/home/knielsen/kicad/kicad-source-mirror/3d-viewer/dialogs/dialog_3D_view_option.cpp:26:
/home/knielsen/kicad/kicad-source-mirror/3d-viewer/./3d_material.h:35:23: fatal 
error: glm/glm.hpp: No such file or directory

I fixed it with a quick hack (ln -s ../glm ~/kicad/include/wx-3.0/), as I
was too lazy to look up how to pass extra -I options with cmake. I should
probably try to come up with a small patch to make cmake add the required -I
automatically for GLM.

Hope this helps, let me know if there is anything further that needs
testing.

 - Kristian.

___
Mailing list: https://launchpad.net/~kicad-developers
Post to : kicad-developers@lists.launchpad.net
Unsubscribe : https://launchpad.net/~kicad-developers
More help   : https://help.launchpad.net/ListHelp

Re: [Kicad-developers] Removal of download_boost.cmake.

[Kristian Nielsen]

Ok, looking further, I think it is just an oversight, GLM_INCLUDE_DIR is
missing in 3d-viewer/CMakeLists.txt. Adding it as per the attached patch
fixes the error for me. The problem will only be seen when GLM is in a
different location from other libraries already included in CMakeLists.txt.

Wayne, will you pick up the attached one-liner patch directly?

Wayne Stambaugh  writes:

> What is the result of `grep GLM < CMakeCache.txt` when run from your
> build folder?  You should see something like:

Yes, it's very similar:

$ grep GLM < CMakeCache.txt 
//GLM library header path.
GLM_INCLUDE_DIR:PATH=/home/knielsen/kicad/include
//Details about finding GLM
FIND_PACKAGE_MESSAGE_DETAILS_GLM:INTERNAL=[/home/knielsen/kicad/include][0.9.6.3][v0.9.6.3(0.9.5.4)]
//ADVANCED property for variable: GLM_INCLUDE_DIR
GLM_INCLUDE_DIR-ADVANCED:INTERNAL=1
GLM_VERSION_MAJOR:INTERNAL=0
GLM_VERSION_MINOR:INTERNAL=9
GLM_VERSION_PATCH:INTERNAL=6
GLM_VERSION_TWEAK:INTERNAL=3

Clearly, cmake is finding my installed GLM library from the
CMAKE_INSTALL_PREFIX (as that is the only place ~/kicad/ is mentioned). I
get the same error if I explicitly put -DGLM_ROOT_DIR=$HOME/kicad/include on
the cmake line. It's probably just the missing GLM_INCLUDE_DIR in
3d-viewer/CMakeLists.txt

 - Kristian.

diff --git a/3d-viewer/CMakeLists.txt b/3d-viewer/CMakeLists.txt
index b48e505..47014f6 100644
--- a/3d-viewer/CMakeLists.txt
+++ b/3d-viewer/CMakeLists.txt
@@ -6,6 +6,7 @@ include_directories(
 ../pcbnew
 ../polygon
 ${GLEW_INCLUDE_DIR}
+${GLM_INCLUDE_DIR}
 ${INC_AFTER}
 )
 
___
Mailing list: https://launchpad.net/~kicad-developers
Post to : kicad-developers@lists.launchpad.net
Unsubscribe : https://launchpad.net/~kicad-developers
More help   : https://help.launchpad.net/ListHelp

Re: [Kicad-developers] [rfc patch] replace avhttp with libcurl

[Kristian Nielsen]

Wayne Stambaugh  writes:

> another place to look.  Has anyone tried building kicad with worker
> threads disabled in footprint_info.cpp to confirm if this resolves the
> issue?

I am not sure if you are thinking of the problem I reported a few days ago
with footprint loading hanging, or of some other problem.

But about the problem I reported: I tried building with worker threads
disabled, and this does not reduce the problem. Nor is the problem related
to DNS. Tcpdump shows it is caused by packet loss. I suspect the packet loss
happens on the github end, it is a very busy site, after all. But it could
also be some strange problem on my network, of course. In any case, that
particular problem I think the KiCAD code can do nothing about, except to
have a timeout (and possibly a retry).

 - Kristian.

___
Mailing list: https://launchpad.net/~kicad-developers
Post to : kicad-developers@lists.launchpad.net
Unsubscribe : https://launchpad.net/~kicad-developers
More help   : https://help.launchpad.net/ListHelp

Re: [Kicad-developers] Occasional hang in Cvpcb when retrieving footprint libraries from github

[Kristian Nielsen]

Kristian Nielsen <kniel...@knielsen-hq.org> writes:

> Since I am able to reproduce, even in a single-threaded configuration, I
> will poke around a bit and see if I can figure out what is going on.

Hm, the problem is actually just that there is no timeout in the code for
the download of the individual footprint .zip files. So if the network
connection gets stuck, Cvpcb hangs forever with an unresponsive GUI.

Unfortunately, I did not spot an easy way to set a timeout for the avhttp
request. Maybe Mark Roszko's libcurl patch will make this easy to add. It
would be nice that the GUI would not get stuck, of course, but if most
people are unaffected, maybe not so critical just now. I wonder if this is
occasional Github overload, or just some problem with my network connection.

For anyone curious: the tcp connection gets stuck in an interesting way,
that is what confused me at first and made me disregard just a broken tcp
connection:

17:02:27.417749 IP codeload.github.com.https > urd.37817: Flags [.], seq 
1:1425, ack 296, win 15, options [nop,nop,TS val 1504320372 ecr 7315069], 
length 1424
17:02:27.417786 IP urd.37817 > codeload.github.com.https: Flags [.], ack 1425, 
win 251, options [nop,nop,TS val 7315104 ecr 1504320372], length 0
17:04:27.416054 IP codeload.github.com.https > urd.37817: Flags [F.], seq 3029, 
ack 296, win 15, options [nop,nop,TS val 1504350372 ecr 7315104], length 0
17:04:27.416079 IP urd.37817 > codeload.github.com.https: Flags [.], ack 1425, 
win 251, options [nop,nop,TS val 7345103 ecr 1504320372,nop,nop,sack 1 
{3029:3030}], length 0

The FIN packet from the github server has sequence number 3029, but the
client has only seen sequence number 1425, so some packets got lost. This
causes the FIN to be disregarded by the client, and the connection to get
stuck.

(Of course a tcp connection can get stuck for many reasons, in the general
case. This just happened to be the pattern I saw in multiple hangs on my
machine, for some odd reason).

Hope this helps,

 - Kristian.

___
Mailing list: https://launchpad.net/~kicad-developers
Post to : kicad-developers@lists.launchpad.net
Unsubscribe : https://launchpad.net/~kicad-developers
More help   : https://help.launchpad.net/ListHelp

Re: [Kicad-developers] [rfc patch] replace avhttp with libcurl

[Kristian Nielsen]

Mark Roszko  writes:

> I have written a replacement implementation that drops avhttp in favor
> of libcurl.

Works for me on Debian stable.

And as a bonus, broken network connection now causes a timeout rather than
an infinite hang:

  Errors were encountered loading footprints
  IO_ERROR: 
https://codeload.github.com/KiCad/Relays_ThroughHole.pretty/zip/master
  Cannot get/download Zip archive: 
'https://github.com/KiCad/Relays_ThroughHole.pretty'
  for library path: 'IO_ERROR: CURL Request Failed: Timeout was reached

You seem to have trailing whitespace lines in your patch, you might want to
remove those.

And you left a comment about avhttp on the function
GITHUB_GETLIBLIST::remoteGetJSON() when you moved/renamed it, probably that
comment should be deleted now?

Thanks,

 - Kristian.

___
Mailing list: https://launchpad.net/~kicad-developers
Post to : kicad-developers@lists.launchpad.net
Unsubscribe : https://launchpad.net/~kicad-developers
More help   : https://help.launchpad.net/ListHelp

[Kicad-developers] Occasional hang in Cvpcb when retrieving footprint libraries from github

[Kristian Nielsen]

Sometimes, but not everytime, Cvpcb hangs for me when started when it tries
to read the footprints from Github. It looks like a race, maybe a bug in
Boost. I suspect this might have been the causeof bugs like
https://bugs.launchpad.net/kicad/+bug/1489128, just the random nature of the
bug making debugging difficult.

Does anyone else experience this? This is not the normal download delay, it
seems stuck for good in recvmsg():

$ strace -p 8161
Process 8161 attached
recvmsg(61, 

$ lsof -p 8145
kicad   8145 knielsen   61u  IPv4  85080  0t0  TCP 
urd:37817->codeload.github.com:https (ESTABLISHED)

$ sudo tcpdump -i any host codeload.github.com
... (other parallel connections omitted for brevity)
17:02:27.146802 IP urd.37817 > codeload.github.com.https: Flags [S], seq 
3346029680, win 29200, options [mss 1460,sackOK,TS val 7315036 ecr 0,nop,wscale 
7], length 0
17:02:27.277308 IP codeload.github.com.https > urd.37817: Flags [S.], seq 
1746405727, ack 3346029681, win 14240, options [mss 1436,sackOK,TS val 
1504320336 ecr 7315036,nop,wscale 10], length 0
17:02:27.277317 IP urd.37817 > codeload.github.com.https: Flags [.], ack 1, win 
229, options [nop,nop,TS val 7315069 ecr 1504320336], length 0
17:02:27.277354 IP urd.37817 > codeload.github.com.https: Flags [P.], seq 
1:296, ack 1, win 229, options [nop,nop,TS val 7315069 ecr 1504320336], length 
295
17:02:27.417749 IP codeload.github.com.https > urd.37817: Flags [.], seq 
1:1425, ack 296, win 15, options [nop,nop,TS val 1504320372 ecr 7315069], 
length 1424
17:02:27.417786 IP urd.37817 > codeload.github.com.https: Flags [.], ack 1425, 
win 251, options [nop,nop,TS val 7315104 ecr 1504320372], length 0
17:04:27.416054 IP codeload.github.com.https > urd.37817: Flags [F.], seq 3029, 
ack 296, win 15, options [nop,nop,TS val 1504350372 ecr 7315104], length 0
17:04:27.416079 IP urd.37817 > codeload.github.com.https: Flags [.], ack 1425, 
win 251, options [nop,nop,TS val 7345103 ecr 1504320372,nop,nop,sack 1 
{3029:3030}], length 0

See how the client sent 295 bytes and Github returned 1424 bytes - and then
nothing, until Github times out and closes the connection, while the client
is still stuck in recvmsg() on that connection. Clearly, Cvpcb should have
continued the communication here, but got stuck somehow.

GDB stacktraces of the stuck threads (full stacktrace at the end of the
mail):

Thread 2 (Thread 0x7f6458d1d700 (LWP 8161)):
#0  0x7f647e11de9d in recvmsg () at ../sysdeps/unix/syscall-template.S:81
#1  0x7f6470a6f96c in boost::asio::detail::socket_ops::sync_recv(int, 
unsigned char, iovec*, unsigned long, int, bool, boost::system::error_code&) ()
   from /kvm/kicad/bin/_cvpcb.kiface
#2  0x7f6470a8189a in unsigned long 
boost::asio::ssl::detail::io, 
boost::asio::ssl::detail::handshake_op>(boost::asio::basic_stream_socket&, 
boost::asio::ssl::detail::stream_core&, boost::asio::ssl::detail::handshake_op 
const&, boost::system::error_code&) () from /kvm/kicad/bin/_cvpcb.kiface
#3  0x7f6470a91c20 in avhttp::http_stream::open(avhttp::url const&, 
boost::system::error_code&) () from /kvm/kicad/bin/_cvpcb.kiface
#4  0x7f6470a69500 in GITHUB_PLUGIN::remote_get_zip(wxString const&) () 
from /kvm/kicad/bin/_cvpcb.kiface

Thread 1 (Thread 0x7f6481164a80 (LWP 8145)):
#0  pthread_cond_wait@@GLIBC_2.3.2 () at 
../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185
#1  0x7f6470a9bcbb in 
boost::condition_variable::wait(boost::unique_lock&) () from 
/kvm/kicad/bin/_cvpcb.kiface
#2  0x7f6470a9a3bc in boost::thread::join_noexcept() () from 
/kvm/kicad/bin/_cvpcb.kiface
#3  0x7f6470806ce0 in FOOTPRINT_LIST::ReadFootprintFiles(FP_LIB_TABLE*, 
wxString const*) () from /kvm/kicad/bin/_cvpcb.kiface

It kind of looks like a bug in Boost here? I think boost opens a number of
parallel HTTP sessions to github, and somehow there seems to be a race that
causes it to get stuck occasionally. Unless it's a bug in the linux network
stack (I am on version 4.1.3 of the Linux kernel).

Any suggestions on what I could do to try to debug this further? I am using
a self-compiled Kicad latest from Github as of today:

commit f4cf050669955c7c75bb760166ffbd9b9c8f2e6b
Author: jean-pierre charras 
Date:   Wed Dec 2 12:46:55 2015 +0100

Eagle plugin: fix typo errors. Use paired fab layers for layers 51 and 52. 
Not perfect, but better than the old choice (drawing layer)

I am on Debian stable / Jessie. Boost version on the system is 1.55.0.2,
while the one in .downloads-by-cmake/ is boost_1_54_0/ (sorry, not sure how
to tell which of the two is used by the build).

Hope this helps,

 - Kristian.

---
Full GDB stack traces, for reference:

Thread 4 (Thread 0x7f6472f23700 (LWP 8146)):
#0  

Re: [Kicad-developers] Occasional hang in Cvpcb when retrieving footprint libraries from github

[Kristian Nielsen]

Nick Østergaard  writes:

> Did you specifically use -DKICAD_SKIP_BOOST=YES ? If now then you are
> using boost 1.54 that kicad downloaded.
>
> You can see the version used in the help meny in any of the main apps,
> help -> copy version info

No, I did not use -DKICAD_SKIP_BOOST=YES. And yes, the version info shows
I'm using Boost 1.54, thanks for the tip:

  Version: (2015-12-02 BZR 6340, Git f4cf050)-product, release build
  wxWidgets 3.0.2 Unicode and Boost 1.54.0
  Platform: Linux 4.1.3-kn x86_64, 64 bit

 - Kristian.

___
Mailing list: https://launchpad.net/~kicad-developers
Post to : kicad-developers@lists.launchpad.net
Unsubscribe : https://launchpad.net/~kicad-developers
More help   : https://help.launchpad.net/ListHelp

Re: [Kicad-developers] Occasional hang in Cvpcb when retrieving footprint libraries from github

[Kristian Nielsen]

Nick Østergaard  writes:

> If you feel like testing you could make a new build with 1.55 and see
> if it makes a difference.

Wayne Stambaugh  writes:

> The other option is compile footprint_info.cpp with
>
> #define USE_WORKER_THREADS  0

As it turns out, I could reproduce also when building with Boost 1.55, and
even with #define USE_WORKER_THREADS 0. So it appears to be a race related
to the arrival of packets on the network or something, not an inter-thread
race.

> multiple threads.  I've never had any issues with this so I'm not sure
> why all of a sudden we are having issues with this.

I'm not sure it is all of a sudden. I remember seeing it before some time
ago, though I did not investigate deeper at the time.

I think it is just a very elusive bug that happens only in special
circumstances. Testing a bit more, I discovered that I seem to only be able
to reproduce while having a specific program (vlc) open in the background -
and even then it is only in 10% of cases or something like that.

Since I am able to reproduce, even in a single-threaded configuration, I
will poke around a bit and see if I can figure out what is going on.

Thanks,

 - Kristian.

___
Mailing list: https://launchpad.net/~kicad-developers
Post to : kicad-developers@lists.launchpad.net
Unsubscribe : https://launchpad.net/~kicad-developers
More help   : https://help.launchpad.net/ListHelp

[Kicad-developers] 3D viewer is unusably slow ( 30 minutes) to open this board

[Kristian Nielsen]

I discovered that for one of my boards, the 3d viewer has become basically
unusably slow:

  https://github.com/knielsen/pcb_pov3d

I reproduced this on bzr revision 6091 as follows:

  git clone g...@github.com:knielsen/pcb_pov3d.git
  cd pcb_pov3d/
  pcbnew pcb_pov3d.kicad_pcb

Choose View - 3D Viewer. This makes KiCAD hang for more than 30 minutes
for me (window completely unresponsive) before the 3D view pops up and
things start working again. This used to take only around 2-3 seconds.

This makes the viewer mostly unusable for this board. The board is of medium
complexity, I wonder if the problem will also show on other boards.

I tracked down the two commits that caused this performance regression:

1. 
http://bazaar.launchpad.net/~kicad-product-committers/kicad/product/revision/5889

---
Bzr revision 5889
commit 24170f558816f9661e9e198ecdc40f84772f583b
Author: Tomasz Włostowski tomasz.wlostow...@cern.ch
Date:   Mon Jul 6 15:15:48 2015 +0200

Fixes to SHAPE_POLY_SET slitting/fracturing algo, some speed optimization.
---

Before this commit, the viewer comes up in seconds. After this commit, it
takes more than one minute.

2. 
http://bazaar.launchpad.net/~kicad-product-committers/kicad/product/revision/5999

---
Bzr revision 5999
commit 63b35f40a7210a8240a609888ac7e28606c44172
Author: Tomasz Wlostowski tomasz.wlostow...@cern.ch
Date:   Mon Jul 27 21:45:57 2015 +0200

Removed all dependencies on boost::polygon except for bitmap2component. 
Replaced almost all instances of CPOLYGONS_LIST with SHAPE_POLY_SET.
---

With this commit, the time to open the 3D viewer for this board goes up to
more than 30 minutes.

The code seems to spend most of its time in this stack trace (abbreviated;
full trace at the end of the mail):

  Thread 1 (Thread 0x7fe966833a80 (LWP 16451)):
  #0  0x7fe9521ed483 in ClipperLib::PointInPolygon(ClipperLib::IntPoint 
const, ClipperLib::OutPt*) () from /kvm/kicad/bin/_pcbnew.kiface
  #1  0x7fe9521ed5f5 in ClipperLib::Poly2ContainsPoly1(ClipperLib::OutPt*, 
ClipperLib::OutPt*) () from /kvm/kicad/bin/_pcbnew.kiface
  #2  0x7fe9521f475e in ClipperLib::Clipper::JoinCommonEdges() () from 
/kvm/kicad/bin/_pcbnew.kiface
  #3  0x7fe9521f92ea in ClipperLib::Clipper::ExecuteInternal() () from 
/kvm/kicad/bin/_pcbnew.kiface
  #4  0x7fe9521f31ce in ClipperLib::Clipper::Execute(ClipperLib::ClipType, 
ClipperLib::PolyTree, ClipperLib::PolyFillType, ClipperLib::PolyFillType) 
[clone .part.287] () from /kvm/kicad/bin/_pcbnew.kiface
  #5  0x7fe9521a3649 in SHAPE_POLY_SET::booleanOp(ClipperLib::ClipType, 
SHAPE_POLY_SET const) () from /kvm/kicad/bin/_pcbnew.kiface
  #6  0x7fe951ed68ec in EDA_3D_CANVAS::buildBoard3DView(unsigned int, 
unsigned int, REPORTER*, REPORTER*) () from /kvm/kicad/bin/_pcbnew.kiface
  #7  0x7fe951ed3ba6 in EDA_3D_CANVAS::CreateDrawGL_List(REPORTER*, 
REPORTER*) () from /kvm/kicad/bin/_pcbnew.kiface
  #8  0x7fe951ed4823 in EDA_3D_CANVAS::Redraw() () from 
/kvm/kicad/bin/_pcbnew.kiface

It seems this code is looping over very large vectors. Eg. I saw
m_Joins.size()==14207 in the outer loop of Clipper::JoinCommonEdges() and
m_PolyOuts.size()==9756 in the inner loop (and then a third nested loop
within that in Poly2ContainsPoly1()). I wonder if something in my board
causes excessive long lists to be generated? Or if it is hitting some case
that causes an O(n**3) complexity from three nested loops?

I can take a closer look at the code or run some profiling, but I wanted to
provide this info now, in case someone who knows the code has some idea. I
saw there was already some discussion around these commits and elimiation of
boost::polygon. Hopefully some way can be found to improve the performance
of the new code to something a bit closer to what it was before.

I suppose if KiCAD can get stuck for many minutes opening the 3D viewer,
most users would think it has gotten completely stuck (I know I did, at
first), which is not very good, obviously.

Hope this helps,

 - Kristian.


Full GDB stack trace of where the code seemed to be spending its time, for
completeness:

Thread 1 (Thread 0x7fe966833a80 (LWP 16451)):
#0  0x7fe9521ed483 in ClipperLib::PointInPolygon(ClipperLib::IntPoint 
const, ClipperLib::OutPt*) () from /kvm/kicad/bin/_pcbnew.kiface
#1  0x7fe9521ed5f5 in ClipperLib::Poly2ContainsPoly1(ClipperLib::OutPt*, 
ClipperLib::OutPt*) () from /kvm/kicad/bin/_pcbnew.kiface
#2  0x7fe9521f475e in ClipperLib::Clipper::JoinCommonEdges() () from 
/kvm/kicad/bin/_pcbnew.kiface
#3  0x7fe9521f92ea in ClipperLib::Clipper::ExecuteInternal() () from 
/kvm/kicad/bin/_pcbnew.kiface
#4  0x7fe9521f31ce in ClipperLib::Clipper::Execute(ClipperLib::ClipType,