Re: [scikit-learn] Heisenbug?

2020-01-17 Thread Dan Stromberg
It's looking, at this point, like: 1) The NaN's are real 2) They're coming from some XGBoost native code, or perhaps a Python<->native boundary, which is interfacing using ctypes. The print's that didn't print were probably because of a misplaced flush. The debugger that didn't debug was probably

Re: [scikit-learn] Heisenbug?

2019-12-18 Thread Dan Stromberg
Any (further) suggestions folks? BTW, when I say pudb fails to start, I mean it's tracebacking trying to get None.fileno() In other pieces of (C)Python code I've tried it in, pudb.set_trace() worked nicely. On Tue, Dec 17, 2019 at 7:50 AM Dan Stromberg wrote: > > Hi. > > Overflow does sound ki

Re: [scikit-learn] Heisenbug?

2019-12-17 Thread Dan Stromberg
Here are the inputs to _assert_all_finite() on one specific failed run. They look finite to me: X: array([0.6150936 , 0.24652782, 0.8880004 , 0.2016928 , 0.80948585, 0.10764928, 0.81631166, 0.25909033, 0.9299345 , 0.10186833, 0.81581795, 0.21659133, 0.8279047 , 0.11432098, 0.7335735 ,

Re: [scikit-learn] Heisenbug?

2019-12-17 Thread Dan Stromberg
Hi. Overflow does sound kind of possible. We're sending semi-random values to the test. I believe our systems are all x86_64, Linux. Some are Ubuntu 16.04, some are Mint 19.2. I realized on the way to work this morning, that I left out some important information; I suspect a heisenbug for 3 re

Re: [scikit-learn] Heisenbug?

2019-12-16 Thread Joel Nothman
Hi Dan, this kind of error can come from overflow. Are all of your test systems the same architecture? On Tue., 17 Dec. 2019, 12:03 pm Dan Stromberg, wrote: > Hi folks. > > I'm new to Scikit-learn. > > I have a very large Python project that seems to have a heisenbug which is > manifesting in sc

[scikit-learn] Heisenbug?

2019-12-16 Thread Dan Stromberg
Hi folks. I'm new to Scikit-learn. I have a very large Python project that seems to have a heisenbug which is manifesting in scikit-learn code. Short of constructing an SSCCE, are there any magical techniques I should try for pinning down the precise cause? Like valgrind or something? An SSCCE