On Fri, Oct 16, 2015 at 8:56 PM, Allan Haldane <allanhald...@gmail.com> wrote:
> On 10/16/2015 05:31 PM, josef.p...@gmail.com wrote: > > > > > > On Fri, Oct 16, 2015 at 2:21 PM, Charles R Harris > > <charlesr.har...@gmail.com <mailto:charlesr.har...@gmail.com>> wrote: > > > > > > > > On Fri, Oct 16, 2015 at 12:20 PM, Charles R Harris > > <charlesr.har...@gmail.com <mailto:charlesr.har...@gmail.com>> > wrote: > > > > > > > > On Fri, Oct 16, 2015 at 11:58 AM, <josef.p...@gmail.com > > <mailto:josef.p...@gmail.com>> wrote: > > > > was there a change with reduce operations with recarrays in > > 1.10 or 1.10.1? > > > > Travis shows a new test failure in the statsmodels testsuite > > with 1.10.1: > > > > ERROR: test suite for <class > > 'statsmodels.base.tests.test_data.TestRecarrays'> > > > > File > > > > "/home/travis/miniconda/envs/statsmodels-test/lib/python2.7/site-packages/statsmodels-0.8.0-py2.7-linux-x86_64.egg/statsmodels/base/data.py", > > line 131, in _handle_constant > > const_idx = np.where(self.exog.ptp(axis=0) == > > 0)[0].squeeze() > > TypeError: cannot perform reduce with flexible type > > > > > > Sorry for asking so late. > > (statsmodels is short on maintainers, and I'm distracted) > > > > > > statsmodels still has code to support recarrays and > > structured dtypes from the time before pandas became > > popular, but I don't think anyone is using them together > > with statsmodels anymore. > > > > > > There were several commits dealing both recarrays and ufuncs, so > > this might well be a regression. > > > > > > A bisection would be helpful. Also, open an issue. > > > > > > > > The reason for the test failure might be somewhere else hiding behind > > several layers of statsmodels, but only started to show up with numpy > 1.10.1 > > > > I already have the reduce exception with my currently installed numpy > > '1.9.2rc1' > > > >>>> x = np.random.random(9*3).view([('const', 'f8'),('x_1', 'f8'), > > ('x_2', 'f8')]).view(np.recarray) > > > >>>> np.ptp(x, axis=0) > > Traceback (most recent call last): > > File "<stdin>", line 1, in <module> > > File > > > "C:\programs\WinPython-64bit-3.4.3.1\python-3.4.3.amd64\lib\site-packages\numpy\core\fromnumeric.py", > > line 2047, in ptp > > return ptp(axis, out) > > TypeError: cannot perform reduce with flexible type > > > > > > Sounds like fun, and I don't even know how to automatically bisect. > > > > Josef > > That example isn't the problem (ptp should definitely fail on structured > arrays), but I've tracked down what is - it has to do with views of > record arrays. > > The fix looks simple, I'll get it in for the next release. > Thanks, I realized that at that point in the statsmodels code we should have only regular ndarrays, so the array conversion fails somewhere. AFAICS, the main helper function to convert is def struct_to_ndarray(arr): return arr.view((float, len(arr.dtype.names))) which doesn't look like it will handle other dtypes than float64. Nobody ever complained, so maybe our test suite is the only user of this. What is now the recommended way of converting structured dtypes/recarrays to ndarrays? Josef > > Allan > > > > > _______________________________________________ > NumPy-Discussion mailing list > NumPy-Discussion@scipy.org > https://mail.scipy.org/mailman/listinfo/numpy-discussion >
_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org https://mail.scipy.org/mailman/listinfo/numpy-discussion
