On Sun, Feb 19, 2012 at 4:03 AM, David Cournapeau <courn...@gmail.com>wrote:
> On Sun, Feb 19, 2012 at 9:28 AM, Mark Wiebe <mwwi...@gmail.com> wrote: > > > Is there anyone who uses a blue gene or small device which needs > up-to-date > > numpy support, that I could talk to directly? We really need a list of > > supported platforms on the numpy wiki we can refer to when discussing > this > > stuff, it all seems very nebulous to me. > > They may not need an up to date numpy version now, but if stopping > support for them is a requirement for C++, it must be kept in mind. I > actually suspect Travis to have more details on the big iron side of > things. On the small side of things: > http://projects.scipy.org/numpy/ticket/1969 > > This may seem like not very useful - but that's part of what a open > source project is all about in my mind. > > > > > Particular styles of using templates can cause this, yes. To properly do > > this kind of advanced C++ library work, it's important to think about the > > big-O notation behavior of your template instantiations, not just the > big-O > > notation of run-time. C++ templates have a turing-complete language > (which > > is said to be quite similar to haskell, but spelled vastly different) > > running at compile time in them. This is what gives template > > meta-programming in C++ great power, but since templates weren't designed > > for this style of programming originally, template meta-programming is > not > > very easy. > > scipy.sparse.sparsetools is quite straightforward in its usage of > templates (would be great if you could suggest improvement BTW, e.g. > scipy/sparse/sparsetools/csr.h), and does not by itself use any > meta-template programming. > I took a look, and I think the reason this is so slow to compile and uses so much memory is visible as follows: [sparsetools]$ wc *.cxx | sort -n 4039 13276 116263 csgraph_wrap.cxx 6464 21385 189537 dia_wrap.cxx 14002 45406 412262 coo_wrap.cxx 32385 102534 963688 csc_wrap.cxx 42997 140896 1313797 bsr_wrap.cxx 50041 161127 1501400 csr_wrap.cxx 149928 484624 4496947 total That's almost 4.5MB of code, in 6 files. C/C++ compilers are not optimized to compile this sort of thing fast, they are focused on more "human-style" coding with smaller individual files. Looking at some of these SWIG-generated files, the way they dispatch based on the input Python types is bloated as well. Probably the main question I would ask is, does scipy really need sparse matrix variants for all of int8, uint8, int16, uint16, etc? Trimming away some of these might be reasonable, and would be a start to improve compile times. The reason for the slowness is not C++ templates in this example. Cheers, Mark > I like that numpy can be built in a few seconds (at least without > optimization), and consider this to be a useful feature. > > cheers, > > David > _______________________________________________ > NumPy-Discussion mailing list > NumPy-Discussion@scipy.org > http://mail.scipy.org/mailman/listinfo/numpy-discussion >
_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
