On Wed, Oct 2, 2013 at 1:47 PM, Joachim Durchholz <[email protected]> wrote: > Am 02.10.2013 20:26, schrieb Ondřej Čertík: > >> The reason is that to truly nail the speed, one needs to use special >> dictionary implementation, >> tweak hashing functions, reference counting, memory allocators etc. > > > Reference counting can improve latency, but it will actually decrease > throughput. > > At best, RC in C++ is faster than what Python does (which has a very crappy > garbage collector). > A good GC, such as what's available in Java, can beat any RC implementation > hands down. Mostly because malloc() and free() do MUCH more work than most > people think; it's not unheard of that allocation-intensive programs (and > Sympy would certainly qualify) spend 20% of their time in memory management, > and a mark-and-sweep collector, even a simplicistic one, simply has a much > better throughput.
Yes, I have found that malloc, resp. the default new/delete operators in C++ can be sped-up by 10% or 20% by using custom allocators. I have tried http://warp.povusers.org/FSBAllocator/ see my implementation here: https://github.com/certik/csympy/compare/master...alloc_fsb and also tried http://goog-perftools.sourceforge.net/doc/tcmalloc.html both of which provide good speedup. At the moment I use neither, because the FSBAllocator does not work for std::unordered_map. Ultimately the best approach seems to allocate a pool of memory and then do our own memory allocation along the lines of FSBAllocator and in-place new(). (C++ allows to implement all this, while this is very hard in Cython.) As to garbage collection approach, I should try it as well. Though what I read it might not beat the above approach. > > >> Besides speed, I came to appreciate having just one language and write >> everything in it. Much simpler to maintain >> and master. > > > Well sure, but that would be a case for setting up an entirely new SymC++ > project, wouldn't it? Right. The only way to maintain single code base to run both in pure Python and compiled in C/C++ for speed that I know of is to use pure Python mode in Cython or some kind of translation as we discussed. But I don't know how to effectively nail the speed with either of these approaches, as discussed. So the other simplest obvious approach is to stick to pure C++. But by having thin Python wrappers, one can use it complementary together with SymPy, and we might be able to incorporate such optional libraries in SymPy more tightly in the future by using some double dispatch or such. > > >> With your sympy2cython approach, you need to master >> >> Python, SymPy code, the sympy2cython >> tool, Cython, C and Python C/API intricacies. That's a lot of layers. > > > Fully agreed that that would become a serious problem over time. > Also, it would deter contributors that don't know all these layers. Exactly. C++, while complex, is still a very very popular language with quite a lot of people able to contribute. Ondrej -- You received this message because you are subscribed to the Google Groups "sympy" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at http://groups.google.com/group/sympy. For more options, visit https://groups.google.com/groups/opt_out.
