Chuck, I will update the wiki page on the Numpy developer site that discusses the refactoring this week. Right now what's there reflects our plans before they met the reality of code. Needless to say, the actual implementation differs in some of the details.
Here is a very brief overview of the structure: - The libndarray directory now contains all of the code for the 'core' library. This library is independent of Python and implements most of the array, descriptor, iterator, and ufunc functionality. The goal is that all non-trivial behavior should be in here, but in reality some parts are tied fairly tightly to the CPython interpreter and will take more work to move into the core. - numpy/core/src/multiarray and numpy/core/src/umath now implement "just" the CPython interface to libndarray. We have preserved both the Python interface and the C API. Ideally each C API function is just a simple wrapper around a call to the core API, though it doesn't always work out that way. However, a large amount of code has been moved out of these modules into the core. - The core is built as a shared library that is independent of any given interface layer. That is, the same shared library/DLL can be used with CPython, IronPython and any other implementation. Each interface is required to pass in a set of callbacks for handling reference counting, object manipulation, and other interface-specific behavior. - The core implements its own reference counting semantics that happen to look very much like CPython's. This was necessary to make the core library independent of the interface layer and preserve performance (ie, limit the number of callbacks). The handshaking between interface and core is a bit complicated but ends up working nicely and efficiently for both reference counted and garbage collected systems. I'll write up the details on the wiki page. - As Travis mentioned we are also working on a .NET back end to Cython. This lets us port the modules such as MTRAND without having to have two separate interfaces, a Cython and a .NET version. Instead, we can modify the existing .pyx file to use the new core API (should improve performance in CPython version slightly). Once done, Cython can generate the .NET and CPython interfaces from the same .pyx file. We have done a fair amount of cleanup on the naming conventions but certainly more needs to be done! I'll write it up for everyone this week but feel free to email me with other questions. Regards, Jason On Mon, Aug 23, 2010 at 9:54 PM, Charles R Harris <charlesr.har...@gmail.com > wrote: > > > On Mon, Aug 23, 2010 at 2:30 PM, Travis Oliphant > <oliph...@enthought.com>wrote: > >> >> Hi all, >> >> I'm curious as to the status of the Github migration and if there is >> anything I can do to help. I have a couple of weeks right now and I would >> love to see us make the transition of both NumPy and SciPy to GIT. >> >> On a slightly related note, it would really help the numpy-refactor >> project if it received more input from others in the community. Right now, >> the numpy-refactor is happening on a public github branch (named >> numpy-refactor) awaiting numpy itself to be on github. It would be more >> useful if the numpy-refactor branch were a regular branch of the github >> NumPy project. >> >> The numpy-refactor project is making great progress and we have a working >> core library that can be built on Windows, Mac, and Linux. The goal is >> for this numpy-refactor to become the basis for NumPy 2.0 which should come >> out sometime this Fall. Already, a lot of unit tests have been written >> and code coverage has increased on the core NumPy code which I think we all >> agree is "a good thing" In addition, some lingering reference count bugs >> (particularly in object arrays) have been found and squashed. >> >> There is also some good progress on the Cython backend for .NET which >> would allow and also put pressure on migration of most of SciPy to >> Cython-or-Fwrap generated extension modules. >> >> I am looking forward to working on NumPy a little more over the coming >> months. >> >> All the best, >> >> > I've been having some fun browsing through the branch but don't have much > to say on such short acquaintance. > > I wonder if the patch in ticket > #1085<http://projects.scipy.org/numpy/ticket/1085>might be something you > folks could look at now that the loops have been > moved about and such? Also, it would be nice if the extended comment style > was rigidly adhered to, although things look pretty good in that department. > Another nit would be to keep an eye out during the cleanups for "if (blah) > foo;" if statements and clean them up by putting the foo on a separate line > when it is convenient to do so. Apart from that it looks like Ilan and Jason > are really getting into it and doing a nice job of regularizing the naming > conventions and such which should make the code easier to read and maintain. > Adding some explanatory comments along the way would also help as it may be > awhile before someone else looks so closely at the code as to have a good > understanding of what the different functions do. I know that gets in the > way of getting the code out, but I throw it out as a wish. > > Would it be possible to sketch out how the general layering of > functionality looks at this point? Is there going to be any cleanup of the > visible macros? > > Chuck > > _______________________________________________ > NumPy-Discussion mailing list > NumPy-Discussion@scipy.org > http://mail.scipy.org/mailman/listinfo/numpy-discussion > >
_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
