On Tue, Jun 27, 2017 at 12:09 AM, Robert McLeod <robbmcl...@gmail.com> wrote:
> Jaime, > > One thing that might be a good selling point would be if you built-in > threaded operations on image stacks, which I think are pretty common in all > forms of microscopy now. I'm thinking particularly of `ni.convolve()`, > where one might want to repeatedly call the function on every image frame > in the stack. Generally multi-threading kernel operations like convolve on > single images is tricky, but if you have a stack it's much more > straight-forward to parallelize. Right now I would do such a thing with a > thread pool in Python, but it would scale a lot better if there was C-level > support. E.g.: > I think the biggest hurdle for adding any such low-level support for multithreading in scipy is not having a portable threading library that will work on the myriad platforms that scipy is supposed to work on. In any case, such a change would have to be a next step after making ndimage something sane to work with, which is the purpose of the refactoring I was describing. Jaime > > import multiprocessing as mp > import numpy as np > import scipy.ndimage as ni > > # funcHandle must release the GIL > def stackFunc( funcHandle, images, funcArgs ): > tPool = mp.ThreadPool( self.n_threads ) > > slices = self.images.shape[0] > # Build parameters list for the threaded processes, consisting of > index > tArgs = [None] * slices > for J in np.arange(slices): > tArgs[J] = (J, images, funcArgs) > > # All operations are done 'in-place' > tPool.map( funcHandle, tArgs ) > tPool.close() > tPool.join() > > > Sincerely, > > Robert > > > On Mon, Jun 26, 2017 at 2:30 AM, Jaime Fernández del Río < > jaime.f...@gmail.com> wrote: > >> Hi all, >> >> I have started working on an ambitious refactor of scipy.ndimage. Some >> more context on what I would like to achieve can be found here >> <https://mail.python.org/pipermail/scipy-dev/2017-June/021986.html>. >> >> scikit-image is probably the biggest "customer" of ndimage, so even >> though the API and performance are not expected to change, you may want to >> have a say. >> >> Assuming there is no major concern with the overall plan, I would also >> like to poke this community for help with: >> >> - figuring out a sensible test strategy: Ralf Gommers points out that >> we should probably use scikit-image's test suite regularly to complement >> the relatively few tests currently in ndimage: is the development version >> of scikit-image stable enough for this purpose or do you suggest using a >> specific stable release? Any other ideas? >> - PR reviewing. The idea is to make this refactor as incremental as >> possible. That means many smaller interdependent PRs. Even if you don't >> have commit rights to the scipy repository you are probably better versed >> than most of us in image processing algorithms, so if anyone can keep an >> eye on ndimage PRs >> >> <https://github.com/scipy/scipy/pulls?q=is%3Apr+is%3Aopen+label%3Ascipy.ndimage> >> and provide occasional reviews it would be greatly appreciated. >> - expert consulting: some of the code in ndimage is pretty hard to >> understand, e.g. I struggle with spline interpolation. If anyone with the >> proper academic knowledge is willing to answer stupid questions, please >> let >> me know. >> - the refactoring itself: if anyone would like to be a part of this, >> it would be my pleasure! A basic command of the C language is needed, but >> I'm more than willing to mentor beginners. >> >> Thanks! >> >> Jaime >> >> -- >> (\__/) >> ( O.o) >> ( > <) Este es Conejo. Copia a Conejo en tu firma y ayúdale en sus planes >> de dominación mundial. >> >> _______________________________________________ >> scikit-image mailing list >> scikit-image@python.org >> https://mail.python.org/mailman/listinfo/scikit-image >> >> > > > -- > Robert McLeod, Ph.D. > robert.mcl...@unibas.ch > robert.mcl...@bsse.ethz.ch <robert.mcl...@ethz.ch> > robbmcl...@gmail.com > > _______________________________________________ > scikit-image mailing list > scikit-image@python.org > https://mail.python.org/mailman/listinfo/scikit-image > > -- (\__/) ( O.o) ( > <) Este es Conejo. Copia a Conejo en tu firma y ayúdale en sus planes de dominación mundial.
_______________________________________________ scikit-image mailing list scikit-image@python.org https://mail.python.org/mailman/listinfo/scikit-image
