Feng Yu <rainwood...@gmail.com> wrote: > In most (half?) situations the result can be directly write back via > preallocated shared array before works are spawned. Then there is no > need to pass data back with named segments.
You can work around it in various ways, this being one of them. Personally I prefer a parallel programming style with queues – either to scatter arrays to workers and collecting arrays from workers, or to chain workers together in a pipeline (without using coroutines). But exactly how you program is a matter of taste. I want to make it as inexpensive as possible to pass a NumPy array through a queue. If anyone else wants to help improve parallel programming with NumPy using a different paradigm, that is fine too. I just wanted to clarify why I stopped working on shared memory arrays. (As for the implementation, I am also experimenting with platform dependent asynchronous I/O (IOCP, GCD or kqueue, epoll) to pass NumPy arrays though a queue as inexpensively and scalably as possible. And no, there is no public repo, as I like to experiment with my pet project undisturbed before I let it out in the wild.) Sturla _______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org https://mail.scipy.org/mailman/listinfo/numpy-discussion
