Package: wnpp Severity: wishlist Owner: Julian Gilbey <j...@debian.org> X-Debbugs-Cc: debian-de...@lists.debian.org, Debian Python Team <team+pyt...@tracker.debian.org>
* Package name : pathos Version : 0.3.1 Upstream Contact: Mike McKerns <mmckerns at uqfoundation dot org> * URL : https://github.com/uqfoundation/pathos * License : BSD-3-clause Programming Lang: Python Description : Framework for heterogeneous parallel computing Pathos provides a consistent high-level interface for configuring and launching parallel computations across heterogeneous resources. It provides configurable launchers for parallel and distributed computing, where each launcher contains the syntactic logic to configure and launch jobs in an execution environment. Examples of launchers include: "pyina", a queue-less MPI-based launcher; "pathos", an ssh-based launcher; "multiprocess", a multi-process launcher. . It provides a consistent interface for parallel and/or distributed versions of "map" and "apply" for each launcher; the guiding design principle is that "map" and "apply" should be drop-in replacements in otherwise serial code, reducing the time to conver a code to parallel, but also enabling a single code-base to be maintained instead of requiring serial, parallel and distributed versions of code. . The "pathos" framework consists of several interoperating packages: * "dill": serialize all of Python (python3-dill) * "pox": utilities for filesystem exploration and automated builds (python3-pox) * "klepto": persistent caching to memory, disk, or database (python3-klepto) * "multiprocess": better multiprocessing and multithreading in Python (python3-multiprocess) * "ppft": distributed and parallel Python (python3-ppft) * "pyina": MPI parallel "map" and cluster scheduling (python3-pyina) * "pathos": graph management and execution in heterogeneous computing (python3-pathos) This is a package I've started using; it provides a very effective framework for parallel computing, allowing for constructs that the standard Python library does not support. I will maintain it within the Debian Python Team.
