Hi all, I am trying to create a Python package containing free functions that use HPX in their implementation. Outputs from functions is passed as inputs into other ones. The result is interdependent tasks that are executed by the HPX runtime.
I know how to create a Python package wrapping C++ code and how to call C++ functions upon Python module start-up and tear-down. I cannot figure out how to make the Python interpreter and the HPX runtime play nice together when multiple localities are used. Can someone please explain the gist of how Python and HPX can be made to cooperate? Or maybe point me to an example that does something similar? My goal is for the Python package to be usable by a regular / unpatched Python interpreter. I can share my code if that is useful, but I would first like to understand the higher level aspects. My Python script runs fine on a single locality, but crashes (assertion in HPX code) when I use multiple localities. I use hpxrun.py for spawning multiple Python processes on my development machine. My current, possibly wrong, ideas are: HPX: - I need to start the HPX runtime on each locality - One locality is special. This root locality is where the top-level functions that use HPX in their implementation are executed. - All localities receive work generated at the root locality. Python: - The Python interpreter is executed on each locality. On the command line the name of the script which imports my new module is passed, along with HPX-specific arguments (about threads, localities, network, ...). - There must be some way to trigger the initialization of the HPX runtime. This could be done upon Python module load or by exposing some init_hpx_runtime function in Python. I am not sure what makes most sense. It must be possible to pass in arguments to the runtime. - Somehow the Python script must only be interpreted on the main locality. The other ones will receive work. Maybe test from Python whether the interpreter runs on the root locality and only then continue? This must then be one of the first things Python does. I have my functions working fine from C++. I would like to make them available to end-users who are familiar with Python, but not with C++. They only need to be able to call my functions in whatever order they like and have the computational load distributed over multiple localities. Thanks in advance for any insights! Kor _______________________________________________ hpx-users mailing list [email protected] https://mail.cct.lsu.edu/mailman/listinfo/hpx-users
