Hi Nixin, It is tough to parse the parallel traceback, but it looks to me like this is unrelated to halo finding and simple that yt is having trouble reading this data. Are you able to load this data with yt and query it? If you are having trouble converting this data into a readable format, it may be worth trying to use the generic particle loader. https://yt-project.org/docs/dev/examining/Loading_Generic_Particle_Data.html If you are able to make that work, then it should be just a matter of setting a few additional attributes for making it work with the halo finder.
Britton On Sun, Dec 21, 2025 at 3:10 PM Nixin Saji Abraham via yt-users < [email protected]> wrote: > I am trying to run rockstar algorithm using yt astro analysis. What I have > with me is gadget 1 data which was converted from big file data format. I > have generated 32, 34, 39 chunks of the same data in gadget 1 format. > Majority of the time, I get the following error at the start itself. File > "/home/user/p1cj_phy/fastpm/yt_astro_analysis/yt_astro_analysis/halo_analysis/halo_catalog/halo_catalog.py", > line 346, in create > self._run(save_halos, save_output, njobs=njobs, dynamic=dynamic) > File > "/home/user/p1cj_phy/fastpm/yt_astro_v/lib/python3.12/site-packages/yt/utilities/parallel_tools/parallel_analysis_interface.py", > line 322, in barrierize > retval = func(*args, **kwargs) > ^^^^^^^^^^^^^^^^^^^^^ > File > "/home/user/p1cj_phy/fastpm/yt_astro_analysis/yt_astro_analysis/halo_analysis/halo_catalog/halo_catalog.py", > line 239, in _run > self.finder_method(self) > File > "/home/user/p1cj_phy/fastpm/yt_astro_analysis/yt_astro_analysis/halo_analysis/halo_catalog/halo_finding_methods.py", > line 36, in __call__ > return self.function(hc, *self.args, **self.kwargs) > ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ > File > "/home/user/p1cj_phy/fastpm/yt_astro_analysis/yt_astro_analysis/halo_analysis/halo_catalog/halo_finding_methods.py", > line 88, in _rockstar_method > rh = RockstarHaloFinder(ds, **finder_kwargs) > ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ > File > "/home/user/p1cj_phy/fastpm/yt_astro_analysis/yt_astro_analysis/halo_analysis/halo_finding/rockstar/rockstar.py", > line 290, in __init__ > p = self._setup_parameters(ts) > ^^^^^^^^^^^^^^^^^^^^^^^^^^ > File > "/home/user/p1cj_phy/fastpm/yt_astro_analysis/yt_astro_analysis/halo_analysis/halo_finding/rockstar/rockstar.py", > line 299, in _setup_parameters > tds.index # noqa: B018 > ^^^^^^^^^ > File > "/home/user/p1cj_phy/fastpm/yt_astro_v/lib/python3.12/site-packages/yt/data_objects/static_output.py", > line 605, in index > self._instantiated_index = self._index_class( > ^^^^^^^^^^^^^^^^^^ > File > "/home/user/p1cj_phy/fastpm/yt_astro_v/lib/python3.12/site-packages/yt/frontends/gadget/data_structures.py", > line 195, in __init__ > super().__init__(ds, dataset_type) > File > "/home/user/p1cj_phy/fastpm/yt_astro_v/lib/python3.12/site-packages/yt/geometry/particle_geometry_handler.py", > line 73, in __init__ > self._initialize_index() > File > "/home/user/p1cj_phy/fastpm/yt_astro_v/lib/python3.12/site-packages/yt/frontends/gadget/data_structures.py", > line 204, in _initialize_index > super()._initialize_index() > File > "/home/user/p1cj_phy/fastpm/yt_astro_v/lib/python3.12/site-packages/yt/frontends/sph/data_structures.py", > line 89, in _initialize_index > super()._initialize_index() > File > "/home/user/p1cj_phy/fastpm/yt_astro_v/lib/python3.12/site-packages/yt/geometry/particle_geometry_handler.py", > line 237, in _initialize_index > max_hsml = self._initialize_coarse_index() > ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ > File > "/home/user/p1cj_phy/fastpm/yt_astro_v/lib/python3.12/site-packages/yt/geometry/particle_geometry_handler.py", > line 280, in _initialize_coarse_index > self.regions.masks = self.comm.mpi_allreduce(self.regions.masks, > op="sum") > > ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ > File > "/home/user/p1cj_phy/fastpm/yt_astro_v/lib/python3.12/site-packages/yt/utilities/parallel_tools/parallel_analysis_interface.py", > line 297, in passage > return func(self, *args, **kwargs) > ^^^^^^^^^^^^^^^^^^^^^^^^^^^ > File > "/home/user/p1cj_phy/fastpm/yt_astro_v/lib/python3.12/site-packages/yt/utilities/parallel_tools/parallel_analysis_interface.py", > line 913, in mpi_allreduce > self.comm.Allreduce( > File "src/mpi4py/MPI.src/Comm.pyx", line 1115, in > mpi4py.MPI.Comm.Allreduce > P004 yt : [ERROR ] 2025-12-19 23:49:08,036 Exception: MPI_ERR_ARG: > invalid argument of some other kind > P004 yt : [ERROR ] 2025-12-19 23:49:08,036 Error occurred on rank 3. > -------------------------------------------------------------------------- > MPI_ABORT was invoked on rank 4 in communicator MPI_COMM_WORLD > with errorcode 1. > > NOTE: invoking MPI_ABORT causes Open MPI to kill all MPI processes. > You may or may not see output from other processes, depending on > exactly when Open MPI kills them. > -------------------------------------------------------------------------- > File "/home/user/p1cj_phy/fastpm/my.example/rockstar.py", line 14, in > <module> > hc.create() > File > "/home/user/p1cj_phy/fastpm/yt_astro_analysis/yt_astro_analysis/halo_analysis/halo_catalog/halo_catalog.py", > line 346, in create > self._run(save_halos, save_output, njobs=njobs, dynamic=dynamic) > File > "/home/user/p1cj_phy/fastpm/yt_astro_v/lib/python3.12/site-packages/yt/utilities/parallel_tools/parallel_analysis_interface.py", > line 322, in barrierize > retval = func(*args, **kwargs) > ^^^^^^^^^^^^^^^^^^^^^ > File > "/home/user/p1cj_phy/fastpm/yt_astro_analysis/yt_astro_analysis/halo_analysis/halo_catalog/halo_catalog.py", > line 239, in _run > self.finder_method(self) > File > "/home/user/p1cj_phy/fastpm/yt_astro_analysis/yt_astro_analysis/halo_analysis/halo_catalog/halo_finding_methods.py", > line 36, in __call__ > return self.function(hc, *self.args, **self.kwargs) > ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ > File > "/home/user/p1cj_phy/fastpm/yt_astro_analysis/yt_astro_analysis/halo_analysis/halo_catalog/halo_finding_methods.py", > line 88, in _rockstar_method > rh = RockstarHaloFinder(ds, **finder_kwargs) > ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ > File > "/home/user/p1cj_phy/fastpm/yt_astro_analysis/yt_astro_analysis/halo_analysis/halo_finding/rockstar/rockstar.py", > line 290, in __init__ > p = self._setup_parameters(ts) > ^^^^^^^^^^^^^^^^^^^^^^^^^^ > File > "/home/user/p1cj_phy/fastpm/yt_astro_analysis/yt_astro_analysis/halo_analysis/halo_finding/rockstar/rockstar.py", > line 299, in _setup_parameters > tds.index # noqa: B018 > ^^^^^^^^^ > File > "/home/user/p1cj_phy/fastpm/yt_astro_v/lib/python3.12/site-packages/yt/data_objects/static_output.py", > line 605, in index > self._instantiated_index = self._index_class( > ^^^^^^^^^^^^^^^^^^ > File > "/home/user/p1cj_phy/fastpm/yt_astro_v/lib/python3.12/site-packages/yt/frontends/gadget/data_structures.py", > line 195, in __init__ > super().__init__(ds, dataset_type) > File > "/home/user/p1cj_phy/fastpm/yt_astro_v/lib/python3.12/site-packages/yt/geometry/particle_geometry_handler.py", > line 73, in __init__ > self._initialize_index() > File > "/home/user/p1cj_phy/fastpm/yt_astro_v/lib/python3.12/site-packages/yt/frontends/gadget/data_structures.py", > line 204, in _initialize_index > super()._initialize_index() > File > "/home/user/p1cj_phy/fastpm/yt_astro_v/lib/python3.12/site-packages/yt/frontends/sph/data_structures.py", > line 89, in _initialize_index > super()._initialize_index() > File > "/home/user/p1cj_phy/fastpm/yt_astro_v/lib/python3.12/site-packages/yt/geometry/particle_geometry_handler.py", > line 237, in _initialize_index > max_hsml = self._initialize_coarse_index() > ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ > File > "/home/user/p1cj_phy/fastpm/yt_astro_v/lib/python3.12/site-packages/yt/geometry/particle_geometry_handler.py", > line 280, in _initialize_coarse_index > self.regions.masks = self.comm.mpi_allreduce(self.regions.masks, > op="sum") > > ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ > File > "/home/user/p1cj_phy/fastpm/yt_astro_v/lib/python3.12/site-packages/yt/utilities/parallel_tools/parallel_analysis_interface.py", > line 297, in passage > return func(self, *args, **kwargs) > ^^^^^^^^^^^^^^^^^^^^^^^^^^^ > File > "/home/user/p1cj_phy/fastpm/yt_astro_v/lib/python3.12/site-packages/yt/utilities/parallel_tools/parallel_analysis_interface.py", > line 913, in mpi_allreduce > self.comm.Allreduce( > File "src/mpi4py/MPI.src/Comm.pyx", line 1115, in > mpi4py.MPI.Comm.Allreduce > P001 yt : [ERROR ] 2025-12-19 23:49:08,554 Exception: MPI_ERR_ARG: > invalid argument of some other kind > P001 yt : [ERROR ] 2025-12-19 23:49:08,554 Error occurred on rank 0. > File "/home/user/p1cj_phy/fastpm/my.example/rockstar.py", line 14, in > <module> > hc.create() > File > "/home/user/p1cj_phy/fastpm/yt_astro_analysis/yt_astro_analysis/halo_analysis/halo_catalog/halo_catalog.py", > line 346, in create > self._run(save_halos, save_output, njobs=njobs, dynamic=dynamic) > File > "/home/user/p1cj_phy/fastpm/yt_astro_v/lib/python3.12/site-packages/yt/utilities/parallel_tools/parallel_analysis_interface.py", > line 322, in barrierize > retval = func(*args, **kwargs) > ^^^^^^^^^^^^^^^^^^^^^ > File > "/home/user/p1cj_phy/fastpm/yt_astro_analysis/yt_astro_analysis/halo_analysis/halo_catalog/halo_catalog.py", > line 239, in _run > self.finder_method(self) > File > "/home/user/p1cj_phy/fastpm/yt_astro_analysis/yt_astro_analysis/halo_analysis/halo_catalog/halo_finding_methods.py", > line 36, in __call__ > return self.function(hc, *self.args, **self.kwargs) > ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ > File > "/home/user/p1cj_phy/fastpm/yt_astro_analysis/yt_astro_analysis/halo_analysis/halo_catalog/halo_finding_methods.py", > line 88, in _rockstar_method > rh = RockstarHaloFinder(ds, **finder_kwargs) > ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ > File > "/home/user/p1cj_phy/fastpm/yt_astro_analysis/yt_astro_analysis/halo_analysis/halo_finding/rockstar/rockstar.py", > line 290, in __init__ > p = self._setup_parameters(ts) > ^^^^^^^^^^^^^^^^^^^^^^^^^^ > File > "/home/user/p1cj_phy/fastpm/yt_astro_analysis/yt_astro_analysis/halo_analysis/halo_finding/rockstar/rockstar.py", > line 299, in _setup_parameters > tds.index # noqa: B018 > ^^^^^^^^^ > File > "/home/user/p1cj_phy/fastpm/yt_astro_v/lib/python3.12/site-packages/yt/data_objects/static_output.py", > line 605, in index > self._instantiated_index = self._index_class( > ^^^^^^^^^^^^^^^^^^ > File > "/home/user/p1cj_phy/fastpm/yt_astro_v/lib/python3.12/site-packages/yt/frontends/gadget/data_structures.py", > line 195, in __init__ > super().__init__(ds, dataset_type) > File > "/home/user/p1cj_phy/fastpm/yt_astro_v/lib/python3.12/site-packages/yt/geometry/particle_geometry_handler.py", > line 73, in __init__ > self._initialize_index() > File > "/home/user/p1cj_phy/fastpm/yt_astro_v/lib/python3.12/site-packages/yt/frontends/gadget/data_structures.py", > line 204, in _initialize_index > super()._initialize_index() > File > "/home/user/p1cj_phy/fastpm/yt_astro_v/lib/python3.12/site-packages/yt/frontends/sph/data_structures.py", > line 89, in _initialize_index > super()._initialize_index() > File > "/home/user/p1cj_phy/fastpm/yt_astro_v/lib/python3.12/site-packages/yt/geometry/particle_geometry_handler.py", > line 237, in _initialize_index > max_hsml = self._initialize_coarse_index() > ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ > File > "/home/user/p1cj_phy/fastpm/yt_astro_v/lib/python3.12/site-packages/yt/geometry/particle_geometry_handler.py", > line 280, in _initialize_coarse_index > self.regions.masks = self.comm.mpi_allreduce(self.regions.masks, > op="sum") > > ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ > File > "/home/user/p1cj_phy/fastpm/yt_astro_v/lib/python3.12/site-packages/yt/utilities/parallel_tools/parallel_analysis_interface.py", > line 297, in passage > return func(self, *args, **kwargs) > ^^^^^^^^^^^^^^^^^^^^^^^^^^^ > File > "/home/user/p1cj_phy/fastpm/yt_astro_v/lib/python3.12/site-packages/yt/utilities/parallel_tools/parallel_analysis_interface.py", > line 913, in mpi_allreduce > self.comm.Allreduce( > File "src/mpi4py/MPI.src/Comm.pyx", line 1115, in > mpi4py.MPI.Comm.Allreduce > P002 yt : [ERROR ] 2025-12-19 23:49:08,936 Exception: MPI_ERR_ARG: > invalid argument of some other kind > P002 yt : [ERROR ] 2025-12-19 23:49:08,936 Error occurred on rank 1. > File "/home/user/p1cj_phy/fastpm/my.example/rockstar.py", line 14, in > <module> > hc.create() > File > "/home/user/p1cj_phy/fastpm/yt_astro_analysis/yt_astro_analysis/halo_analysis/halo_catalog/halo_catalog.py", > line 346, in create > self._run(save_halos, save_output, njobs=njobs, dynamic=dynamic) > File > "/home/user/p1cj_phy/fastpm/yt_astro_v/lib/python3.12/site-packages/yt/utilities/parallel_tools/parallel_analysis_interface.py", > line 322, in barrierize > retval = func(*args, **kwargs) > ^^^^^^^^^^^^^^^^^^^^^ > File > "/home/user/p1cj_phy/fastpm/yt_astro_analysis/yt_astro_analysis/halo_analysis/halo_catalog/halo_catalog.py", > line 239, in _run > self.finder_method(self) > File > "/home/user/p1cj_phy/fastpm/yt_astro_analysis/yt_astro_analysis/halo_analysis/halo_catalog/halo_finding_methods.py", > line 36, in __call__ > return self.function(hc, *self.args, **self.kwargs) > ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ > File > "/home/user/p1cj_phy/fastpm/yt_astro_analysis/yt_astro_analysis/halo_analysis/halo_catalog/halo_finding_methods.py", > line 88, in _rockstar_method > rh = RockstarHaloFinder(ds, **finder_kwargs) > ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ > File > "/home/user/p1cj_phy/fastpm/yt_astro_analysis/yt_astro_analysis/halo_analysis/halo_finding/rockstar/rockstar.py", > line 290, in __init__ > p = self._setup_parameters(ts) > ^^^^^^^^^^^^^^^^^^^^^^^^^^ > File > "/home/user/p1cj_phy/fastpm/yt_astro_analysis/yt_astro_analysis/halo_analysis/halo_finding/rockstar/rockstar.py", > line 299, in _setup_parameters > tds.index # noqa: B018 > ^^^^^^^^^ > File > "/home/user/p1cj_phy/fastpm/yt_astro_v/lib/python3.12/site-packages/yt/data_objects/static_output.py", > line 605, in index > self._instantiated_index = self._index_class( > ^^^^^^^^^^^^^^^^^^ > File > "/home/user/p1cj_phy/fastpm/yt_astro_v/lib/python3.12/site-packages/yt/frontends/gadget/data_structures.py", > line 195, in __init__ > super().__init__(ds, dataset_type) > File > "/home/user/p1cj_phy/fastpm/yt_astro_v/lib/python3.12/site-packages/yt/geometry/particle_geometry_handler.py", > line 73, in __init__ > self._initialize_index() > File > "/home/user/p1cj_phy/fastpm/yt_astro_v/lib/python3.12/site-packages/yt/frontends/gadget/data_structures.py", > line 204, in _initialize_index > super()._initialize_index() > File > "/home/user/p1cj_phy/fastpm/yt_astro_v/lib/python3.12/site-packages/yt/frontends/sph/data_structures.py", > line 89, in _initialize_index > super()._initialize_index() > File > "/home/user/p1cj_phy/fastpm/yt_astro_v/lib/python3.12/site-packages/yt/geometry/particle_geometry_handler.py", > line 237, in _initialize_index > max_hsml = self._initialize_coarse_index() > ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ > File > "/home/user/p1cj_phy/fastpm/yt_astro_v/lib/python3.12/site-packages/yt/geometry/particle_geometry_handler.py", > line 280, in _initialize_coarse_index > self.regions.masks = self.comm.mpi_allreduce(self.regions.masks, > op="sum") > > ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ > File > "/home/user/p1cj_phy/fastpm/yt_astro_v/lib/python3.12/site-packages/yt/utilities/parallel_tools/parallel_analysis_interface.py", > line 297, in passage > return func(self, *args, **kwargs) > ^^^^^^^^^^^^^^^^^^^^^^^^^^^ > File > "/home/user/p1cj_phy/fastpm/yt_astro_v/lib/python3.12/site-packages/yt/utilities/parallel_tools/parallel_analysis_interface.py", > line 913, in mpi_allreduce > self.comm.Allreduce( > File "src/mpi4py/MPI.src/Comm.pyx", line 1115, in > mpi4py.MPI.Comm.Allreduce > P003 yt : [ERROR ] 2025-12-19 23:49:08,961 Exception: MPI_ERR_ARG: > invalid argument of some other kind > P003 yt : [ERROR ] 2025-12-19 23:49:08,961 Error occurred on rank 2. > [Node001:85381] 3 more processes have sent help message help-mpi-api.txt / > mpi-abort > [Node001:85381] Set MCA parameter "orte_base_help_aggregate" to 0 to see > all help / error messages > -- I have tried different reader and writer combinations (4&9, 4&11, 4&13, > 8&11 etc.). I am not able to understand this error and solve it. Also I > didn't succeed in creating a code that converts data from big file to > gadget 2 data format. Plese help. > _______________________________________________ > yt-users mailing list -- [email protected] > To unsubscribe send an email to [email protected] > https://mail.python.org/mailman3//lists/yt-users.python.org > Member address: [email protected] >
_______________________________________________ yt-users mailing list -- [email protected] To unsubscribe send an email to [email protected] https://mail.python.org/mailman3//lists/yt-users.python.org Member address: [email protected]
