Mikael,
Thanks for putting this together. This looks good to me. May I suggest that you separate the two parts of this into their own emails when sending it to the HPX lists? Thanks! Regards Hartmut --------------- <https://stellar.cct.lsu.edu> https://stellar.cct.lsu.edu <https://github.com/STEllAR-GROUP/hpx> https://github.com/STEllAR-GROUP/hpx From: [email protected] <[email protected]> On Behalf Of Simberg Mikael Sent: Monday, May 31, 2021 8:22 AM To: [email protected] Subject: [hpx-pmc] [Proposal for VOTE] Proposal to split HPX into at least two repositories Hi all, We discussed in the last PMC meeting again the possibility of splitting HPX into one or more repositories, but concluded that we and the community should vote on it before making any decisions. I would like to send the following email to the hpx-users and hpx-devel mailing lists. I've attempted to keep the message as neutral as possible, with the exception for my own vote at the end where I explain why I and the other CSCS developers support the proposal. Please let me know if you'd like to have anything anything changed, added, or removed. --- begin message --- Dear HPX users and developers, The HPX users and developers at CSCS (that includes myself) have expressed an interest in separating the local-only and distributed functionality of HPX into two separate projects and repositories. This is a contentious topic, so before we do a large change like this we again want to consult the community through a vote. My personal vote and motivation for the change is at the bottom of the message. Practically speaking, the proposal is to move the on-node functionality of HPX (this includes futures, algorithms, basic CUDA/HIP support, a local-only runtime, and all the utilities required to support this) into a separate repository. The remaining distributed functionality of HPX would remain in the current STEllAR-GROUP/hpx repository and it would gain one new dependency, called (e.g.) hpx-local. Releases of hpx and hpx-local would often be done together, but could be done independently of each other. As a reminder, voting works as follows: If a formal vote on a proposal is called (signaled simply by sending a email with [VOTE] in the subject line), all participants on the HPX user's mailing list may express an opinion and vote. They do this by sending an email in reply to the original [VOTE] email, with the following vote and information: - +1: 'yes', 'agree': also willing to help bring about the proposed action - +0: 'yes', 'agree': not willing or able to help bring about the proposed action - -0: 'no', 'disagree': but will not oppose the action's going forward - -1: 'no', 'disagree': opposes the action's going forward and must propose an alternative action to address the issue (or a justification for not addressing the issue) Responses from developers and users alike are encouraged. Please vote as soon as possible, but we will leave the voting open until Thursday 10th June. My own vote is +1, for the following reasons: - Local-only HPX already contains a huge amount of functionality useful for many people and making it a separate project gives it more visibility outside of "HPX is a library for distributed computing"; it makes it explicit for potentially curious users that HPX is useful even when one is not interested in distributed computing. This may bring in new users and developers. - Build times of HPX itself for local-only users will be reduced, without additional configuration. - There is no chance of accidentally including distributed headers that may slow down builds for local-only users; a local-only user has to explicitly opt-in to using distributed features of HPX. - Importantly, for users of distributed HPX there is no additional effort if they are using a package manager. For those who build HPX manually there is at most one additional project to configure and build. This can be zero additional projects if using git submodules or CMake's fetchcontent. Names of headers, functionality, and libraries that currently expose distributed functionality will continue to do so under the same names (modulo refactoring that might be done even if the project stayed in one repository) and will require no adaptations for users of distributed HPX. - Development - Testing and development can be done more quickly, not having to build and run over 1000 tests (the split is roughly a third each for local-only non-parallel-algorithm tests, local-only parallel-algorithm tests, and distributed tests). This makes testing everything locally before opening pull requests more feasible, and gives faster feedback through CI. - In cases where it seems awkward or annoying to do changes across two repositories I believe it's a sign that the two projects are too tightly coupled and that coupling needs to be addressed. However, I think these cases will be few (or are already resolved). Simple changes like renamings can easily be done in two stages across two repositories. - Releases can be made independently and more frequently with two or more smaller projects. My personal preference would be to split HPX even further such that it e.g. futures/senders/receivers would only be interfaces available as a separate library. This may encourage alternative runtimes to be implemented and used behind the standards-conforming interfaces, without having to depend on and build the HPX runtime. The same goes for the parallel algorithms. In general, I think providing smaller (within reason) reusable libraries can encourage use of and contribution to those libraries in a way that a large monolithic project can't. It also encourages separation of concerns (I'm not arguing that this can't be done in a monolithic project, but accidentally introducing unwanted coupling between separately developed libraries is very difficult). However, for the purposes of this vote, this finer split is not on the table. CSCS has unfortunately decided not to continue to support the full distributed version of HPX long term, and will therefore be withdrawing development of HPX in its current form after the next HPX release. We hope to continue working with the HPX community as part of a reduced hpx-local project and will continue to maintain and develop modules that are part of that reduced project. Kind regards, Mikael Simberg --- end message --- Mikael
_______________________________________________ hpx-pmc mailing list [email protected] https://mail.cct.lsu.edu/mailman/listinfo/hpx-pmc
