Ahmed I'm a little bit frightened by the idea of you working on hwloc+hpx - the reason is the same as the one that caused problems in gsoc.
* we don't have a well defined plan on what we want to do with hwloc yet. I would like to clean it up and make it better, but only because it is missing a couple of functions that I would like - but I don't really know hwloc that well either. (same as I didn't really know fflib) If you are working on a a problem and the tools you have don't quite solve the problem, then you can tweak the tools to work - for this reason I am looking at hwloc and asking myself - how do I find out which socket on the node is 'closest' to the gpu? Should I create a thread pool for GPU managment on socket 0, or socket 1 (or 2,3.4 ...). I will be working on a machine with 6 gpu's and 2 processors. it is probably safe to assume that gpu's 0,1,2 are closest to socket 0, and 3,4,5 to socket 1 - but maybe it's 0,2,4 and 1,3,5 - I'd like to be able to query hwloc and get this info and build it into our resource partitioned in such a way that the programmer can just say "which gpus should I use from this socket" or vice versa. So I know what I want - but I don't really have a plan yet, and usually, one knocks up a bit of code, get's something that half works and is good enough, then tries to make it fit into the existing framework and realize that something else needs to be tweaked to make it fit nicely with the rest of HPX. This requires a deal of understanding of the hpx internals and how it all fits together. So in summary, its better for you to find a project that already interests you (part of your coursework?) and use HPX for that project - this gives you time to learn how hpx works and use it a bit - then extend it gradually as your knowledge grows. hwloc/topology is quite a low level part of hpx and does require a bit of help. If you were here in the office next door to me, it'd be no problem - cos you could get help any time, but working on your own is going to be tough. I'm not saying do't do it (I did suggest it after all), but it will be hard to get going. I will try to think of some other less low level things we could use help with. (Ideas Thomas?) JB ________________________________ From: hpx-users-boun...@stellar.cct.lsu.edu [hpx-users-boun...@stellar.cct.lsu.edu] on behalf of Ahmed Ali [ahmed.al...@eng-st.cu.edu.eg] Sent: 24 October 2018 14:12 To: hartmut.kai...@gmail.com Cc: hpx-users@stellar.cct.lsu.edu Subject: Re: [hpx-users] Contributing to HPX John, Hartmut, Well I can go with that hwloc part. Where should I start? I think I should start by reading about hwloc from the open-mpi document. Is there any document about the HPX interface with hwloc? Best, Ahmed Samir On Tue, Oct 23, 2018 at 1:46 PM Hartmut Kaiser <hartmut.kai...@gmail.com<mailto:hartmut.kai...@gmail.com>> wrote: John, Ahmed, The hwloc integration sounds like a good idea to me. But anything else would be fine as well.,, Regards Hartmut --------------- http://stellar.cct.lsu.edu https://github.com/STEllAR-GROUP/hpx From: hpx-users-boun...@stellar.cct.lsu.edu<mailto:hpx-users-boun...@stellar.cct.lsu.edu> <hpx-users-boun...@stellar.cct.lsu.edu<mailto:hpx-users-boun...@stellar.cct.lsu.edu>> On Behalf Of Biddiscombe, John A. Sent: Tuesday, October 23, 2018 3:43 AM To: hpx-users@stellar.cct.lsu.edu<mailto:hpx-users@stellar.cct.lsu.edu> Subject: Re: [hpx-users] Contributing to HPX Ahmed Good to hear from you again. >I was a participant in GSoC'18 in HPX but I've failed the program. I still feel bad about that. >However, I liked the project and the contribution to HPX. I wanted to continue >contributing to HPX so I want someone to guide me. Nice. >What part should I work on now? and where should I start? Well, there is the obvious option of continuing the parcelport work, but I suspect you want to do something else since we didn’t help you enough first time around. I’d certainly like to carry on with that and get it up and running. It’s on my list, but I have a full plate already unfortunately, so it has to wait if I’m doing it myself. There should still be a list of ‘projects’ that were compiled for GSoC that you could look through and if something looks interesting, have a go at it. If you have a project that you are already working on for your studies or hobbies - why not try involving that somehow. Experience tells me that if someone has a problem to solve and they use a library like HPX to solve it, then they get much better results than if they just make up a project and try to implement it. The real advantage to this is that you can USE hpx to work on a project without having to understand all of hpx under the hood - then when you find that a feature you need in hpx doesn’t exist, then you atsrt poking around to find a way of implementing it or adding support. One thing that is catching my eye at the moment is our interface between hwloc (https://www.open-mpi.org/projects/hwloc/) and hpx::topology classes. the code in there is not very well maintained and could do with a thorough cleaning up. I’d like to be able to create a topology object from hwloc and query not just numa domains (which we have implemented in the resource partitioner), but also PCI bus connections to GPUs and network cards. If I have a 2 socket machine and only 1 gpu - which socket is closest to the gpu etc. That hasn’t been implemented and it would be very useful. Hartmut, Thomas Do you have anything in mind that Ahmed could work on? JB
_______________________________________________ hpx-users mailing list hpx-users@stellar.cct.lsu.edu https://mail.cct.lsu.edu/mailman/listinfo/hpx-users