Hi, and thanks again for your detailed reply. If I decided to go with Chapel, how would you recommend me to do load balancing with existing tools when using rectangular grids? In earlier post you said that could be done implicitly by distributions of arrays, which would require custom domain map, or explicitly via on clauses. Wouldn't doing that with on clauses cause a lot of unnecessary data transfers between machines, or is there some clever mechanism that avoids that? Also, would spawning multiple locales onto those more powerful machines be a viable option?
>> How difficult would you consider it to be to write such domain map? I >> looked at the already available distributions and was surprised by how >> much code was needed, I would have thought that all that would be needed >> would be functions to map between global indices and rank, local >> index-pairs.... > > You are correct that writing a fully-functional domain map is a > significant endeavor. The reason is that while providing mappings from > indices to locales is sufficient to write a domain map, it isn't (in my > technical opinion) sufficient for writing one that is space and time > efficient. In designing Chapel, we wanted the domain map author to be > able to specify the data structures and memory layout used to store a > locale's local indices and data in order to maximize efficiency, and once > the storage model is open, all the methods for accessing, iterating over, > and operating on that storage must also be open. I don't have enough knowledge to say anything about the efficiency issue. However, as an end-user I think that implementing custom domain map seems way too difficult. I think that either writing custom domain maps should be easier, or there should be much more and more flexible already implemented maps included, however I understand well that this language is still work in progress. > In practice, it is often the case that one can avoid supporting the full > domain map interface; for example, slicing and reindexing tend to be two > of the more complex operations to implement, but there's no need to > implement them if you don't do any slicing or reindexing of arrays using > that domain map in your program. > > Typically, our guidance for writing a new domain map involves: > > (a) prototyping the key data structures, iterators, etc. outside of the > domain map framework first, for simplicity, and then moving it inside the > framework as it begins to work. This is similar to the suggestion to do > load balancing manually in my mail from yesterday; once you had that > working, moving it inside of a domain map would make it accessible to > others via a simpler interface. Would you expand this a bit, what do you mean by those key data structures? What would those be in block dist, for example? I looked at README.dsi on docs/technical, is document that up to date? > (b) starting from an existing domain map and customizing it -- e.g., if my > unevenly partitioned array was going to look a lot like a block > distribution, yet with different-sized blocks per locale, the block > distribution would be a good place to start from. Would copying Block dist and making such changes be very difficult? Basically, the kind of distribution I was thinking of would be very similar to Block dist. Differences would be that the data would be partitioned only along one axis, the longest axis. That would make it easier to assign different sized blocks to locales. The constructor would take a map as an argument which describes (relative) number of elements assigned to each locale. Later I'd also like to add functionality to change distribution during execution by passing a similar map. That way I could do load balancing at some other place, assuming I have a way to time the executions on each locale there. >> Also, do you think there is some other language, library or framework >> that would suit my needs better? > Charm++ comes to mind as a reasonably mature programming model that is > inherently very dynamic in its mapping of data and tasking to compute > nodes, so you might take a look at it. It's the main thing that springs > to mind as a viable alternative to me at present within the HPC space. Thanks, I'll have a look at that. ------------------------------------------------------------------------------ _______________________________________________ Chapel-users mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/chapel-users
