Hi John,
For both your questions, the answer is that there isn't an existing
implementation of distributed arrays in the standard library that does
exactly what you want. But, X10 is certainly capable of being used to
define such kinds of distributed data structures, and get them to perform
well. So, with some programming, it is certainly possible.
To design the right data structure, you'll need to think about which
operations are on your critical path and which operations you can afford to
make a little slow to gain flexibility.
John MacFrenz <macfr...@yandex.com> wrote on 10/03/2014 05:29:26 AM:
>
> I've two questions regarding distributed arrays.
>
> First is how could I distribute a 2d or 3d array in block
> distribution so that sizes of distributed blocks could be varied
> dynamically during runtime? Blocks should be distributed along just
> a single axis, the longest axis in the array. Distributed blocks
> would be continuous, but vary in length along longest axis. This
> would be needed since I am planning to run this program in
> heterogenous environment of consumer grade computers, and dynamic
> load balancing will be paramount. If it matters, this program would
> solve NS equations.
>
The distributed arrays in x10.array (subclasses of DistArray) are
designed to optimize indexing performance of individual element access.
The existing implementations (including two I added late this afternoon
because of your email (http://sourceforge.net/p/x10/code/28348)), (a) have
a fixed distribution of the longest axis among the places and (b)
distribute the indices of that axis as evenly as possible.
It would be really easy to take one of the arrays I just checked in
and change (b) so that you could distribute the longest axis unevenly when
the array was constructed. You'd just need to change the few places where
BlockingUtils.partitionBlock is called to do something else. The basic
design idea with these classes is to compute key values needed for indexing
operations at array construction and cache them in the local state of each
array so they don't need to be recomputed on access. This is key to making
the translation from the global index to the local offset use as few
instructions as possible.
One warning...I wrote these two new array implementations in about 45
minutes...I did do some basic testing, but they might not be 100% correct
yet.
If you really need to be able to dynamically redistribute the longest
axis between places after the array is created, you might consider
something that was more like an array of arrays in each place to make it
easier to redistribute the longest axis while maintaining spatial locality
for the shorter axis.
>
> Another question concerns distribution of unstructured arrays. This
> would be for a particle system. Particles should be dynamically
> distributed according to their location. Probably this would be done
> by dividing the domain into regular sized cuboids and checking into
> which cuboid each particle fits, and then distributing particles
> residing inside given cuboid to one place. Question is how this
> could be achieved in X10? Again, ability to dynamically balance the
> load is required.
>
Here, I'd probably start by looking into some of the data structures used
by ANUChem (http://sourceforge.net/projects/anuchem/). Josh might chime in
with more exact suggestions later, but I think his code does quite similar
things. There's a distributed OctTree in his FMM code for example.
--dave------------------------------------------------------------------------------
Meet PCI DSS 3.0 Compliance Requirements with EventLog Analyzer
Achieve PCI DSS 3.0 Compliant Status with Out-of-the-box PCI DSS Reports
Are you Audit-Ready for PCI DSS 3.0 Compliance? Download White paper
Comply to PCI DSS 3.0 Requirement 10 and 11.5 with EventLog Analyzer
http://pubads.g.doubleclick.net/gampad/clk?id=154622311&iu=/4140/ostg.clktrk
_______________________________________________
X10-users mailing list
X10-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/x10-users
