Hi Sara - As Dave said earlier, on a network which does not support RDMA,
such as you are using, AsncCopy and At Rail numbers should be similar. In
general, it's good practice to use AsyncCopy, because if you happen to run
your program on hardware which supports RDMA, you will get a performance
boost.
As for the performance question, there shouldn't be a mismatch, and if
there is, I'd like to look into it. Are you running X10 v2.5.0?
- Ben
From: Sara Salem Hamouda <sara.sa...@anu.edu.au>
To: "x10-users@lists.sourceforge.net"
<x10-users@lists.sourceforge.net>
Date: 11/05/2014 08:49 PM
Subject: Re: [X10-users] asyncCopy versus at for Rail copying
Hi All,
I have an update regarding the experiment I sent before. The rail size I
used was too small for the comparison, also I didn't measure how much time
"at" takes without copying any data.
I repeated the experiments with bigger rails (400K, 1M, 128M) and measured
the time "at" takes without copying any data by adding this code snippet to
the beginning of my program.
val timeBeforeAt1 = Timer.milliTime();
at (destPlace){
Console.OUT.println("At Time["+(Timer.milliTime
()-timeBeforeAt1)+"] ...");
}
Description of the below results:
- The times are in milliseconds and are the average of 3 runs.
- At: time taken by "at" without copying any extra variables than the one
used for measuring the time.
- At GR: time taken by "at" which copies a GlobalRef to the rail (method
1)
- AsyncCopy: time taken to remote copy the rail (method 1)
- At Rail: time taken by "at" which copies the whole rail (method 2)
- Copy: local copying the rail (method 2)
Expriment 1: Rail Size 400K
Native:
At : 4386
At GR : 4384
AsyncCopy : 4
At Rail : 4393
Copy : 0
Managed:
At : 4599
At GR : 4390
AsyncCopy : 86
At Rail : 4398
Copy : 0
Expriment 2: Rail Size 1M
Native:
At : 4386
At GR : 4385
AsyncCopy : 10
At Rail : at hangs
Copy : at hangs
Managed:
At : 4600
At GR : 4391
AsyncCopy : 100
At Rail : 4410
Copy : 1
Expriment 3: Rail Size 128M
Native:
At : 4387
At GR : 4386
AsyncCopy : 1257
At Rail : at hangs
Copy : at hangs
Managed:
At : 4600
At GR : 4392
AsyncCopy : 11358
At Rail : at hangs
Copy : at hangs
Conclusions:
- In the runs when "at" doesn't hang: "at" which copies a rail takes more
time than "at" that copies a GR, this matches the expected behaviour.
- Time taken by asyncCopy in the native backend is reasonable considering
the network speed (1000Mb/s). Copying 128M took 1257 milliseconds in
experiment 3.
- Time taken by asyncCopy in managed X10 is too slow. I used a simple java
program that transfers 128MB using sockets, it took around 1185
milliseconds. I don't know why the managed version is about 10 times
slower, any explanation for this point?
Best Regards,
Sara
From: Sara Salem Hamouda
Sent: Wednesday, November 5, 2014 8:26 PM
To: x10-users@lists.sourceforge.net
Subject: asyncCopy versus at for Rail copying
I was trying to compare the performance of copying a Rail[Byte] between 2
places using Rail.async and using "at". Each place is running on a
different host, and the network speed is 1000Mb/s.
The first method:
1. A rail is created
2. A GlobalRef to that rail is created
3. Used At to move to the second place -> here I expect that "at" will
need to copy only the GlobalRef, not the whole array.
4. Used Rail.asyncCopy from the second place to copy the array using the
GlobalRef
The second method:
1. A rail is created
2. Used At to move to the second place -> here I expect that "at" will
implicitly copy the whole rail, that is why I expected it to be slower than
the "at" in the first method.
3. Used Rail.copy from the second place to copy the array
That is the used program code:
import x10.util.Timer;
public class TestRemoteCopy {
public static def main(args:Rail[String]) {
//Create Rail and Initialize it
val size = Long.parse(args(0));
val source:Rail[Byte] = new Rail[Byte](size);
val sourcePlace = Place.places()(0);
val destPlace = Place.places()(1);
for (var i:Long = 0; i < size; i++){
source(i) = (i%128) as Byte;
}
//Copy using Rail.asyncCopy
val gr = GlobalRail[Byte](source);
val timeBeforeAt = Timer.milliTime();
at (destPlace){
Console.OUT.println("At Time["+(Timer.milliTime
()-timeBeforeAt)+"] ...");
val dest:Rail[Byte] = new Rail[Byte](size);//1<
val timeBeforeAsyncCopy = Timer.milliTime();
finish Rail.asyncCopy(gr, 0, dest, 0, size);
Console.OUT.println("AsyncCopy Time["+(Timer.milliTime
()-timeBeforeAsyncCopy)+"] ...");
}
//Copy using at
val timeBeforeSecondAt = Timer.milliTime();
at(destPlace){
Console.OUT.println("Second At Time["+(Timer.milliTime
()-timeBeforeSecondAt)+"] ...");
val dest:Rail[Byte] = new Rail[Byte](size);
val timeBeforeSecondCopy = Timer.milliTime();
Rail.copy(source, 0, dest, 0, size);
Console.OUT.println("Second Copy Time["+(Timer.milliTime
()-timeBeforeSecondCopy)+"] ...");
}
}
}
These are some results, time is printed in milliseconds:
Rail Size: 1024 bytes
Native X10:
At Time[4305] ...
AsyncCopy Time[1] ...
Second At Time[4305] ...
Second Copy Time[0] ...
Managed X10:
At Time[4521] ...
AsyncCopy Time[31] ...
Second At Time[4308] ...
Second Copy Time[1] ...
Rail Size: 1048576 bytes (1024*1024)
Native X10:
At Time[4306] ...
AsyncCopy Time[10] ...
--> using "at" hangs
Managed X10:
At Time[4534] ...
AsyncCopy Time[106] ...
Second At Time[4331] ...
Second Copy Time[0] ...
I have some questions based on these numbers:
1) Why "at" in the first method, which is supposed to copy only a
GlobalRef, is taking as much time or even more time than the second method
which copies the whole rail?
2) Is there a maximum limit on the data size that "at" can copy in native
X10?
3) The above numbers suggests that using "at" for copying rails is more
preferable than using Rail.asyncCopy. Is this conclusion correct?
​Best Regards,
Sara
------------------------------------------------------------------------------
_______________________________________________
X10-users mailing list
X10-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/x10-users
------------------------------------------------------------------------------
_______________________________________________
X10-users mailing list
X10-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/x10-users
