Hi, Ben, thanks for the hints.
When X10_HOSTFILE (or X10_HOSTLIST) is populated with one entry per core (that is in my case 12 entries per one node), the processes are properly distributed among nodes. I've also tested the MPI version, where MPI is configured to run over InfiniBand and it runs significantly faster (thanks to InfiniBand low latency). And I've also tried to use X10_NTHREADS > 1, and yes, I get the results faster, however, the result are not correct. I guess I'll have to look into the code ... I've put the code of the N-body problem (http://en.wikipedia.org/wiki/N-body_problem) below (the code is from my colleague...) ... if somebody finds any error in the code regarding the usage of async, I'll be very glad ;) ! import x10.io.Console; import x10.io.File; import x10.io.FileReader; import x10.io.FileWriter; import x10.util.Random; import x10.compiler.Native; public class nbody { public static def main(args : Array[String](1)){ val fileName = args.size > 0 ? args(0) : "problem1"; var nTimeSteps : Int; var eps : Double; var tol : Double; var bodyMass : Array[Double](1); var bodyPos : Array[Double](1); var bodyVel : Array[Double](1); var bodyAcc : Array[Double](1); ////////// READ FILE ////////// var f : File = new File(fileName); val reader = f.openRead(); val nBodies = Int.parse(reader.readLine().trim()); nTimeSteps = Int.parse(reader.readLine().trim()); val dTime = Double.parse(reader.readLine().trim()); eps = Double.parse(reader.readLine().trim()); val epssq=eps*eps; tol = Double.parse(reader.readLine().trim()); Console.OUT.println(nBodies); bodyMass = new Array[Double](nBodies); bodyPos = new Array[Double](nBodies*3); bodyVel = new Array[Double](nBodies*3); bodyAcc = new Array[Double](nBodies*3,0.0); for (var i : Int = 0; i<nBodies; i++){ val line = reader.readLine().trim(); val value = line.split(" "); bodyMass(i) = Double.parse(value(0).toString()); bodyPos(3*i+0) = Double.parse(value(1).toString()); bodyPos(3*i+1) = Double.parse(value(2).toString()); bodyPos(3*i+2) = Double.parse(value(3).toString()); bodyVel(3*i+0) = Double.parse(value(4).toString()); bodyVel(3*i+1) = Double.parse(value(5).toString()); bodyVel(3*i+2) = Double.parse(value(6).toString()); } if(reader != null) reader.close(); ////////// END READ FILE ////////// val dist = Dist.makeBlock(bodyMass.region); val remBodyPos = new RemoteArray(bodyPos); val remBodyVel = new RemoteArray(bodyVel); val remBodyAcc = new RemoteArray(bodyAcc); val bodyPoss = bodyPos; val bodyVell = bodyVel; val bodyAccc = bodyAcc; val bodyMasss = bodyMass; for(var step: Int = 0; step < nTimeSteps; step++) { finish ateach (p in dist){ //Console.OUT.println("Befor Place:" + here.id + " Item:"+p+" Value:" + bodyPoss(p)); //finish async{ //computeForce(p); var dr : Array[Double](1) = new Array[Double](3); var drsq : Double; var idr : Double; var scale : Double; for(var k : Int=0;k<3;k++){ bodyAccc(3*p+k) = 0.0; } finish for(var j: Int=0; j<nBodies; j++){ val jj=j; async{ for(var k : Int=0; k<3; k++){ dr(k) = bodyPoss(3*jj+k) - bodyPoss(3*p+k); } drsq = dr(0)*dr(0) + dr(1)*dr(1)+dr(2)*dr(2)+epssq; idr = 1/Math.sqrt(drsq); scale = bodyMasss(jj)*idr*idr*idr; for(var k : Int=0; k<3; k++){ bodyAccc(3*p+k) += scale*dr(k); } } } //} //finish async { //advanceBody(p) for(var k : Int=0;k<3;k++){ bodyPoss(3*p+k) += bodyVell(3*p+k)*dTime + 0.5*bodyAccc(3*p+k)*dTime*dTime; bodyVell(3*p+k) += bodyAccc(3*p+k)*dTime; } //} //Console.OUT.println("After Place:" + here.id + " Item:"+p+" Value:" + bodyPoss(p)); finish Array.asyncCopy(bodyPoss, 3*p, remBodyPos, 3*p, 3); finish Array.asyncCopy(bodyVell, 3*p, remBodyVel, 3*p, 3); finish Array.asyncCopy(bodyAccc, 3*p, remBodyAcc, 3*p, 3); //Console.OUT.println("aCopy Place:" + here.id + " Item:"+p+" Value:" + bodyPoss(p)); } //for(var i:Int=0;i<bodyPos.size;i++)Console.OUT.print(bodyPos(i)+", "); //Console.OUT.println(); } //// WRITE OUTPUT // f = new File(fileName + ".out"); val writer = f.printer(); try{ for (var i : Int = 0; i<nBodies; i++){ writer.printf("%+1.6E\t%+1.6E\t%+1.6E\n", bodyPos(3*i+0), bodyPos(3*i+1), bodyPos(3*i+2)); } writer.flush(); } catch(ioe: x10.io.IOException){ Console.OUT.println("IO error"); } finally{ if(writer != null) writer.close(); } Console.OUT.println("-------------"); } } Kind regards, Marko Kobal -----Original Message----- From: Benjamin W Herta [mailto:bhe...@us.ibm.com] Sent: Tuesday, August 09, 2011 7:23 PM To: Marko Kobal Cc: x10-users@lists.sourceforge.net Subject: Re: [X10-users] runing on multiple places (cluster) with sockets RT implementation Hi Marko - unfortunately, attachments are also removed on this mailing list. You can send an email directly to me if you would like me to look at it. You can specify which host each place is located on most easily by using X10_HOSTFILE instead of X10_HOSTLIST. If you are running with 48 places, then you can create a text file with 48 lines, with a hostname on each line. Place 0 will run on the host specified on the first line, place 1 on the second line, and so on. This can also be done with a hostlist, but it will be a very long command line. Both the hostfile and hostlist wrap, so when you specify only 4 nodes as per your email below, you should be getting 12 places on each (node001 should have places 0,4,8,12,etc). Depending on your program, you may get better performance running with 4 places instead of 48, and use async to increase the parallelism within each place. You may also want to explicitly set the X10_NTHREADS environment variable to 1 if you're using 48 places, or 12 if using 4 places. Others on this mailing list may have additional comments on this. - Ben From: Marko Kobal <marko.ko...@arctur.si> To: "x10-users@lists.sourceforge.net" <x10-users@lists.sourceforge.net> Date: 08/09/2011 12:35 Subject: Re: [X10-users] runing on multiple places (cluster) with sockets RT implementation Hi, Me again ;) I have an example (N-body) written to execute in parallel. It works just fine, scales good on more cores. However, when I try to run it on more than one machine, more exactly on 4 nodes, I can see that X10 does not properly distributes the load to the nodes. I have nodes with 2 Intel processors, 6 cores each, that makes 12 cores per node, 48 cores per 4 nodes: export X10_HOSTLIST=node001,node002,node003,node004 export X10_NPLACES=48 I did compile for the sockets RT Implementation: # x10c++ -x10rt sockets -o nbody.parallel.sockets nbody.parallel.x10 When I execute the program, I can see that processes are spawn through the 4 nodes, however the load is not distributed evenly. On some nodes there are more than 12 processes running, on some less than 12. This is obviously not good as some nodes are overloaded (and as such processing is not optimal) and some are under loaded (that's not one would wish for). See the print screen from my monitoring software: (sorry, the picture was embedded wich is obviusly not supported by the mailing list, I've put it into attachement now) The usage for X10Laucher says: X10Launcher [-np NUM_OF_PLACES] [-hostlist HOST1,HOST2,ETC] [-hostfile FILENAME] COMMAND_TO_LAUNCH [ARG1 ARG2 ...] . so there is no parameter to set "processes per node" . I would expect something similiar as is the "-perhost" parameter in the MPI world. Is there any way to achieve this with the X10 sockets RT Implementation? Thanks for help! Kind regards, Marko Kobal ---------------------------------------------------------------------------- -- uberSVN's rich system and user administration capabilities and model configuration take the hassle out of deploying and managing Subversion and the tools developers use with it. Learn more about uberSVN and get a free download at: http://p.sf.net/sfu/wandisco-dev2dev _______________________________________________ X10-users mailing list X10-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/x10-users
------------------------------------------------------------------------------ uberSVN's rich system and user administration capabilities and model configuration take the hassle out of deploying and managing Subversion and the tools developers use with it. Learn more about uberSVN and get a free download at: http://p.sf.net/sfu/wandisco-dev2dev
_______________________________________________ X10-users mailing list X10-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/x10-users
