Hi Sascha,
It looks like the X10 scheduler has nothing to do with your performance
issue.
As far as I can tell on my mac (10.6) the following happens:
- whether you have 1 or many X10_NTHREADS, a single thread reads the file
(as expected from your source code).
- internally the file is read byte by byte using fgetc (c++ backend).
- if there is more than one thread for each read operation: fgetc invokes
flockfile to lock the file before reading a byte.
- if there is exactly one thread, there is no locking...
So what I measure is a significant perf degradation going from 1 to 2
threads and more or less (aka minor degradation) when the number of threads
increases beyond that.
I am somewhat surprised by this flockfile thing and not sure how portable
my observations are.
In any case, this suggests you should be reading the file in larger chunks
(I believe we support that...) and shuffle the data separately.
That should bring the perf of the multi-threaded runs back in line with the
single-threaded case.
We will keep this issue in mind as we improve the design of the io lib.
Let me know if your own observations contradict this diagnostic.
Olivier
Sascha Roloff <sascha.rol...@googlemail.com> wrote on 01/13/2011 06:17:04
PM:
> From: Sascha Roloff <sascha.rol...@googlemail.com>
> To: Mailing list for users of the X10 programming language <x10-
> us...@lists.sourceforge.net>
> Date: 01/13/2011 06:18 PM
> Subject: Re: [X10-users] X10_NTHREADS slowdown with IO
>
> Hi Oliver,
>
> thank you for your help. I can give you some parts of my code, it's not
> complete! This code just tries to read the data of an ppm image. I
skipped
> the calculations, because only the IO transfers cause the slowdown. I
hope
> it's enough that you can reproduce it. Thank you very much.
>
> Sascha
>
> // encode image
> public static def encodeImage(file:String):Image {
> // read image data
> var time:Long = Timer.nanoTime();
> val image = new Image(file);
> time = (Timer.nanoTime() - time) / 1000000;
>
> Console.OUT.println("read image data: " + time + " ms");
> }
>
> public class Image {
> public val width:Int;
> public val height:Int;
> public val comp1:Array[Int](2){self.rect==true};
> public val comp2:Array[Int](2){self.rect==true};
> public val comp3:Array[Int](2){self.rect==true};
>
> // constructor
> public def this(file:String) {
> val fileobject = new File(file);
> val filereader = fileobject.openRead();
>
> comp1 = new Array[Int]((0..(height-1))*(0..(width-1)));
> comp2 = new Array[Int]((0..(height-1))*(0..(width-1)));
> comp3 = new Array[Int]((0..(height-1))*(0..(width-1)));
>
> for([i,j] in comp1) {
> comp1(i,j) = (filereader.readByte() & 0xFF) - 128;
> comp2(i,j) = (filereader.readByte() & 0xFF) - 128;
> comp3(i,j) = (filereader.readByte() & 0xFF) - 128;
> }
> }
> }
>
>
> 2011/1/13 Olivier Tardieu <tard...@us.ibm.com>
>
> > Hi Sascha,
> >
> > Could you share the code with me so that I can try reproducing the
issue?
> > At least the reading part (including the timer code).
> >
> > Thanks,
> >
> > Olivier
> >
> > Sascha Roloff <sascha.rol...@googlemail.com> wrote on 01/13/2011
05:25:45
> > AM:
> >
> > > From: Sascha Roloff <sascha.rol...@googlemail.com>
> > > To: Mailing list for users of the X10 programming language <x10-
> > > us...@lists.sourceforge.net>
> > > Date: 01/13/2011 05:30 AM
> > > Subject: [X10-users] X10_NTHREADS slowdown with IO
> > >
> > > Dear list,
> > >
> > > I'm writing an X10 application which is suitable to parallelise. I
work
> > on
> > > an Intel Quad Core CPU and want to use 4 Threads. I changed
X10_NTHREADS
> > > accordingly. The results are ok, I got almost a linear speedup at the
> > > calculations. But there is one problem. I also have to read a big
> > imagefile
> > > at the beginning and this takes a lot more time, when I use more
Threads.
> > >
> > > $ x10c++ -STATIC_CALLS -O -NO_CHECKS -o x10app Code.x10
> > >
> > > $ export X10_NTHREADS=1
> > > $ ./x10app
> > > read image data: 2336 ms
> > > calculation: 8290 ms
> > >
> > > $ export X10_NTHREADS=4
> > > $ ./x10app
> > > read image data: 6129 ms
> > > calculation: 2330 ms
> > >
> > > Here is a code snippet of reading the image:
> > >
> > > comp1 = new Array[Int]((0..(height-1))*(0..(width-1)));
> > > comp2 = new Array[Int]((0..(height-1))*(0..(width-1)));
> > > comp3 = new Array[Int]((0..(height-1))*(0..(width-1)));
> > >
> > > for([i,j] in comp1) {
> > > comp1(i,j) = filereader.readByte() & 0xFF;
> > > comp2(i,j) = filereader.readByte() & 0xFF;
> > > comp3(i,j) = filereader.readByte() & 0xFF;
> > > }
> > >
> > > I don't understand, maybe someone can help me?
> > >
> > > Sascha
> > >
> >
> >
>
------------------------------------------------------------------------------
> >
> > > Protect Your Site and Customers from Malware Attacks
> > > Learn about various malware tactics and how to avoid them. Understand
> > > malware threats, the impact they can have on your business, and how
you
> > > can protect your company and customers by using code signing.
> > > http://p.sf.net/sfu/oracle-sfdevnl
> > > _______________________________________________
> > > X10-users mailing list
> > > X10-users@lists.sourceforge.net
> > > https://lists.sourceforge.net/lists/listinfo/x10-users
> >
> >
>
------------------------------------------------------------------------------
> > Protect Your Site and Customers from Malware Attacks
> > Learn about various malware tactics and how to avoid them. Understand
> > malware threats, the impact they can have on your business, and how you
> > can protect your company and customers by using code signing.
> > http://p.sf.net/sfu/oracle-sfdevnl
> > _______________________________________________
> > X10-users mailing list
> > X10-users@lists.sourceforge.net
> > https://lists.sourceforge.net/lists/listinfo/x10-users
> >
>
------------------------------------------------------------------------------
> Protect Your Site and Customers from Malware Attacks
> Learn about various malware tactics and how to avoid them. Understand
> malware threats, the impact they can have on your business, and how you
> can protect your company and customers by using code signing.
> http://p.sf.net/sfu/oracle-sfdevnl
> _______________________________________________
> X10-users mailing list
> X10-users@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/x10-users
------------------------------------------------------------------------------
The ultimate all-in-one performance toolkit: Intel(R) Parallel Studio XE:
Pinpoint memory and threading errors before they happen.
Find and fix more than 250 security defects in the development cycle.
Locate bottlenecks in serial and parallel code that limit performance.
http://p.sf.net/sfu/intel-dev2devfeb
_______________________________________________
X10-users mailing list
X10-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/x10-users
