Am Samstag, 4. September 2010, 18:05:29 schrieb Juan Pablo Carbajal: > Hi, > > I think the work of the authors is not to be discourage, since is good > that benchmarks are performed (correct ones, of course). If there is > as strong opinion about the contents of the papers the community > representatives should write the authors to make things clear. Their > e-mails can be found on the article itself. I have already send a > heads up to the first author. > > Regards, > > JPi > > On Sat, Sep 4, 2010 at 5:39 PM, Laurent Hoeltgen <hoeltg...@gmail.com> wrote: > > Hi, > > > > Am Samstag, den 04.09.2010, 16:08 +0200 schrieb Thomas Weber: > >> On Sat, Sep 04, 2010 at 01:43:44PM +0200, Martin Helm wrote: > >> > Am Samstag, 4. September 2010, 13:22:36 schrieb Jaroslav Hajek: > >> > > Hmm, the Octave-related information is sadly outdated :( Probably > >> > > still based on Octave 3.0.x. I believe 3.2.x would also perform > >> > > better in the benchmarks. > >> > > >> > In addition I have some severe doubts about the reliability of the > >> > tests, I look through the table and found for a 2000x2000 matrix > >> > multiply octave 18.664 > >> > R 0.070 > >> > >> It's not just Octave, R beats every other product by a factor of at > >> least 100. > >> > >> > Comparing the performance of a matrix multiply in R and octave does > >> > not give me any hint that R outperforms (even the old 3.0 version of > >> > octave) by a factor 266 (?!?) but gives comparable speed. > >> > Looks like the testers did not recognize that a*b is not the same in > >> > octave and R but that in R one has to write %*% for matrix > >> > multiplication and they compare component wise multiplication in R > >> > with full matrix multiplication in octave. > >> > Not very promising from my point of view. > >> > >> I have the same feeling. LU decomposition is faster by a factor of at > >> least 500(!) in R than in any other product. > >> > >> Sorry, I don't believe this. > >> > >> Thomas > > > > Furthermore, they state that there's no online documentation for octave. > > Simply googling for "octave documentation" returns the octave manual and > > the octave forge documentation as the first two search results. > > > > Laurent > > > > The concern is not if something is discouraging or encouraging, but that some of the numbers are simply wrong, and I would like to see how they were exactly made. The linear algebra routines in Gnu R have approximately the same speed as the ones in octave (I can not argue about the other systems since I use only R and octave), depends of course against which libs one links the software but not to that extent. And in my opinion if I read a paper there numbers which I can easily compare myself are not correct then I cannot trust the rest of the numbers which I cannot easily compare myself, that is the point. I personally do not care at all if one of the systems is slightly slower or faster than the other since I decide based on the purpose which system I use not based on the average speed of a default installation (esp. because normally I build it on my own), but someone who cares will get wrong impressions. I will for sure test next week some of the results with the same versions of some of the software mentioned there and ask to the authors how they came to the results if I see more than the two wrong results I found already (but I am of course not a community representative but a private person who is concerned about incorrect benchmarks and will ask that not on behalf my own). That does not automatically say that all the numbers are wrong, but at least some of them and for a published paper which will probably be cited elsewhere this is really bad.
- Martin ------------------------------------------------------------------------------ This SF.net Dev2Dev email is sponsored by: Show off your parallel programming skills. Enter the Intel(R) Threading Challenge 2010. http://p.sf.net/sfu/intel-thread-sfd _______________________________________________ Octave-dev mailing list Octave-dev@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/octave-dev
