And I think we should change this benchmark from random to a stable implementation. So we should communicate with all of the other peers, not just a random peername. Then we can precompute (or cache) the tag message and we save the string concat operations. We can put them into a directly allocated ByteBuffer as well, this will save the serialization. But then we cannot compare the results with the versions before.
Additional to the log level change, we can receive a superior performance improvement ;) 2011/9/22 Edward J. Yoon <[email protected]> > Haha, obviously it can't be ignored. > > Unfortunately, I can't access my test machines now. I'll check tomorrow. > > On Thu, Sep 22, 2011 at 5:05 PM, Thomas Jungblut > <[email protected]> wrote: > > You're going to laugh, but we spend 80% of the time, logging the > messages. > > Let's change the log level to debug or remove the logging in the bench > > example. > > > > Sadly I still receive > > > > org.apache.zookeeper.KeeperException$NoNodeException: KeeperErrorCode = > >> NoNode for /bsp/job_201109220959_0001/224/ready > >> > > > > and it hangs forever. Current version is after you committed ChiaHung's > > patch. > > I'm in pseudo-distributed mode with 3 tasks. > > > > Are you going to bench this without the logging? That would be > interesting > > though ;D > > > > 2011/9/22 Thomas Jungblut <[email protected]> > > > >> That is great. I think we can push this under 200s. > >> I attach a profiler and send you a list of hotspots. > >> > >> lg. > >> > >> 2011/9/22 Edward J. Yoon <[email protected]> > >> > >> By ChiaHung's HAMA-387.patch, hang problem is fixed. > >>> > >>> And also, on same environment (1 rack, 256 cores), a bench example > >>> result is dramatically improved. (184.076 seconds from 307.129 > >>> seconds) > >>> > >>> ---- > >>> # core/bin/hama jar > >>> examples/target/hama-examples-0.4.0-incubating-SNAPSHOT.jar bench 16 > >>> 1000 512 > >>> .. > >>> 11/09/22 10:27:32 INFO bsp.BSPJobClient: Current supersteps number: 504 > >>> 11/09/22 10:27:35 INFO bsp.BSPJobClient: Current supersteps number: 508 > >>> 11/09/22 10:27:38 INFO bsp.BSPJobClient: Current supersteps number: 512 > >>> 11/09/22 10:27:38 INFO bsp.BSPJobClient: The total number of > supersteps: > >>> 512 > >>> Job Finished in 184.076 seconds > >>> > >>> Hama 0.4 (r.1163903) was: > >>> > >>> 16 bytes | 1000 | 512 | 307.129 seconds > >>> > >>> -- > >>> Best Regards, Edward J. Yoon > >>> @eddieyoon > >>> > >> > >> > >> > >> -- > >> Thomas Jungblut > >> Berlin > >> > >> mobile: 0170-3081070 > >> > >> business: [email protected] > >> private: [email protected] > >> > > > > > > > > -- > > Thomas Jungblut > > Berlin > > > > mobile: 0170-3081070 > > > > business: [email protected] > > private: [email protected] > > > > > > -- > Best Regards, Edward J. Yoon > @eddieyoon > -- Thomas Jungblut Berlin mobile: 0170-3081070 business: [email protected] private: [email protected]
