Scripted a fix version: http://pastebin.com/WbWWxd2R
You can test this as well if you like. 2011/9/22 Thomas Jungblut <[email protected]> > And I think we should change this benchmark from random to a stable > implementation. > So we should communicate with all of the other peers, not just a random > peername. > Then we can precompute (or cache) the tag message and we save the string > concat operations. We can put them into a directly allocated ByteBuffer as > well, this will save the serialization. > But then we cannot compare the results with the versions before. > > Additional to the log level change, we can receive a superior performance > improvement ;) > > > 2011/9/22 Edward J. Yoon <[email protected]> > >> Haha, obviously it can't be ignored. >> >> Unfortunately, I can't access my test machines now. I'll check tomorrow. >> >> On Thu, Sep 22, 2011 at 5:05 PM, Thomas Jungblut >> <[email protected]> wrote: >> > You're going to laugh, but we spend 80% of the time, logging the >> messages. >> > Let's change the log level to debug or remove the logging in the bench >> > example. >> > >> > Sadly I still receive >> > >> > org.apache.zookeeper.KeeperException$NoNodeException: KeeperErrorCode = >> >> NoNode for /bsp/job_201109220959_0001/224/ready >> >> >> > >> > and it hangs forever. Current version is after you committed ChiaHung's >> > patch. >> > I'm in pseudo-distributed mode with 3 tasks. >> > >> > Are you going to bench this without the logging? That would be >> interesting >> > though ;D >> > >> > 2011/9/22 Thomas Jungblut <[email protected]> >> > >> >> That is great. I think we can push this under 200s. >> >> I attach a profiler and send you a list of hotspots. >> >> >> >> lg. >> >> >> >> 2011/9/22 Edward J. Yoon <[email protected]> >> >> >> >> By ChiaHung's HAMA-387.patch, hang problem is fixed. >> >>> >> >>> And also, on same environment (1 rack, 256 cores), a bench example >> >>> result is dramatically improved. (184.076 seconds from 307.129 >> >>> seconds) >> >>> >> >>> ---- >> >>> # core/bin/hama jar >> >>> examples/target/hama-examples-0.4.0-incubating-SNAPSHOT.jar bench 16 >> >>> 1000 512 >> >>> .. >> >>> 11/09/22 10:27:32 INFO bsp.BSPJobClient: Current supersteps number: >> 504 >> >>> 11/09/22 10:27:35 INFO bsp.BSPJobClient: Current supersteps number: >> 508 >> >>> 11/09/22 10:27:38 INFO bsp.BSPJobClient: Current supersteps number: >> 512 >> >>> 11/09/22 10:27:38 INFO bsp.BSPJobClient: The total number of >> supersteps: >> >>> 512 >> >>> Job Finished in 184.076 seconds >> >>> >> >>> Hama 0.4 (r.1163903) was: >> >>> >> >>> 16 bytes | 1000 | 512 | 307.129 seconds >> >>> >> >>> -- >> >>> Best Regards, Edward J. Yoon >> >>> @eddieyoon >> >>> >> >> >> >> >> >> >> >> -- >> >> Thomas Jungblut >> >> Berlin >> >> >> >> mobile: 0170-3081070 >> >> >> >> business: [email protected] >> >> private: [email protected] >> >> >> > >> > >> > >> > -- >> > Thomas Jungblut >> > Berlin >> > >> > mobile: 0170-3081070 >> > >> > business: [email protected] >> > private: [email protected] >> > >> >> >> >> -- >> Best Regards, Edward J. Yoon >> @eddieyoon >> > > > > -- > Thomas Jungblut > Berlin > > mobile: 0170-3081070 > > business: [email protected] > private: [email protected] > -- Thomas Jungblut Berlin mobile: 0170-3081070 business: [email protected] private: [email protected]
