Tested many times debug mode vs. non-debug mode. There is only a small difference: 170 ~ 180 secs vs. 210 ~ 220 secs.
---- 11/09/23 09:15:52 INFO bsp.BSPJobClient: Current supersteps number: 493 11/09/23 09:15:55 INFO bsp.BSPJobClient: Current supersteps number: 504 11/09/23 09:15:58 INFO bsp.BSPJobClient: Current supersteps number: 506 11/09/23 09:16:01 INFO bsp.BSPJobClient: Current supersteps number: 512 11/09/23 09:16:01 INFO bsp.BSPJobClient: The total number of supersteps: 512 Job Finished in 214.089 seconds On Thu, Sep 22, 2011 at 8:58 PM, Thomas Jungblut <[email protected]> wrote: > Scripted a fix version: > > http://pastebin.com/WbWWxd2R > > You can test this as well if you like. > > 2011/9/22 Thomas Jungblut <[email protected]> > >> And I think we should change this benchmark from random to a stable >> implementation. >> So we should communicate with all of the other peers, not just a random >> peername. >> Then we can precompute (or cache) the tag message and we save the string >> concat operations. We can put them into a directly allocated ByteBuffer as >> well, this will save the serialization. >> But then we cannot compare the results with the versions before. >> >> Additional to the log level change, we can receive a superior performance >> improvement ;) >> >> >> 2011/9/22 Edward J. Yoon <[email protected]> >> >>> Haha, obviously it can't be ignored. >>> >>> Unfortunately, I can't access my test machines now. I'll check tomorrow. >>> >>> On Thu, Sep 22, 2011 at 5:05 PM, Thomas Jungblut >>> <[email protected]> wrote: >>> > You're going to laugh, but we spend 80% of the time, logging the >>> messages. >>> > Let's change the log level to debug or remove the logging in the bench >>> > example. >>> > >>> > Sadly I still receive >>> > >>> > org.apache.zookeeper.KeeperException$NoNodeException: KeeperErrorCode = >>> >> NoNode for /bsp/job_201109220959_0001/224/ready >>> >> >>> > >>> > and it hangs forever. Current version is after you committed ChiaHung's >>> > patch. >>> > I'm in pseudo-distributed mode with 3 tasks. >>> > >>> > Are you going to bench this without the logging? That would be >>> interesting >>> > though ;D >>> > >>> > 2011/9/22 Thomas Jungblut <[email protected]> >>> > >>> >> That is great. I think we can push this under 200s. >>> >> I attach a profiler and send you a list of hotspots. >>> >> >>> >> lg. >>> >> >>> >> 2011/9/22 Edward J. Yoon <[email protected]> >>> >> >>> >> By ChiaHung's HAMA-387.patch, hang problem is fixed. >>> >>> >>> >>> And also, on same environment (1 rack, 256 cores), a bench example >>> >>> result is dramatically improved. (184.076 seconds from 307.129 >>> >>> seconds) >>> >>> >>> >>> ---- >>> >>> # core/bin/hama jar >>> >>> examples/target/hama-examples-0.4.0-incubating-SNAPSHOT.jar bench 16 >>> >>> 1000 512 >>> >>> .. >>> >>> 11/09/22 10:27:32 INFO bsp.BSPJobClient: Current supersteps number: >>> 504 >>> >>> 11/09/22 10:27:35 INFO bsp.BSPJobClient: Current supersteps number: >>> 508 >>> >>> 11/09/22 10:27:38 INFO bsp.BSPJobClient: Current supersteps number: >>> 512 >>> >>> 11/09/22 10:27:38 INFO bsp.BSPJobClient: The total number of >>> supersteps: >>> >>> 512 >>> >>> Job Finished in 184.076 seconds >>> >>> >>> >>> Hama 0.4 (r.1163903) was: >>> >>> >>> >>> 16 bytes | 1000 | 512 | 307.129 seconds >>> >>> >>> >>> -- >>> >>> Best Regards, Edward J. Yoon >>> >>> @eddieyoon >>> >>> >>> >> >>> >> >>> >> >>> >> -- >>> >> Thomas Jungblut >>> >> Berlin >>> >> >>> >> mobile: 0170-3081070 >>> >> >>> >> business: [email protected] >>> >> private: [email protected] >>> >> >>> > >>> > >>> > >>> > -- >>> > Thomas Jungblut >>> > Berlin >>> > >>> > mobile: 0170-3081070 >>> > >>> > business: [email protected] >>> > private: [email protected] >>> > >>> >>> >>> >>> -- >>> Best Regards, Edward J. Yoon >>> @eddieyoon >>> >> >> >> >> -- >> Thomas Jungblut >> Berlin >> >> mobile: 0170-3081070 >> >> business: [email protected] >> private: [email protected] >> > > > > -- > Thomas Jungblut > Berlin > > mobile: 0170-3081070 > > business: [email protected] > private: [email protected] > -- Best Regards, Edward J. Yoon @eddieyoon
