I am going to write yourkit this evening if we can get an open source license. I'm going to publish the key on the private list once I get it, so everyone can use it if he needs it.
2011/11/4 Edward J. Yoon <[email protected]> > > Although removing Thread.sleep() solves the current issue, is there any > chance that we can (a bit) formally profile the system? I agree Thomas's > point and I think profiling would give us a bit more ideas about the > current state of system. > Agree. > > Please open new ticket. > > On Fri, Nov 4, 2011 at 7:19 PM, ChiaHung Lin <[email protected]> wrote: > > > > Although removing Thread.sleep() solves the current issue, is there any > chance that we can (a bit) formally profile the system? I agree Thomas's > point and I think profiling would give us a bit more ideas about the > current state of system. > > > > -----Original message----- > > From:Edward J. Yoon <[email protected]> > > To:[email protected],[email protected] > > Date:Fri, 4 Nov 2011 18:43:17 +0900 > > Subject:Re: Experiences with large hama cluster. > > > > After deleting Thread.sleep(), I'm very very satisfied. Haha! Now my > > Hama is running on 200 nodes using 2 thousand tasks. It's comedy. We > > have to clean up the codes. > > > >> We should remove that legacy code e.g. using another thread for > reporting task. > > > > I'm +1. Please open new ticket. > > > > On Fri, Nov 4, 2011 at 6:33 PM, ChiaHung Lin <[email protected]> wrote: > >> We should remove that legacy code e.g. using another thread for > reporting task. > >> > >> This just reminds me of another issue. When doing checkpoint, the patch > provided also uses the main thread for checkpointing. This would slow down > the task completion as the messages size grows. We should perhaps move it > to another thread for saving data in queue as well. > >> > >> -----Original message----- > >> From:Edward J. Yoon <[email protected]> > >> To:[email protected],[email protected] > >> Date:Fri, 4 Nov 2011 18:01:00 +0900 > >> Subject:Re: Experiences with large hama cluster. > >> > >> I found, "Thread.sleep(REPORT_INTERVAL)" in a loop :/ > >> > >> ---- > >> for (Map.Entry<TaskAttemptID, TaskInProgress> e : runningTasks > >> .entrySet()) { Thread.sleep(REPORT_INTERVAL); > >> TaskInProgress tip = e.getValue(); TaskStatus taskStatus > >> = tip.getStatus(); > >> if (taskStatus.getRunState() == TaskStatus.State.RUNNING) { > >> taskStatus.setProgress(taskStatus.getSuperstepCount()); > >> if (!tip.runner.isAlive()) { if > >> (taskStatus.getRunState() != TaskStatus.State.FAILED) { > >> taskStatus.setRunState(TaskStatus.State.SUCCEEDED); > >> LOG.info("Task '" + taskStatus.getTaskId().toString() > >> + "' has completed."); } > >> taskStatus.setPhase(TaskStatus.Phase.CLEANUP); } } > >> taskStatuses.add(taskStatus); } > >> doReport(taskStatuses); Thread.sleep(REPORT_INTERVAL); > >> 2011/11/4 ChiaHung Lin <[email protected]>: > >>> The current way how groom reports status back to master is done by > periodically rpc using groom server's main thread. Probably this action can > be executed by another thread. > >>> > >>> If I remember correctly, client would only receive the final superstep > report if child process reports status in done() method. > >>> > >>> But I think profiling would better tell us what parts is the > bottleneck. > >>> > >>> -----Original message----- > >>> From:Tommaso Teofili <[email protected]> > >>> To:[email protected] > >>> Date:Thu, 3 Nov 2011 12:18:41 +0100 > >>> Subject:Re: Experiences with large hama cluster. > >>> > >>> 2011/11/3 Thomas Jungblut <[email protected]> > >>> > >>>> It is difficult to say what the real bottleneck is. > >>>> > >>>> >What If each child processes reports directly using done() method? > >>>> > >>>> You want to report directly from a task to the master? > >>>> We need the status update to detect failure and stuff. We can make it > less > >>>> frequent, e.G. every n-th superstep. > >>>> But I don't think that this is going to make the whole computation > soo much > >>>> faster. > >>>> > >>>> We simply need to profile it... Yourkit has these OpenSource licenses, > >>>> maybe we write them an email. > >>>> There are several other Apache projects using it. Flume for example: > >>>> https://cwiki.apache.org/FLUME/ (scroll down to the bottom). > >>>> > >>>> It could only be helpful. > >>>> > >>>> > >>> +1 > >>> Tommaso > >>> > >>> > >>>> 2011/11/3 Edward J. Yoon <[email protected]> > >>>> > >>>> > > We have the advantage that we don't have to constantly launch new > task > >>>> > for> each input.> Once we have our n-task launched, they run for the > >>>> whole > >>>> > computation. > >>>> > > >>>> > You're right. > >>>> > > >>>> > > How should we fix/improve this? > >>>> > > >>>> > GroomServer checks task statuses and reports to master periodically. > >>>> > > >>>> > What If each child processes reports directly using done() method? > >>>> > On Thu, Nov 3, 2011 at 7:57 PM, Thomas Jungblut > >>>> > <[email protected]> wrote: > >>>> > > We have the advantage that we don't have to constantly launch new > task > >>>> > for > >>>> > > each input. > >>>> > > Once we have our n-task launched, they run for the whole > computation. > >>>> > > > >>>> > > How should we fix/improve this? > >>>> > > > >>>> > > 2011/11/3 Edward J. Yoon <[email protected]> > >>>> > > > >>>> > >> Launching Map tasks is also slow as you know. And, in our case, > + ZK > >>>> > >> client instance creation time. > >>>> > >> > >>>> > >> On Thu, Nov 3, 2011 at 7:42 PM, Thomas Jungblut > >>>> > >> <[email protected]> wrote: > >>>> > >> > Are you sure? I think we inherited the costly task setup from > Hadoop > >>>> > now > >>>> > >> by > >>>> > >> > copying everything. > >>>> > >> > > >>>> > >> > 2011/11/3 Edward J. Yoon <[email protected]> > >>>> > >> > > >>>> > >> >> Not yet. but I think one of reasons is the creation time of a > zk > >>>> > >> >> client instance. > >>>> > >> >> > >>>> > >> >> On Thu, Nov 3, 2011 at 6:35 PM, Tommaso Teofili > >>>> > >> >> <[email protected]> wrote: > >>>> > >> >> > Thanks for sharing Edward, were you also able to debug why > that > >>>> was > >>>> > >> the > >>>> > >> >> > case? > >>>> > >> >> > Tommaso > >>>> > >> >> > > >>>> > >> >> > 2011/11/3 Edward J. Yoon <[email protected]> > >>>> > >> >> > > >>>> > >> >> >> I tested Hama r.1197018 on 100 nodes cluster using 2 tasks > per > >>>> > node. > >>>> > >> >> >> > >>>> > >> >> >> The all the functions are works well, but task launching > and > >>>> > >> finishing > >>>> > >> >> >> are *very slow*. > >>>> > >> >> >> > >>>> > >> >> >> [root@hnode-3-7 hama-trunk]# core/bin/start-bspd.sh > >>>> > >> >> >> hnode-3-7: starting zookeeper, logging to > >>>> > >> >> >> > >>>> > >> >> >> > >>>> > >> >> > >>>> > >> > >>>> > > >>>> > /usr/local/src/hama-trunk/core/bin/../logs/hama-root-zookeeper-hnode-3-7.out > >>>> > >> >> >> starting bspmaster, logging to > >>>> > >> >> >> > >>>> > >> >> >> > >>>> > >> >> > >>>> > >> > >>>> > > >>>> > /usr/local/src/hama-trunk/core/bin/../logs/hama-root-bspmaster-hnode-3-7.out > >>>> > >> >> >> 2011-11-03 16:29:14.382::INFO: Logging to STDERR via > >>>> > >> >> >> org.mortbay.log.StdErrLog > >>>> > >> >> >> 2011-11-03 16:29:14.430::INFO: jetty-6.1.14 > >>>> > >> >> >> 2011-11-03 16:29:14.684::INFO: Started > >>>> > >> SelectChannelConnector@hnode-3-7 > >>>> > >> >> >> :40013 > >>>> > >> >> >> hnode-3-7: starting groom, logging to > >>>> > >> >> >> > >>>> > >> > >>>> > /usr/local/src/hama-trunk/core/bin/../logs/hama-root-groom-hnode-3-7.out > >>>> > >> >> >> hnode-3-8: starting groom, logging to > >>>> > >> >> >> > >>>> > >> > >>>> > /usr/local/src/hama-trunk/core/bin/../logs/hama-root-groom-hnode-3-8.out > >>>> > >> >> >> hnode-3-10: starting groom, logging to > >>>> > >> >> >> > >>>> > >> >> > >>>> > >> > >>>> > > /usr/local/src/hama-trunk/core/bin/../logs/hama-root-groom-hnode-3-10.out > >>>> > >> >> >> hnode-3-9: starting groom, logging to > >>>> > >> >> >> > >>>> > >> > >>>> > /usr/local/src/hama-trunk/core/bin/../logs/hama-root-groom-hnode-3-9.out > >>>> > >> >> >> hnode-3-12: starting groom, logging to > >>>> > >> >> >> > >>>> > >> >> > >>>> > >> > >>>> > > /usr/local/src/hama-trunk/core/bin/../logs/hama-root-groom-hnode-3-12.out > >>>> > >> >> >> hnode-3-13: starting groom, logging to > >>>> > >> >> >> > >>>> > >> >> > >>>> > >> > >>>> > > /usr/local/src/hama-trunk/core/bin/../logs/hama-root-groom-hnode-3-13.out > >>>> > >> >> >> hnode-3-11: starting groom, logging to > >>>> > >> >> >> > >>>> > >> >> > >>>> > >> > >>>> > > /usr/local/src/hama-trunk/core/bin/../logs/hama-root-groom-hnode-3-11.out > >>>> > >> >> >> hnode-3-15: starting groom, logging to > >>>> > >> >> >> > >>>> > >> >> > >>>> > >> > >>>> > > /usr/local/src/hama-trunk/core/bin/../logs/hama-root-groom-hnode-3-15.out > >>>> > >> >> >> hnode-3-16: starting groom, logging to > >>>> > >> >> >> > >>>> > >> >> > >>>> > >> > >>>> > > /usr/local/src/hama-trunk/core/bin/../logs/hama-root-groom-hnode-3-16.out > >>>> > >> >> >> hnode-3-18: starting groom, logging to > >>>> > >> >> >> > >>>> > >> >> > >>>> > >> > >>>> > > /usr/local/src/hama-trunk/core/bin/../logs/hama-root-groom-hnode-3-18.out > >>>> > >> >> >> hnode-3-14: starting groom, logging to > >>>> > >> >> >> > >>>> > >> >> > >>>> > >> > >>>> > > /usr/local/src/hama-trunk/core/bin/../logs/hama-root-groom-hnode-3-14.out > >>>> > >> >> >> hnode-3-17: starting groom, logging to > >>>> > >> >> >> > >>>> > >> >> > >>>> > >> > >>>> > > /usr/local/src/hama-trunk/core/bin/../logs/hama-root-groom-hnode-3-17.out > >>>> > >> >> >> hnode-3-20: starting groom, logging to > >>>> > >> >> >> > >>>> > >> >> > >>>> > >> > >>>> > > /usr/local/src/hama-trunk/core/bin/../logs/hama-root-groom-hnode-3-20.out > >>>> > >> >> >> hnode-3-21: starting groom, logging to > >>>> > >> >> >> > >>>> > >> >> > >>>> > >> > >>>> > > /usr/local/src/hama-trunk/core/bin/../logs/hama-root-groom-hnode-3-21.out > >>>> > >> >> >> hnode-3-19: starting groom, logging to > >>>> > >> >> >> > >>>> > >> >> > >>>> > >> > >>>> > > /usr/local/src/hama-trunk/core/bin/../logs/hama-root-groom-hnode-3-19.out > >>>> > >> >> >> hnode-3-22: starting groom, logging to > >>>> > >> >> >> > >>>> > >> >> > >>>> > >> > >>>> > > /usr/local/src/hama-trunk/core/bin/../logs/hama-root-groom-hnode-3-22.out > >>>> > >> >> >> hnode-3-23: starting groom, logging to > >>>> > >> >> >> > >>>> > >> >> > >>>> > >> > >>>> > > /usr/local/src/hama-trunk/core/bin/../logs/hama-root-groom-hnode-3-23.out > >>>> > >> >> >> hnode-3-24: starting groom, logging to > >>>> > >> >> >> > >>>> > >> >> > >>>> > >> > >>>> > > /usr/local/src/hama-trunk/core/bin/../logs/hama-root-groom-hnode-3-24.out > >>>> > >> >> >> hnode-3-28: starting groom, logging to > >>>> > >> >> >> > >>>> > >> >> > >>>> > >> > >>>> > > /usr/local/src/hama-trunk/core/bin/../logs/hama-root-groom-hnode-3-28.out > >>>> > >> >> >> hnode-3-25: starting groom, logging to > >>>> > >> >> >> > >>>> > >> >> > >>>> > >> > >>>> > > /usr/local/src/hama-trunk/core/bin/../logs/hama-root-groom-hnode-3-25.out > >>>> > >> >> >> hnode-3-27: starting groom, logging to > >>>> > >> >> >> > >>>> > >> >> > >>>> > >> > >>>> > > /usr/local/src/hama-trunk/core/bin/../logs/hama-root-groom-hnode-3-27.out > >>>> > >> >> >> hnode-3-26: starting groom, logging to > >>>> > >> >> >> > >>>> > >> >> > >>>> > >> > >>>> > > /usr/local/src/hama-trunk/core/bin/../logs/hama-root-groom-hnode-3-26.out > >>>> > >> >> >> hnode-3-30: starting groom, logging to > >>>> > >> >> >> > >>>> > >> >> > >>>> > >> > >>>> > > /usr/local/src/hama-trunk/core/bin/../logs/hama-root-groom-hnode-3-30.out > >>>> > >> >> >> hnode-3-29: starting groom, logging to > >>>> > >> >> >> > >>>> > >> >> > >>>> > >> > >>>> > > /usr/local/src/hama-trunk/core/bin/../logs/hama-root-groom-hnode-3-29.out > >>>> > >> >> >> hnode-3-31: starting groom, logging to > >>>> > >> >> >> > >>>> > >> >> > >>>> > >> > >>>> > > /usr/local/src/hama-trunk/core/bin/../logs/hama-root-groom-hnode-3-31.out > >>>> > >> >> >> hnode-3-32: starting groom, logging to > >>>> > >> >> >> > >>>> > >> >> > >>>> > >> > >>>> > > /usr/local/src/hama-trunk/core/bin/../logs/hama-root-groom-hnode-3-32.out > >>>> > >> >> >> hnode-3-33: starting groom, logging to > >>>> > >> >> >> > >>>> > >> >> > >>>> > >> > >>>> > > /usr/local/src/hama-trunk/core/bin/../logs/hama-root-groom-hnode-3-33.out > >>>> > >> >> >> hnode-3-35: starting groom, logging to > >>>> > >> >> >> > >>>> > >> >> > >>>> > >> > >>>> > > /usr/local/src/hama-trunk/core/bin/../logs/hama-root-groom-hnode-3-35.out > >>>> > >> >> >> hnode-3-34: starting groom, logging to > >>>> > >> >> >> > >>>> > >> >> > >>>> > >> > >>>> > > /usr/local/src/hama-trunk/core/bin/../logs/hama-root-groom-hnode-3-34.out > >>>> > >> >> >> hnode-3-37: starting groom, logging to > >>>> > >> >> >> > >>>> > >> >> > >>>> > >> > >>>> > > /usr/local/src/hama-trunk/core/bin/../logs/hama-root-groom-hnode-3-37.out > >>>> > >> >> >> hnode-3-38: starting groom, logging to > >>>> > >> >> >> > >>>> > >> >> > >>>> > >> > >>>> > > /usr/local/src/hama-trunk/core/bin/../logs/hama-root-groom-hnode-3-38.out > >>>> > >> >> >> hnode-3-36: starting groom, logging to > >>>> > >> >> >> > >>>> > >> >> > >>>> > >> > >>>> > > /usr/local/src/hama-trunk/core/bin/../logs/hama-root-groom-hnode-3-36.out > >>>> > >> >> >> hnode-3-39: starting groom, logging to > >>>> > >> >> >> > >>>> > >> >> > >>>> > >> > >>>> > > /usr/local/src/hama-trunk/core/bin/../logs/hama-root-groom-hnode-3-39.out > >>>> > >> >> >> hnode-3-40: starting groom, logging to > >>>> > >> >> >> > >>>> > >> >> > >>>> > >> > >>>> > > /usr/local/src/hama-trunk/core/bin/../logs/hama-root-groom-hnode-3-40.out > >>>> > >> >> >> hnode-3-42: starting groom, logging to > >>>> > >> >> >> > >>>> > >> >> > >>>> > >> > >>>> > > /usr/local/src/hama-trunk/core/bin/../logs/hama-root-groom-hnode-3-42.out > >>>> > >> >> >> hnode-3-43: starting groom, logging to > >>>> > >> >> >> > >>>> > >> >> > >>>> > >> > >>>> > > /usr/local/src/hama-trunk/core/bin/../logs/hama-root-groom-hnode-3-43.out > >>>> > >> >> >> hnode-3-45: starting groom, logging to > >>>> > >> >> >> > >>>> > >> >> > >>>> > >> > >>>> > > /usr/local/src/hama-trunk/core/bin/../logs/hama-root-groom-hnode-3-45.out > >>>> > >> >> >> hnode-3-41: starting groom, logging to > >>>> > >> >> >> > >>>> > >> >> > >>>> > >> > >>>> > > /usr/local/src/hama-trunk/core/bin/../logs/hama-root-groom-hnode-3-41.out > >>>> > >> >> >> hnode-3-44: starting groom, logging to > >>>> > >> >> >> > >>>> > >> >> > >>>> > >> > >>>> > > /usr/local/src/hama-trunk/core/bin/../logs/hama-root-groom-hnode-3-44.out > >>>> > >> >> >> hnode-3-46: starting groom, logging to > >>>> > >> >> >> > >>>> > >> >> > >>>> > >> > >>>> > > /usr/local/src/hama-trunk/core/bin/../logs/hama-root-groom-hnode-3-46.out > >>>> > >> >> >> hnode-3-47: starting groom, logging to > >>>> > >> >> >> > >>>> > >> >> > >>>> > >> > >>>> > > /usr/local/src/hama-trunk/core/bin/../logs/hama-root-groom-hnode-3-47.out > >>>> > >> >> >> hnode-3-49: starting groom, logging to > >>>> > >> >> >> > >>>> > >> >> > >>>> > >> > >>>> > > /usr/local/src/hama-trunk/core/bin/../logs/hama-root-groom-hnode-3-49.out > >>>> > >> >> >> hnode-3-50: starting groom, logging to > >>>> > >> >> >> > >>>> > >> >> > >>>> > >> > >>>> > > /usr/local/src/hama-trunk/core/bin/../logs/hama-root-groom-hnode-3-50.out > >>>> > >> >> >> hnode-3-51: starting groom, logging to > >>>> > >> >> >> > >>>> > >> >> > >>>> > >> > >>>> > > /usr/local/src/hama-trunk/core/bin/../logs/hama-root-groom-hnode-3-51.out > >>>> > >> >> >> hnode-3-48: starting groom, logging to > >>>> > >> >> >> > >>>> > >> >> > >>>> > >> > >>>> > > /usr/local/src/hama-trunk/core/bin/../logs/hama-root-groom-hnode-3-48.out > >>>> > >> >> >> hnode-3-55: starting groom, logging to > >>>> > >> >> >> > >>>> > >> >> > >>>> > >> > >>>> > > /usr/local/src/hama-trunk/core/bin/../logs/hama-root-groom-hnode-3-55.out > >>>> > >> >> >> hnode-3-53: starting groom, logging to > >>>> > >> >> >> > >>>> > >> >> > >>>> > >> > >>>> > > /usr/local/src/hama-trunk/core/bin/../logs/hama-root-groom-hnode-3-53.out > >>>> > >> >> >> hnode-3-52: starting groom, logging to > >>>> > >> >> >> > >>>> > >> >> > >>>> > >> > >>>> > > /usr/local/src/hama-trunk/core/bin/../logs/hama-root-groom-hnode-3-52.out > >>>> > >> >> >> hnode-3-56: starting groom, logging to > >>>> > >> >> >> > >>>> > >> >> > >>>> > >> > >>>> > > /usr/local/src/hama-trunk/core/bin/../logs/hama-root-groom-hnode-3-56.out > >>>> > >> >> >> hnode-3-54: starting groom, logging to > >>>> > >> >> >> > >>>> > >> >> > >>>> > >> > >>>> > > /usr/local/src/hama-trunk/core/bin/../logs/hama-root-groom-hnode-3-54.out > >>>> > >> >> >> hnode-3-60: starting groom, logging to > >>>> > >> >> >> > >>>> > >> >> > >>>> > >> > >>>> > > /usr/local/src/hama-trunk/core/bin/../logs/hama-root-groom-hnode-3-60.out > >>>> > >> >> >> hnode-3-59: starting groom, logging to > >>>> > >> >> >> > >>>> > >> >> > >>>> > >> > >>>> > > /usr/local/src/hama-trunk/core/bin/../logs/hama-root-groom-hnode-3-59.out > >>>> > >> >> >> hnode-3-58: starting groom, logging to > >>>> > >> >> >> > >>>> > >> >> > >>>> > >> > >>>> > > /usr/local/src/hama-trunk/core/bin/../logs/hama-root-groom-hnode-3-58.out > >>>> > >> >> >> hnode-3-57: starting groom, logging to > >>>> > >> >> >> > >>>> > >> >> > >>>> > >> > >>>> > > /usr/local/src/hama-trunk/core/bin/../logs/hama-root-groom-hnode-3-57.out > >>>> > >> >> >> hnode-3-62: starting groom, logging to > >>>> > >> >> >> > >>>> > >> >> > >>>> > >> > >>>> > > /usr/local/src/hama-trunk/core/bin/../logs/hama-root-groom-hnode-3-62.out > >>>> > >> >> >> hnode-3-63: starting groom, logging to > >>>> > >> >> >> > >>>> > >> >> > >>>> > >> > >>>> > > /usr/local/src/hama-trunk/core/bin/../logs/hama-root-groom-hnode-3-63.out > >>>> > >> >> >> hnode-3-64: starting groom, logging to > >>>> > >> >> >> > >>>> > >> >> > >>>> > >> > >>>> > > /usr/local/src/hama-trunk/core/bin/../logs/hama-root-groom-hnode-3-64.out > >>>> > >> >> >> hnode-3-65: starting groom, logging to > >>>> > >> >> >> > >>>> > >> >> > >>>> > >> > >>>> > > /usr/local/src/hama-trunk/core/bin/../logs/hama-root-groom-hnode-3-65.out > >>>> > >> >> >> hnode-3-67: starting groom, logging to > >>>> > >> >> >> > >>>> > >> >> > >>>> > >> > >>>> > > /usr/local/src/hama-trunk/core/bin/../logs/hama-root-groom-hnode-3-67.out > >>>> > >> >> >> hnode-3-70: starting groom, logging to > >>>> > >> >> >> > >>>> > >> >> > >>>> > >> > >>>> > > /usr/local/src/hama-trunk/core/bin/../logs/hama-root-groom-hnode-3-70.out > >>>> > >> >> >> hnode-3-61: starting groom, logging to > >>>> > >> >> >> > >>>> > >> >> > >>>> > >> > >>>> > > /usr/local/src/hama-trunk/core/bin/../logs/hama-root-groom-hnode-3-61.out > >>>> > >> >> >> hnode-3-66: starting groom, logging to > >>>> > >> >> >> > >>>> > >> >> > >>>> > >> > >>>> > > /usr/local/src/hama-trunk/core/bin/../logs/hama-root-groom-hnode-3-66.out > >>>> > >> >> >> hnode-3-68: starting groom, logging to > >>>> > >> >> >> > >>>> > >> >> > >>>> > >> > >>>> > > /usr/local/src/hama-trunk/core/bin/../logs/hama-root-groom-hnode-3-68.out > >>>> > >> >> >> hnode-3-72: starting groom, logging to > >>>> > >> >> >> > >>>> > >> >> > >>>> > >> > >>>> > > /usr/local/src/hama-trunk/core/bin/../logs/hama-root-groom-hnode-3-72.out > >>>> > >> >> >> hnode-3-69: starting groom, logging to > >>>> > >> >> >> > >>>> > >> >> > >>>> > >> > >>>> > > /usr/local/src/hama-trunk/core/bin/../logs/hama-root-groom-hnode-3-69.out > >>>> > >> >> >> hnode-3-71: starting groom, logging to > >>>> > >> >> >> > >>>> > >> >> > >>>> > >> > >>>> > > /usr/local/src/hama-trunk/core/bin/../logs/hama-root-groom-hnode-3-71.out > >>>> > >> >> >> hnode-3-75: starting groom, logging to > >>>> > >> >> >> > >>>> > >> >> > >>>> > >> > >>>> > > /usr/local/src/hama-trunk/core/bin/../logs/hama-root-groom-hnode-3-75.out > >>>> > >> >> >> hnode-3-74: starting groom, logging to > >>>> > >> >> >> > >>>> > >> >> > >>>> > >> > >>>> > > /usr/local/src/hama-trunk/core/bin/../logs/hama-root-groom-hnode-3-74.out > >>>> > >> >> >> hnode-3-73: starting groom, logging to > >>>> > >> >> >> > >>>> > >> >> > >>>> > >> > >>>> > > /usr/local/src/hama-trunk/core/bin/../logs/hama-root-groom-hnode-3-73.out > >>>> > >> >> >> hnode-3-77: starting groom, logging to > >>>> > >> >> >> > >>>> > >> >> > >>>> > >> > >>>> > > /usr/local/src/hama-trunk/core/bin/../logs/hama-root-groom-hnode-3-77.out > >>>> > >> >> >> hnode-3-76: starting groom, logging to > >>>> > >> >> >> > >>>> > >> >> > >>>> > >> > >>>> > > /usr/local/src/hama-trunk/core/bin/../logs/hama-root-groom-hnode-3-76.out > >>>> > >> >> >> hnode-3-79: starting groom, logging to > >>>> > >> >> >> > >>>> > >> >> > >>>> > >> > >>>> > > /usr/local/src/hama-trunk/core/bin/../logs/hama-root-groom-hnode-3-79.out > >>>> > >> >> >> hnode-3-78: starting groom, logging to > >>>> > >> >> >> > >>>> > >> >> > >>>> > >> > >>>> > > /usr/local/src/hama-trunk/core/bin/../logs/hama-root-groom-hnode-3-78.out > >>>> > >> >> >> hnode-3-81: starting groom, logging to > >>>> > >> >> >> > >>>> > >> >> > >>>> > >> > >>>> > > /usr/local/src/hama-trunk/core/bin/../logs/hama-root-groom-hnode-3-81.out > >>>> > >> >> >> hnode-3-80: starting groom, logging to > >>>> > >> >> >> > >>>> > >> >> > >>>> > >> > >>>> > > /usr/local/src/hama-trunk/core/bin/../logs/hama-root-groom-hnode-3-80.out > >>>> > >> >> >> hnode-3-82: starting groom, logging to > >>>> > >> >> >> > >>>> > >> >> > >>>> > >> > >>>> > > /usr/local/src/hama-trunk/core/bin/../logs/hama-root-groom-hnode-3-82.out > >>>> > >> >> >> hnode-3-83: starting groom, logging to > >>>> > >> >> >> > >>>> > >> >> > >>>> > >> > >>>> > > /usr/local/src/hama-trunk/core/bin/../logs/hama-root-groom-hnode-3-83.out > >>>> > >> >> >> hnode-3-85: starting groom, logging to > >>>> > >> >> >> > >>>> > >> >> > >>>> > >> > >>>> > > /usr/local/src/hama-trunk/core/bin/../logs/hama-root-groom-hnode-3-85.out > >>>> > >> >> >> hnode-3-86: starting groom, logging to > >>>> > >> >> >> > >>>> > >> >> > >>>> > >> > >>>> > > /usr/local/src/hama-trunk/core/bin/../logs/hama-root-groom-hnode-3-86.out > >>>> > >> >> >> hnode-3-84: starting groom, logging to > >>>> > >> >> >> > >>>> > >> >> > >>>> > >> > >>>> > > /usr/local/src/hama-trunk/core/bin/../logs/hama-root-groom-hnode-3-84.out > >>>> > >> >> >> hnode-3-87: starting groom, logging to > >>>> > >> >> >> > >>>> > >> >> > >>>> > >> > >>>> > > /usr/local/src/hama-trunk/core/bin/../logs/hama-root-groom-hnode-3-87.out > >>>> > >> >> >> hnode-3-88: starting groom, logging to > >>>> > >> >> >> > >>>> > >> >> > >>>> > >> > >>>> > > /usr/local/src/hama-trunk/core/bin/../logs/hama-root-groom-hnode-3-88.out > >>>> > >> >> >> hnode-3-89: starting groom, logging to > >>>> > >> >> >> > >>>> > >> >> > >>>> > >> > >>>> > > /usr/local/src/hama-trunk/core/bin/../logs/hama-root-groom-hnode-3-89.out > >>>> > >> >> >> hnode-3-90: starting groom, logging to > >>>> > >> >> >> > >>>> > >> >> > >>>> > >> > >>>> > > /usr/local/src/hama-trunk/core/bin/../logs/hama-root-groom-hnode-3-90.out > >>>> > >> >> >> hnode-3-91: starting groom, logging to > >>>> > >> >> >> > >>>> > >> >> > >>>> > >> > >>>> > > /usr/local/src/hama-trunk/core/bin/../logs/hama-root-groom-hnode-3-91.out > >>>> > >> >> >> hnode-3-95: starting groom, logging to > >>>> > >> >> >> > >>>> > >> >> > >>>> > >> > >>>> > > /usr/local/src/hama-trunk/core/bin/../logs/hama-root-groom-hnode-3-95.out > >>>> > >> >> >> hnode-3-93: starting groom, logging to > >>>> > >> >> >> > >>>> > >> >> > >>>> > >> > >>>> > > /usr/local/src/hama-trunk/core/bin/../logs/hama-root-groom-hnode-3-93.out > >>>> > >> >> >> hnode-3-94: starting groom, logging to > >>>> > >> >> >> > >>>> > >> >> > >>>> > >> > >>>> > > /usr/local/src/hama-trunk/core/bin/../logs/hama-root-groom-hnode-3-94.out > >>>> > >> >> >> hnode-3-92: starting groom, logging to > >>>> > >> >> >> > >>>> > >> >> > >>>> > >> > >>>> > > /usr/local/src/hama-trunk/core/bin/../logs/hama-root-groom-hnode-3-92.out > >>>> > >> >> >> hnode-3-96: starting groom, logging to > >>>> > >> >> >> > >>>> > >> >> > >>>> > >> > >>>> > > /usr/local/src/hama-trunk/core/bin/../logs/hama-root-groom-hnode-3-96.out > >>>> > >> >> >> hnode-3-97: starting groom, logging to > >>>> > >> >> >> > >>>> > >> >> > >>>> > >> > >>>> > > /usr/local/src/hama-trunk/core/bin/../logs/hama-root-groom-hnode-3-97.out > >>>> > >> >> >> hnode-3-98: starting groom, logging to > >>>> > >> >> >> > >>>> > >> >> > >>>> > >> > >>>> > > /usr/local/src/hama-trunk/core/bin/../logs/hama-root-groom-hnode-3-98.out > >>>> > >> >> >> hnode-3-99: starting groom, logging to > >>>> > >> >> >> > >>>> > >> >> > >>>> > >> > >>>> > > /usr/local/src/hama-trunk/core/bin/../logs/hama-root-groom-hnode-3-99.out > >>>> > >> >> >> hnode-3-102: starting groom, logging to > >>>> > >> >> >> > >>>> > >> >> > >>>> > >> > >>>> > > >>>> > /usr/local/src/hama-trunk/core/bin/../logs/hama-root-groom-hnode-3-102.out > >>>> > >> >> >> hnode-3-100: starting groom, logging to > >>>> > >> >> >> > >>>> > >> >> > >>>> > >> > >>>> > > >>>> > /usr/local/src/hama-trunk/core/bin/../logs/hama-root-groom-hnode-3-100.out > >>>> > >> >> >> hnode-3-104: starting groom, logging to > >>>> > >> >> >> > >>>> > >> >> > >>>> > >> > >>>> > > >>>> > /usr/local/src/hama-trunk/core/bin/../logs/hama-root-groom-hnode-3-104.out > >>>> > >> >> >> hnode-3-101: starting groom, logging to > >>>> > >> >> >> > >>>> > >> >> > >>>> > >> > >>>> > > >>>> > /usr/local/src/hama-trunk/core/bin/../logs/hama-root-groom-hnode-3-101.out > >>>> > >> >> >> hnode-3-105: starting groom, logging to > >>>> > >> >> >> > >>>> > >> >> > >>>> > >> > >>>> > > >>>> > /usr/local/src/hama-trunk/core/bin/../logs/hama-root-groom-hnode-3-105.out > >>>> > >> >> >> hnode-3-106: starting groom, logging to > >>>> > >> >> >> > >>>> > >> >> > >>>> > >> > >>>> > > >>>> > /usr/local/src/hama-trunk/core/bin/../logs/hama-root-groom-hnode-3-106.out > >>>> > >> >> >> hnode-3-103: starting groom, logging to > >>>> > >> >> >> > >>>> > >> >> > >>>> > >> > >>>> > > >>>> > /usr/local/src/hama-trunk/core/bin/../logs/hama-root-groom-hnode-3-103.out > >>>> > >> >> >> [root@hnode-3-7 hama-trunk]# ls > >>>> > >> >> >> CHANGES.txt core DISCLAIMER.txt examples graph lib > >>>> > LICENSE.txt > >>>> > >> >> >> NOTICE.txt pom.xml README.txt src target yarn > >>>> > >> >> >> [root@hnode-3-7 hama-trunk]# core/bin/hama jar > >>>> > >> >> >> > examples/target/hama-examples-0.4.0-incubating-SNAPSHOT.jar pi > >>>> > >> >> >> 11/11/03 16:36:45 INFO bsp.BSPJobClient: Running job: > >>>> > >> >> job_201111031629_0001 > >>>> > >> >> >> 11/11/03 16:36:48 INFO bsp.BSPJobClient: Current supersteps > >>>> > number: 0 > >>>> > >> >> >> 11/11/03 16:38:09 INFO bsp.BSPJobClient: Current supersteps > >>>> > number: 1 > >>>> > >> >> >> 11/11/03 16:38:12 INFO bsp.BSPJobClient: The total number > of > >>>> > >> >> supersteps: 1 > >>>> > >> >> >> Estimated value of PI is 3.1420720000000006 > >>>> > >> >> >> Job Finished in 90.868 seconds > >>>> > >> >> >> [root@hnode-3-7 hama-trunk]# core/bin/hama jar > >>>> > >> >> >> examples/target/hama-examples-0.4.0-incubating-SNAPSHOT.jar > >>>> bench > >>>> > 16 > >>>> > >> >> >> 1000 30 > >>>> > >> >> >> 11/11/03 16:40:05 INFO bsp.BSPJobClient: Running job: > >>>> > >> >> job_201111031629_0002 > >>>> > >> >> >> 11/11/03 16:40:08 INFO bsp.BSPJobClient: Current supersteps > >>>> > number: 0 > >>>> > >> >> >> 11/11/03 16:41:47 INFO bsp.BSPJobClient: Current supersteps > >>>> > number: 1 > >>>> > >> >> >> 11/11/03 16:41:50 INFO bsp.BSPJobClient: Current supersteps > >>>> > number: 3 > >>>> > >> >> >> 11/11/03 16:41:53 INFO bsp.BSPJobClient: Current supersteps > >>>> > number: 5 > >>>> > >> >> >> 11/11/03 16:41:59 INFO bsp.BSPJobClient: Current supersteps > >>>> > number: 7 > >>>> > >> >> >> 11/11/03 16:42:02 INFO bsp.BSPJobClient: Current supersteps > >>>> > number: > >>>> > >> 10 > >>>> > >> >> >> 11/11/03 16:42:05 INFO bsp.BSPJobClient: Current supersteps > >>>> > number: > >>>> > >> 14 > >>>> > >> >> >> 11/11/03 16:42:08 INFO bsp.BSPJobClient: Current supersteps > >>>> > number: > >>>> > >> 19 > >>>> > >> >> >> 11/11/03 16:42:11 INFO bsp.BSPJobClient: Current supersteps > >>>> > number: > >>>> > >> 24 > >>>> > >> >> >> 11/11/03 16:42:14 INFO bsp.BSPJobClient: Current supersteps > >>>> > number: > >>>> > >> 28 > >>>> > >> >> >> 11/11/03 16:42:17 INFO bsp.BSPJobClient: Current supersteps > >>>> > number: > >>>> > >> 30 > >>>> > >> >> >> 11/11/03 16:42:20 INFO bsp.BSPJobClient: The total number > of > >>>> > >> >> supersteps: 30 > >>>> > >> >> >> Job Finished in 136.781 seconds > >>>> > >> >> >> [root@hnode-3-7 hama-trunk]# core/bin/hama jar > >>>> > >> >> >> examples/target/hama-examples-0.4.0-incubating-SNAPSHOT.jar > >>>> bench > >>>> > 16 > >>>> > >> >> >> 10000 30 > >>>> > >> >> >> 11/11/03 16:43:56 INFO bsp.BSPJobClient: Running job: > >>>> > >> >> job_201111031629_0003 > >>>> > >> >> >> 11/11/03 16:43:59 INFO bsp.BSPJobClient: Current supersteps > >>>> > number: 0 > >>>> > >> >> >> 11/11/03 16:45:42 INFO bsp.BSPJobClient: Current supersteps > >>>> > number: 1 > >>>> > >> >> >> 11/11/03 16:45:48 INFO bsp.BSPJobClient: Current supersteps > >>>> > number: 2 > >>>> > >> >> >> 11/11/03 16:45:54 INFO bsp.BSPJobClient: Current supersteps > >>>> > number: 3 > >>>> > >> >> >> 11/11/03 16:45:57 INFO bsp.BSPJobClient: Current supersteps > >>>> > number: 4 > >>>> > >> >> >> 11/11/03 16:46:03 INFO bsp.BSPJobClient: Current supersteps > >>>> > number: 5 > >>>> > >> >> >> 11/11/03 16:46:06 INFO bsp.BSPJobClient: Current supersteps > >>>> > number: 6 > >>>> > >> >> >> 11/11/03 16:46:09 INFO bsp.BSPJobClient: Current supersteps > >>>> > number: 7 > >>>> > >> >> >> 11/11/03 16:46:12 INFO bsp.BSPJobClient: Current supersteps > >>>> > number: 8 > >>>> > >> >> >> 11/11/03 16:46:15 INFO bsp.BSPJobClient: Current supersteps > >>>> > number: 9 > >>>> > >> >> >> 11/11/03 16:46:18 INFO bsp.BSPJobClient: Current supersteps > >>>> > number: > >>>> > >> 10 > >>>> > >> >> >> 11/11/03 16:46:21 INFO bsp.BSPJobClient: Current supersteps > >>>> > number: > >>>> > >> 11 > >>>> > >> >> >> 11/11/03 16:46:24 INFO bsp.BSPJobClient: Current supersteps > >>>> > number: > >>>> > >> 12 > >>>> > >> >> >> 11/11/03 16:46:27 INFO bsp.BSPJobClient: Current supersteps > >>>> > number: > >>>> > >> 13 > >>>> > >> >> >> 11/11/03 16:46:30 INFO bsp.BSPJobClient: Current supersteps > >>>> > number: > >>>> > >> 14 > >>>> > >> >> >> 11/11/03 16:46:33 INFO bsp.BSPJobClient: Current supersteps > >>>> > number: > >>>> > >> 15 > >>>> > >> >> >> 11/11/03 16:46:36 INFO bsp.BSPJobClient: Current supersteps > >>>> > number: > >>>> > >> 16 > >>>> > >> >> >> 11/11/03 16:46:39 INFO bsp.BSPJobClient: Current supersteps > >>>> > number: > >>>> > >> 17 > >>>> > >> >> >> 11/11/03 16:46:42 INFO bsp.BSPJobClient: Current supersteps > >>>> > number: > >>>> > >> 19 > >>>> > >> >> >> 11/11/03 16:46:45 INFO bsp.BSPJobClient: Current supersteps > >>>> > number: > >>>> > >> 20 > >>>> > >> >> >> 11/11/03 16:46:51 INFO bsp.BSPJobClient: Current supersteps > >>>> > number: > >>>> > >> 21 > >>>> > >> >> >> 11/11/03 16:46:54 INFO bsp.BSPJobClient: Current supersteps > >>>> > number: > >>>> > >> 22 > >>>> > >> >> >> 11/11/03 16:46:57 INFO bsp.BSPJobClient: Current supersteps > >>>> > number: > >>>> > >> 23 > >>>> > >> >> >> 11/11/03 16:47:00 INFO bsp.BSPJobClient: Current supersteps > >>>> > number: > >>>> > >> 24 > >>>> > >> >> >> 11/11/03 16:47:03 INFO bsp.BSPJobClient: Current supersteps > >>>> > number: > >>>> > >> 25 > >>>> > >> >> >> 11/11/03 16:47:06 INFO bsp.BSPJobClient: Current supersteps > >>>> > number: > >>>> > >> 26 > >>>> > >> >> >> 11/11/03 16:47:09 INFO bsp.BSPJobClient: Current supersteps > >>>> > number: > >>>> > >> 27 > >>>> > >> >> >> 11/11/03 16:47:12 INFO bsp.BSPJobClient: Current supersteps > >>>> > number: > >>>> > >> 28 > >>>> > >> >> >> 11/11/03 16:47:15 INFO bsp.BSPJobClient: Current supersteps > >>>> > number: > >>>> > >> 29 > >>>> > >> >> >> 11/11/03 16:47:18 INFO bsp.BSPJobClient: Current supersteps > >>>> > number: > >>>> > >> 30 > >>>> > >> >> >> 11/11/03 16:47:24 INFO bsp.BSPJobClient: The total number > of > >>>> > >> >> supersteps: 30 > >>>> > >> >> >> Job Finished in 209.516 seconds > >>>> > >> >> >> [root@hnode-3-7 hama-trunk]# core/bin/hama jar > >>>> > >> >> >> examples/target/hama-examples-0.4.0-incubating-SNAPSHOT.jar > >>>> bench > >>>> > 16 > >>>> > >> >> >> 100000 30 > >>>> > >> >> >> 11/11/03 16:48:59 INFO bsp.BSPJobClient: Running job: > >>>> > >> >> job_201111031629_0004 > >>>> > >> >> >> 11/11/03 16:49:02 INFO bsp.BSPJobClient: Current supersteps > >>>> > number: 0 > >>>> > >> >> >> 11/11/03 16:50:50 INFO bsp.BSPJobClient: Current supersteps > >>>> > number: 1 > >>>> > >> >> >> 11/11/03 16:51:08 INFO bsp.BSPJobClient: Current supersteps > >>>> > number: 2 > >>>> > >> >> >> 11/11/03 16:51:26 INFO bsp.BSPJobClient: Current supersteps > >>>> > number: 3 > >>>> > >> >> >> 11/11/03 16:51:44 INFO bsp.BSPJobClient: Current supersteps > >>>> > number: 4 > >>>> > >> >> >> 11/11/03 16:51:59 INFO bsp.BSPJobClient: Current supersteps > >>>> > number: 5 > >>>> > >> >> >> 11/11/03 16:52:17 INFO bsp.BSPJobClient: Current supersteps > >>>> > number: 6 > >>>> > >> >> >> 11/11/03 16:52:39 INFO bsp.BSPJobClient: Current supersteps > >>>> > number: 7 > >>>> > >> >> >> 11/11/03 16:52:54 INFO bsp.BSPJobClient: Current supersteps > >>>> > number: 8 > >>>> > >> >> >> 11/11/03 16:53:12 INFO bsp.BSPJobClient: Current supersteps > >>>> > number: 9 > >>>> > >> >> >> 11/11/03 16:53:39 INFO bsp.BSPJobClient: Current supersteps > >>>> > number: > >>>> > >> 10 > >>>> > >> >> >> 11/11/03 16:53:57 INFO bsp.BSPJobClient: Current supersteps > >>>> > number: > >>>> > >> 11 > >>>> > >> >> >> 11/11/03 16:54:12 INFO bsp.BSPJobClient: Current supersteps > >>>> > number: > >>>> > >> 12 > >>>> > >> >> >> 11/11/03 16:54:30 INFO bsp.BSPJobClient: Current supersteps > >>>> > number: > >>>> > >> 13 > >>>> > >> >> >> 11/11/03 16:54:48 INFO bsp.BSPJobClient: Current supersteps > >>>> > number: > >>>> > >> 14 > >>>> > >> >> >> 11/11/03 16:55:03 INFO bsp.BSPJobClient: Current supersteps > >>>> > number: > >>>> > >> 15 > >>>> > >> >> >> 11/11/03 16:55:18 INFO bsp.BSPJobClient: Current supersteps > >>>> > number: > >>>> > >> 16 > >>>> > >> >> >> 11/11/03 16:55:36 INFO bsp.BSPJobClient: Current supersteps > >>>> > number: > >>>> > >> 17 > >>>> > >> >> >> 11/11/03 16:55:51 INFO bsp.BSPJobClient: Current supersteps > >>>> > number: > >>>> > >> 18 > >>>> > >> >> >> 11/11/03 16:56:09 INFO bsp.BSPJobClient: Current supersteps > >>>> > number: > >>>> > >> 19 > >>>> > >> >> >> 11/11/03 16:56:24 INFO bsp.BSPJobClient: Current supersteps > >>>> > number: > >>>> > >> 20 > >>>> > >> >> >> 11/11/03 16:56:42 INFO bsp.BSPJobClient: Current supersteps > >>>> > number: > >>>> > >> 21 > >>>> > >> >> >> 11/11/03 16:56:57 INFO bsp.BSPJobClient: Current supersteps > >>>> > number: > >>>> > >> 22 > >>>> > >> >> >> 11/11/03 16:57:15 INFO bsp.BSPJobClient: Current supersteps > >>>> > number: > >>>> > >> 23 > >>>> > >> >> >> 11/11/03 16:57:30 INFO bsp.BSPJobClient: Current supersteps > >>>> > number: > >>>> > >> 24 > >>>> > >> >> >> 11/11/03 16:57:48 INFO bsp.BSPJobClient: Current supersteps > >>>> > number: > >>>> > >> 25 > >>>> > >> >> >> 11/11/03 16:58:09 INFO bsp.BSPJobClient: Current supersteps > >>>> > number: > >>>> > >> 26 > >>>> > >> >> >> 11/11/03 16:58:24 INFO bsp.BSPJobClient: Current supersteps > >>>> > number: > >>>> > >> 27 > >>>> > >> >> >> 11/11/03 16:58:39 INFO bsp.BSPJobClient: Current supersteps > >>>> > number: > >>>> > >> 28 > >>>> > >> >> >> 11/11/03 16:58:57 INFO bsp.BSPJobClient: Current supersteps > >>>> > number: > >>>> > >> 29 > >>>> > >> >> >> 11/11/03 16:59:12 INFO bsp.BSPJobClient: Current supersteps > >>>> > number: > >>>> > >> 30 > >>>> > >> >> >> 11/11/03 16:59:36 INFO bsp.BSPJobClient: The total number > of > >>>> > >> >> supersteps: 30 > >>>> > >> >> >> Job Finished in 639.003 seconds > >>>> > >> >> >> > >>>> > >> >> >> > >>>> > >> >> >> -- > >>>> > >> >> >> Best Regards, Edward J. Yoon > >>>> > >> >> >> @eddieyoon > >>>> > >> >> >> > >>>> > >> >> > > >>>> > >> >> > >>>> > >> >> > >>>> > >> >> > >>>> > >> >> -- > >>>> > >> >> Best Regards, Edward J. Yoon > >>>> > >> >> @eddieyoon > >>>> > >> >> > >>>> > >> > > >>>> > >> > > >>>> > >> > > >>>> > >> > -- > >>>> > >> > Thomas Jungblut > >>>> > >> > Berlin <[email protected]> > >>>> > >> > > >>>> > >> > >>>> > >> > >>>> > >> > >>>> > >> -- > >>>> > >> Best Regards, Edward J. Yoon > >>>> > >> @eddieyoon > >>>> > >> > >>>> > > > >>>> > > > >>>> > > > >>>> > > -- > >>>> > > Thomas Jungblut > >>>> > > Berlin <[email protected]> > >>>> > > > >>>> > > >>>> > > >>>> > > >>>> > -- > >>>> > Best Regards, Edward J. Yoon > >>>> > @eddieyoon > >>>> > > >>>> > >>>> > >>>> > >>>> -- > >>>> Thomas Jungblut > >>>> Berlin <[email protected]> > >>>> > >>> > >>> > >>> -- > >>> ChiaHung Lin > >>> Department of Information Management > >>> National University of Kaohsiung > >>> Taiwan > >>> > >> > >> > >> > >> -- > >> Best Regards, Edward J. Yoon > >> @eddieyoon > >> > >> > >> -- > >> ChiaHung Lin > >> Department of Information Management > >> National University of Kaohsiung > >> Taiwan > >> > > > > > > > > -- > > Best Regards, Edward J. Yoon > > @eddieyoon > > > > > > -- > > ChiaHung Lin > > Department of Information Management > > National University of Kaohsiung > > Taiwan > > > > > > -- > Best Regards, Edward J. Yoon > @eddieyoon > -- Thomas Jungblut Berlin <[email protected]>
