Questions on Hama
Hi. I am a Hama user, who is analyzing source code now. I have some questions. *1. Build Fail* When I tried build with mvn clean install with version 0.6.4, it succeeded clearly. However, when I tried build with mvn --projects core,examples install, it failed on test task. I think there is something I don't understand yet, but It would be better to update http://wiki.apache.org/hama/HowToContribute, to prevent confusion. I got above command from that wiki page. *2. Why BSPJob and GraphJob is separated?* I read BSPJob GraphJob class, and feel its design is a little bit weird. Why they are separated? Is there any design decision I don't know? Thanks in Advance. - Dongjin -- *Dongjin Lee* *So interested in massive-scale machine learning.facebook: www.facebook.com/dongjin.lee.kr http://www.facebook.com/dongjin.lee.krlinkedin: kr.linkedin.com/in/dongjinleekr http://kr.linkedin.com/in/dongjinleekrgithub: http://goog_969573159github.com/dongjinleekr http://github.com/dongjinleekrtwitter: www.twitter.com/dongjinleekr http://www.twitter.com/dongjinleekr*
Re: Questions on Hama
Perhaps you can check ${module-name}/target/surefire-report/ for more
detail about which test cases fail.
Apache Hama is a BSP engine, meaning it's not only capable of
performing graph computation. Instead, it's suitable for general
purpose parallel computing as long as the algorithm can be expressed
as an iterative application. The benefit of separating Graph Job from
BSP Job is that users can perform their tasks without too much
restriction. For example, a user not merely can write a program to
perform graph computation, but also can write general BSP jobs when
it's required.
On 10 August 2014 22:15, Dongjin Lee dongjin.lee...@gmail.com wrote:
Hi. I am a Hama user, who is analyzing source code now. I have some
questions.
*1. Build Fail*
When I tried build with mvn clean install with version 0.6.4, it succeeded
clearly. However, when I tried build with mvn --projects core,examples
install, it failed on test task.
I think there is something I don't understand yet, but It would be better
to update http://wiki.apache.org/hama/HowToContribute, to prevent
confusion. I got above command from that wiki page.
*2. Why BSPJob and GraphJob is separated?*
I read BSPJob GraphJob class, and feel its design is a little bit weird.
Why they are separated? Is there any design decision I don't know?
Thanks in Advance.
- Dongjin
--
*Dongjin Lee*
*So interested in massive-scale machine learning.facebook:
www.facebook.com/dongjin.lee.kr
http://www.facebook.com/dongjin.lee.krlinkedin:
kr.linkedin.com/in/dongjinleekr
http://kr.linkedin.com/in/dongjinleekrgithub:
http://goog_969573159github.com/dongjinleekr
http://github.com/dongjinleekrtwitter: www.twitter.com/dongjinleekr
http://www.twitter.com/dongjinleekr*
[ANN] Welcome to new Hama committers
The Apache Hama PMC is pleased to announce the following additions: * Victor Lee is now a Hama Committer * ByungSeok Min is now a Hama committer. Thanks again to both for their efforts, we hope to see them continue to move Apache Hama forward. Thanks! -- Best Regards, Edward J. Yoon CEO at DataSayer Co., Ltd.
Build failed in Jenkins: Hama-Nightly-for-Hadoop-2.x #329
See https://builds.apache.org/job/Hama-Nightly-for-Hadoop-2.x/329/ -- [...truncated 5070 lines...] 14/08/10 23:45:38 INFO server.ZooKeeperServer: Server environment:user.dir=/tmp 14/08/10 23:45:38 INFO server.ZooKeeperServer: tickTime set to 3000 14/08/10 23:45:38 INFO server.ZooKeeperServer: minSessionTimeout set to -1 14/08/10 23:45:38 INFO server.ZooKeeperServer: maxSessionTimeout set to 120 14/08/10 23:45:38 INFO server.NIOServerCnxnFactory: binding to port 0.0.0.0/0.0.0.0:2 14/08/10 23:45:38 INFO persistence.FileSnap: Reading snapshot /tmp/hama-testzookeeper/version-2/snapshot.42 14/08/10 23:45:38 INFO persistence.FileTxnSnapLog: Snapshotting: 0x47 to /tmp/hama-testzookeeper/version-2/snapshot.47 14/08/10 23:45:48 INFO zookeeper.ZooKeeper: Client environment:zookeeper.version=3.4.5-1392090, built on 09/30/2012 17:52 GMT 14/08/10 23:45:48 INFO zookeeper.ZooKeeper: Client environment:host.name=hemera.apache.org 14/08/10 23:45:48 INFO zookeeper.ZooKeeper: Client environment:java.version=1.7.0_25 14/08/10 23:45:48 INFO zookeeper.ZooKeeper: Client environment:java.vendor=Oracle Corporation 14/08/10 23:45:48 INFO zookeeper.ZooKeeper: Client environment:java.home=/x1/jenkins/tools/java/jdk1.7.0_25-32/jre 14/08/10 23:45:48 INFO zookeeper.ZooKeeper: Client environment:java.class.path=/tmp/surefirebooter6922403531058828251.jar 14/08/10 23:45:48 INFO zookeeper.ZooKeeper: Client environment:java.library.path=/usr/java/packages/lib/i386:/lib:/usr/lib 14/08/10 23:45:48 INFO zookeeper.ZooKeeper: Client environment:java.io.tmpdir=/tmp 14/08/10 23:45:48 INFO zookeeper.ZooKeeper: Client environment:java.compiler=NA 14/08/10 23:45:48 INFO zookeeper.ZooKeeper: Client environment:os.name=Linux 14/08/10 23:45:48 INFO zookeeper.ZooKeeper: Client environment:os.arch=i386 14/08/10 23:45:48 INFO zookeeper.ZooKeeper: Client environment:os.version=3.2.0-67-generic 14/08/10 23:45:48 INFO zookeeper.ZooKeeper: Client environment:user.name=jenkins 14/08/10 23:45:48 INFO zookeeper.ZooKeeper: Client environment:user.home=/home/jenkins 14/08/10 23:45:48 INFO zookeeper.ZooKeeper: Client environment:user.dir=/tmp 14/08/10 23:45:48 INFO zookeeper.ZooKeeper: Initiating client connection, connectString=localhost:2 sessionTimeout=120 watcher=org.apache.hama.bsp.sync.ZKSyncBSPMasterClient@149ecb4 14/08/10 23:45:48 INFO sync.ZKSyncBSPMasterClient: Initialized ZK false 14/08/10 23:45:48 INFO sync.ZKSyncClient: Initializing ZK Sync Client 14/08/10 23:45:48 INFO zookeeper.ClientCnxn: Opening socket connection to server localhost/0:0:0:0:0:0:0:1:2. Will not attempt to authenticate using SASL (unknown error) 14/08/10 23:45:48 INFO server.NIOServerCnxnFactory: Accepted socket connection from /0:0:0:0:0:0:0:1:35100 14/08/10 23:45:48 INFO zookeeper.ClientCnxn: Socket connection established to localhost/0:0:0:0:0:0:0:1:2, initiating session 14/08/10 23:45:48 INFO server.ZooKeeperServer: Client attempting to establish new session at /0:0:0:0:0:0:0:1:35100 14/08/10 23:45:48 INFO persistence.FileTxnLog: Creating new log file: log.48 14/08/10 23:45:48 INFO server.ZooKeeperServer: Established session 0x147c2502792 with negotiated timeout 120 for client /0:0:0:0:0:0:0:1:35100 14/08/10 23:45:48 INFO zookeeper.ClientCnxn: Session establishment complete on server localhost/0:0:0:0:0:0:0:1:2, sessionid = 0x147c2502792, negotiated timeout = 120 14/08/10 23:45:48 INFO mortbay.log: Logging to org.slf4j.impl.Log4jLoggerAdapter(org.mortbay.log) via org.mortbay.log.Slf4jLog 14/08/10 23:45:48 INFO mortbay.log: Created master and client sync clients 14/08/10 23:45:48 INFO mortbay.log: BSP root exists 14/08/10 23:45:48 INFO zookeeper.ZooKeeper: Initiating client connection, connectString=localhost:2 sessionTimeout=120 watcher=org.apache.hama.bsp.sync.ZooKeeperSyncClientImpl@407d11 14/08/10 23:45:48 INFO zookeeper.ClientCnxn: Opening socket connection to server localhost/127.0.0.1:2. Will not attempt to authenticate using SASL (unknown error) 14/08/10 23:45:48 INFO sync.ZKSyncClient: Initializing ZK Sync Client 14/08/10 23:45:48 INFO sync.ZooKeeperSyncClientImpl: Start connecting to Zookeeper! At /0.0.0.0:21815 14/08/10 23:45:48 INFO zookeeper.ClientCnxn: Socket connection established to localhost/127.0.0.1:2, initiating session 14/08/10 23:45:48 INFO server.NIOServerCnxnFactory: Accepted socket connection from /127.0.0.1:59495 14/08/10 23:45:48 INFO server.ZooKeeperServer: Client attempting to establish new session at /127.0.0.1:59495 14/08/10 23:45:48 INFO server.ZooKeeperServer: Established session 0x147c25027920001 with negotiated timeout 120 for client /127.0.0.1:59495 14/08/10 23:45:48 INFO zookeeper.ClientCnxn: Session establishment complete on server localhost/127.0.0.1:2, sessionid = 0x147c25027920001, negotiated timeout = 120 14/08/10 23:45:48 INFO sync.ZKSyncClient:
Build failed in Jenkins: Hama-Nightly-for-Hadoop-1.x #1333
See https://builds.apache.org/job/Hama-Nightly-for-Hadoop-1.x/1333/ -- [...truncated 198 lines...] at org.tmatesoft.svn.core.internal.io.dav.http.HTTPConnection._request(HTTPConnection.java:775) ... 32 more Caused by: svn: E175002: timed out waiting for server at org.tmatesoft.svn.core.SVNErrorMessage.create(SVNErrorMessage.java:208) at org.tmatesoft.svn.core.internal.io.dav.http.HTTPConnection._request(HTTPConnection.java:514) ... 32 more Caused by: java.net.SocketTimeoutException: connect timed out at java.net.PlainSocketImpl.socketConnect(Native Method) at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) at java.net.Socket.connect(Socket.java:579) at org.tmatesoft.svn.core.internal.util.SVNSocketConnection.run(SVNSocketConnection.java:57) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) ... 4 more java.io.IOException: remote file operation failed: https://builds.apache.org/job/Hama-Nightly-for-Hadoop-1.x/ws/ at hudson.remoting.Channel@7be778dd:ubuntu-5 at hudson.FilePath.act(FilePath.java:910) at hudson.FilePath.act(FilePath.java:887) at hudson.scm.SubversionSCM.checkout(SubversionSCM.java:936) at hudson.scm.SubversionSCM.checkout(SubversionSCM.java:871) at hudson.model.AbstractProject.checkout(AbstractProject.java:1414) at hudson.model.AbstractBuild$AbstractBuildExecution.defaultCheckout(AbstractBuild.java:671) at jenkins.scm.SCMCheckoutStrategy.checkout(SCMCheckoutStrategy.java:88) at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:580) at hudson.model.Run.execute(Run.java:1676) at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43) at hudson.model.ResourceController.execute(ResourceController.java:88) at hudson.model.Executor.run(Executor.java:231) Caused by: java.io.IOException: Failed to check out http://svn.apache.org/repos/asf/hama/trunk at hudson.scm.subversion.CheckoutUpdater$1.perform(CheckoutUpdater.java:110) at hudson.scm.subversion.WorkspaceUpdater$UpdateTask.delegateTo(WorkspaceUpdater.java:161) at hudson.scm.SubversionSCM$CheckOutTask.perform(SubversionSCM.java:1030) at hudson.scm.SubversionSCM$CheckOutTask.invoke(SubversionSCM.java:1011) at hudson.scm.SubversionSCM$CheckOutTask.invoke(SubversionSCM.java:987) at hudson.FilePath$FileCallableWrapper.call(FilePath.java:2462) at hudson.remoting.UserRequest.perform(UserRequest.java:118) at hudson.remoting.UserRequest.perform(UserRequest.java:48) at hudson.remoting.Request$2.run(Request.java:328) at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72) at java.util.concurrent.FutureTask.run(FutureTask.java:262) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:745) Caused by: org.tmatesoft.svn.core.SVNException: svn: E175002: OPTIONS /repos/asf/hama/trunk failed at org.tmatesoft.svn.core.internal.io.dav.http.HTTPConnection.request(HTTPConnection.java:388) at org.tmatesoft.svn.core.internal.io.dav.http.HTTPConnection.request(HTTPConnection.java:373) at org.tmatesoft.svn.core.internal.io.dav.http.HTTPConnection.request(HTTPConnection.java:361) at org.tmatesoft.svn.core.internal.io.dav.DAVConnection.performHttpRequest(DAVConnection.java:707) at org.tmatesoft.svn.core.internal.io.dav.DAVConnection.exchangeCapabilities(DAVConnection.java:627) at org.tmatesoft.svn.core.internal.io.dav.DAVConnection.open(DAVConnection.java:102) at org.tmatesoft.svn.core.internal.io.dav.DAVRepository.openConnection(DAVRepository.java:1020) at org.tmatesoft.svn.core.internal.io.dav.DAVRepository.getLatestRevision(DAVRepository.java:180) at org.tmatesoft.svn.core.internal.wc16.SVNBasicDelegate.getRevisionNumber(SVNBasicDelegate.java:480) at org.tmatesoft.svn.core.internal.wc16.SVNBasicDelegate.getLocations(SVNBasicDelegate.java:833) at org.tmatesoft.svn.core.internal.wc16.SVNBasicDelegate.createRepository(SVNBasicDelegate.java:527) at org.tmatesoft.svn.core.internal.wc16.SVNUpdateClient16.doCheckout(SVNUpdateClient16.java:875) at org.tmatesoft.svn.core.internal.wc2.old.SvnOldCheckout.run(SvnOldCheckout.java:19) at
Re: [ANN] Welcome to new Hama committers
Congrats to join the community! On 11 August 2014 06:46, Edward J. Yoon edwardy...@apache.org wrote: The Apache Hama PMC is pleased to announce the following additions: * Victor Lee is now a Hama Committer * ByungSeok Min is now a Hama committer. Thanks again to both for their efforts, we hope to see them continue to move Apache Hama forward. Thanks! -- Best Regards, Edward J. Yoon CEO at DataSayer Co., Ltd.
