Your version? On Wed, Apr 17, 2013 at 12:07 AM, Lvxuedong <[email protected]> wrote: > Hi, Edward, thank you. But your suggestion seems do not work, do you have > some other advice ? > > Is java.lang.NullPointerException related with heap size ? > > > > 在 2013-4-16，22:13，"Edward J. Yoon" <[email protected]> 写道： > >> I guess you need to increase the child processor JVM heap size. >> >> - conf/hama-site.xml: >> >> <property> >> <name>bsp.child.java.opts</name> >> <value>-Xmx2048m</value> >> </property> >> >> On Tue, Apr 16, 2013 at 10:55 PM, Lyu Xuedong <[email protected]> wrote: >>> My project can work well in eclipse, but when I export it as a jar file and >>> submit it to a two-node hama cluster, some errors and fatal occur if the >>> input file is over 64M. >>> I suspect that my configure files of hadoop or hama is somewhere not right, >>> but a pi estimator can run normally on my cluster and >>> GroomServer$BSPPerChild can be seen on each nodes. I debug for a whole day, >>> nothing improved. >>> What in my input file are a large number of RDF triples: "<subject> >>> <predicate> <object> ." My task is to create vertices for subjects and >>> objects. Predicates are subjects' edges. >>> What should I do ? >>> >>> Terminal output: >>> >>> 13/04/16 21:13:36 INFO bgp.HamaBgpComplete: Job begain. >>> 13/04/16 21:13:37 INFO bsp.FileInputFormat: Total input paths to process : 2 >>> 13/04/16 21:13:38 INFO bsp.BSPJobClient: Running job: job_201304161357_0015 >>> 13/04/16 21:13:41 INFO bsp.BSPJobClient: Current supersteps number: 0 >>> 13/04/16 21:13:47 INFO bsp.BSPJobClient: Current supersteps number: 2 >>> 13/04/16 21:13:53 INFO bsp.BSPJobClient: Current supersteps number: 3 >>> 13/04/16 21:13:59 INFO bsp.BSPJobClient: Current supersteps number: 4 >>> 13/04/16 21:14:05 INFO bsp.BSPJobClient: Current supersteps number: 5 >>> attempt_201304161357_0015_000000_0: 13/04/16 21:13:43 INFO >>> sync.ZKSyncClient: Initializing ZK Sync Client >>> attempt_201304161357_0015_000000_0: 13/04/16 21:13:43 INFO >>> sync.ZooKeeperSyncClientImpl: Start connecting to Zookeeper! At >>> hadoop1/1.2.3.4:61002 >>> attempt_201304161357_0015_000000_0: 13/04/16 21:13:43 INFO ipc.Server: >>> Starting SocketReader >>> attempt_201304161357_0015_000000_0: 13/04/16 21:13:43 INFO ipc.Server: IPC >>> Server Responder: starting >>> attempt_201304161357_0015_000000_0: 13/04/16 21:13:43 INFO ipc.Server: IPC >>> Server handler 0 on 61002: starting >>> attempt_201304161357_0015_000000_0: 13/04/16 21:13:43 INFO >>> message.HadoopMessageManagerImpl: BSPPeer address:hadoop1 port:61002 >>> attempt_201304161357_0015_000000_0: 13/04/16 21:13:43 INFO ipc.Server: IPC >>> Server listener on 61002: starting >>> attempt_201304161357_0015_000000_0: 13/04/16 21:14:03 ERROR bsp.BSPTask: >>> Error running bsp setup and bsp function. >>> attempt_201304161357_0015_000000_0: java.lang.NullPointerException >>> attempt_201304161357_0015_000000_0: 13/04/16 21:14:04 INFO ipc.Server: >>> Stopping server on 61002 >>> attempt_201304161357_0015_000000_0: 13/04/16 21:14:04 INFO ipc.Server: IPC >>> Server handler 0 on 61002: exiting >>> attempt_201304161357_0015_000000_0: 13/04/16 21:14:04 INFO ipc.Server: >>> Stopping IPC Server listener on 61002 >>> attempt_201304161357_0015_000000_0: 13/04/16 21:14:04 INFO ipc.Server: >>> Stopping IPC Server Responder >>> attempt_201304161357_0015_000000_0: 13/04/16 21:14:04 INFO >>> metrics.RpcInstrumentation: shut down >>> attempt_201304161357_0015_000000_0: 13/04/16 21:14:04 ERROR bsp.BSPTask: >>> Shutting down ping service. >>> attempt_201304161357_0015_000000_0: 13/04/16 21:14:04 FATAL bsp.GroomServer: >>> Error running child >>> attempt_201304161357_0015_000000_0: java.lang.NullPointerException >>> attempt_201304161357_0015_000000_0: java.lang.NullPointerException >>> 13/04/16 21:15:11 INFO bsp.BSPJobClient: Job failed. >>> >>> tasklogs: >>> attempt_201304161357_0015_000000_0.log >>> 13/04/16 21:13:43 INFO sync.ZKSyncClient: Initializing ZK Sync Client >>> 13/04/16 21:13:43 INFO sync.ZooKeeperSyncClientImpl: Start connecting to >>> Zookeeper! At hadoop1/1.2.3.4:61002 >>> 13/04/16 21:13:43 INFO ipc.Server: Starting SocketReader >>> 13/04/16 21:13:43 INFO ipc.Server: IPC Server Responder: starting >>> 13/04/16 21:13:43 INFO ipc.Server: IPC Server handler 0 on 61002: starting >>> 13/04/16 21:13:43 INFO message.HadoopMessageManagerImpl: BSPPeer >>> address:hadoop1 port:61002 >>> 13/04/16 21:13:43 INFO ipc.Server: IPC Server listener on 61002: starting >>> 13/04/16 21:14:03 ERROR bsp.BSPTask: Error running bsp setup and bsp >>> function. >>> java.lang.NullPointerException >>> 13/04/16 21:14:04 INFO ipc.Server: Stopping server on 61002 >>> 13/04/16 21:14:04 INFO ipc.Server: IPC Server handler 0 on 61002: exiting >>> 13/04/16 21:14:04 INFO ipc.Server: Stopping IPC Server listener on 61002 >>> 13/04/16 21:14:04 INFO ipc.Server: Stopping IPC Server Responder >>> 13/04/16 21:14:04 INFO metrics.RpcInstrumentation: shut down >>> 13/04/16 21:14:04 ERROR bsp.BSPTask: Shutting down ping service. >>> 13/04/16 21:14:04 FATAL bsp.GroomServer: Error running child >>> java.lang.NullPointerException >>> java.lang.NullPointerException >>> >>> attempt_201304161357_0015_000001_0.log >>> 13/04/16 21:13:42 INFO sync.ZKSyncClient: Initializing ZK Sync Client >>> 13/04/16 21:13:42 INFO sync.ZooKeeperSyncClientImpl: Start connecting to >>> Zookeeper! At hadoop1/1.2.3.4:61001 >>> 13/04/16 21:13:42 ERROR sync.ZooKeeperSyncClientImpl: >>> org.apache.zookeeper.KeeperException$NoNodeException: KeeperErrorCode = >>> NoNode for /bsp/job_201304161357_0015/peers >>> 13/04/16 21:13:42 INFO ipc.Server: Starting SocketReader >>> 13/04/16 21:13:42 INFO ipc.Server: IPC Server Responder: starting >>> 13/04/16 21:13:42 INFO message.HadoopMessageManagerImpl: BSPPeer >>> address:hadoop1 port:61001 >>> 13/04/16 21:13:42 INFO ipc.Server: IPC Server listener on 61001: starting >>> 13/04/16 21:13:42 INFO ipc.Server: IPC Server handler 0 on 61001: starting >>> 13/04/16 21:14:06 ERROR bsp.BSPPeerImpl: Error while sending messages >>> java.io.IOException: Call to hadoop1/1.2.3.4:61002 failed on local >>> exception: java.io.EOFException >>> at org.apache.hadoop.ipc.Client.wrapException(Client.java:1103) >>> at org.apache.hadoop.ipc.Client.call(Client.java:1071) >>> at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:225) >>> at $Proxy3.put(Unknown Source) >>> at >>> org.apache.hama.bsp.message.HadoopMessageManagerImpl.transfer(HadoopMessageManagerImpl.java:108) >>> at org.apache.hama.bsp.BSPPeerImpl.sync(BSPPeerImpl.java:410) >>> at org.apache.hama.graph.GraphJobRunner.bsp(GraphJobRunner.java:118) >>> at org.apache.hama.bsp.BSPTask.runBSP(BSPTask.java:166) >>> at org.apache.hama.bsp.BSPTask.run(BSPTask.java:143) >>> at >>> org.apache.hama.bsp.GroomServer$BSPPeerChild.main(GroomServer.java:1266) >>> Caused by: java.io.EOFException >>> at java.io.DataInputStream.readInt(DataInputStream.java:375) >>> at >>> org.apache.hadoop.ipc.Client$Connection.receiveResponse(Client.java:800) >>> at org.apache.hadoop.ipc.Client$Connection.run(Client.java:745) >> >> >> >> -- >> Best Regards, Edward J. Yoon >> @eddieyoon
-- Best Regards, Edward J. Yoon @eddieyoon
