[
https://issues.apache.org/jira/browse/GIRAPH-198?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13291443#comment-13291443
]
Roman Shaposhnik commented on GIRAPH-198:
-----------------------------------------
Aha! Thanks a million Claudio!
I got much further with the following (although I'm surprised that Current
master task partition==0 and Last checkpointed superstep==0):
{noformat}
$ hadoop jar /usr/lib/giraph/giraph-jar-with-dependencies.jar
org.apache.giraph.benchmark.PageRankBenchmark -Dgiraph.useNetty=true
-Dgiraph.zkList=localhost -v -e 1 -s 3 -V 100000 -w 3
....
Giraph Stats
Aggregate edges=100000
Aggregate finished vertices=100000
Aggregate vertices=100000
Current master task partition=0
Current workers=3
Last checkpointed superstep=0
Sent messages=0
Superstep=4
Giraph Timers
Setup (milliseconds)=313
Shutdown (milliseconds)=434
Superstep 0 (milliseconds)=2337
Superstep 1 (milliseconds)=2171
Superstep 2 (milliseconds)=1091
Superstep 3 (milliseconds)=453
Total (milliseconds)=13332
Vertex input superstep (milliseconds)=6525
{noformat}
I got so excited that I wanted to try all examples and that's where it started
to fail again:
{noformat}
$ hadoop jar /usr/lib/giraph/giraph-jar-with-dependencies.jar
org.apache.giraph.examples.SimpleCheckpointVertex -Dgiraph.useNetty=true
-Dgiraph.zkList=localhost -v -w 3 -s 10 -o testGiraph
....
12/06/07 17:07:03 INFO mapreduce.Job: Running job: job_1339098338323_0011
12/06/07 17:07:17 INFO mapreduce.Job: Job job_1339098338323_0011 running in
uber mode : false
12/06/07 17:07:17 INFO mapreduce.Job: map 0% reduce 0%
12/06/07 17:07:34 INFO mapreduce.Job: Task Id :
attempt_1339098338323_0011_m_000000_0, Status : FAILED
12/06/07 17:07:34 WARN mapreduce.Job: Error reading task output Server returned
HTTP response code: 400 for URL:
http://ahmed-laptop:8080/tasklog?plaintext=true&attemptid=attempt_1339098338323_0011_m_000000_0&filter=stdout
12/06/07 17:07:34 WARN mapreduce.Job: Error reading task output Server returned
HTTP response code: 400 for URL:
http://ahmed-laptop:8080/tasklog?plaintext=true&attemptid=attempt_1339098338323_0011_m_000000_0&filter=stderr
12/06/07 17:07:46 INFO mapreduce.Job: map 25% reduce 0%
{noformat}
And here's what I see in the logs of the failed tasks:
{noformat}
2012-06-07 17:07:30,918 INFO [main-SendThread()]
org.apache.zookeeper.ClientCnxn: Opening socket connection to server
/127.0.0.1:2181
2012-06-07 17:07:30,928 WARN [main-SendThread(ahmed-laptop:2181)]
org.apache.zookeeper.client.ZooKeeperSaslClient: SecurityException:
java.lang.SecurityException: Unable to locate a login configuration occurred
when trying to find JAAS configuration.
2012-06-07 17:07:30,928 INFO [main-SendThread(ahmed-laptop:2181)]
org.apache.zookeeper.client.ZooKeeperSaslClient: Client will not
SASL-authenticate because the default JAAS configuration section 'Client' could
not be found. If you are not using SASL, you may ignore this. On the other
hand, if you expected SASL to work, please fix your JAAS configuration.
2012-06-07 17:07:30,929 INFO [main-SendThread(ahmed-laptop:2181)]
org.apache.zookeeper.ClientCnxn: Socket connection established to
ahmed-laptop/127.0.0.1:2181, initiating session
2012-06-07 17:07:30,943 INFO [main-SendThread(ahmed-laptop:2181)]
org.apache.zookeeper.ClientCnxn: Session establishment complete on server
ahmed-laptop/127.0.0.1:2181, sessionid = 0x137c87b48140025, negotiated timeout
= 40000
2012-06-07 17:07:30,946 INFO [main-EventThread]
org.apache.giraph.graph.BspService: process: Asynchronous connection complete.
2012-06-07 17:07:30,952 INFO [main] org.apache.giraph.graph.GraphMapper: map:
No need to do anything when not a worker
2012-06-07 17:07:30,952 INFO [main] org.apache.giraph.graph.GraphMapper:
cleanup: Starting for MASTER_ONLY
2012-06-07 17:07:31,019 INFO [org.apache.giraph.graph.MasterThread]
org.apache.giraph.graph.BspService: getJobState: Job state already exists
(/_hadoopBsp/job_1339098338323_0011/_masterJobState)
2012-06-07 17:07:31,027 INFO [org.apache.giraph.graph.MasterThread]
org.apache.giraph.graph.BspServiceMaster: becomeMaster: First child is
'/_hadoopBsp/job_1339098338323_0011/_masterElectionDir/ahmed-laptop_00000000000'
and my bid is
'/_hadoopBsp/job_1339098338323_0011/_masterElectionDir/ahmed-laptop_00000000000'
2012-06-07 17:07:31,033 INFO [org.apache.giraph.graph.MasterThread]
org.apache.giraph.graph.BspService: getApplicationAttempt: Node
/_hadoopBsp/job_1339098338323_0011/_applicationAttemptsDir already exists!
2012-06-07 17:07:31,034 INFO [org.apache.giraph.graph.MasterThread]
org.apache.giraph.graph.BspServiceMaster: becomeMaster: I am now the master!
2012-06-07 17:07:31,042 INFO [org.apache.giraph.graph.MasterThread]
org.apache.giraph.graph.BspService: getApplicationAttempt: Node
/_hadoopBsp/job_1339098338323_0011/_applicationAttemptsDir already exists!
2012-06-07 17:07:31,418 FATAL [org.apache.giraph.graph.MasterThread]
org.apache.giraph.graph.GraphMapper: uncaughtException:
OverrideExceptionHandler on thread org.apache.giraph.graph.MasterThread, msg =
java.lang.InstantiationException, exiting...
java.lang.RuntimeException: java.lang.InstantiationException
at
org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:128)
at
org.apache.giraph.graph.BspUtils.createVertexInputFormat(BspUtils.java:141)
at
org.apache.giraph.graph.BspServiceMaster.generateInputSplits(BspServiceMaster.java:225)
at
org.apache.giraph.graph.BspServiceMaster.createInputSplits(BspServiceMaster.java:506)
at org.apache.giraph.graph.MasterThread.run(MasterThread.java:98)
Caused by: java.lang.InstantiationException
at
sun.reflect.InstantiationExceptionConstructorAccessorImpl.newInstance(InstantiationExceptionConstructorAccessorImpl.java:30)
at java.lang.reflect.Constructor.newInstance(Constructor.java:513)
at
org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:126)
... 4 more
{noformat}
Is this a known issue with Giraph?
> running Giraph trunk on Hadoop 2.0.0-alpha leads to an exception
> ----------------------------------------------------------------
>
> Key: GIRAPH-198
> URL: https://issues.apache.org/jira/browse/GIRAPH-198
> Project: Giraph
> Issue Type: Bug
> Components: mapreduce
> Affects Versions: 0.2.0
> Reporter: Roman Shaposhnik
> Assignee: Jakob Homan
> Fix For: 0.2.0
>
>
> Here's how we compile:
> {noformat}
> $ mvn -Phadoop_trunk -DskipTests -Dhadoop.version=2.0.0-alpha site package
> {noformat}
> Here's how we run:
> {noformat}
> $ export GIRAPH_HOME=/usr/lib/giraph
> $ hadoop jar /usr/lib/giraph/giraph-jar-with-dependencies.jar
> org.apache.giraph.benchmark.PageRankBenchmark -e 1 -s 3 -V 100000 -w 3
> {noformat}
> Here's what we get in the task logs (as a side note -- once 3 of the 8 tasks
> fail that way the entire job hangs forever which could be a bug in and of
> itself):
> {noformat}
> 2012-05-31 10:31:28,518 WARN [main]
> org.apache.hadoop.mapred.YarnChild: Exception running child :
> java.lang.RuntimeException: java.lang.NoSuchFieldException: versionID
> at org.apache.hadoop.ipc.RPC.getProtocolVersion(RPC.java:170)
> at
> org.apache.hadoop.ipc.WritableRpcEngine$Invocation.(WritableRpcEngine.java:110)
> at
> org.apache.hadoop.ipc.WritableRpcEngine$Invoker.invoke(WritableRpcEngine.java:224)
> at $Proxy11.putVertexList(Unknown Source)
> at
> org.apache.giraph.comm.BasicRPCCommunications.sendPartitionReq(BasicRPCCommunications.java:921)
> at
> org.apache.giraph.graph.BspServiceWorker.loadVertices(BspServiceWorker.java:322)
> at
> org.apache.giraph.graph.BspServiceWorker.setup(BspServiceWorker.java:594)
> at org.apache.giraph.graph.GraphMapper.setup(GraphMapper.java:473)
> at org.apache.giraph.graph.GraphMapper.run(GraphMapper.java:645)
> at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:725)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:332)
> at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:152)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:396)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1232)
> at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:147)
> Caused by: java.lang.NoSuchFieldException: versionID
> at java.lang.Class.getField(Class.java:1520)
> at org.apache.hadoop.ipc.RPC.getProtocolVersion(RPC.java:166)
> ... 15 more
> {noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
