[
https://issues.apache.org/jira/browse/HAMA-443?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13146953#comment-13146953
]
Thomas Jungblut commented on HAMA-443:
--------------------------------------
This could be a bit more difficult.
First I have to regenerate the input to a sequencefile which it was before.
Second there is very likely a problem with the partitioning: after partitioning
the splits are inside an array, JobInProgress is going to setup new tasks and
assigns the split to the task. This assignment is NOT consistent with the
peername indices in our array of peers (you know, sorted by name for master
tasks).
Two solutions:
- Make the partition index visible for the user, then in the first superstep
each task has to broadcast the partition <- not very good..
- Instead of sorting by name we should sort ascending by the partition index
if it is available. This would mean, that the partition id would be stored in
zookeeper as well. <- not a good solution for us, but I guess it would be more
convinient for the user.
More generall sorting by name seems to be not a good strategy in the long run.
Consider we have multiple tasks and get in touch with fault tolerance. A groom
handles 3 tasks on port 33, 34, 35. groom:33 is faulty and gets killed. Now it
will be respawned, because the port is blocked it will be groom:36.
Now the task isn't the master anymore, because the whole sequence is sorted by
the name.
In my opinion we should sort ascending by taskId, this would actually solve the
problem with partitioning as well, because the index is the same index in the
taskid.
> SSSP doesn't works with multi-tasks
> -----------------------------------
>
> Key: HAMA-443
> URL: https://issues.apache.org/jira/browse/HAMA-443
> Project: Hama
> Issue Type: Bug
> Components: examples
> Affects Versions: 0.3.0
> Reporter: Edward J. Yoon
> Assignee: Thomas Jungblut
> Fix For: 0.4.0
>
>
> {code}
> root@Cnode1:/usr/local/src/hama-trunk# core/bin/hama jar
> examples/target/hama-examples-0.4.0-incubating-SNAPSHOT.jar sssp Klewno xx
> /user/root/edward/sssp-adjacencylist.txt
> Single Source Shortest Path Example:
> <Startvertex name> <optional: output path> <optional: path to own adjacency
> list textfile!>
> Setting default start vertex to "Frankfurt"!
> Setting start vertex to Klewno!
> Using new output folder: xx
> 11/09/26 09:46:24 INFO graph.ShortestPaths: Starting data partitioning...
> 11/09/26 09:47:12 INFO graph.ShortestPaths: Finished!
> 11/09/26 09:47:12 INFO bsp.BSPJobClient: Running job: job_201109260929_0004
> 11/09/26 09:47:15 INFO bsp.BSPJobClient: Current supersteps number: 0
> 11/09/26 09:47:21 INFO bsp.BSPJobClient: Current supersteps number: 1
> 11/09/26 09:47:33 INFO bsp.BSPJobClient: The total number of supersteps: 1
> Job Finished in 21.553 seconds
> -------------------- RESULTS --------------------
> java.lang.NullPointerException
> at
> org.apache.hama.examples.graph.ShortestPathsBase.printOutput(ShortestPathsBase.java:93)
> at
> org.apache.hama.examples.graph.ShortestPaths.main(ShortestPaths.java:239)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at
> org.apache.hadoop.util.ProgramDriver$ProgramDescription.invoke(ProgramDriver.java:68)
> at org.apache.hadoop.util.ProgramDriver.driver(ProgramDriver.java:139)
> at org.apache.hama.examples.ExampleDriver.main(ExampleDriver.java:37)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hama.util.RunJar.main(RunJar.java:145)
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
