[
https://issues.apache.org/jira/browse/GIRAPH-208?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13396369#comment-13396369
]
Hudson commented on GIRAPH-208:
-------------------------------
Integrated in giraph-trunk-hadoop-0.23 #6 (See
[https://builds.apache.org/job/giraph-trunk-hadoop-0.23/6/])
GIRAPH-208: LocalTestMode's zookeeper directory is not being cleaned up
after job runs. (Revision 1351533)
Result = SUCCESS
ekoontz :
http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1351533
Files :
* /giraph/trunk/src/main/java/org/apache/giraph/zk/ZooKeeperManager.java
> LocalTestMode's zookeeper directory is not being cleaned up after job runs
> --------------------------------------------------------------------------
>
> Key: GIRAPH-208
> URL: https://issues.apache.org/jira/browse/GIRAPH-208
> Project: Giraph
> Issue Type: Bug
> Affects Versions: 0.2.0
> Reporter: Eugene Koontz
> Assignee: Eugene Koontz
> Priority: Minor
> Attachments: GIRAPH-208.patch, GIRAPH-208.patch
>
>
> Discovered bug while running PageRankBenchmark in localTestMode, with
> -Phadoop_1.0, with the following giraph-site.xml:
> {code}
> <configuration>
> <property>
> <name>giraph.SplitMasterWorker</name>
> <value>false</value>
> </property>
> <property>
> <name>giraph.localTestMode</name>
> <value>true</value>
> </property>
> <property>
> <name>giraph.zkJar</name>
>
> <value>/home/eugene/giraph/target/giraph-0.2-SNAPSHOT-jar-with-dependencies.jar</value>
> </property>
> </configuration>
> {code}
> With this configuration, I ran PageRankBenchmark as follows:
> {code}
> java -cp (all the jars..) org.apache.giraph.benchmark.PageRankBenchmark -c 0
> -e 3 -s 5 -v -w 1 -V 10
> {code}
> This worked the first time:
> {code}
> 12/06/18 15:33:51 INFO mapred.JobClient: Job complete: job_local_0001
> 12/06/18 15:33:51 INFO mapred.JobClient: Counters: 31
> 12/06/18 15:33:51 INFO mapred.JobClient: Giraph Timers
> 12/06/18 15:33:51 INFO mapred.JobClient: Total (milliseconds)=5361
> 12/06/18 15:33:51 INFO mapred.JobClient: Superstep 3 (milliseconds)=305
> 12/06/18 15:33:51 INFO mapred.JobClient: Vertex input superstep
> (milliseconds)=207
> 12/06/18 15:33:51 INFO mapred.JobClient: Superstep 4 (milliseconds)=317
> 12/06/18 15:33:51 INFO mapred.JobClient: Superstep 10 (milliseconds)=297
> 12/06/18 15:33:51 INFO mapred.JobClient: Setup (milliseconds)=459
> 12/06/18 15:33:51 INFO mapred.JobClient: Shutdown (milliseconds)=875
> 12/06/18 15:33:51 INFO mapred.JobClient: Superstep 7 (milliseconds)=305
> 12/06/18 15:33:51 INFO mapred.JobClient: Superstep 0 (milliseconds)=553
> 12/06/18 15:33:51 INFO mapred.JobClient: Superstep 8 (milliseconds)=304
> 12/06/18 15:33:51 INFO mapred.JobClient: Superstep 9 (milliseconds)=306
> 12/06/18 15:33:51 INFO mapred.JobClient: Superstep 6 (milliseconds)=339
> 12/06/18 15:33:51 INFO mapred.JobClient: Superstep 5 (milliseconds)=268
> 12/06/18 15:33:51 INFO mapred.JobClient: Superstep 2 (milliseconds)=313
> 12/06/18 15:33:51 INFO mapred.JobClient: Superstep 1 (milliseconds)=503
> 12/06/18 15:33:51 INFO mapred.JobClient: File Output Format Counters
> 12/06/18 15:33:51 INFO mapred.JobClient: Bytes Written=0
> 12/06/18 15:33:51 INFO mapred.JobClient: Giraph Stats
> 12/06/18 15:33:51 INFO mapred.JobClient: Aggregate edges=100
> 12/06/18 15:33:51 INFO mapred.JobClient: Superstep=11
> 12/06/18 15:33:51 INFO mapred.JobClient: Current workers=1
> 12/06/18 15:33:51 INFO mapred.JobClient: Last checkpointed superstep=0
> 12/06/18 15:33:51 INFO mapred.JobClient: Current master task partition=0
> 12/06/18 15:33:51 INFO mapred.JobClient: Sent messages=0
> 12/06/18 15:33:51 INFO mapred.JobClient: Aggregate finished vertices=10
> 12/06/18 15:33:51 INFO mapred.JobClient: Aggregate vertices=10
> 12/06/18 15:33:51 INFO mapred.JobClient: File Input Format Counters
> 12/06/18 15:33:51 INFO mapred.JobClient: Bytes Read=0
> 12/06/18 15:33:51 INFO mapred.JobClient: FileSystemCounters
> 12/06/18 15:33:51 INFO mapred.JobClient: FILE_BYTES_READ=88
> 12/06/18 15:33:51 INFO mapred.JobClient: FILE_BYTES_WRITTEN=32525
> 12/06/18 15:33:51 INFO mapred.JobClient: Map-Reduce Framework
> 12/06/18 15:33:51 INFO mapred.JobClient: Map input records=1
> 12/06/18 15:33:51 INFO mapred.JobClient: Spilled Records=0
> 12/06/18 15:33:51 INFO mapred.JobClient: SPLIT_RAW_BYTES=44
> 12/06/18 15:33:51 INFO mapred.JobClient: Map output records=0
> {code}
> but trying to run it again yields the following:
> {code}
> 12/06/18 15:35:01 INFO mapred.Task: Task 'attempt_local_0001_m_000000_0' done.
> 12/06/18 15:35:01 WARN mapred.FileOutputCommitter: Output path is null in
> cleanup
> 12/06/18 15:35:02 INFO mapred.JobClient: map 100% reduce 0%
> 12/06/18 15:35:02 INFO mapred.JobClient: Job complete: job_local_0001
> 12/06/18 15:35:02 INFO mapred.JobClient: Counters: 8
> 12/06/18 15:35:02 INFO mapred.JobClient: File Output Format Counters
> 12/06/18 15:35:02 INFO mapred.JobClient: Bytes Written=0
> 12/06/18 15:35:02 INFO mapred.JobClient: File Input Format Counters
> 12/06/18 15:35:02 INFO mapred.JobClient: Bytes Read=0
> 12/06/18 15:35:02 INFO mapred.JobClient: FileSystemCounters
> 12/06/18 15:35:02 INFO mapred.JobClient: FILE_BYTES_READ=88
> 12/06/18 15:35:02 INFO mapred.JobClient: FILE_BYTES_WRITTEN=32493
> 12/06/18 15:35:02 INFO mapred.JobClient: Map-Reduce Framework
> 12/06/18 15:35:02 INFO mapred.JobClient: Map input records=1
> 12/06/18 15:35:02 INFO mapred.JobClient: Spilled Records=0
> 12/06/18 15:35:02 INFO mapred.JobClient: SPLIT_RAW_BYTES=44
> 12/06/18 15:35:02 INFO mapred.JobClient: Map output records=0
> Disconnected from the target VM, address: '127.0.0.1:33268', transport:
> 'socket'
> {code}
> which is wrong because the Giraph mapper never got called (note the lack of
> Superstep timers and "Giraph Stats" section in the above)
> A workaround for this bug is to run "rm -rf
> ~/giraph/_bsp/_defaultZkManagerDir" before re-running PageRankBenchmark -
> then it will run correctly afterwards.
> The problem in the code is that the ZookeeperManager's directory is not being
> removed as it should be. This is because the zkDirDefault in
> ZooKeeperManager.java is not being set correctly - it is currently:
> {code}
> System.getProperty("user.dir") + "/_bspZooKeeper";
> {code}
> but it should be:
> {code}
> System.getProperty("user.dir") + GiraphJob.ZOOKEEPER_MANAGER_DIR_DEFAULT;
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
