mike anderson wrote:
This would be really useful for my current projects. I'd be more than happy
to help out if needed.
well the first bit of code to play with then is this
http://smartfrog.svn.sourceforge.net/viewvc/smartfrog/trunk/core/extras/citerank/
the standalone.xml file is the one you want to build and run with, the
other would require you to check out and build two levels up, but gives
you the ability to bring up local or remote clusters to test. Call
run-local to run it locally., which should give you some stats like this:
[java] 09/06/25 17:09:22 INFO citerank.CiteRankTool: Counters: 11
[java] 09/06/25 17:09:22 INFO citerank.CiteRankTool: File Systems
[java] 09/06/25 17:09:22 INFO citerank.CiteRankTool: Local
bytes read=209445683448
[java] 09/06/25 17:09:22 INFO citerank.CiteRankTool: Local
bytes written=173943642259
[java] 09/06/25 17:09:22 INFO citerank.CiteRankTool: Map-Reduce
Framework
[java] 09/06/25 17:09:22 INFO citerank.CiteRankTool: Reduce
input groups=9985124
[java] 09/06/25 17:09:22 INFO citerank.CiteRankTool: Combine
output records=34
[java] 09/06/25 17:09:22 INFO citerank.CiteRankTool: Map input
records=24383448
[java] 09/06/25 17:09:22 INFO citerank.CiteRankTool: Reduce
output records=16494967
[java] 09/06/25 17:09:22 INFO citerank.CiteRankTool: Map
output bytes=1243216870
[java] 09/06/25 17:09:22 INFO citerank.CiteRankTool: Map input
bytes=1528854187
[java] 09/06/25 17:09:22 INFO citerank.CiteRankTool: Combine
input records=4528655
[java] 09/06/25 17:09:22 INFO citerank.CiteRankTool: Map
output records=41958636
[java] 09/06/25 17:09:22 INFO citerank.CiteRankTool: Reduce
input records=37430015
======================================================================
Exiting project "citerank"
======================================================================
BUILD SUCCESSFUL - at 25/06/09 17:09
Total time: 9 minutes 1 second
--
Steve Loughran http://www.1060.org/blogxter/publish/5
Author: Ant in Action http://antbook.org/
