0.21 is here, can we use this repo for Hadoop 0.21 poms? If so, how to change ivy stuff to do so? I can't seem to tell Ivy to fetch hadoop from here:
https://repository.apache.org/content/groups/snapshots/org/apache/hadoop/hadoop- common/ On Wednesday 14 December 2011 16:05:07 Markus Jelsma wrote: > Andrzej, > > I cannot continue with testing migration on 0.20 because things like > MapFileOutputFormat are missing in the new API. I cannot compile with 0.22 > because it no longer has the old mapred API. And i cannot build with 0.21 > because it is not in maven central!? > > Any help? > > Thanks! > > On Tuesday 13 December 2011 18:57:48 Andrzej Bialecki wrote: > > On 13/12/2011 18:04, Markus Jelsma wrote: > > > Hi > > > > > > I did a quick test to see what happens and it won't compile. It cannot > > > find our old mapred API's in 0.22. I've also tried 0.20.205.0 which > > > compiles but won't run and many tests fail with stuff like. > > > > > > Exception in thread "main" java.lang.NoClassDefFoundError: > > > org/codehaus/jackson/map/JsonMappingException > > > > > > at > > > > > > org.apache.nutch.util.dupedb.HostDeduplicator.deduplicator(HostDeduplic > > > at or.java:421) > > > > Hmm... what's that? I don't see this class (or this package) in the > > Nutch tree. Also, trunk doesn't use JSON for anything as far as I know. > > > > > at > > > > > > org.apache.nutch.util.dupedb.HostDeduplicator.run(HostDeduplicator.java > > > :4 43) > > > > > > at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65) > > > at > > > > > > org.apache.nutch.util.dupedb.HostDeduplicator.main(HostDeduplicator.jav > > > a: 431) Caused by: java.lang.ClassNotFoundException: > > > org.codehaus.jackson.map.JsonMappingException > > > > > > at java.net.URLClassLoader$1.run(URLClassLoader.java:202) > > > at java.security.AccessController.doPrivileged(Native Method) > > > at java.net.URLClassLoader.findClass(URLClassLoader.java:190) > > > at java.lang.ClassLoader.loadClass(ClassLoader.java:306) > > > at > > > sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301) > > > at java.lang.ClassLoader.loadClass(ClassLoader.java:247) ... > > > 4 more > > > > > > I think this can be overcome but we cannot hide from the fact that all > > > jobs must be ported to the new API at some point. > > > > > > You did some work on the new API's, did you come across any cumbersome > > > issues when working on it? > > > > It was quite some time ago .. but I don't remember anything being really > > complicated, it was just tedious - and once you've done one class the > > other classes follow roughly the same pattern. -- Markus Jelsma - CTO - Openindex
