I don't see anything that obvious, except there are two servlet API JARS -one of those should go
On 10 November 2012 19:42, Koert Kuipers <[email protected]> wrote: > ant sets classpath for me but i have same issues when i set it myself. > for example i can run it like this: > java -cp lib/test/*:build/classes:build/test > com.tresata.hadoop.mapred.MapRedTest > > my dependencies are hadoop-core, hadoop-test and i also had to add > jsr311api to make it work. > ls lib/test gives: > > ant-1.6.5.jar > commons-beanutils-1.7.0.jar > commons-beanutils-core-1.8.0.jar > commons-cli-1.2.jar > commons-codec-1.4.jar > commons-collections-3.2.1.jar > commons-configuration-1.6.jar > commons-digester-1.8.jar > commons-el-1.0.jar > commons-httpclient-3.0.1.jar > commons-lang-2.4.jar > commons-logging-1.1.1.jar > commons-math-2.1.jar > commons-net-1.4.1.jar > core-3.1.1.jar > ftplet-api-1.0.0.jar > ftpserver-core-1.0.0.jar > ftpserver-deprecated-1.0.0-M2.jar > guava-r06.jar > hadoop-core-1.0.4.jar > hadoop-test-1.0.4.jar > hsqldb-1.8.0.10.jar > jackson-core-asl-1.0.1.jar > jackson-mapper-asl-1.0.1.jar > jasper-compiler-5.5.12.jar > jasper-runtime-5.5.12.jar > jets3t-0.7.1.jar > jetty-6.1.26.jar > jetty-util-6.1.26.jar > joda-time-2.1.jar > jsp-2.1-6.1.14.jar > jsp-api-2.1-6.1.14.jar > jsr311-api-1.1.1.jar > junit-3.8.1.jar > kfs-0.3.jar > log4j-1.2.16.jar > mina-core-2.0.0-M5.jar > oro-2.0.8.jar > servlet-api-2.5-20081211.jar > servlet-api-2.5-6.1.14.jar > slf4j-api-1.6.1.jar > slf4j-log4j12-1.6.1.jar > xmlenc-0.52.jar > > > On Sat, Nov 10, 2012 at 5:24 AM, Steve Loughran <[email protected]>wrote: > >> >> >> On 10 November 2012 07:43, Koert Kuipers <[email protected]> wrote: >> >>> i am porting a map-reduce library from CDH3 to apache hadoop 1.0.4. the >>> unit tests used to run fine but are now misbehaving. i do a simple setup in >>> the unit test like this: >>> >>> private static MiniDFSCluster dfsCluster; >>> private static FileSystem fs; >>> private static MiniMRCluster mrCluster; >>> >>> @BeforeClass >>> public static void oneTimeSetUp() throws Exception { >>> new File("tmp/logs").mkdirs(); >>> System.setProperty("hadoop.log.dir", "tmp/logs"); >>> System.setProperty("javax.xml.parsers.SAXParserFactory", >>> >>> "com.sun.org.apache.xerces.internal.jaxp.SAXParserFactoryImpl"); >>> final Configuration conf = new Configuration(); >>> conf.set("io.skip.checksum.errors", "true"); >>> dfsCluster = new MiniDFSCluster(conf, 2, true, null); >>> fs = dfsCluster.getFileSystem(); >>> mrCluster = new MiniMRCluster(2, >>> dfsCluster.getFileSystem().getUri().toString(), 1, null, null, new >>> org.apache.hadoop.mapred.JobConf(conf)); >>> } >>> >>> however with hadoop 1.0.4 this results in an exception where the >>> jobtracker dies: >>> >>> 12/11/10 02:38:29 ERROR mapred.MiniMRCluster: Job tracker crashed >>> java.lang.NoSuchFieldError: IS_SECURITY_ENABLED >>> at >>> org.apache.jasper.compiler.JspRuntimeContext.<init>(JspRuntimeContext.java:197) >>> at org.apache.jasper.servlet.JspServlet.init(JspServlet.java:150) >>> at >>> org.mortbay.jetty.servlet.ServletHolder.initServlet(ServletHolder.java:440) >>> >>> not sure what this error is about, but i managed to get around it for >>> now by removing the 2 hadoop apache jasper dependencies which seem to have >>> something to do with it: jasper-compiler-5.5.12.jar and >>> jasper-runtime-5.5.12.jar. with this "fix" i get further and my jobs >>> starts, but now all the tasks fail with messages like this: >>> >>> attempt_20121110024037623_0001_r_000002_0: Exception in thread "main" >>> java.lang.NoClassDefFoundError: org/apache/hadoop/mapred/Child >>> >>> am i doing something completely wrong, and was i just lucky that this >>> used to run with CDH3? any ideas? thanks! koert >>> >>> >> This sounds like some classpath/versioning thing -MiniMR cluster works >> for me; I even fixed that system property setup in trunk as I got fed up >> with having to do it myself. >> >> What is your CP in your test run? >> > >
