anything in the oozie/log/catalina.out file that could hint at the reason for the spontaneous shutdown?
Thx On Mon, Oct 8, 2012 at 10:26 AM, Mona Chitnis <[email protected]> wrote: > Amelia, > > No it is not supposed to be shutting down. There are another set of logs from > tomcat sys out. Can you check either logs/catalina.out and > logs/localhost-<date>.log? > > -- > Mona Chitnis > > From: Amelia Jones <[email protected]<mailto:[email protected]>> > To: "[email protected]<mailto:[email protected]>" > <[email protected]<mailto:[email protected]>> > Cc: Mona Chitnis <[email protected]<mailto:[email protected]>>, > "[email protected]<mailto:[email protected]>" > <[email protected]<mailto:[email protected]>> > Subject: Re: Oozie and Datastax > > Further to this log, after the warning about the native-hadoop library we > have this: > > 2012-10-08 13:40:45,064 DEBUG DagEngine:545 - USER[amelia] GROUP[-] TOKEN[-] > APP[-] JOB[-] ACTION[-] Oozie DagEngine is using XCommands. > 2012-10-08 13:40:45,066 DEBUG SubmitXCommand:545 - USER[amelia] GROUP[-] > TOKEN[-] APP[-] JOB[-] ACTION[-] Load state for [null] > 2012-10-08 13:40:45,066 DEBUG SubmitXCommand:545 - USER[amelia] GROUP[-] > TOKEN[-] APP[-] JOB[-] ACTION[-] Precondition check for command [submit] key > [null] > 2012-10-08 13:40:45,066 DEBUG SubmitXCommand:545 - USER[amelia] GROUP[-] > TOKEN[-] APP[-] JOB[-] ACTION[-] Execute command [submit] key [null] > 2012-10-08 13:40:48,213 INFO Services:539 - Shutdown > > That is after attempting to run a job, which still hasn't returned. Nor does > the server respond to 'bin/oozie admin -ooziehttp://localhost:11000/oozie > -status'. The last line concerns me. Is it supposed to be shutting down? > > On 8 October 2012 10:07, Amelia Jones > <[email protected]<mailto:[email protected]>> wrote: > Thanks, I didn't notice the duplicates. I've removed them all (I think): > > ant-1.6.5.jar > antlr-3.2.jar > antlr-runtime-3.2.jar > avro-1.4.0-cassandra-1.jar > cassandra-all-1.0.10.jar > cassandra-clientutil-1.0.10.jar > cassandra-jdbc-1.0.5-20120514.145502-2.jar > cassandra-thrift-1.0.10.jar > commons-beanutils-1.7.0.jar > commons-beanutils-core-1.8.0.jar > commons-cli-1.2.jar > commons-codec-1.4.jar > commons-collections-3.2.1.jar > commons-configuration-1.6.jar > commons-digester-1.8.jar > commons-el-1.0.jar > commons-httpclient-3.0.1.jar > commons-lang-2.4.jar > commons-logging-1.1.1.jar > commons-math-2.1.jar > commons-net-1.4.1.jar > compress-lzf-0.8.4.jar > concurrentlinkedhashmap-lru-1.2.jar > core-3.1.1.jar > dse-2.1.jar > ext-2.2.zip > ftplet-api-1.0.0.jar > ftpserver-core-1.0.0.jar > ftpserver-deprecated-1.0.0-M2.jar > guava-r08.jar > hadoop-core-1.0.2-dse-20120522.195802-4.jar > hadoop-examples-1.0.2-dse-20120522.195802-4.jar > hadoop-fairscheduler-0.20.203.1-brisk1.jar > hadoop-streaming-1.0.2-dse-20120522.195802-4.jar > hadoop-test-1.0.2-dse-20120522.195802-4.jar > hadoop-tools-1.0.2-dse-20120522.195802-4.jar > high-scale-lib-1.1.2.jar > hsqldb-1.8.0.10.jar > httpclient-4.0.1.jar > httpcore-4.0.1.jar > jackson-core-asl-1.4.0.jar > jackson-mapper-asl-1.4.0.jar > jamm-0.2.5.jar > jasper-compiler-5.5.12.jar > jasper-runtime-5.5.12.jar > jets3t-0.7.1.jar > jetty-6.1.26.jar > jetty-util-6.1.26.jar > jline-0.9.94.jar > joda-time-1.6.2.jar > json-simple-1.1.jar > jsp-2.1-6.1.14.jar > jsp-api-2.1-6.1.14.jar > kfs-0.3.jar > libthrift-0.6.1.jar > log4j-1.2.16.jar > mina-core-2.0.0-M5.jar > oro-2.0.8.jar > servlet-api-2.5-6.1.14.jar > slf4j-api-1.6.1.jar > snakeyaml-1.6.jar > snappy-java-1.0.4.1.jar > stringtemplate-3.2.jar > xmlenc-0.52.jar > > > Re-ran setup.sh, still hanging when trying to run a job. I'm getting some > output in the log now though! > 2012-10-08 10:01:28,853 WARN AuthenticationFilter:381 - Authentication > exception: AuthenticationToken expired > org.apache.hadoop.security.authentication.client.AuthenticationException: > AuthenticationToken expired > at > org.apache.hadoop.security.authentication.server.AuthenticationFilter.getToken(AuthenticationFilter.java:310) > at > org.apache.hadoop.security.authentication.server.AuthenticationFilter.doFilter(AuthenticationFilter.java:334) > at org.apache.oozie.servlet.AuthFilter.doFilter(AuthFilter.java:131) > at > org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235) > at > org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) > at > org.apache.oozie.servlet.HostnameFilter.doFilter(HostnameFilter.java:67) > at > org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235) > at > org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) > at > org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233) > at > org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191) > at > org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127) > at > org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102) > at > org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109) > at > org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:298) > at > org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:859) > at > org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:588) > at > org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:489) > at java.lang.Thread.run(Thread.java:679) > 2012-10-08 10:01:28,853 WARN AuthenticationFilter:381 - Authentication > exception: AuthenticationToken expired > org.apache.hadoop.security.authentication.client.AuthenticationException: > AuthenticationToken expired > at > org.apache.hadoop.security.authentication.server.AuthenticationFilter.getToken(AuthenticationFilter.java:310) > at > org.apache.hadoop.security.authentication.server.AuthenticationFilter.doFilter(AuthenticationFilter.java:334) > at org.apache.oozie.servlet.AuthFilter.doFilter(AuthFilter.java:131) > at > org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235) > at > org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) > at > org.apache.oozie.servlet.HostnameFilter.doFilter(HostnameFilter.java:67) > at > org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235) > at > org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) > at > org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233) > at > org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191) > at > org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127) > at > org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102) > at > org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109) > at > org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:298) > at > org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:859) > at > org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:588) > at > org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:489) > at java.lang.Thread.run(Thread.java:679) > > > 2012-10-08 10:01:28,882 DEBUG AuthenticationFilter:337 - Request > [http://localhost:11000/oozie/versions] triggering authentication > 2012-10-08 10:01:28,882 DEBUG AuthenticationFilter:337 - Request > [http://localhost:11000/oozie/versions] triggering authentication > 2012-10-08 10:01:28,884 DEBUG AuthenticationFilter:347 - Request > [http://localhost:11000/oozie/versions] user [null] authenticated > 2012-10-08 10:01:28,884 DEBUG AuthenticationFilter:347 - Request > [http://localhost:11000/oozie/versions] user [null] authenticated > 2012-10-08 10:01:28,895 DEBUG AuthenticationFilter:337 - Request > [http://localhost:11000/oozie/versions?user.name=amelia] triggering > authentication > 2012-10-08 10:01:28,895 DEBUG AuthenticationFilter:337 - Request > [http://localhost:11000/oozie/versions?user.name=amelia] triggering > authentication > 2012-10-08 10:01:28,896 DEBUG AuthenticationFilter:347 - Request > [http://localhost:11000/oozie/versions?user.name=amelia] user [amelia] > authenticated > 2012-10-08 10:01:28,896 DEBUG AuthenticationFilter:347 - Request > [http://localhost:11000/oozie/versions?user.name=amelia] user [amelia] > authenticated > 2012-10-08 10:01:28,900 DEBUG AuthenticationFilter:347 - Request > [http://localhost:11000/oozie/versions] user [amelia] authenticated > 2012-10-08 10:01:28,900 DEBUG AuthenticationFilter:347 - Request > [http://localhost:11000/oozie/versions] user [amelia] authenticated > 2012-10-08 10:01:28,932 DEBUG AuthenticationFilter:347 - Request > [http://localhost:11000/oozie/v1/jobs?action=start] user [amelia] > authenticated > 2012-10-08 10:01:28,932 DEBUG AuthenticationFilter:347 - Request > [http://localhost:11000/oozie/v1/jobs?action=start] user [amelia] > authenticated > 2012-10-08 10:01:29,151 DEBUG AuthenticationFilter:347 - Request > [http://localhost:11000/oozie/v1/jobs?action=start] user [amelia] > authenticated > 2012-10-08 10:01:29,151 DEBUG AuthenticationFilter:347 - Request > [http://localhost:11000/oozie/v1/jobs?action=start] user [amelia] > authenticated > 2012-10-08 10:01:29,625 WARN NativeCodeLoader:52 - Unable to load > native-hadoop library for your platform... using builtin-java classes where > applicable > > > I'm not sure if the problem here is the authentication or the native-hadoop > library? > > > On 8 October 2012 09:47, Edward Capriolo > <[email protected]<mailto:[email protected]>> wrote: > It you are seeing a lib folder with two versions of the same jar you > are likely to have very bad results. It means that classes are built > against conflicting libraries, since java has dynamic binding > sometimes you are ok, but usually this causes issues. > > -- Alejandro
