----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/38589/#review100815 -----------------------------------------------------------
My last round of testing on real cluster: dist/src/main/bin/sqoop.sh (line 27) <https://reviews.apache.org/r/38589/#comment158068> In the old Tomcat world [1] we had default of SQOOP_HOME/lib (masked as ${catalina.home}/../lib/*.jar) that allowed user to create lib/ directory, put random jars (mostly JDBC drivers) and inject them easily to Sqoop 2 server classpath. I'm wondering whether we can provide similar functionality? Let's perhaps define variable SQOOP_SERVER_EXTRA_LIB that defaults to "lib/" that enables admin to easily inject the jars? I can see the variable SQOOP_SERVER_LIB pointing to our own directory, but I would prefer user not to mess with our own directories and have a dedicated directory for "inserted" jars. It's better for upgrade as you can simply point the new version to the same "extra" directory without need to see what jars belongs to Sqoop and what were added and re-add them to the new version. Links: 1: https://github.com/apache/sqoop/blob/sqoop2/dist/src/main/server/conf/catalina.properties dist/src/main/bin/sqoop.sh (lines 28 - 31) <https://reviews.apache.org/r/38589/#comment158066> Thank you for incorporating my suggestion here! Might I suggest to be more defensive and verify that those variables are defined? I've tried running without them and I got this output: Sqoop home directory: /root/sqoop-2.0.0-SNAPSHOT-bin-hadoop200 PID: /tmp/sqoop-root-jetty-server.pid Starting the Sqoop2 server... Sqoop2 server started. [root@streamsets-1 sqoop-2.0.0-SNAPSHOT-bin-hadoop200]# 0 [main] INFO org.apache.sqoop.core.SqoopServer - Initializing Sqoop server. 33 [main] INFO org.apache.sqoop.core.PropertiesConfigurationProvider - Starting config file poller thread Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/hadoop/conf/Configuration at org.apache.sqoop.security.authentication.SimpleAuthenticationHandler.secureLogin(SimpleAuthenticationHandler.java:36) at org.apache.sqoop.security.AuthenticationManager.initialize(AuthenticationManager.java:98) at org.apache.sqoop.core.SqoopServer.initialize(SqoopServer.java:54) at org.apache.sqoop.server.SqoopJettyServer.<init>(SqoopJettyServer.java:50) at org.apache.sqoop.server.SqoopJettyServer.main(SqoopJettyServer.java:121) Caused by: java.lang.ClassNotFoundException: org.apache.hadoop.conf.Configuration at java.net.URLClassLoader$1.run(URLClassLoader.java:366) at java.net.URLClassLoader$1.run(URLClassLoader.java:355) at java.security.AccessController.doPrivileged(Native Method) at java.net.URLClassLoader.findClass(URLClassLoader.java:354) at java.lang.ClassLoader.loadClass(ClassLoader.java:425) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308) at java.lang.ClassLoader.loadClass(ClassLoader.java:358) ... 5 more Which is kind of confusing - the scripts exists with success even though that we immediately fail. dist/src/main/bin/sqoop.sh (lines 77 - 82) <https://reviews.apache.org/r/38589/#comment158061> From some reason on my cluster this method was returning "0" if the file didn't existed at all. I had to add a "return 1" at the end of the method to workaround it. dist/src/main/bin/sqoop.sh (line 141) <https://reviews.apache.org/r/38589/#comment158062> Can we add somewhere check that EXEC_JAVA is a valid command? I got this output of my cluster when "java" is not available on the path: [root@streamsets-1 sqoop-2.0.0-SNAPSHOT-bin-hadoop200]# ./bin/sqoop2-server start Sqoop home directory: /root/sqoop-2.0.0-SNAPSHOT-bin-hadoop200 PID: /tmp/sqoop-root-jetty-server.pid Starting the Sqoop2 server... Sqoop2 server started. [root@streamsets-1 sqoop-2.0.0-SNAPSHOT-bin-hadoop200]# /root/sqoop-2.0.0-SNAPSHOT-bin-hadoop200/bin/sqoop.sh: line 167: java: command not found echo $? 0 Seems very deceiving as the scripts reports "0" (e.g. success) and prints out "Sqoop2 server started" even though that the server didn't started at all. dist/src/main/bin/sqoop.sh (line 183) <https://reviews.apache.org/r/38589/#comment158067> This one I'm not clear about, but I'm looking for your thoughts. Should we also remove the the pid file when stopping the server? Jarcec - Jarek Cecho On Sept. 25, 2015, 7:10 a.m., Colin Ma wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/38589/ > ----------------------------------------------------------- > > (Updated Sept. 25, 2015, 7:10 a.m.) > > > Review request for Sqoop. > > > Repository: sqoop-sqoop2 > > > Description > ------- > > Update script for Jetty server > > > Diffs > ----- > > dist/src/main/bin/sqoop-sys.sh 64dd0bf > dist/src/main/bin/sqoop.sh 707c3fc > > Diff: https://reviews.apache.org/r/38589/diff/ > > > Testing > ------- > > > Thanks, > > Colin Ma > >
