Don't use hadoop -config ... Use sqoop import -libjars .... And I am not sure why you need libjars option. For your purpose, copy mysql connector jarfile to $SQOOP_HOME/lib and just run sqoop import
Venkat On Thu, Jun 6, 2013 at 5:56 AM, Jane Wayne <[email protected]> wrote: > i am running the Sqoop tool as follows. > > export > LIBJARS=sqoop-1.4.3.jar,mysql-connector-java-5.1.25-bin.jar,ant-contrib-1.0b3.jar,ant-eclipse-1.0-jvm1.2.jar,avro-1.5.3.jar,avro-ipc-1.5.3.jar,avro-mapred-1.5.3.jar,commons-io-1.4.jar,hsqldb-1.8.0.10.jar,ivy-2.1.0.jar,jackson-core-asl-1.7.3.jar,jackson-mapper-asl-1.7.3.jar,jopt-simple-3.2.jar,paranamer-2.3.jar,snappy-java-1.0.3.2.jar > > hadoop --config path/to/sqoop-cluster-config-dir jar sqoop-1.4.3.jar > org.apache.sqoop.Sqoop import -libjars ${LIBJARS} --connect > jdbc:mysql://mysql/mydb --username myuser --password mypass --table data > --verbose --append --warehouse-dir /sqoop > > i am getting this stack trace. > > java.lang.RuntimeException: java.lang.ClassNotFoundException: > org.apache.sqoop.mapreduce.RawKeyTextOutputFormat > at > org.apache.hadoop.conf.Configuration.getClass(Configuration.java:996) > at > org.apache.hadoop.mapreduce.JobContext.getOutputFormatClass(JobContext.java:248) > at org.apache.hadoop.mapred.Task.initialize(Task.java:501) > at org.apache.hadoop.mapred.MapTask.run(MapTask.java:306) > at org.apache.hadoop.mapred.Child$4.run(Child.java:270) > at java.security.AccessController.doPrivileged(Native Method) > at javax.security.auth.Subject.doAs(Subject.java:396) > at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1127) > at org.apache.hadoop.mapred.Child.main(Child.java:264) > Caused by: java.lang.ClassNotFoundException: > org.apache.sqoop.mapreduce.RawKeyTextOutputFormat > at java.net.URLClassLoader$1.run(URLClassLoader.java:202) > at java.security.AccessController.doPrivileged(Native Method) > at java.net.URLClassLoader.findClass(URLClassLoader.java:190) > > when i looked at the sqoop-1.4.3.jar, clearly, > org.apache.sqoop.mapreduce.RawKeyTextOutputFormat.class exists. > > any ideas what's going on? > > > > > On Thu, Jun 6, 2013 at 1:54 AM, Venkat Ranganathan < > [email protected]> wrote: > >> sqoop tools implement the ToolRunner interface. So, you can pass >> -libjars to sqoop (but make sure it is the first option among the list of >> the options >> >> For example, sqoop import -libjars $LIBJARS other options >> >> Venkat >> >> >> On Wed, Jun 5, 2013 at 5:23 PM, Jane Wayne <[email protected]>wrote: >> >>> hi, >>> >>> i am wondering if i can run the org.apache.sqoop.Sqoop tool like other >>> tools from the command line using the -libjars flag/option/parameter? >>> >>> i have tried the following. >>> >>> export >>> LIBJARS=ant-contrib-1.0b3.jar,ant-eclipse-1.0-jvm1.2.jar,avro-1.5.3.jar,avro-ipc-1.5.3.jar,avro-mapred-1.5.3.jar,commons-io-1.4.jar,hsqldb-1.8.0.10.jar,ivy-2.1.0.jar,jackson-core-asl-1.7.3.jar,jackson-mapper-asl-1.7.3.jar,jopt-simple-3.2.jar,paranamer-2.3.jar,snappy-java-1.0.3.2.jar >>> >>> hadoop --config path/to/sqoop-cluster-config-dir jar sqoop-1.4.3.jar >>> org.apache.sqoop.Sqoop -libjars ${LIBJARS} import --connect >>> jdbc:mysql://mysql/mydb --username myuser --password mypass --table data >>> --verbose --append --warehouse-dir /sqoop >>> >>> what i get is the message below. >>> >>> No such sqoop tool: -libjars. See 'sqoop help'. >>> >>> obviously, this problem has something to do with sqoop's parsing of the >>> command line options. shouldn't sqoop allow hadoop's command line options >>> and its own at the same time? >>> >>> in the meanwhile, i will try to tinker a bit with that Sqoop class to >>> see if i can get it to work with -libjars. >>> >>> references that i found regarding -libjars. >>> 1. http://grepalex.com/2013/02/25/hadoop-libjars/ >>> 2. >>> http://blog.cloudera.com/blog/2011/01/how-to-include-third-party-libraries-in-your-map-reduce-job/ >>> >>> thanks, >>> >> >> >
