Yes it should but it is more of a hack :). I see no reasons why libjars option wouldn't work. And yes besides using libjars option, the jdbc driver also needs to be in the class path as sqoop will use it for fetching metadata in order to determine the splits.
Thanks, Anirudh Sent from my iPhone On Apr 23, 2012, at 11:57, Cheolsoo Park <[email protected]> wrote: > Hi Dipesh, > > I was looking for such an alternative because currently i don't have > permissions to add jars in Sqoop/lib. > > OK, that's unfortunate. > > If none of these suggestions is feasible for your case, there is the last > thing that you can do: install Sqoop from the tarball in a directory where > you have full permissions and use it instead. As long as HADOOP_HOME is set > to the installed hadoop, everything will work. > > Thanks, > Cheolsoo > > > On Mon, Apr 23, 2012 at 11:17 AM, Anirudh <[email protected]> wrote: > Dipesh, > > It would be ideal if the jdbc driver is placed under Sqoop/lib as other users > too would be able to leverage the same. But given your situation, use the > -libjars option to point to the driver. > > Hope this helps. > > Thanks, > Anirudh > > > On Mon, Apr 23, 2012 at 11:01 AM, DIPESH KUMAR SINGH <[email protected]> > wrote: > Hi Cheolsoo, > > Yeah, I know. I was looking for an alternative way by which we could tell > Sqoop to load jars from a location different than Sqoop/lib. I was looking > for such an alternative because currently i don't have > permissions to add jars in Sqoop/lib, though i have initiated the process. > > Thanks & Regards, > > Dipesh > > > > On Mon, Apr 23, 2012 at 11:13 PM, Cheolsoo Park <[email protected]> wrote: > Hi Dipesh, > > 12/04/23 01:02:42 ERROR sqoop.Sqoop: Got exception running Sqoop: > java.lang.RuntimeException: Could not load db driver class: > com.teradata.jdbc.TeraDriver > java.lang.RuntimeException: Could not load db driver class: > com.teradata.jdbc.TeraDrive > > Your Teradata jdbc jar is not found by Sqoop. The easiest way to fix it is to > put terajdbc4.jar and tdgssconfig.jar in /usr/lib/sqoop/lib. > > Thanks, > Cheolsoo > > On Mon, Apr 23, 2012 at 10:32 AM, DIPESH KUMAR SINGH <[email protected]> > wrote: > Hi Cheolsoo, > > As you suggested, I tried after removing "TeraDriver" line from connectors > file but that didn't worked > too. Please help..!! > > Here is the stack trace: > > 12/04/23 01:02:35 DEBUG tool.BaseSqoopTool: Enabled debug logging. > Enter password: > 12/04/23 01:02:42 DEBUG util.ClassLoaderStack: Checking for existing class: > com.cloudera.sqoop.manager.TeradataManagerFactory > 12/04/23 01:02:42 DEBUG util.ClassLoaderStack: Attempting to load jar through > URL: jar:file:///ms/user/a/sqoop-user/Sqoop/sqoop-td-connector-1.0.5.jar!/ > 12/04/23 01:02:42 DEBUG util.ClassLoaderStack: Previous classloader is > java.net.URLClassLoader@27b15692 > 12/04/23 01:02:42 DEBUG util.ClassLoaderStack: Testing class in jar: > com.cloudera.sqoop.manager.TeradataManagerFactory > 12/04/23 01:02:42 DEBUG util.ClassLoaderStack: Loaded jar into current JVM: > jar:file:///ms/user/a/sqoop-user/Sqoop/sqoop-td-connector-1.0.5.jar!/ > 12/04/23 01:02:42 DEBUG util.ClassLoaderStack: Added classloader for jar > /ms/user/a/sqoop-user/Sqoop/sqoop-td-connector-1.0.5.jar: > java.net.FactoryURLClassLoader@19e3cd51 > 12/04/23 01:02:42 DEBUG sqoop.ConnFactory: Added factory > com.cloudera.sqoop.manager.TeradataManagerFactory in jar > /ms/user/a/sqoop-user/Sqoop/sqoop-td-connector-1.0.5.jar specified by > /ms/user/a/sqoop-user/Sqoop/conf/managers.d/connectors > 12/04/23 01:02:42 DEBUG sqoop.ConnFactory: Loaded manager factory: > com.cloudera.sqoop.manager.TeradataManagerFactory > 12/04/23 01:02:42 DEBUG sqoop.ConnFactory: Loaded manager factory: > com.cloudera.sqoop.manager.DefaultManagerFactory > 12/04/23 01:02:42 DEBUG sqoop.ConnFactory: Trying ManagerFactory: > com.cloudera.sqoop.manager.TeradataManagerFactory > 12/04/23 01:02:42 DEBUG manager.TeradataManagerFactory: Trying with scheme: > jdbc:teradata: > 12/04/23 01:02:42 INFO manager.SqlManager: Using default fetchSize of 1000 > 12/04/23 01:02:42 DEBUG sqoop.ConnFactory: Instantiated ConnManager > com.cloudera.sqoop.teradata.TeradataManager@220ca470 > 12/04/23 01:02:42 INFO tool.CodeGenTool: Beginning code generation > 12/04/23 01:02:42 ERROR sqoop.Sqoop: Got exception running Sqoop: > java.lang.RuntimeException: Could not load db driver class: > com.teradata.jdbc.TeraDriver > java.lang.RuntimeException: Could not load db driver class: > com.teradata.jdbc.TeraDriver > at com.cloudera.sqoop.manager.SqlManager.makeConnection(SqlManager.java:634) > at > com.cloudera.sqoop.manager.GenericJdbcManager.getConnection(GenericJdbcManager.java:51) > at com.cloudera.sqoop.manager.SqlManager.execute(SqlManager.java:456) > at com.cloudera.sqoop.manager.SqlManager.execute(SqlManager.java:479) > at > com.cloudera.sqoop.manager.SqlManager.getColumnTypesForRawQuery(SqlManager.java:192) > at com.cloudera.sqoop.manager.SqlManager.getColumnTypes(SqlManager.java:176) > at com.cloudera.sqoop.orm.ClassWriter.generate(ClassWriter.java:1017) > at com.cloudera.sqoop.tool.CodeGenTool.generateORM(CodeGenTool.java:82) > at com.cloudera.sqoop.tool.ImportTool.importTable(ImportTool.java:337) > at com.cloudera.sqoop.tool.ImportTool.run(ImportTool.java:423) > at com.cloudera.sqoop.Sqoop.run(Sqoop.java:144) > at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65) > at com.cloudera.sqoop.Sqoop.runSqoop(Sqoop.java:180) > at com.cloudera.sqoop.Sqoop.runTool(Sqoop.java:219) > at com.cloudera.sqoop.Sqoop.runTool(Sqoop.java:228) > at com.cloudera.sqoop.Sqoop.main(Sqoop.java:237) > > > Thanks, > > Dipesh > > > > On Sun, Apr 22, 2012 at 3:01 PM, Cheolsoo Park <[email protected]> wrote: > Hi Dispesh, > > I added the line com.teradata.jdbc.TeraDriver=<path to jar file > terajdbc4.jar> along with > com.cloudera.sqoop.manager.TeradataManagerFactory=<path to > sqoop-td-connector-1.0.5.jar> > in connectors file inside managers.d directory. > > Please remove "com.teradata.jdbc.TeraDriver=<path to jar file terajdbc4.jar>" > from your connector file. > > It should only contain > "com.cloudera.sqoop.manager.TeradataManagerFactory=<path to > sqoop-td-connector-1.0.5.jar>". > > Thanks, > Cheolsoo > > > On Sat, Apr 21, 2012 at 10:21 PM, DIPESH KUMAR SINGH <[email protected]> > wrote: > I am trying to import the data from Teradata to hdfs referring Cloudera > Connector guide. I added the property > > (The Hadoop environment i am working in is a shared Hadoop installation, So > users are not allowed to > add any jars in lib directory or make changes in Sqoop-site.xml of central > installation. > However, users can use .profile i.e. user's configuration files to specify > CLASSPATH ) > > I added the line com.teradata.jdbc.TeraDriver=<path to jar file > terajdbc4.jar> along with > com.cloudera.sqoop.manager.TeradataManagerFactory=<path to > sqoop-td-connector-1.0.5.jar> > in connectors file inside managers.d directory. In my CLASSPATH, i have added > path to > SQOOP_CONF i.e. a Sqoop folder where i have kept my managers.d and all jars. > > > With these I suppose i am able to connect to teradata using the teradata > manager factory using the teradriver as well > but, still i am facing this when i issue the sqoop import command as below > > sqoop import > -libjars <path to tdgssconfig.jar> > --driver com.teradata.jdbc.TeraDriver > --connect jdbc:teradata://xxxxxx/DATABASE=dbname > --username hadoop --P > --table SQOOP_TEST > --num-mappers 3 --verbose > > 12/04/13 02:29:32 DEBUG tool.BaseSqoopTool: Enabled debug logging. > 12/04/13 02:29:32 WARN tool.BaseSqoopTool: Setting your password on the > command-line is insecure. Consider using -P instead. > 12/04/13 02:29:32 DEBUG util.ClassLoaderStack: Checking for existing class: > com.cloudera.sqoop.manager.TeradataManagerFactory > 12/04/13 02:29:32 DEBUG util.ClassLoaderStack: Attempting to load jar through > URL: jar:file:///Sqoop/sqoop-td-connector-1.0.5.jar!/ > 12/04/13 02:29:32 DEBUG util.ClassLoaderStack: Previous classloader is > java.net.URLClassLoader@6a8c436b > 12/04/13 02:29:32 DEBUG util.ClassLoaderStack: Testing class in jar: > com.cloudera.sqoop.manager.TeradataManagerFactory > 12/04/13 02:29:32 DEBUG util.ClassLoaderStack: Loaded jar into current JVM: > jar:file:///Sqoop/sqoop-td-connector-1.0.5.jar!/ > 12/04/13 02:29:32 DEBUG util.ClassLoaderStack: Added classloader for jar > /Sqoop/sqoop-td-connector-1.0.5.jar: java.net.FactoryURLClassLoader@45db05b2 > 12/04/13 02:29:32 DEBUG sqoop.ConnFactory: Added factory > com.cloudera.sqoop.manager.TeradataManagerFactory in jar > /Sqoop/sqoop-td-connector-1.0.5.jar specified by > /Sqoop/conf/managers.d/connectors > 12/04/13 02:29:32 DEBUG util.ClassLoaderStack: Checking for existing class: > com.teradata.jdbc.TeraDriver > 12/04/13 02:29:32 DEBUG util.ClassLoaderStack: Attempting to load jar through > URL: jar:file:///Sqoop/lib/terajdbc4.jar!/ > 12/04/13 02:29:32 DEBUG util.ClassLoaderStack: Previous classloader is > java.net.FactoryURLClassLoader@45db05b2 > 12/04/13 02:29:32 DEBUG util.ClassLoaderStack: Testing class in jar: > com.teradata.jdbc.TeraDriver > 12/04/13 02:29:32 DEBUG util.ClassLoaderStack: Loaded jar into current JVM: > jar:file:///Sqoop/lib/terajdbc4.jar!/ > 12/04/13 02:29:32 DEBUG util.ClassLoaderStack: Added classloader for jar > /Sqoop/lib/terajdbc4.jar: java.net.FactoryURLClassLoader@518f5824 > 12/04/13 02:29:32 DEBUG sqoop.ConnFactory: Added factory > com.teradata.jdbc.TeraDriver in jar /Sqoop/lib/terajdbc4.jar specified by > /Sqoop/conf/managers.d/connectors > 12/04/13 02:29:32 DEBUG sqoop.ConnFactory: Loaded manager factory: > com.cloudera.sqoop.manager.TeradataManagerFactory > 12/04/13 02:29:32 ERROR tool.BaseSqoopTool: Got error creating database > manager: java.lang.ClassCastException: com.teradata.jdbc.TeraDriver cannot be > cast to com.cloudera.sqoop.manager.ManagerFactory > at com.cloudera.sqoop.ConnFactory.instantiateFactories(ConnFactory.java:90) > at com.cloudera.sqoop.ConnFactory.<init>(ConnFactory.java:60) > at com.cloudera.sqoop.tool.BaseSqoopTool.init(BaseSqoopTool.java:186) > at com.cloudera.sqoop.tool.ImportTool.init(ImportTool.java:81) > at com.cloudera.sqoop.tool.ImportTool.run(ImportTool.java:411) > at com.cloudera.sqoop.Sqoop.run(Sqoop.java:144) > at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65) > at com.cloudera.sqoop.Sqoop.runSqoop(Sqoop.java:180) > at com.cloudera.sqoop.Sqoop.runTool(Sqoop.java:219) > at com.cloudera.sqoop.Sqoop.runTool(Sqoop.java:228) > at com.cloudera.sqoop.Sqoop.main(Sqoop.java:237) > > > Please help. > > Thanks & Regards, > -- > Dipesh Kr. Singh > > > > > > > > > -- > Dipesh Kr. Singh > > > > > > > > > -- > Dipesh Kr. Singh > > > > > >
