Hi Abe - Thanks for the response. I've updated my sqoop-env.sh file to have the following export vars
export HADOOP_HOME="/usr/local/Cellar/hadoop/2.4.0/libexec" export HBASE_HOME="/usr/local/Cellar/hbase/0.98.1/libexec" export HIVE_HOME="/usr/local/Cellar/hive/0.13.0/libexec" export ZOOCFGDIR="/usr/local/etc/zookeeper" export HCAT_HOME="/usr/local/Cellar/hive/0.13.0/libexec/hcatalog" and to answer your question the hbase directory has the following file / directories CHANGES.txt LICENSE.txt README.txt libexec INSTALL_RECEIPT.json NOTICE.txt bin Now, when I try and run my import I get a new error (and the hbase table is actually created, just no data going into it) The error is Exception in thread "main" java.lang.IncompatibleClassChangeError: Found interface org.apache.hadoop.mapreduce.JobContext, but class was expected at org.apache.sqoop.mapreduce.DelegatingOutputFormat.checkOutputSpecs(DelegatingOutputFormat.java:63) at org.apache.hadoop.mapreduce.JobSubmitter.checkSpecs(JobSubmitter.java:458) at org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:343) at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1285) at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1282) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:396) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1548) at org.apache.hadoop.mapreduce.Job.submit(Job.java:1282) at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:1303) at org.apache.sqoop.mapreduce.ImportJobBase.doSubmitJob(ImportJobBase.java:186) at org.apache.sqoop.mapreduce.ImportJobBase.runJob(ImportJobBase.java:159) at org.apache.sqoop.mapreduce.ImportJobBase.runImport(ImportJobBase.java:239) at org.apache.sqoop.manager.SqlManager.importQuery(SqlManager.java:645) a I've read this is because of incompatible jar files but, like I said I'm a complete newb at all of this and if I could get a little more direction on what exactly I'm missing / is incompatible that would be fantastic. This is all on homebrew mac os x 10.9 hadoop 2.4 hbase .98 sqoop 1.4.4 On Thu, Jun 5, 2014 at 2:14 AM, Abraham Elmahrek <[email protected]> wrote: > Hey there, > > What does the contents of the directory '/usr/local/Cellar/hbase/0.98.1' > look like? The command 'find /usr/local/Cellar/hbase/0.98.1' should > assist in figuring that out. Please paste the result of that command here. > > The environment variables listed above are used in many different areas of > Hadoop. I've typically seen them in startup scripts to resolve libraries > and binaries. In this case, it's used to include jars in the classpath when > you call Sqoop. > > Are you able to import to HDFS? > > -Abe > > > On Wed, Jun 4, 2014 at 6:59 PM, Josh Millstein <[email protected] > > wrote: > >> I've got hadoop / hbase / hive / set up and running (can create files on >> hdfs, run map reduce jobs, create a "table" in hbase and also in hive) on >> my mac with osx 10.9. I'm now trying to import data from a mysql table into >> sqoop (using query, not table-name etc). I am getting this error with this >> command. >> >> --COMMAND >> sqoop import --connect jdbc:mysql:///joshLocal --username root --query >> "SELECT * FROM BITLOG WHERE \$CONDITIONS" --split-by oozie_job.id >> --hbase-table bitlogTest --hbase-create- table --column-family >> bitLogColumn >> --ERROR >> ERROR tool.ImportTool: Error during import: HBase jars are not present in >> classpath, cannot import to HBase! >> >> I believe that all the export vars are correctly setup. I have the >> following in swoop-env.sh >> >> export HADOOP_HOME="/usr/local/Cellar/hadoop/2.4.0" >> export HBASE_HOME="/usr/local/Cellar/hbase/0.98.1" >> export HIVE_HOME="/usr/local/Cellar/hive/0.13.0" >> export ZOOCFGDIR="/usr/local/etc/zookeeper" >> export HCAT_HOME="/usr/local/Cellar/hive/0.13.0/libexec/hcatalog" >> >> one thing I did that gave a different message was to change the hbase >> home to point to HBASE_HOME/libexec/lib in swoop-env.sh. That gave me a >> >> Exception in thread "main" java.lang.NoClassDefFoundError: >> org/apache/hadoop/util/Tool >> at java.lang.ClassLoader.defineClass1(Native Method) >> at java.lang.ClassLoader.defineClassCond(ClassLoader.java:631) >> at java.lang.ClassLoader.defineClass(ClassLoader.java:615) >> at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:141) >> >> error. I have seen some advice given that says I need to copy over >> hadoop security jar files to hbase's installation. I don't exactly know >> what files need to go over and if that is even an issue. The only reason I >> thought it might be is because java.security.SecureClassLoader is in the >> stacktrace. >> >> I'm sorry if this is a really basic java question but I'm a complete >> novice with it. >> >> One other even more basic java question. When we define HADOOP_HOME >> HBASE_HOME etc., what are we "telling" the other java programs that rely on >> that info. Are we saying "here is the executable java file" or are we >> saying "here are the jar file in the lib folder" . I don't quite understand >> what I should actually be pointing to because I don't know how that path is >> used. >> > >
