That is the first thing I checked - jar is accessible via command. I guess this is some classpath problem but isn't obvious - the same settings are working in another single node cluster.
Regards, Omkar Joshi From: Babu, Prashanth [mailto:[email protected]] Sent: Tuesday, September 10, 2013 12:55 PM To: [email protected] Subject: RE: Weird issue during HBase import Looks like none of your jars are proper or the current user does not have permissions. error: error reading /home/bigdata/hadoop-1.0.2/libexec/../lib/commons-io-1.3.2.jar; cannot read zip file Can you try accessing this file by running ls -l /home/bigdata/hadoop-1.0.2/libexec/../lib/commons-io-1.3.2.jar Or jar -tvf /home/bigdata/hadoop-1.0.2/libexec/../lib/commons-io-1.3.2.jar. From: Omkar Joshi [mailto:[email protected]] Sent: Tuesday, September 10, 2013 12:36 PM To: [email protected] Subject: Weird issue during HBase import I have used Sqoop several times before but now I'm facing a weird issue on a 3-node Hadoop/Hbase cluster. The .bashrc/.profile file entries are as follows : export HADOOP_HOME=/home/bigdata/hadoop-1.0.2 # Set JAVA_HOME export JAVA_HOME=/home/bigdata/jdk1.7.0_21 export PATH=$PATH:$JAVA_HOME/bin # Add Hadoop bin/ directory to PATH export PATH=$PATH:$HADOOP_HOME/bin export HADOOP_CONF_DIR=/home/bigdata/hdconf #HBase export HBASE_HOME=/home/bigdata/hadoop_ecosystem/apache_hbase/hbase-0.94.11 export PATH=$PATH:$HBASE_HOME/bin export SQOOP_HOME=/home/bigdata/hadoop_ecosystem/apache_sqoop/sqoop-1.4.3.bin__hadoop-1.0.0 export PATH=$PATH:$SQOOP_HOME/bin But when I try to import a MySQL table in HBase, I get this : bigdata@cldx-1230-1116:~$ sqoop import --connect jdbc:mysql://172.17.89.231/sentimizer_user_db --username root --password root --table nounphrasesentries --hbase-table nounphrasesentries --column-family word_attributes --hbase-create-table -m 1 Warning: $HADOOP_HOME is deprecated. 13/09/10 17:09:28 WARN tool.BaseSqoopTool: Setting your password on the command-line is insecure. Consider using -P instead. 13/09/10 17:09:29 INFO manager.MySQLManager: Preparing to use a MySQL streaming resultset. 13/09/10 17:09:29 INFO tool.CodeGenTool: Beginning code generation 13/09/10 17:09:32 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM `nounphrasesentries` AS t LIMIT 1 13/09/10 17:09:32 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM `nounphrasesentries` AS t LIMIT 1 13/09/10 17:09:32 INFO orm.CompilationManager: HADOOP_MAPRED_HOME is /home/bigdata/hadoop-1.0.2 error: error reading /home/bigdata/hadoop-1.0.2/libexec/../lib/activemq-core-5.6.0.jar; cannot read zip file error: error reading /home/bigdata/hadoop-1.0.2/libexec/../lib/apache-mime4j-core-0.7.jar; cannot read zip file error: error reading /home/bigdata/hadoop-1.0.2/libexec/../lib/apache-mime4j-dom-0.7.jar; cannot read zip file error: error reading /home/bigdata/hadoop-1.0.2/libexec/../lib/asm-3.1.jar; cannot read zip file error: error reading /home/bigdata/hadoop-1.0.2/libexec/../lib/boilerpipe-1.1.0.jar; cannot read zip file error: error reading /home/bigdata/hadoop-1.0.2/libexec/../lib/commons-beanutils-1.8.0.jar; cannot read zip file error: error reading /home/bigdata/hadoop-1.0.2/libexec/../lib/commons-collections-2.1.1.jar; cannot read zip file error: error reading /home/bigdata/hadoop-1.0.2/libexec/../lib/commons-compress-1.3.jar; cannot read zip file error: error reading /home/bigdata/hadoop-1.0.2/libexec/../lib/commons-dbcp-1.4.jar; cannot read zip file error: error reading /home/bigdata/hadoop-1.0.2/libexec/../lib/commons-fileupload-1.2.1.jar; cannot read zip file error: error reading /home/bigdata/hadoop-1.0.2/libexec/../lib/commons-io-1.3.2.jar; cannot read zip file error: error reading /home/bigdata/hadoop-1.0.2/libexec/../lib/commons-javaflow-20060411.jar; cannot read zip file error: error reading /home/bigdata/hadoop-1.0.2/libexec/../lib/commons-logging-1.1.1.jar; cannot read zip file error: error reading /home/bigdata/hadoop-1.0.2/libexec/../lib/freemarker-2.3.15.jar; cannot read zip file error: error reading /home/bigdata/hadoop-1.0.2/libexec/../lib/geronimo-stax-api_1.0_spec-1.0.1.jar; cannot read zip file error: error reading /home/bigdata/hadoop-1.0.2/libexec/../lib/gson-2.2.2.jar; cannot read zip file error: error reading /home/bigdata/hadoop-1.0.2/libexec/../lib/gson-2.2.2-javadoc.jar; cannot read zip file error: error reading /home/bigdata/hadoop-1.0.2/libexec/../lib/gson-2.2.2-sources.jar; cannot read zip file error: error reading /home/bigdata/hadoop-1.0.2/libexec/../lib/httpclient-4.1.2.jar; cannot read zip file error: error reading /home/bigdata/hadoop-1.0.2/libexec/../lib/httpcore-4.1.4.jar; cannot read zip file error: error reading /home/bigdata/hadoop-1.0.2/libexec/../lib/itextpdf-5.1.0.jar; cannot read zip file error: error reading /home/bigdata/hadoop-1.0.2/libexec/../lib/j2ee-1.4.jar; cannot read zip file error: error reading /home/bigdata/hadoop-1.0.2/libexec/../lib/jaws-bin.jar; cannot read zip file error: error reading /home/bigdata/hadoop-1.0.2/libexec/../lib/jcommon-1.0.15.jar; cannot read zip file error: error reading /home/bigdata/hadoop-1.0.2/libexec/../lib/je-4.0.92.jar; cannot read zip file error: error reading /home/bigdata/hadoop-1.0.2/libexec/../lib/jfreechart-1.0.0-rc1.jar; cannot read zip file error: error reading /home/bigdata/hadoop-1.0.2/libexec/../lib/jfreechart-1.0.12.jar; cannot read zip file error: error reading /home/bigdata/hadoop-1.0.2/libexec/../lib/jgraph.jar; cannot read zip file error: error reading /home/bigdata/hadoop-1.0.2/libexec/../lib/jsapi.jar; cannot read zip file error: error reading /home/bigdata/hadoop-1.0.2/libexec/../lib/jsch-0.1.49.jar; cannot read zip file error: error reading /home/bigdata/hadoop-1.0.2/libexec/../lib/js.jar; cannot read zip file error: error reading /home/bigdata/hadoop-1.0.2/libexec/../lib/junit.jar; cannot read zip file error: error reading /home/bigdata/hadoop-1.0.2/libexec/../lib/lingpipe-4.1.0.jar; cannot read zip file error: error reading /home/bigdata/hadoop-1.0.2/libexec/../lib/lucene-analyzers-3.1.0.jar; cannot read zip file error: error reading /home/bigdata/hadoop-1.0.2/libexec/../lib/lucene-core-3.6.0.jar; cannot read zip file error: error reading /home/bigdata/hadoop-1.0.2/libexec/../lib/metadata-extractor-2.4.0-beta-1.jar; cannot read zip file error: error reading /home/bigdata/hadoop-1.0.2/libexec/../lib/mysql-connector-java-5.0.8-bin.jar; cannot read zip file error: error reading /home/bigdata/hadoop-1.0.2/libexec/../lib/ognl-2.6.11.jar; cannot read zip file error: error reading /home/bigdata/hadoop-1.0.2/libexec/../lib/opennlp-maxent-3.0.2-incubating.jar; cannot read zip file error: error reading /home/bigdata/hadoop-1.0.2/libexec/../lib/opennlp-tools-1.5.2-incubating.jar; cannot read zip file error: error reading /home/bigdata/hadoop-1.0.2/libexec/../lib/opennlp-uima-1.5.2-incubating.jar; cannot read zip file error: error reading /home/bigdata/hadoop-1.0.2/libexec/../lib/poi-ooxml-3.6.jar; cannot read zip file error: error reading /home/bigdata/hadoop-1.0.2/libexec/../lib/signpost-core-1.1-SNAPSHOT.jar; cannot read zip file error: error reading /home/bigdata/hadoop-1.0.2/libexec/../lib/sphinx4.jar; cannot read zip file error: error reading /home/bigdata/hadoop-1.0.2/libexec/../lib/StemIt.jar; cannot read zip file error: error reading /home/bigdata/hadoop-1.0.2/libexec/../lib/struts-1.2.7.jar; cannot read zip file error: error reading /home/bigdata/hadoop-1.0.2/libexec/../lib/svnant.jar; cannot read zip file error: error reading /home/bigdata/hadoop-1.0.2/libexec/../lib/svnClientAdapter.jar; cannot read zip file error: error reading /home/bigdata/hadoop-1.0.2/libexec/../lib/tagsoup-1.2.1.jar; cannot read zip file error: error reading /home/bigdata/hadoop-1.0.2/libexec/../lib/tagsoup-1.2.jar; cannot read zip file error: error reading /home/bigdata/hadoop-1.0.2/libexec/../lib/TGGraphLayout.jar; cannot read zip file error: error reading /home/bigdata/hadoop-1.0.2/libexec/../lib/tika-core-0.10.jar; cannot read zip file error: error reading /home/bigdata/hadoop-1.0.2/libexec/../lib/tika-core-1.0.jar; cannot read zip file error: error reading /home/bigdata/hadoop-1.0.2/libexec/../lib/tika-parsers-0.10.jar; cannot read zip file error: error reading /home/bigdata/hadoop-1.0.2/libexec/../lib/tika-parsers-1.0.jar; cannot read zip file error: error reading /home/bigdata/hadoop-1.0.2/libexec/../lib/WSJ_8gau_13dCep_16k_40mel_130Hz_6800Hz.jar; cannot read zip file error: error reading /home/bigdata/hadoop-1.0.2/libexec/../lib/xmlunit1.0.jar; cannot read zip file error: error reading /home/bigdata/hadoop-1.0.2/libexec/../lib/xwork-2.1.2.jar; cannot read zip file Note: /tmp/sqoop-bigdata/compile/939747ca6dab7c25898cd5058e0ed0ab/nounphrasesentries.java uses or overrides a deprecated API. Note: Recompile with -Xlint:deprecation for details. Exception in thread "main" java.lang.NoSuchMethodError: org.apache.commons.io.FileUtils.moveFile(Ljava/io/File;Ljava/io/File;)V at org.apache.sqoop.orm.CompilationManager.compile(CompilationManager.java:228) at org.apache.sqoop.tool.CodeGenTool.generateORM(CodeGenTool.java:83) at org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:390) at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:476) at org.apache.sqoop.Sqoop.run(Sqoop.java:145) at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65) at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:181) at org.apache.sqoop.Sqoop.runTool(Sqoop.java:220) at org.apache.sqoop.Sqoop.runTool(Sqoop.java:229) at org.apache.sqoop.Sqoop.main(Sqoop.java:238) I'm clueless about why is Sqoop trying to pick a zip(?) file from the libexec folder of Hadoop? Regards, Omkar Joshi ________________________________ The contents of this e-mail and any attachment(s) may contain confidential or privileged information for the intended recipient(s). Unintended recipients are prohibited from taking action on the basis of information in this e-mail and using or disseminating the information, and must notify the sender and delete it from their system. L&T Infotech will not accept responsibility or liability for the accuracy or completeness of, or the presence of any virus or disabling code in this e-mail" ______________________________________________________________________ Disclaimer:This email and any attachments are sent in strictest confidence for the sole use of the addressee and may contain legally privileged, confidential, and proprietary data. If you are not the intended recipient, please advise the sender by replying promptly to this email and then delete and destroy this email and any attachments without any further use, copying or forwarding
