Ok, Madhan, why not. Could you kindly provide you availabilities of time and communication tools. I will be more than happy to help you out with this.
Best, Mengwei On Thu, Jul 4, 2013 at 1:15 AM, Madhanmohan Savadamuthu <[email protected]>wrote: > After doing changes in catalina.properties also, same issue is coming. > > is there any possibility for interactive discussion on this issue? > > Regards, > Madhan > > On Wed, Jul 3, 2013 at 11:05 PM, Mengwei Ding > <[email protected]>wrote: > >> Thank you for your prompt response, sir. Please don't worry, I can help >> you out with this until your problem is done. >> >> Well, let's try out our new method of adding dependency jar files, and >> forget about the addtowar.sh script. >> >> Please following these instructions: >> >> " >> Installing Dependencies >> >> Hadoop libraries must be available on node where you are planning to run >> Sqoop server with proper configuration for major services - NameNode and >> either JobTracker or ResourceManager depending whether you are running >> Hadoop 1 or 2. There is no need to run any Hadoop service on the same node >> as Sqoop server, just the libraries and configuration must be available. >> >> Path to Hadoop libraries is stored in file catalina.properties inside >> directory server/conf. You need to change property called common.loader to >> contain all directories with your Hadoop libraries. The default expected >> locations are /usr/lib/hadoop and /usr/lib/hadoop/lib/. Please check out >> the comments in the file for further description how to configure different >> locations. >> >> Lastly you might need to install JDBC drivers that are not bundled with >> Sqoop because of incompatible licenses. You can add any arbitrary Java jar >> file to Sqoop server by copying it into lib/ directory. You can create this >> directory if it do not exists already. >> " >> >> I can give you my configuration as an example. So in my >> catalina.properties file, I have the following line: >> >> * >> common.loader=${catalina.base}/lib,${catalina.base}/lib/*.jar,${catalina.home}/lib,${catalina.home}/lib/*.jar,${catalina.home}/../lib/*.jar,/usr/lib/hadoop/client-0.20/*.jar,/home/mengweid/Downloads/mysql-connector-java-5.1.25-bin.jar >> * >> >> The */usr/lib/hadoop/client-0.20/*.jar *is used to include all >> hadoop-related jars, and *mysql-connector-java-5.1.25-bin.jar *is used >> for JDBC driver. >> >> Please try this, and let me know whether it works. Thank you. >> >> Best, >> Mengwei >> >> >> On Wed, Jul 3, 2013 at 9:18 AM, Madhanmohan Savadamuthu < >> [email protected]> wrote: >> >>> I did deployment as sugggested in below thread. I am not able to >>> successfully use sqoop2. I am attaching the services log for your >>> references. >>> >>> I made sure that exact same set of JAR files in appropriate location and >>> also deleted sqoop folder before starting the sqoop server. >>> >>> *Error Message:* >>> Exception has occurred during processing command >>> Exception: com.sun.jersey.api.client.UniformInterfaceException Message: >>> GET http://<ipaddress>:12013/sqoop/version returned a response status >>> of 404 Not Found >>> >>> Regards, >>> Madhan >>> >>> On Wed, Jul 3, 2013 at 7:30 PM, Mengwei Ding <[email protected] >>> > wrote: >>> >>>> Hi Madhanmohan, >>>> >>>> Thank you for providing all these detailed information. Help a lot to >>>> diagnose the problem. >>>> >>>> First, the addtowar.sh is not good enough for every situation, >>>> we apologize for that. We have already figured out a new way to add >>>> dependency library, which will coming out along with next version of >>>> Sqoop2. >>>> >>>> Currently, it seems like the hadoop-core.jar has not been added. I >>>> could show you all the libraries existing in the webapps/sqoop/WEB-INF/lib >>>> folder, please check below: >>>> avro-1.7.4.jar >>>> commons-cli-1.2.jar >>>> commons-configuration-1.6.jar >>>> commons-dbcp-1.4.jar >>>> commons-lang-2.5.jar >>>> commons-logging-1.1.1.jar >>>> commons-pool-1.5.4.jar >>>> derby-10.8.2.2.jar >>>> guava-11.0.2.jar >>>> hadoop-auth-2.0.0-cdh4.3.0.jar >>>> hadoop-common-2.0.0-cdh4.3.0.jar >>>> *hadoop-core-2.0.0-mr1-cdh4.3.0.jar* >>>> hadoop-hdfs-2.0.0-cdh4.3.0.jar >>>> hadoop-mapreduce-client-app-2.0.0-cdh4.3.0.jar >>>> hadoop-mapreduce-client-common-2.0.0-cdh4.3.0.jar >>>> hadoop-mapreduce-client-core-2.0.0-cdh4.3.0.jar >>>> hadoop-mapreduce-client-jobclient-2.0.0-cdh4.3.0.jar >>>> hadoop-yarn-api-2.0.0-cdh4.3.0.jar >>>> hadoop-yarn-common-2.0.0-cdh4.3.0.jar >>>> jackson-core-asl-1.8.8.jar >>>> jackson-mapper-asl-1.8.8.jar >>>> json-simple-1.1.jar >>>> log4j-1.2.16.jar >>>> mysql-connector-java-5.1.25-bin.jar >>>> protobuf-java-2.4.0a.jar >>>> slf4j-api-1.6.1.jar >>>> slf4j-log4j12-1.6.1.jar >>>> sqoop-common-1.99.2.jar >>>> sqoop-connector-generic-jdbc-1.99.2.jar >>>> sqoop-core-1.99.2.jar >>>> sqoop-execution-mapreduce-1.99.2-hadoop200.jar >>>> sqoop-repository-derby-1.99.2.jar >>>> sqoop-spi-1.99.2.jar >>>> sqoop-submission-mapreduce-1.99.2-hadoop200.jar >>>> >>>> I have the same Hadoop and Sqoop2 installation directories with you. >>>> And I am running a pseudo cluster in a single Ubuntu virtual machine. >>>> >>>> So, now, you could try to add the hadoop-core.jar manually, and then go >>>> ahead to see whether the sqoop2 server could run. Please follow the >>>> following steps: >>>> >>>> *./bin/addtowar.sh >>>> -jars /usr/lib/hadoop-0.20-mapreduce/hadoop-core-2.0.0-mr1-cdh4.3.0.jar >>>> * >>>> >>>> Please find the hadoop-core.jar in your own machine. It should be in a >>>> similar place. But still, if you have problem, please let me know. >>>> >>>> >>>> The reason why it's better to remove the "sqoop" folder is to clear the >>>> cached old servlet. Because Tomcat cannot alway extract the sqoop.war file >>>> immediately after you add dependency library to sqoop.war file. By removing >>>> the sqoop folder, the Tomcat is forced to extract the sqoop.war to keep the >>>> sqoop folder up-to-date. So in this way, you could know whether you have >>>> correctly setup the dependency library. Does this explanation help? >>>> >>>> Best, >>>> Mengwei >>>> >>>> >>>> On Tue, Jul 2, 2013 at 9:19 PM, Madhanmohan Savadamuthu < >>>> [email protected]> wrote: >>>> >>>>> Hi Mengwei, >>>>> >>>>> Following are details >>>>> >>>>> Hadoop Version: Hadoop 2.0.0-cdh4.2.1 >>>>> Linux Version: Linux version 2.6.32-358.2.1.el6.x86_64 ( >>>>> [email protected]) (gcc version 4.4.7 >>>>> 20120313 (Red Hat 4.4.7-3) (GCC) ) #1 SMP Wed Feb 20 12:17:37 EST 2013 >>>>> Hadoop Installation Location: /usr/lib/hadoop >>>>> Sqoop2 Installation Location: /usr/lib/sqoop2 >>>>> Sqoop2 Dependency Configuration Command Used: ./bin/addtowar.sh >>>>> -hadoop-auto >>>>> Files in : >>>>> >>>>> avro-1.7.3.jar >>>>> commons-cli-1.2.jar >>>>> commons-configuration-1.6.jar >>>>> commons-dbcp-1.4.jar >>>>> commons-lang-2.5.jar >>>>> commons-logging-1.1.1.jar >>>>> commons-pool-1.5.4.jar >>>>> derby-10.8.2.2.jar >>>>> guava-11.0.2.jar >>>>> hadoop-auth-2.0.0-cdh4.2.1.jar >>>>> hadoop-common-2.0.0-cdh4.2.1-tests.jar >>>>> hadoop-hdfs-2.0.0-cdh4.2.1.jar >>>>> hadoop-mapreduce-client-app-2.0.0-cdh4.2.1.jar >>>>> hadoop-mapreduce-client-common-2.0.0-cdh4.2.1.jar >>>>> hadoop-mapreduce-client-core-2.0.0-cdh4.2.1.jar >>>>> hadoop-mapreduce-client-jobclient-2.0.0-cdh4.2.1.jar >>>>> hadoop-yarn-api-2.0.0-cdh4.2.1.jar >>>>> hadoop-yarn-common-2.0.0-cdh4.2.1.jar >>>>> jackson-core-asl-1.8.8.jar >>>>> jackson-mapper-asl-1.8.8.jar >>>>> json-simple-1.1.jar >>>>> log4j-1.2.16.jar >>>>> mysql-connector-java-5.1.25-bin.jar >>>>> protobuf-java-2.4.0a.jar >>>>> slf4j-api-1.6.1.jar >>>>> slf4j-log4j12-1.6.1.jar >>>>> sqoop-common-1.99.2.jar >>>>> sqoop-connector-generic-jdbc-1.99.2.jar >>>>> sqoop-core-1.99.2.jar >>>>> sqoop-execution-mapreduce-1.99.2-hadoop200.jar >>>>> sqoop-repository-derby-1.99.2.jar >>>>> sqoop-spi-1.99.2.jar >>>>> sqoop-submission-mapreduce-1.99.2-hadoop200.jar >>>>> >>>>> Can you elaborate more about deletion of 'sqoop' folder? >>>>> >>>>> Regards, >>>>> Madhanmohan S >>>>> >>>>> On Tue, Jul 2, 2013 at 10:50 PM, Mengwei Ding < >>>>> [email protected]> wrote: >>>>> >>>>>> Hi Madhanmohan, >>>>>> >>>>>> Thank you for your interest in Sqoop2. It's really great to hear >>>>>> this. And thank you for providing details for your question. Let me help >>>>>> you out with this. >>>>>> >>>>>> This main reason for your situation is that the Sqoop servlet has not >>>>>> been started successfully, so the client get connection refused. I have >>>>>> gone through you attachments. The reason of servlet failure is that your >>>>>> Hadoop dependency library has not be configured correctly. Could you >>>>>> kindly >>>>>> answer my following questions, so that I could help with you further. >>>>>> >>>>>> 1. Your Hadoop version and installation location? You operating >>>>>> system? >>>>>> 2. The details of how you configure the dependency library for sqoop? >>>>>> 3. Could you kindly go to >>>>>> [sqoop_install_dir]/server/server/webapps/sqoop/WEB-INF/lib and list all >>>>>> the jar files? >>>>>> >>>>>> PS: remember to delete the sqoop folder under server/server/webapps >>>>>> every time after you configure the dependency library. >>>>>> >>>>>> Best, >>>>>> Mengwei >>>>>> >>>>>> >>>>>> On Tue, Jul 2, 2013 at 10:05 AM, Madhanmohan Savadamuthu < >>>>>> [email protected]> wrote: >>>>>> >>>>>>> I have setup Sqoop 1.99.2 as mentioned in sqoop.apache.orginstruction. >>>>>>> When I try to show version --all command, following error is >>>>>>> coming. >>>>>>> >>>>>>> Sqoop 1.99.2 revision 3e31b7d3eefb3696d4970704364dea05a9ea2a59 >>>>>>> Compiled by homeuser on Mon Apr 15 20:50:13 PDT 2013 >>>>>>> Exception has occurred during processing command >>>>>>> Exception: com.sun.jersey.api.client.ClientHandlerException Message: >>>>>>> java.net.ConnectException: Connection refused >>>>>>> >>>>>>> all log files are attached for reference. >>>>>>> >>>>>>> >>>>>>> >>>>>>> -- >>>>>>> Thanks and Regards, >>>>>>> Madhanmohan S >>>>>>> >>>>>> >>>>>> >>>>> >>>>> >>>>> -- >>>>> Thanks and Regards, >>>>> Madhanmohan S >>>>> >>>> >>>> >>> >>> >>> -- >>> Thanks and Regards, >>> Madhanmohan S >>> >> >> > > > -- > Thanks and Regards, > Madhanmohan S >
