Thanks John and Minh for sharing your thoughts and hepling provide the
direction to resolve the issue.
I will add the connector as part of Hadoop Jar command. However there is one
thought that crossed my mind as I was reading these emails.
My initial problem was related to connecting with Oracle DB using the Oracle
RAC environment. I am still able to connect directly to the physical database
however due to network issues between Access Node and DB, a routing deamon is
unable to route the requests appropriately.
Are we sure we are really addressing the issue at hand? I mean I could
definately discover if I can really use Oracle RAC connection URL as opposed to
direct physical connection by writing the Java program. How will this new
implementation be different from say sqoop tool which does not work when I
source Oracle RAC url (proxy) instead of direct url.
Any help is appreciated.
-Akash
> From: mdngu...@gmail.com
> Date: Tue, 26 Jun 2012 22:41:24 -0400
> Subject: Re: Sqoop Issue
> To: common-user@hadoop.apache.org
>
> Akash,
>
> Instead of adding the connector jar to $HADOOP_HOME/lib, when
> running your map-reduce job using "hadoop jar" you can pass your
> connector jar using the "-libjars" flag.
>
> For example: hadoop jar hadoop-examples.jar wordcount -files
> cachefile.txt -libjars mylib.jar input output
>
> ~ Minh
>
> On Tue, Jun 26, 2012 at 9:52 PM, John Hancock <jhancock1...@gmail.com> wrote:
> > Akash,
> >
> > I forgot to mention you will need to make sure the connector for your
> > oracle db is available to Hadoop. There are many ways to do this, but what
> > works for me is I copy the library to <hadoop-install>/lib in all nodes in
> > my cluster.
> >
> > I've written a similar program to do what you are asking about but using a
> > mysql db, so I have mysql-connector-java-5.0.8-bin.jar in the lib directory
> > on all my hadoop nodes.
> >
> > -John
> >
> > On Tue, Jun 26, 2012 at 8:38 AM, Akash Sharma <sharma...@hotmail.com> wrote:
> >
> >>
> >> Thanks for the response John,
> >>
> >> This is exactly what I was looking for. I will pursue this direction and
> >> create a Java program to connect with Oracle DB, extract relevant data and
> >> push it into HDFS.
> >>
> >> I am guessing the Hadoop/HDFS API's will contain all the relevant calls
> >> required to execute these tasks.
> >>
> >> Thanks,
> >>
> >> -Akash
> >>
> >> > Date: Tue, 26 Jun 2012 05:36:27 -0400
> >> > Subject: Re: Sqoop Issue
> >> > From: jhancock1...@gmail.com
> >> > To: common-user@hadoop.apache.org
> >> >
> >> > Akash,
> >> >
> >> > You can write a simple Java program that queries your Oracle DB and uses
> >> > whatever kind of file output object from java.io that you like to write
> >> the
> >> > data to a file.
> >> >
> >> > Compile the program and package it into a jar file.
> >> >
> >> > Then run the program using <path-to-hadoop>/'hadoop jar <jar-file-name>
> >> > <output-file-name> in your Hadoop cluster.
> >> >
> >> > The result will be in HDFS.
> >> >
> >> > -John
> >> >
> >> > On Mon, Jun 25, 2012 at 8:02 AM, Akash Sharma <sharma...@hotmail.com>
> >> wrote:
> >> >
> >> > >
> >> > >
> >> > > Hi,
> >> > >
> >> > > Seeking some advise/options on sqoop of data from Oracle DB to HDFS. We
> >> > > are on Cloudera 3.
> >> > >
> >> > > Sqoop/JDBC connection to Oracle RAC fails within the Integration
> >> cluster.
> >> > > In the Oracle RAC there is an additional layer between the Edge
> >> > > Node/Server and Database. Routing the connection occurs to appropriate
> >> > > database listener. The specific issue is ORA-12516, TNS:listener could
> >> not
> >> > > find available handler with matching protocol stack. This issue is
> >> > > primarily a network configuration issue which is stopping the team to
> >> > > progress further.
> >> > >
> >> > > I would like to know other than sqoop, what are the other options to
> >> > > bring the data into HDFS. Our primary goal is to land data from Oracle
> >> DB
> >> > > into HDFS. Since sqoop does not work, we are thinking of using Oracle
> >> > > unload, ftp, and then put command to import the data into HDFS. Please
> >> > > guide if there is any other option other than this. Will it help in
> >> anyway
> >> > > to code these options in java using HDFS/FTP Api within Java itself
> >> rather
> >> > > than using tools?
> >> > >
> >> > > Any help is appreciative.
> >> > >
> >> > > -Akash
> >> > >
> >> > >
> >> > >
> >>
> >>
