No - I didn't add anything to my service which should load the configs. The service exposes four operations (methods) - for connecting to Hive using JDBC, showing the list of tables, describing the schema of a given table and executing queries. And the code is very similar to the sample code. Still, I'll try to build Hive from the trunk - not sure whether I'll be able to do it or not - because every time I tried that, the build process started downloading all versions of hadoop and failed with an error saying that the downloaded size didn't match the expected size...
I'll have a look again. Arijit 2009/10/23 Bill Graham <[email protected]>: > Sorry Araijit, I see what you're trying to do now. > > Yes, your webservice is the standalone JDBC client in this case, since it's > configured to connect to the Hive server at localhost:10000. You should > therefore not need the jpox jars and hive configs for the JDBC client. > > When you followed the HWI tutorial, did you add anything else to your server > that would try to load hive/hadoop configs? If so, I'd say to remove it. > > Despite the exception, you're still able to establish a connection to the > HiveServer and execute JDBC queries? That's strange. I would recommend > trying to build hive from the trunk and try using those jars (looking at > your stack trace you seem to be using an older release). > > If that doesn't fix the issue, we can look at what's happening in the code > when the exception is thrown. > > > On Wed, Oct 21, 2009 at 9:09 PM, Arijit Mukherjee <[email protected]> > wrote: >> >> Yes - I understood that. But this is what I am doing - implementing a >> Web Service which will connect to the Hive/Thrift server via JDBC and >> execute queries. For that, I used some statements to connect as shown >> in the sample JDBC application. >> >> Arijit >> >> 2009/10/21 Bill Graham <[email protected]>: >> > There is no J2EE web server or SOAP web service in this equation. The >> > Hive >> > JDBC client connects to the Hive Server, which can be started with a >> > script >> > like so run from your $HIVE_HOME/build/dist directory: >> > >> > export HADOOP_HOME=/path/to/hadoop >> > HIVE_PORT=10000 ./bin/hive --service hiveserver >> > >> > No war files, or WEB-INF/ directories at all in this case. >> > >> > >> > On Wed, Oct 21, 2009 at 5:24 AM, Arijit Mukherjee <[email protected]> >> > wrote: >> >> >> >> If I understood the concept of "standalone" and "embedded" properly, >> >> then a Web Service which connects to the Hive/Thrift server via JDBC >> >> (jdbc:hive://host:port...) is actually a standalone client. The >> >> difference from a Java standalone client is that - in this case, the >> >> whole thing is packaged as a Web Service and deployed on a web server >> >> such as JBoss/GlassFish - and the connection is initiated only after a >> >> SOAP request is received from the web service client. If that is the >> >> case, then the Web Service should not require the conf files, or jpox >> >> libraries - is it not? Or did I misunderstand the concept? >> >> >> >> Arijit >> >> >> >> 2009/10/21 Arijit Mukherjee <[email protected]>: >> >> > Update: I did a clean/build/deploy - the config files are within the >> >> > Web Service WEB-INF/classes folder, and the libraries (including the >> >> > jpox ones) are inside WEB-INF/lib - which are standard for any web >> >> > application. But the config related exception is still there:-(( >> >> > >> >> > Arijit >> >> > >> >> > 2009/10/21 Arijit Mukherjee <[email protected]>: >> >> >> Thanx Bill. I copied the jpox jars from the 0.3.0 distribution and >> >> >> added them to the web service archive, and they are in the >> >> >> classpath, >> >> >> but the config related exception is still there. Let me do a clean >> >> >> build/deploy and I'll get back again. >> >> >> >> >> >> Arijit >> >> >> >> >> >> 2009/10/20 Bill Graham <[email protected]>: >> >> >>> The Hive JDBC client can run in two different modes: standalone and >> >> >>> embedded. >> >> >>> >> >> >>> Standalone mode is where the client connects to a separate >> >> >>> standalone >> >> >>> HiveServer by specifying the host:port of the server in the jdbc >> >> >>> URL >> >> >>> like >> >> >>> this: jdbc:hive://localhost:10000/default. In this case the hive >> >> >>> configs are >> >> >>> not needed by the client, since the client is making thrift >> >> >>> requests >> >> >>> to the >> >> >>> server which has the Hive configs. the Hive Server knows how to >> >> >>> resolve the >> >> >>> metastore. >> >> >>> >> >> >>> Embedded mode is where the JDBC client "connects to itself" so to >> >> >>> speak >> >> >>> using a JDBC url like this: jdbc:hive://. It's as if the client is >> >> >>> running >> >> >>> an embedded server that only it communicates with. In this case the >> >> >>> client >> >> >>> needs the Hive configs since it needs to resolve the metastore, >> >> >>> amongst >> >> >>> other things. The metastore dependency in this case is what will >> >> >>> cause >> >> >>> you >> >> >>> to see jpox errors appear if those jars aren't found. >> >> >>> >> >> >>> HTH, >> >> >>> Bill >> >> >>> >> >> >>> On Tue, Oct 20, 2009 at 4:14 AM, Arijit Mukherjee >> >> >>> <[email protected]> >> >> >>> wrote: >> >> >>>> >> >> >>>> BTW - the service is working though, in spite of those exceptions. >> >> >>>> I'm >> >> >>>> able to run queries and get results. >> >> >>>> >> >> >>>> Arijit >> >> >>>> >> >> >>>> 2009/10/20 Arijit Mukherjee <[email protected]>: >> >> >>>> > I created a hive-site.xml using the outline given in the Hive >> >> >>>> > Web >> >> >>>> > Interface tutorial - now that file is in the classpath of the >> >> >>>> > Web >> >> >>>> > Service - and the service can find the file. But, now there's >> >> >>>> > another >> >> >>>> > exception - >> >> >>>> > >> >> >>>> > 2009-10-20 14:27:30,914 DEBUG [httpSSLWorkerThread-14854-0] >> >> >>>> > HiveQueryService - connecting to Hive using URL: >> >> >>>> > jdbc:hive://localhost:10000/default >> >> >>>> > 2009-10-20 14:27:30,969 DEBUG [httpSSLWorkerThread-14854-0] >> >> >>>> > Configuration - java.io.IOException: config() >> >> >>>> > at >> >> >>>> > >> >> >>>> > org.apache.hadoop.conf.Configuration.<init>(Configuration.java:176) >> >> >>>> > at >> >> >>>> > >> >> >>>> > org.apache.hadoop.conf.Configuration.<init>(Configuration.java:164) >> >> >>>> > at >> >> >>>> > org.apache.hadoop.hive.conf.HiveConf.<init>(HiveConf.java:287) >> >> >>>> > at >> >> >>>> > >> >> >>>> > >> >> >>>> > org.apache.hadoop.hive.jdbc.HiveConnection.<init>(HiveConnection.java:63) >> >> >>>> > at >> >> >>>> > >> >> >>>> > org.apache.hadoop.hive.jdbc.HiveDriver.connect(HiveDriver.java:109) >> >> >>>> > at >> >> >>>> > java.sql.DriverManager.getConnection(DriverManager.java:582) >> >> >>>> > at >> >> >>>> > java.sql.DriverManager.getConnection(DriverManager.java:185) >> >> >>>> > at >> >> >>>> > >> >> >>>> > >> >> >>>> > com.ctva.poc.hive.service.HiveQueryService.getConnection(HiveQueryService.java:134) >> >> >>>> > at >> >> >>>> > >> >> >>>> > >> >> >>>> > com.ctva.poc.hive.service.HiveQueryService.connectDB(HiveQueryService.java:43) >> >> >>>> > >> >> >>>> > Apparently, something goes wrong during the config routine. Do I >> >> >>>> > need >> >> >>>> > something more within the service? >> >> >>>> > >> >> >>>> > Regards >> >> >>>> > Arijit >> >> >>>> > >> >> >>>> > 2009/10/20 Arijit Mukherjee <[email protected]>: >> >> >>>> >> Hi >> >> >>>> >> >> >> >>>> >> I'm trying to create a Web Service which will access Hive >> >> >>>> >> (0.4.0 >> >> >>>> >> release) using JDBC. I used to sample JDBC code from the wiki >> >> >>>> >> >> >> >>>> >> >> >> >>>> >> >> >> >>>> >> (http://wiki.apache.org/hadoop/Hive/HiveClient#head-fd2d8ae9e17fdc3d9b7048d088b2c23a53a6857d), >> >> >>>> >> but when I'm trying to connect the the DB using the >> >> >>>> >> DriverManager, >> >> >>>> >> there's an exception which seems to relate to hive-site.xml >> >> >>>> >> (HiveConf >> >> >>>> >> - hive-site.xml not found.). But I could not find any >> >> >>>> >> hive-site.xml in >> >> >>>> >> $HIVE_HOME/conf - there's only hive-default.xml. The wiki page >> >> >>>> >> also >> >> >>>> >> speaks about couple of jpox JAR files, which aren't in the lib >> >> >>>> >> folder >> >> >>>> >> either. >> >> >>>> >> >> >> >>>> >> Am I missing something here? >> >> >>>> >> >> >> >>>> >> Regards >> >> >>>> >> Arijit >> >> >>>> >> >> >> >>>> >> -- >> >> >>>> >> "And when the night is cloudy, >> >> >>>> >> There is still a light that shines on me, >> >> >>>> >> Shine on until tomorrow, let it be." >> >> >>>> >> >> >> >>>> > >> >> >>>> > >> >> >>>> > >> >> >>>> > -- >> >> >>>> > "And when the night is cloudy, >> >> >>>> > There is still a light that shines on me, >> >> >>>> > Shine on until tomorrow, let it be." >> >> >>>> > >> >> >>>> >> >> >>>> >> >> >>>> >> >> >>>> -- >> >> >>>> "And when the night is cloudy, >> >> >>>> There is still a light that shines on me, >> >> >>>> Shine on until tomorrow, let it be." >> >> >>> >> >> >>> >> >> >> >> >> >> >> >> >> >> >> >> -- >> >> >> "And when the night is cloudy, >> >> >> There is still a light that shines on me, >> >> >> Shine on until tomorrow, let it be." >> >> >> >> >> > >> >> > >> >> > >> >> > -- >> >> > "And when the night is cloudy, >> >> > There is still a light that shines on me, >> >> > Shine on until tomorrow, let it be." >> >> > >> >> >> >> >> >> >> >> -- >> >> "And when the night is cloudy, >> >> There is still a light that shines on me, >> >> Shine on until tomorrow, let it be." >> > >> > >> >> >> >> -- >> "And when the night is cloudy, >> There is still a light that shines on me, >> Shine on until tomorrow, let it be." > > -- "And when the night is cloudy, There is still a light that shines on me, Shine on until tomorrow, let it be."
