Sorry I rexplain my situation. I try to connect from an Java app to a Hive server.
I start with the JDBC solution and I found the Java Heap Space problem. After that, I try with the Thrift client and I had also the same problem. After adding more memory I have this problem exception in the both case : > org.apache.thrift.transport.TTransportException > at > org.apache.thrift.transport.TIOStreamTransport.read(TIOStreamTransport.java:132) > at org.apache.thrift.transport.TTransport.readAll(TTransport.java:84) > at > org.apache.thrift.protocol.TBinaryProtocol.readStringBody(TBinaryProtocol.java:352) > at > org.apache.thrift.protocol.TBinaryProtocol.readMessageBegin(TBinaryProtocol.java:213) > > at > org.apache.hadoop.hive.service.ThriftHive$Client.recv_execute(ThriftHive.java:72) > at > org.apache.hadoop.hive.service.ThriftHive$Client.execute(ThriftHive.java:57) > at com.tubemogul.rtb.datastore.HiveDAO.testThrift(HiveDAO.java:62) > > at com.tubemogul.rtb.daemon.ImportDataThread.run(ImportDataThread.java:54) > at java.lang.Thread.run(Thread.java:619) Thanks On 28 juil. 2010, at 20:19, Todd Lee wrote: > sorry I am a little bit confused - so are you trying to connect to Hive > server via a JDBC client *or* a Thrift client? > > were you able to run the above JDBC client code without any problem? > > Cheers, > Todd > > On Wed, Jul 28, 2010 at 3:18 PM, Francois LAGIER <[email protected]> > wrote: > First of all, thanks for you answer. > > I have exactly the same JDBC code. After increase the heap size, it's getting > better but I have now a new error : > > org.apache.thrift.transport.TTransportException > at > org.apache.thrift.transport.TIOStreamTransport.read(TIOStreamTransport.java:132) > at org.apache.thrift.transport.TTransport.readAll(TTransport.java:84) > at > org.apache.thrift.protocol.TBinaryProtocol.readStringBody(TBinaryProtocol.java:352) > at > org.apache.thrift.protocol.TBinaryProtocol.readMessageBegin(TBinaryProtocol.java:213) > > at > org.apache.hadoop.hive.service.ThriftHive$Client.recv_execute(ThriftHive.java:72) > at > org.apache.hadoop.hive.service.ThriftHive$Client.execute(ThriftHive.java:57) > at com.tubemogul.rtb.datastore.HiveDAO.testThrift(HiveDAO.java:62) > > at com.tubemogul.rtb.daemon.ImportDataThread.run(ImportDataThread.java:54) > at java.lang.Thread.run(Thread.java:619) > > On this one, I have no idea ... > > Francois > > Le 7/28/2010 3:04 PM, Todd Lee a écrit : >> to use Hive with JDBC, you should probably do something like this, >> >> Connection con = >> DriverManager.getConnection("jdbc:hive://localhost:10000/default", "", ""); >> Statement stmt = con.createStatement(); >> String sql = "show tables;" >> ResultSet res = stmt.executeQuery(sql); >> ... >> >> for OOM error, try increase the heap size in Hadoop by uncommenting this >> line in hadoop-env.sh >> >> export HADOOP_HEAPSIZE=2000 >> >> Cheers, >> T >> >> On Wed, Jul 28, 2010 at 1:00 PM, Francois LAGIER <[email protected]> >> wrote: >> Hello everybody, >> >> I just start working on HIVE so maybe my question will be stupid but for the >> moment I am completely stuck so ... >> >> I try to create a Java app, and try to launch some query on my Hive Server. >> After reading the wiki, I try with JDBC and Thrift. >> >> For the moment, I just try to connect to the server, to launch a very basic >> query ("Show tables") and display the result in my console. >> In the both case, JDBC and Thrift, I have always the same result : >> >> Exception in thread "ImportDataModule" java.lang.OutOfMemoryError: Java heap >> space >> at >> org.apache.thrift.protocol.TBinaryProtocol.readStringBody(TBinaryProtocol.java:296) >> at >> org.apache.thrift.protocol.TBinaryProtocol.readMessageBegin(TBinaryProtocol.java:203) >> at >> org.apache.hadoop.hive.service.ThriftHive$Client.recv_execute(ThriftHive.java:72) >> at >> org.apache.hadoop.hive.service.ThriftHive$Client.execute(ThriftHive.java:57) >> at com.tubemogul.rtb.datastore.HiveDAO.testThrift(HiveDAO.java:59) >> at com.tubemogul.rtb.daemon.ImportDataThread.run(ImportDataThread.java:54) >> at java.lang.Thread.run(Thread.java:619) >> >> And this is my function : >> >> TTransport transport; >> try { >> transport = new TSocket("my-server", 9999); >> TProtocol protocol = new TBinaryProtocol(transport); >> HiveClient client = new HiveClient(protocol); >> transport.open(); >> client.execute("SHOW TABLES"); >> //List<String> a = client.fetchAll(); >> transport.close(); >> } catch (HiveServerException e) { >> // TODO Auto-generated catch block >> e.printStackTrace(); >> } catch (TException e) { >> // TODO Auto-generated catch block >> e.printStackTrace(); >> } >> >> >> Now, I really hope one of you already have this problem and already fix it. >> I am sorry if you already talk about that in one of your email but I didn't >> find something to fix that, >> >> Thanks in advance, >> >> Francois >> > > Francois LAGIER [email protected]
