Re: Thirft + JDBC : java.lang.OutOfMemoryError: Java heap space

[Francois LAGIER]

 First of all, thanks for you answer.

I have exactly the same JDBC code. After increase the heap size, it's 
getting better but I have now a new error :

org.apache.thrift.transport.TTransportException
    at 
org.apache.thrift.transport.TIOStreamTransport.read(TIOStreamTransport.java:132)
    at org.apache.thrift.transport.TTransport.readAll(TTransport.java:84)
    at 
org.apache.thrift.protocol.TBinaryProtocol.readStringBody(TBinaryProtocol.java:352)
    at 
org.apache.thrift.protocol.TBinaryProtocol.readMessageBegin(TBinaryProtocol.java:213)
    at 
org.apache.hadoop.hive.service.ThriftHive$Client.recv_execute(ThriftHive.java:72)
    at 
org.apache.hadoop.hive.service.ThriftHive$Client.execute(ThriftHive.java:57)
    at com.tubemogul.rtb.datastore.HiveDAO.testThrift(HiveDAO.java:62)
    at 
com.tubemogul.rtb.daemon.ImportDataThread.run(ImportDataThread.java:54)
    at java.lang.Thread.run(Thread.java:619)

On this one, I have no idea ...

Francois

Le 7/28/2010 3:04 PM, Todd Lee a écrit :
to use Hive with JDBC, you should probably do something like this,

Connection con = 
DriverManager.getConnection("jdbc:hive://localhost:10000/default", "", 
"");
Statement stmt = con.createStatement();
String sql = "show tables;"
ResultSet res = stmt.executeQuery(sql);
...

for OOM error, try increase the heap size in Hadoop by uncommenting 
this line in hadoop-env.sh

export HADOOP_HEAPSIZE=2000

Cheers,
T

On Wed, Jul 28, 2010 at 1:00 PM, Francois LAGIER 
<francois.lag...@gmail.com <mailto:francois.lag...@gmail.com>> wrote:

     Hello everybody,

    I just start working on HIVE so maybe my question will be stupid
    but for the moment I am completely stuck so ...

    I try to create a Java app, and try to launch some query on my
    Hive Server.
    After reading the wiki, I try with JDBC and Thrift.

    For the moment, I just try to connect to the server, to launch a
    very basic query ("Show tables") and display the result in my console.
    In the both case, JDBC and Thrift, I have always the same result :

    Exception in thread "ImportDataModule" java.lang.OutOfMemoryError:
    Java heap space
       at
    
org.apache.thrift.protocol.TBinaryProtocol.readStringBody(TBinaryProtocol.java:296)
       at
    
org.apache.thrift.protocol.TBinaryProtocol.readMessageBegin(TBinaryProtocol.java:203)
       at
    
org.apache.hadoop.hive.service.ThriftHive$Client.recv_execute(ThriftHive.java:72)
       at
    org.apache.hadoop.hive.service.ThriftHive$Client.execute(ThriftHive.java:57)
       at com.tubemogul.rtb.datastore.HiveDAO.testThrift(HiveDAO.java:59)
       at
    com.tubemogul.rtb.daemon.ImportDataThread.run(ImportDataThread.java:54)
       at java.lang.Thread.run(Thread.java:619)

    And this is my function :

    TTransport transport;
    try {
       transport = new TSocket("my-server", 9999);
       TProtocol protocol = new TBinaryProtocol(transport);
       HiveClient client = new HiveClient(protocol);
       transport.open();
       client.execute("SHOW TABLES");
       //List<String> a = client.fetchAll();
       transport.close();
    } catch (HiveServerException e) {
       // TODO Auto-generated catch block
       e.printStackTrace();
    } catch (TException e) {
       // TODO Auto-generated catch block
       e.printStackTrace();
    }


    Now, I really hope one of you already have this problem and
    already fix it.
    I am sorry if you already talk about that in one of your email but
    I didn't find something to fix that,

    Thanks in advance,

    Francois