1. I have no process with hiveserver2 ...

"ps -ef | grep -i hive"  return some pretty long command with a -Xmx8192
and that's the value set in hive-env.sh


2. The "select * from table limit 1" or even 100 is working correctly.


David.


On Tue, Feb 18, 2014 at 4:16 PM, Stephen Sprague <[email protected]> wrote:

> He lives on after all! and thanks for the continued feedback.
>
> We need the answers to these questions using HS2:
>
>
>
>    1. what is the output of "ps -ef | grep -i hiveserver2" on your system?
> in particular what is the value of -Xmx ?
>
>    2. does "select * from table limit 1" work?
>
> Thanks,
> Stephen.
>
>
>
> On Tue, Feb 18, 2014 at 6:32 AM, David Gayou <[email protected]> wrote:
>
>> I'm so sorry, i wrote an answer, and i forgot to sent it....
>> And i haven't been able to work on this for a few days.
>>
>>
>> So far :
>>
>> I have a 15k columns table and 50k rows.
>>
>> I do not see any changes if i change the storage.
>>
>>
>> *Hive 12.0*
>>
>> My test query is "select * from bigtable"
>>
>>
>> If i use the hive cli, it works fine.
>>
>> If i use hiveserver1 + ODBC : it works fine
>>
>> If i use hiverserver2 + odbc or hiverserver2 + beeline,i have this java
>> exception :
>>
>> 2014-02-18 13:22:22,571 ERROR thrift.ProcessFunction
>> (ProcessFunction.java:process(41)) - Internal error processing FetchResults
>>
>> java.lang.OutOfMemoryError: Java heap space
>>         at java.util.Arrays.copyOf(Arrays.java:2734)
>>         at java.util.ArrayList.ensureCapacity(ArrayList.java:167)
>>         at java.util.ArrayList.add(ArrayList.java:351)
>>          at
>> org.apache.hive.service.cli.thrift.TRow.addToColVals(TRow.java:160)
>>         at
>> org.apache.hive.service.cli.RowBasedSet.addRow(RowBasedSet.java:60)
>>         at
>> org.apache.hive.service.cli.RowBasedSet.addRow(RowBasedSet.java:32)
>>         at
>> org.apache.hive.service.cli.operation.SQLOperation.prepareFromRow(SQLOperation.java:270)
>>         at
>> org.apache.hive.service.cli.operation.SQLOperation.decode(SQLOperation.java:262)
>>         at
>> org.apache.hive.service.cli.operation.SQLOperation.getNextRowSet(SQLOperation.java:246)
>>         at
>> org.apache.hive.service.cli.operation.OperationManager.getOperationNextRowSet(OperationManager.java:171)
>>         at
>> org.apache.hive.service.cli.session.HiveSessionImpl.fetchResults(HiveSessionImpl.java:438)
>>         at
>> org.apache.hive.service.cli.CLIService.fetchResults(CLIService.java:346)
>>         at
>> org.apache.hive.service.cli.thrift.ThriftCLIService.FetchResults(ThriftCLIService.java:407)
>>         at
>> org.apache.hive.service.cli.thrift.TCLIService$Processor$FetchResults.getResult(TCLIService.java:1373)
>>
>>
>>
>>
>> *From the SVN trunk* : (for the HIVE-3746)
>>
>> With the maven change, most of the documentation and wiki are out of
>> date.
>> Compiling from trunk was not that easy and i may have failed some steps
>> but :
>>
>> It has the same behavior. It works in CLI and hiveserver1.
>> It fails with hiveserver 2.
>>
>>
>> Regards
>>
>> David Gayou
>>
>>
>>
>>
>>
>> On Thu, Feb 13, 2014 at 3:11 AM, Navis류승우 <[email protected]> wrote:
>>
>>> With HIVE-3746, which will be included in hive-0.13, HiveServer2 takes
>>> less memory than before.
>>>
>>> Could you try it with the version in trunk?
>>>
>>>
>>> 2014-02-13 10:49 GMT+09:00 Stephen Sprague <[email protected]>:
>>>
>>> question to the original poster.  closure appreciated!
>>>>
>>>>
>>>> On Fri, Jan 31, 2014 at 12:22 PM, Stephen Sprague 
>>>> <[email protected]>wrote:
>>>>
>>>>> thanks Ed. And on a separate tact lets look at Hiveserver2.
>>>>>
>>>>>
>>>>> @OP>
>>>>>
>>>>> *I've tried to look around on how i can change the thrift heap size
>>>>> but haven't found anything.*
>>>>>
>>>>>
>>>>> looking at my hiveserver2 i find this:
>>>>>
>>>>>    $ ps -ef | grep -i hiveserver2
>>>>>    dwr       9824 20479  0 12:11 pts/1    00:00:00 grep -i hiveserver2
>>>>>    dwr      28410     1  0 00:05 ?        00:01:04
>>>>> /usr/lib/jvm/java-6-sun/jre/bin/java 
>>>>> *-Xmx256m*-Dhadoop.log.dir=/usr/lib/hadoop/logs 
>>>>> -Dhadoop.log.file=hadoop.log
>>>>> -Dhadoop.home.dir=/usr/lib/hadoop -Dhadoop.id.str=
>>>>> -Dhadoop.root.logger=INFO,console
>>>>> -Djava.library.path=/usr/lib/hadoop/lib/native
>>>>> -Dhadoop.policy.file=hadoop-policy.xml -Djava.net.preferIPv4Stack=true
>>>>> -Dhadoop.security.logger=INFO,NullAppender org.apache.hadoop.util.RunJar
>>>>> /usr/lib/hive/lib/hive-service-0.12.0.jar
>>>>> org.apache.hive.service.server.HiveServer2
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> questions:
>>>>>
>>>>>    1. what is the output of "ps -ef | grep -i hiveserver2" on your
>>>>> system? in particular what is the value of -Xmx ?
>>>>>
>>>>>    2. can you restart your hiveserver with -Xmx1g? or some value that
>>>>> makes sense to your system?
>>>>>
>>>>>
>>>>>
>>>>> Lots of questions now.  we await your answers! :)
>>>>>
>>>>>
>>>>>
>>>>> On Fri, Jan 31, 2014 at 11:51 AM, Edward Capriolo <
>>>>> [email protected]> wrote:
>>>>>
>>>>>> Final table compression should not effect the de serialized size of
>>>>>> the data over the wire.
>>>>>>
>>>>>>
>>>>>> On Fri, Jan 31, 2014 at 2:49 PM, Stephen Sprague 
>>>>>> <[email protected]>wrote:
>>>>>>
>>>>>>> Excellent progress David.   So.  What the most important thing here
>>>>>>> we learned was that it works (!) by running hive in local mode and that
>>>>>>> this error is a limitation in the HiveServer2.  That's important.
>>>>>>>
>>>>>>> so textfile storage handler and having issues converting it to ORC.
>>>>>>> hmmm.
>>>>>>>
>>>>>>> follow-ups.
>>>>>>>
>>>>>>> 1. what is your query that fails?
>>>>>>>
>>>>>>> 2. can you add a "limit 1" to the end of your query and tell us if
>>>>>>> that works? this'll tell us if it's column or row bound.
>>>>>>>
>>>>>>> 3. bonus points. run these in local mode:
>>>>>>>       > set hive.exec.compress.output=true;
>>>>>>>       > set mapred.output.compression.type=BLOCK;
>>>>>>>       > set
>>>>>>> mapred.output.compression.codec=org.apache.hadoop.io.compress.GzipCodec;
>>>>>>>       > create table blah stored as ORC as select * from <your
>>>>>>> table>;   #i'm curious if this'll work.
>>>>>>>       > show create table blah;  #send output back if previous step
>>>>>>> worked.
>>>>>>>
>>>>>>> 4. extra bonus.  change ORC to SEQUENCEFILE in #3 see if that works
>>>>>>> any differently.
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> I'm wondering if compression would have any effect on the size of
>>>>>>> the internal ArrayList the thrift server uses.
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> On Fri, Jan 31, 2014 at 9:21 AM, David Gayou 
>>>>>>> <[email protected]>wrote:
>>>>>>>
>>>>>>>> Ok, so here are some news :
>>>>>>>>
>>>>>>>> I tried to boost the HADOOP_HEAPSIZE to 8192,
>>>>>>>> I also setted the mapred.child.java.opts to 512M
>>>>>>>>
>>>>>>>> And it doesn't seem's to have any effect.
>>>>>>>>  ------
>>>>>>>>
>>>>>>>> I tried it using an ODBC driver => fail after few minutes.
>>>>>>>> Using a local JDBC (beeline) => running forever without any error.
>>>>>>>>
>>>>>>>> Both through hiveserver 2
>>>>>>>>
>>>>>>>> If i use the local mode : it works!   (but that not really what i
>>>>>>>> need, as i don't really how to access it with my software)
>>>>>>>>
>>>>>>>> ------
>>>>>>>> I use a text file as storage.
>>>>>>>> I tried to use ORC, but i can't populate it with a load data  (it
>>>>>>>> return an error of file format).
>>>>>>>>
>>>>>>>> Using an "ALTER TABLE orange_large_train_3 SET FILEFORMAT ORC"
>>>>>>>> after populating the table, i have a file format error on select.
>>>>>>>>
>>>>>>>> ------
>>>>>>>>
>>>>>>>> @Edward :
>>>>>>>>
>>>>>>>> I've tried to look around on how i can change the thrift heap size
>>>>>>>> but haven't found anything.
>>>>>>>> Same thing for my client (haven't found how to change the heap size)
>>>>>>>>
>>>>>>>> My usecase is really to have the most possible columns.
>>>>>>>>
>>>>>>>>
>>>>>>>> Thanks a lot for your help
>>>>>>>>
>>>>>>>>
>>>>>>>> Regards
>>>>>>>>
>>>>>>>> David
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> On Fri, Jan 31, 2014 at 1:12 AM, Edward Capriolo <
>>>>>>>> [email protected]> wrote:
>>>>>>>>
>>>>>>>>> Ok here are the problem(s). Thrift has frame size limits, thrift
>>>>>>>>> has to buffer rows into memory.
>>>>>>>>>
>>>>>>>>> Hove thrift has a heap size, it needs to big in this case.
>>>>>>>>>
>>>>>>>>> Your client needs a big heap size as well.
>>>>>>>>>
>>>>>>>>> The way to do this query if it is possible may be turning row
>>>>>>>>> lateral, potwntially by treating it as a list, it will make queries 
>>>>>>>>> on it
>>>>>>>>> awkward.
>>>>>>>>>
>>>>>>>>> Good luck
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Thursday, January 30, 2014, Stephen Sprague <[email protected]>
>>>>>>>>> wrote:
>>>>>>>>> > oh. thinking some more about this i forgot to ask some other
>>>>>>>>> basic questions.
>>>>>>>>> >
>>>>>>>>> > a) what storage format are you using for the table (text,
>>>>>>>>> sequence, rcfile, orc or custom)?   "show create table <table>" would 
>>>>>>>>> yield
>>>>>>>>> that.
>>>>>>>>> >
>>>>>>>>> > b) what command is causing the stack trace?
>>>>>>>>> >
>>>>>>>>> > my thinking here is rcfile and orc are column based (i think)
>>>>>>>>> and if you don't select all the columns that could very well limit 
>>>>>>>>> the size
>>>>>>>>> of the "row" being returned and hence the size of the internal 
>>>>>>>>> ArrayList.
>>>>>>>>> OTOH, if you're using "select *", um, you have my sympathies. :)
>>>>>>>>> >
>>>>>>>>> >
>>>>>>>>> >
>>>>>>>>> >
>>>>>>>>> > On Thu, Jan 30, 2014 at 11:33 AM, Stephen Sprague <
>>>>>>>>> [email protected]> wrote:
>>>>>>>>> >
>>>>>>>>> > thanks for the information. Up-to-date hive. Cluster on the
>>>>>>>>> smallish side. And, well, sure looks like a memory issue. :)  rather 
>>>>>>>>> than
>>>>>>>>> an inherent hive limitation that is.
>>>>>>>>> >
>>>>>>>>> > So.  I can only speak as a user (ie. not a hive developer) but
>>>>>>>>> what i'd be interested in knowing next is is this via running hive in 
>>>>>>>>> local
>>>>>>>>> mode, correct? (eg. not through hiveserver1/2).  And it looks like it
>>>>>>>>> boinks on array processing which i assume to be internal code arrays 
>>>>>>>>> and
>>>>>>>>> not hive data arrays - your 15K columns are all scalar/simple types,
>>>>>>>>> correct?  Its clearly fetching results and looks be trying to store 
>>>>>>>>> them in
>>>>>>>>> a java array  - and not just one row but a *set* of rows (ArrayList)
>>>>>>>>> >
>>>>>>>>> > two things to try.
>>>>>>>>> >
>>>>>>>>> > 1. boost the heap-size. try 8192. And I don't know if
>>>>>>>>> HADOOP_HEAPSIZE is the controller of that. I woulda hoped it was 
>>>>>>>>> called
>>>>>>>>> something like "HIVE_HEAPSIZE". :)  Anyway, can't hurt to try.
>>>>>>>>> >
>>>>>>>>> > 2. trim down the number of columns and see where the breaking
>>>>>>>>> point is.  is it 10K? is it 5K?   The idea is to confirm its _the 
>>>>>>>>> number of
>>>>>>>>> columns_ that is causing the memory to blow and not some other 
>>>>>>>>> artifact
>>>>>>>>> unbeknownst to us.
>>>>>>>>> >
>>>>>>>>> > 3. Google around the Hive namespace for something that might
>>>>>>>>> limit or otherwise control the number of rows stored at once in Hive's
>>>>>>>>> internal buffer. I snoop around too.
>>>>>>>>> >
>>>>>>>>> >
>>>>>>>>> > That's all i got for now and maybe we'll get lucky and someone
>>>>>>>>> on this list will know something or another about this. :)
>>>>>>>>> >
>>>>>>>>> > cheers,
>>>>>>>>> > Stephen.
>>>>>>>>> >
>>>>>>>>> >
>>>>>>>>> >
>>>>>>>>> > On Thu, Jan 30, 2014 at 2:32 AM, David Gayou <
>>>>>>>>> [email protected]> wrote:
>>>>>>>>> >
>>>>>>>>> > We are using the Hive 0.12.0, but it doesn't work better on hive
>>>>>>>>> 0.11.0 or hive 0.10.0
>>>>>>>>> > Our hadoop version is 1.1.2.
>>>>>>>>> > Our cluster is 1 master + 4 slaves with 1 dual core xeon CPU
>>>>>>>>> (with hyperthreading so 4 cores per machine) + 16Gb Ram each
>>>>>>>>> >
>>>>>>>>> > The error message i get is :
>>>>>>>>> >
>>>>>>>>> > 2014-01-29 12:41:09,086 ERROR thrift.ProcessFunction
>>>>>>>>> (ProcessFunction.java:process(41)) - Internal error processing 
>>>>>>>>> FetchResults
>>>>>>>>> > java.lang.OutOfMemoryError: Java heap space
>>>>>>>>> >         at java.util.Arrays.copyOf(Arrays.java:2734)
>>>>>>>>> >         at java.util.ArrayList.ensureCapacity(ArrayList.java:167)
>>>>>>>>> >         at java.util.ArrayList.add(ArrayList.java:351)
>>>>>>>>> >         at org.apache.hive.service.cli.Row.<init>(Row.java:47)
>>>>>>>>> >         at
>>>>>>>>> org.apache.hive.service.cli.RowSet.addRow(RowSet.java:61)
>>>>>>>>> >         at
>>>>>>>>> org.apache.hive.service.cli.operation.SQLOperation.getNextRowSet(SQLOperation.java:235)
>>>>>>>>> >         at
>>>>>>>>> org.apache.hive.service.cli.operation.OperationManager.getOperationNextRowSet(OperationManager.java:170)
>>>>>>>>> >         at
>>>>>>>>> org.apache.hive.service.cli.session.HiveSessionImpl.fetchResults(HiveSessionImpl.java:417)
>>>>>>>>> >         at
>>>>>>>>> org.apache.hive.service.cli.CLIService.fetchResults(CLIService.java:306)
>>>>>>>>> >         at
>>>>>>>>> org.apache.hive.service.cli.thrift.ThriftCLIService.FetchResults(ThriftCLIService.java:386)
>>>>>>>>> >         at
>>>>>>>>> org.apache.hive.service.cli.thrift.TCLIService$Processor$FetchResults.getResult(TCLIService.java:1373)
>>>>>>>>> >         at
>>>>>>>>> org.apache.hive.service.cli.thrift.TCLIService$Processor$FetchResults.getResult(TCLIService.java:1358)
>>>>>>>>> >         at
>>>>>>>>> org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)
>>>>>>>>> >         at
>>>>>>>>> org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39)
>>>>>>>>> >         at
>>>>>>>>> org.apache.hive.service.auth.TUGIContainingProcessor$1.run(TUGIContainingProcessor.java:58)
>>>>>>>>> >         at
>>>>>>>>> org.apache.hive.service.auth.TUGIContainingProcessor$1.run(TUGIContainingProcessor.java:55)
>>>>>>>>> >         at java.security.AccessCont
>>>>>>>>>
>>>>>>>>> --
>>>>>>>>> Sorry this was sent from mobile. Will do less grammar and spell
>>>>>>>>> check than usual.
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>
>

Reply via email to