Re: limit clause + fetch optimization

2015-07-21 Thread Gopal Vijayaraghavan
> Just want to make sure I understand the behavior once that bug is >fixed...a 'select *' with no limit will run without a M/R job and instead >stream. Is that correct? Yes, that¹s the intended behaviour. I can help you get a fix in, if you have some time to test out my WIP patches. > That may

Re: limit clause + fetch optimization

2015-07-21 Thread Adam Silberstein
Thanks for the quick answer Gopal, and also for the details on that param. I indeed use JDBC in production, so will stay away from it. Just want to make sure I understand the behavior once that bug is fixed...a 'select *' with no limit will run without a M/R job and instead stream. Is that correc

Re: limit clause + fetch optimization

2015-07-21 Thread Gopal Vijayaraghavan
> I've been experimenting with 'select *' and 'select * limit X' in >beeline and watching the hive-server2 log to understand when a M/R job is >triggered and when not. It seems like whenever I set a limit, the job is >avoided, but with no limit, it is run. https://issues.apache.org/jira/browse/H

limit clause + fetch optimization

2015-07-21 Thread Adam Silberstein
Hi, I've been experimenting with 'select *' and 'select * limit X' in beeline and watching the hive-server2 log to understand when a M/R job is triggered and when not. It seems like whenever I set a limit, the job is avoided, but with no limit, it is run. I found this param: hive.limit.optimize.f

Re: Hive on Tez query failed with “wrong key class"

2015-07-21 Thread Jim Green
Sample stacktrace is : [Error: Failure while running task:java.lang.RuntimeException: org.apache.hadoop.hive.ql.metadata.HiveException: java.io.IOException: java.io.IOException: wrong key class: org.apache.hadoop.io.BytesWritable is not class org.apache.hadoop.io.NullWritable at org.apache.

Re: Hive on Tez query failed with ³wrong key class"

2015-07-21 Thread Gopal Vijayaraghavan
> Query is a simple group-by on top of sequence table. ... > java.io.IOException: java.io.IOException: wrong key class: >org.apache.hadoop.io.BytesWritable is not class >org.apache.hadoop.io.NullWritable I have seen this issue when mixing Sequence files written by PIG with Sequence files written

RE: Hive on Tez query failed with “wrong key class"

2015-07-21 Thread Bikas Saha
A full stack trace would help determine is this is a Tez issue or hive issue. From: Jim Green [mailto:openkbi...@gmail.com] Sent: Tuesday, July 21, 2015 11:12 AM To: u...@tez.apache.org; user@hive.apache.org Subject: Hive on Tez query failed with “wrong key class" Hi Team, Env: Hive 1.0 on Tez 0

Hive on Tez query failed with “wrong key class"

2015-07-21 Thread Jim Green
Hi Team, Env: Hive 1.0 on Tez 0.5.3 Query is a simple group-by on top of sequence table. It fails with below error on tez mode: *java.lang.RuntimeException: org.apache.hadoop.hive.ql.metadata.HiveException: * *java.io.IOException: java.io.IOException: wrong key class: org.apache.hadoop.io.BytesWr

Re: hbase column without prefix

2015-07-21 Thread Wojciech Indyk
Hello! I've posted a bug on this issue: https://issues.apache.org/jira/browse/HIVE-11329 What do you think? I can prepare a patch. Kindly regards Wojciech Indyk 2015-07-07 9:51 GMT+02:00 Wojciech Indyk : > Hi! > I use hbase column regex matching to create map column in hive, like: > "hbase.colum

Simulating a login trigger in hive

2015-07-21 Thread Steve Howard
We would like to assign a YARN queue to a user upon login. Is there any way to do this outside of the box? If not, is anyone aware of any development effort to do this? It sounds like it would be pretty simple to extend the Connection class to lookup a queue in a custom table in the metastore fo