Re: Pulling off server's debug info to Client

2012-07-05 Thread Navis류승우
I've modified hive to notify jobId for monitoring purpose. Check the issue and vote for acceptance ^^ https://issues.apache.org/jira/browse/HIVE-3235 2012/7/6 VanHuy Pham huy.pham...@gmail.com Hi all, I have been interested in getting the debug information and display it to clients.

Re: Hive to read sequencefile

2012-07-09 Thread Navis류승우
You can try 'STORED AS SEQUENCEFILE' when creating table. https://cwiki.apache.org/Hive/languagemanual-ddl.html#LanguageManualDDL-Create%252FDropTable 2012/7/10 Daning Wang dan...@netseer.com Hi all, New to here. I have sequence files and I want to read in Hive, I did some search on the

Re: Hive Thrift Client

2012-07-17 Thread Navis류승우
Can I ask the hive version you are currently using? 2012/7/17 Ben Kim benkimkim...@gmail.com Did you get around to solve this problem? I running into the same problem and can't find a fix anywhere. For Hiveserver you will need to create your own when running the server such as running hive

Re: Hive Thrift Client

2012-07-17 Thread Navis류승우
, 2012 at 3:03 PM, Navis류승우 navis@nexr.com wrote: Can I ask the hive version you are currently using? 2012/7/17 Ben Kim benkimkim...@gmail.com Did you get around to solve this problem? I running into the same problem and can't find a fix anywhere. For Hiveserver you will need to create

Re: cmd for to know the buckets

2012-07-18 Thread Navis류승우
Currently, configuring bucket num per partition is not allowed. If you want know the bucket num of table, use 'desc extended' or 'desc formatted' 2012/7/18 shaik ahamed shaik5...@gmail.com Hi users, As i would like to know the syntax or the cmd to know the buckets. For example

Re: Hive error: Unable to deserialize reduce input key

2012-09-06 Thread Navis류승우
I've tried to deserialize your data. 0 = bigint = -6341068275337623706 1 = string = TTFVUFHFH 2 = int = -1037822201 3 = int = -1467607277 4 = int = -1473682089 5 = int = -1337884091 6 = string = I 7 = string = IVH ISH 8 = int = -1321908327 9 = int = -1475321453 10 = int = -1476394752 11 = string

Re: What does ROW__OFFSET__INSIDE__BLOCK FROM mean?

2012-10-03 Thread Navis류승우
It seemed that ROW__OFFSET__INSIDE__BLOCK is meaningful only with SequenceFileFormat (+block compression) or RCFileFormat. 2012/10/3 Edward Capriolo edlinuxg...@gmail.com Make sure virtual column support is turned on in your hive-site.xml. I have a feeling that this field is only supported

Re: Book 'Programming Hive' from O'Reilly now available!

2012-10-10 Thread Navis류승우
Great works! Ant I've heard our team will translate your book into Korean. Let's sell it a lot. ^^ Regards, Navis 2012/10/1 Aniket Mokashi aniket...@gmail.com +1. Great work guys. Congrats! I just placed an order. ~Aniket On Sun, Sep 30, 2012 at 11:37 AM, varun kumar varun@gmail.com

Re: Hive Query Unable to distribute load evenly in reducers

2012-10-15 Thread Navis류승우
How about using MapJoin? 2012/10/16 Saurabh Mishra saurabhmishra.i...@outlook.com no there is apparently no heavy skewing. also another stats i wanted to point was, following is approximate table contents in this 4 table join query : tableA : 170 million (actual number, + i am also exploding

Re: reflect UDF can't handle both String and int

2012-10-25 Thread Navis류승우
It seemed a bug. I've booked it on https://issues.apache.org/jira/browse/HIVE-3622 2012/10/26 Alexander Ershov vohs...@gmail.com Query select reflect('java.lang.Integer', 'parseInt', 'a', 16) from src limit 1; throws java.lang.NoSuchMethodException: java.lang.Integer.parseInt(null, int) A

Re: MapJoin error: .hashtable file not found

2012-11-27 Thread Navis류승우
hi, The hashtable file you mentioned is pre-staged file for MapJoin. If you remove MAPJOIN hint and set auto.convert.join=false, it will be disabled 2012/11/28 Vivek Mishra vivek.mis...@impetus.co.in Hi, I am using 0.9.0 hive with Windows 2008 server. I did try debugging into code, but no

Re: Array index support non-constant expresssion

2012-12-12 Thread Navis류승우
Could you try it with CP/PPD disabled? set hive.optimize.cp=false; set hive.optimize.ppd=false; 2012/12/13 java8964 java8964 java8...@hotmail.com: Hi, I played my query further, and found out it is very puzzle to explain the following behaviors: 1) The following query works: select

Re: Array index support non-constant expresssion

2012-12-12 Thread Navis류승우
Different error messages but seemed from same problem. Could you do that with later versions of hive? I think these kind of bugs are fixed. 2012/12/13 java8964 java8964 java8...@hotmail.com: ExprNodeGenericFuncEvaluator

Re: Running commands at hive cli or hive thirft startup

2012-12-14 Thread Navis류승우
I've booked it on https://issues.apache.org/jira/browse/HIVE-3797 2012/12/14 Dean Wampler dean.wamp...@thinkbiganalytics.com: What about using hive -i ~/.hiverc -f script? I haven't tried it, but I assume it reads the specified rc file in the batch mode case. True? On Fri, Dec 14, 2012 at

Re: A few JIRAs closed in v0.10.0 that don't actually appear to be working

2013-01-29 Thread Navis류승우
HIVE-446 - Implement TRUNCATE : is on trunk (v0.11.0) HIVE-887 - Allow SELECT col without a mapreduce job : It needs set hive.fetch.task.conversion=more 2013/1/29 Dean Wampler dean.wamp...@thinkbiganalytics.com: Oh, another one is https://issues.apache.org/jira/browse/HIVE-446 - Implement

Re: unbalanced transaction calls

2013-02-22 Thread Navis류승우
Could you try to make transaction related methods synchronized? It's openTransaction/commitTransaction/isActiveTransaction/rollbackTransaction on ObjectStore.java 2013/2/22 Hemanth Yamijala yhema...@gmail.com: Hi, We are running into the same problem as well. Is there any clue what could be

Re: Get the job id for a hive query

2013-02-28 Thread Navis류승우
https://issues.apache.org/jira/browse/HIVE-3235 could be what you are searching for. 2013/3/1 Nitin Pawar nitinpawar...@gmail.com you can set this property mapred.job.name and this should set the name for the job On Thu, Feb 28, 2013 at 8:26 PM, Tim Bittersohl t...@innoplexia.comwrote:

Re: Use Hive reflect() method to call non-static JDK fuctions

2013-03-18 Thread Navis류승우
It's implemented in trunk, named as reflect2. https://issues.apache.org/jira/browse/HIVE-4025 2013/3/19 David Lee seek...@gmail.com: It's relatively straight forward to call static functions in JDK using reflect. For example, select reflect(java.lang.Math, max, 2, 3) from mytable limit 1;

Re: Important information required about Hive metastore connections

2013-03-18 Thread Navis류승우
Hive uses DBCP as a connection pool. Ref http://www.datanucleus.org/products/accessplatform_2_2/rdbms/dbcp.html 2013/3/19 Siddharth Tiwari siddharth.tiw...@live.com: Hi team, What we have observed is, hive creates so many connections to metastore (mysql). This makes us change the numer of

Re: different outer join plan between hive 0.9 and hive 0.10

2013-03-27 Thread Navis류승우
It's a bug (https://issues.apache.org/jira/browse/HIVE-4206). Thanks for reporting it. 2013/3/24 wzc wzc1...@gmail.com: Recently we tried to upgrade our hive from 0.9 to 0.10, but found some of our hive queries almost 7 times slow. One of such query consists multiple table outer join on the

Re: different outer join plan between hive 0.9 and hive 0.10

2013-03-28 Thread Navis류승우
The problem is mixture of issues (HIVE-3411, HIVE-4209, HIVE-4212, HIVE-3464) and still not completely fixed even in trunk. Will be fixed shortly. 2013/3/29 wzc wzc1...@gmail.com: The bug remains even if I apply the patch in HIVE-4206 :( The explain result hasn't change. 2013/3/28 Navis류승우

Re: A GenericUDF Function to Extract a Field From an Array of Structs

2013-04-02 Thread Navis류승우
try to change codes in evaluate method like, for (int i = 0; i numElements; i++) { Object element = listOI.getListElement(arguments[0].get(), i); Object product = structOI.getStructFieldData(element, structOI.getStructFieldRef(productCategory));

Re: Hive external table to HBase multiple versions

2013-04-29 Thread Navis류승우
https://issues.apache.org/jira/browse/HIVE-2906 seemed what you are looking for. 2013/4/30 Richard Payne rjp...@gmail.com Hi Is it possible to create a Hive external table pointing to an HBase column family and for all versions of the data for the keys to be returned rather than just I

Re: Very poor read performance with composite keys in hbase

2013-05-01 Thread Navis류승우
Currently, hive storage handler reads rows one by one. https://issues.apache.org/jira/browse/HIVE-3603 is for setting cache size, which is not yet fixed. 2013/5/1 kulkarni.swar...@gmail.com kulkarni.swar...@gmail.com: That depends on how dynamic your data is. If it is pretty static, you can

Re: 回复: different outer join plan between hive 0.9 and hive 0.10

2013-05-13 Thread Navis류승우
default.test_join z on a.key = z.key -- wzc1989 已使用 Sparrow 在 2013年3月29日星期五,上午9:34,Navis류승우 写道: The problem is mixture of issues (HIVE-3411, HIVE-4209, HIVE-4212, HIVE-3464) and still not completely fixed even in trunk. Will be fixed shortly. 2013/3/29 wzc wzc1...@gmail.com: The bug remains

Re: 回复: different outer join plan between hive 0.9 and hive 0.10

2013-05-14 Thread Navis류승우
cannot merge join 16+ aliases into single stage while hive 0.9 doesn't have such issue. could you explain this or give me some hint? Thanks! -- wzc1989 已使用 Sparrow 在 2013年5月14日星期二,下午12:26,Navis류승우 写道: The error message means hive cannot merge joins of 16+ aliases with outer join into single

Re: Access context from UDF

2013-06-13 Thread Navis류승우
If VCs are not on query, it'll not be made in ExecMapper from start. If you can do something with Reporter instance in MR, https://issues.apache.org/jira/browse/HIVE-3628 would be helpful. 2013/5/30 Peter Marron peter.mar...@trilliumsoftware.com: Hi, Using Hive 0.10.0 over Hadoop 1.0.4.

Re: How to terminate a running HIve Query? (Executed with JDBC, Hive Server 2)

2013-06-25 Thread Navis류승우
We uses https://issues.apache.org/jira/browse/HIVE-3235 and kill jobs if needed. 2013/6/26 Stephen Sprague sprag...@gmail.com all it is is a comment on the line above the first statement - and that'll show up in the jobtracker. Just as he shows in his example. On Tue, Jun 25, 2013 at 11:05

Re: 回复: different outer join plan between hive 0.9 and hive 0.10

2013-07-01 Thread Navis류승우
5月14日星期二,下午2:17,Navis류승우 写道: In short, hive tags rows a filter mask as a short for outer join, which can contain 16 flags. (see HIVE-3411, plz) I'll survey for a solution. 2013/5/14 wzc1989 wzc1...@gmail.com: hive cannot merge joins of 16+ aliases with outer join into single stage. In our

Re: Override COUNT() function

2013-07-02 Thread Navis류승우
MetadataOnlyOptimizer changes GBY on partition columns to simple TableScan with one line dummy. I think similar things can be done with stats. 2013/6/28 Peter Marron peter.mar...@trilliumsoftware.com: Hi, I feel sure that someone has asked for this before, but here goes… In the case

Re: Override COUNT() function

2013-07-02 Thread Navis류승우
have found. Are there any descriptions, motivations, documentation or examples anywhere? I suspect that there's nothing other than the source itself, but I had to ask. Regards, Z -Original Message- From: Navis류승우 [mailto:navis@nexr.com] Sent: 02 July 2013 08:50 To: user

Re: Possible to specify reducers for each stage?

2013-07-02 Thread Navis류승우
Currently it's not. https://issues.apache.org/jira/browse/HIVE-3946 2013/7/3 Felix.徐 ygnhz...@gmail.com: Hi all, Is it possible to specify reducer number for each stage ? how? thanks!

Re: Fetching Results from Hive Select (JDBC ResultSet.next() vs HiveClient.fetchN())

2013-07-03 Thread Navis류승우
It seemed stmt.setFetchSize(1); can be called before execution (without casting) 2013/7/3 Christian Schneider cschneiderpub...@gmail.com: Hi, i browsed through the sources and found a way to tune the JDBC ResultSet.next() performance. final Connection con =

Re: Strange error in hive

2013-07-09 Thread Navis류승우
Could you try to remove NULL as FLG_DEM_INC_PRX_CS_VALs in the query? It seemed not related to HIVE-4650 but still a bug (I'll book this) 2013/7/9 Jérôme Verdier verdier.jerom...@gmail.com: Hi, Thanks for your help. You can see logs below : java.lang.RuntimeException: Error in configuring

Re: export csv, use ',' as split

2013-07-10 Thread Navis류승우
Fixed in hive-0.11.0 https://issues.apache.org/jira/browse/HIVE-3682 2013/7/11 Sanjay Subramanian sanjay.subraman...@wizecommerce.com: Hive does not have a output delimiter specifier yet (not sure if 0.11.x may have it) But for now please try the following hive -e myquery | sed 's/\t/,/g'

Re: Strange error in hive

2013-07-15 Thread Navis류승우
for this ? 2013/7/10 Navis류승우 navis@nexr.com Attached patch for this in https://issues.apache.org/jira/browse/HIVE-4837 2013/7/10 Navis류승우 navis@nexr.com: Could you try to remove NULL as FLG_DEM_INC_PRX_CS_VALs in the query? It seemed not related to HIVE-4650 but still a bug (I'll

Re: [ANNOUNCE] New Hive Committer - Gunther Hagleitner

2013-07-22 Thread Navis류승우
I'm a little late. Congratulations Gunther and Brock! 2013/7/21 Prasanth J j.prasant...@gmail.com: Congrats Gunther! Thanks -- Prasanth On Jul 21, 2013, at 1:00 AM, Carl Steinbach c...@apache.org wrote: The Apache Hive PMC has voted to make Gunther Hagleitner a committer on the Apache

Re: Calling same UDF multiple times in a SELECT query

2013-07-23 Thread Navis류승우
It will be called 4 times whatever you annotated on the UDF if you are using released version of hive. https://issues.apache.org/jira/browse/HIVE-4209 , which will be included in 0.12.0, will make that single UDF call by caching result. 2013/7/24 Sanjay Subramanian

Re: Problem with the windowing function ntile (Exceptions)

2013-07-25 Thread Navis류승우
I've booked this and attached patch for it. https://issues.apache.org/jira/browse/HIVE-4932 Could you test with that? thanks. 2013/7/25 Lars Francke lars.fran...@gmail.com: We're still being bitten by this problem without a workaround. Does anyone have an idea? Thanks, Lars On Wed, Jul

Re: Get arguments' names in Hive's UDF

2013-08-08 Thread Navis류승우
I've booked this on https://issues.apache.org/jira/browse/HIVE-5025. 2013/7/22 Felix.徐 ygnhz...@gmail.com: Hi all, Is there any api to retrieve the parameter's column name in GenericUDF? For example: Select UDFTEST(columnA,columnB) from test; I want to get the column names(columnA and

Re: 回复: hive 0.11 auto convert join bug report

2013-08-11 Thread Navis류승우
Hi, I've booked this on https://issues.apache.org/jira/browse/HIVE-5056 and attached patch for it. It needs full test for confirmation but you can try it. Thanks. 2013/8/11 wzc1...@gmail.com: Hi all: when I change the table alias dim_pay_date to A, the query pass in hive

Re: 回复: hive 0.11 auto convert join bug report

2013-08-11 Thread Navis류승우
of 'deal' and 'dim_pay_date' are the same and the code in MapJoinProcessor.java ignores the order of rowschema. I look at your patch and it's exactly the same place we are working on. Thanks for your patch. 在 2013年8月11日星期日,下午9:38,Navis류승우 写道: Hi, I've booked this on https://issues.apache.org

Re: Problem in Hive Right Outer Join

2013-08-11 Thread Navis류승우
Could you upload DDLs for those tables? Thanks. 2013/8/8 Jérôme Verdier verdier.jerom...@gmail.com: Hi, I encountered a problem with Right Outer Join in Hive. Here is where is the problem : FROM default.ca ca JOIN default.kpi_magasin mtransf ON

Re: Bug when adding multiple partitions

2013-08-19 Thread Navis류승우
Looks like a bug. I'll fix that. 2013/8/15 Jan Dolinár dolik@gmail.com: Hi everyone, Consider following DDL: CREATE TABLE partition_test (a INT) PARTITIONED BY (b INT); ALTER TABLE partition_test ADD PARTITION (b=1) location '/tmp/test1' PARTITION

Re: Bug when adding multiple partitions

2013-08-19 Thread Navis류승우
https://issues.apache.org/jira/browse/HIVE-5122 2013/8/20 Navis류승우 navis@nexr.com: Looks like a bug. I'll fix that. 2013/8/15 Jan Dolinár dolik@gmail.com: Hi everyone, Consider following DDL: CREATE TABLE partition_test (a INT) PARTITIONED BY (b INT); ALTER

Re: [ANNOUNCE] New Hive Committer - Thejas Nair

2013-08-20 Thread Navis류승우
Congratulations! 2013/8/20 Clark Yang (杨卓荦) yangzhuo...@gmail.com: Congrats Thejas! 在 2013年8月20日星期二,Carl Steinbach 写道: The Apache Hive PMC has voted to make Thejas Nair a committer on the Apache Hive project. Please join me in congratulating Thejas!

Re: Interesting claims that seem untrue

2013-09-13 Thread Navis류승우
It looks like counting codes from company of committer, not of the author. Considering the huge amount of works by Ashutosh, it's not strange. 2013/9/13 Sanjay Subramanian sanjay.subraman...@wizecommerce.com I have not read the full blogs but in the year 2013 , IMHO , LOC is a very old

Re: 回复: hive 0.11 auto convert join bug report

2013-09-15 Thread Navis류승우
how might map to similar hashcode values? Also is changing the alias the only workaround for this problem or is there any other workaround possible? Thanks, Amit On Sun, Aug 11, 2013 at 9:22 PM, Navis류승우 navis@nexr.com wrote: Hi, Hive is notorious making different result

Re: [ANNOUNCE] New Hive PMC Members - Thejas Nair and Brock Noland

2013-10-24 Thread Navis류승우
Congrats! 2013/10/25 Gunther Hagleitner ghagleit...@hortonworks.com Congrats Thejas and Brock! Thanks, Gunther. On Thu, Oct 24, 2013 at 3:25 PM, Prasad Mujumdar pras...@cloudera.com wrote: Congratulations Thejas and Brock ! thanks Prasad On Thu, Oct 24, 2013 at

Re: Using Cluster by to improve Group by Performance

2013-11-01 Thread Navis류승우
From perspective of RS, two query is just different in hash code of RS key. The cost of calculating hash of col3 and col4 might be negligible, I think. 2013/11/1 KayVajj vajjalak...@gmail.com Any response or pointers to understand how Cluster By in sub queries can affect the performance/speed

Re: Developing a GenericUDAF

2013-11-11 Thread Navis류승우
in handling PARTIAL1, inputOI = (StandardListObjectInspector) parameters[0]; return ObjectInspectorFactory.getStandardListObjectInspector(inputOI); 1. inputOI is not guaranteed to be a StandardListObjectInspector. Use ListObjectInspector instead. 2.

Re: HiveServer2

2013-11-19 Thread Navis류승우
I've booked on https://issues.apache.org/jira/browse/HIVE-5858 for the ALTER TABLE issue, mentioned by David Morel (Thanks). 2013/11/20 David Morel dmore...@gmail.com On 18 Nov 2013, at 21:59, Stephen Sprague wrote: A word of warning for users of HiveServer2 - version 0.11 at least. This

Re: hive.query.string not reflecting the current query

2013-12-03 Thread Navis류승우
Looks like a bug. I've booked this on https://issues.apache.org/jira/browse/HIVE-5935. 2013/12/4 Adam Kawa kawa.a...@gmail.com Maybe you can parse the output of EXPLAIN operator applied on your query https://cwiki.apache.org/confluence/display/Hive/LanguageManual+Explain or look for other

Re: requesting access to hive confluence wiki

2013-12-10 Thread Navis류승우
Is there someone who knows how to do this? 2013/11/30 Xiao Meng xi...@simba.com Hi, I would like update/fix some contents on the performance test part. My user name is xiaom. Thanks, Xiao

Re: Limitations in the IndexPredicateAnalyzer

2013-12-17 Thread Navis류승우
IndexPredicateAnalyzer in hive supports AND conjunction only because it's simple. Anyone can implement one that supports conjunctions like OR, CASE, etc. if needed. If you might provide that to hive community, it would be really appreciated. ps. There is a draft patch handling OR conjuncted

Re: Pointing multiple external tables to the same location

2013-12-17 Thread Navis류승우
Hive uses path to table(or partition) mapping internally (you can see that in MapredWork, etc.), which might caused first table overwritten by other. I didn't tried symlink on hdfs, which could be a solution. 2013/12/12 Petter von Dolwitz (Hem) petter.von.dolw...@gmail.com Hi, I have

Re: OOM/GC limit Error

2013-12-29 Thread Navis류승우
Could you post hive version and execution plan for the query? 2013/12/21 Martin, Nick nimar...@pssd.com Hi all, I have two tables: tbl1: 81m rows tbl2: 4m rows tbl1 is partitioned on one column and tbl2 has none. I’m attempting the following query: SELECT tbl1.col_pk,

Re: Pointing multiple external tables to the same location

2014-01-13 Thread Navis류승우
Navis류승우 navis@nexr.com Hive uses path to table(or partition) mapping internally (you can see that in MapredWork, etc.), which might caused first table overwritten by other. I didn't tried symlink on hdfs, which could be a solution. 2013/12/12 Petter von Dolwitz (Hem) petter.von.dolw

Re: Why hiveserver2 is much slower than hiveserver1?

2014-01-18 Thread Navis류승우
HIVE-3746 modified IDL to handle nulls in more performant manner. (hiveserver1 does not handle nulls) Thanks. 2013/11/6 B C ahatest2...@yahoo.com hi, We are building ms sql cube by linkedserver connecting to hiveserver with Cloudera's ODBC driver. There are two test results: 1.

Re: DESCRIBE EXTENDED show numRows=0

2014-01-26 Thread Navis류승우
If the data is loaded into table with insert clause with hive.stats.autogather=true, it will be updated. If it's not, it's zero (or -1 sometimes). 2014-01-23 Haroon Muhammad muhammad.har...@live.com Hi, I have observed that DESCRIBE EXTENDED always shows number of rows to be zero despite

Re: DESCRIBE EXTENDED show numRows=0

2014-02-02 Thread Navis류승우
if hive.stats.autogather was 'false' when the data was loaded? (See the wiki's Statistics in Hive doc: Existing Tableshttps://cwiki.apache.org/confluence/display/Hive/StatsDev#StatsDev-ExistingTables .) -- Lefty On Sun, Jan 26, 2014 at 8:01 PM, Navis류승우 navis@nexr.com wrote: If the data

Re: Performance problem with HBase

2014-02-02 Thread Navis류승우
1. current implementation of hbase handler cannot pushdown filter with 'like' expression. You might rewrite the query some thing like key = 0010_0 AND key = 0010_9 2. Each of all tasks seemed to be scanning whole table (ie. 1000+ time), which is fixed in

Re: Map-side join memory limit is too low

2014-02-02 Thread Navis류승우
try set hive.mapred.local.mem=7000 or add it to hive-site.xml instead of modifying hive-env.sh HADOOP_HEAPSIZE is not in use. Should fix documentation of it. Thanks, Navis 2014-01-31 Avrilia Floratou avrilia.flora...@gmail.com: Hi, I'm running hive 0.12 on yarn and I'm trying to convert a

Re: External table reference subDirectories

2014-02-05 Thread Navis류승우
It's supposed to be implemented after https://issues.apache.org/jira/browse/HIVE-1662 would be checked in. But it's not in progress for a year. Thanks, 2014-02-06 John Meza j_meza...@hotmail.com: A couple of simple questions on logfile organization in HDFS and referenced by an external

Re: Hbase + Hive scan performance

2014-02-10 Thread Navis류승우
HBase storage handler uses it's own InputFormat. So, hbase.client.scanner.caching (which is used in hbase.TableInputFormat) does not work. It might be configurable via HIVE-2906, something like select empno, ename from hbase_emp ('hbase.scan.cache'='1000'). But I've not tried. bq. Is there any

Re: Issue with Hive and table with lots of column

2014-02-12 Thread Navis류승우
With HIVE-3746, which will be included in hive-0.13, HiveServer2 takes less memory than before. Could you try it with the version in trunk? 2014-02-13 10:49 GMT+09:00 Stephen Sprague sprag...@gmail.com: question to the original poster. closure appreciated! On Fri, Jan 31, 2014 at 12:22

Re: Sampling from a single column

2014-02-12 Thread Navis류승우
If it should be sampled using subquery would be inevitable, something like, select x from (select distinct key as x from src)a where rand() 0.9 limit 10; 2014-02-12 6:07 GMT+09:00 Oliver Keyes oke...@wikimedia.org: Hey all So, what I'm looking to do is get N randomly-sampled distinct

Re: Hive Query :: Implementing case statement

2014-02-18 Thread Navis류승우
If key is unique, you might overwrite values by using hbase handler. 2014-02-18 22:05 GMT+09:00 yogesh dhari yogeshh...@gmail.com: Yes, Hive does not provide update statement, I am just looking for the work arround it, how to implement it On Tue, Feb 18, 2014 at 6:27 PM, Peter Marron

Re: KryoException -missing no-arg constructor ASTNodeOrigin

2014-02-26 Thread Navis류승우
It's HIVE-5779 and will be fixed in hive-0.13.0. Thanks, Navis 2014-02-21 21:07 GMT+09:00 Rafal Janik rafal.ja...@softwaremind.pl: Hi All, I've just started my adventure with Hive so I'm not sure if it's an issue here or just my misunderstanding... I'm using Hortonworks Sandbox 2.0 (Hive

Re: [ANNOUNCE] New Hive PMC Member - Xuefu Zhang

2014-03-01 Thread Navis류승우
Congratulations, Xuefu! 2014-03-01 14:38 GMT+09:00 Lefty Leverenz leftylever...@gmail.com: Congrats Xuefu! -- Lefty On Fri, Feb 28, 2014 at 2:52 PM, Eric Hanson (BIG DATA) eric.n.han...@microsoft.com wrote: Congratulations Xuefu! -Original Message- From: Remus Rusanu

Re: Limited capabilities of a custom input format

2014-03-03 Thread Navis류승우
You might be interested in https://issues.apache.org/jira/browse/HIVE-1662, using predicate on file-name vc to filter out inputs. For example, select key,INPUT__FILE__NAME from srcbucket2 where INPUT__FILE__NAME rlike '.*/srcbucket2[03].txt' But it's not committed, yet. Thanks, 2014-03-03

Re: Hive hbase handler composite key - hbase full scan on key

2014-03-03 Thread Navis류승우
https://issues.apache.org/jira/browse/HIVE-6411 is exactly for the cases. The bad new is that it seemed not included even in 0.13.0 and you should implement own predicate analyzer. Thanks, Navis 2014-03-03 20:52 GMT+09:00 Juraj jiv fatcap@gmail.com: Hello, im currently testing Hbase

Re: How to solve the garbage problem

2014-03-04 Thread Navis류승우
Declare it as a binary column and use decoding UDF when accessing it. Thanks, Navis 2014-02-20 12:21 GMT+09:00 kun yan yankunhad...@gmail.com: Hi all GBK encoding data files, but the hive is UTF-8 encoding select * from table display normal I try to modify the following configuration

Re: Limited capabilities of a custom input format

2014-03-04 Thread Navis류승우
partitioning on file level rather than on directory level? Thanks, Petter 2014-03-04 7:32 GMT+01:00 Navis류승우 navis@nexr.com: You might be interested in https://issues.apache.org/jira/browse/HIVE-1662, using predicate on file-name vc to filter out inputs. For example, select key

Re: Using an UDF in the WHERE (IN) clause

2014-03-10 Thread Navis류승우
(KW_IN expressions) - ^(TOK_FUNCTION KW_IN $precedenceEqualExpression expressions) expressions : LPAREN expression (COMMA expression)* RPAREN - expression* ; You should have arguments of IN wrapped by parentheses. But It seemed not possible to use array returning expression in

Re: Using an UDF in the WHERE (IN) clause

2014-03-11 Thread Navis류승우
Then you should use BETWEEN, not IN. BETWEEN can be used for PPD, afaik. 2014-03-11 16:33 GMT+09:00 Petter von Dolwitz (Hem) petter.von.dolw...@gmail.com: Hi Young, I must argue that the partition pruning do actually work if I don't use the IN clause. What I wanted to achieve in my original

Re: full outer join result

2014-03-12 Thread Navis류승우
The 100 100 is the sole matching row for the join condition, so it would be right result, NULLNULLNULL40 NULLNULL12 35 NULLNULL48 NULL NULL40 NULLNULL 12 35 NULLNULL 48 NULLNULLNULL 100 100 100 100 It's

Re: ALTER TABLE RENAME TO fully qualified name

2014-03-13 Thread Navis류승우
HIVE-4064 is expected to solve this kind of problems in hive but not in progress even it's not that hard to implement. I'll take a look at it when I'm free. Thanks, Navis 2014-03-13 22:08 GMT+09:00 Clay McDonald stuart.mcdon...@bateswhite.com: Hello everyone, I'm running Hive 0.12.0 and I

Re: Job killed on HiveServer2 restart

2014-03-13 Thread Navis류승우
User provided classes (by adding jars) should be unloaded when the session is closed. https://issues.apache.org/jira/browse/HIVE-3969 is about that but it's not resolved yet. Thanks, Navis 2014-03-12 8:56 GMT+09:00 Ashu Pachauri a...@rocketfuelinc.com: We use Hive-0.12 and are planning to use

Re: ALTER TABLE RENAME TO fully qualified name

2014-03-14 Thread Navis류승우
-2584 -- chenchun On Friday, 14 March, 2014 at 11:50 AM, Navis류승우 wrote: HIVE-4064 is expected to solve this kind of problems in hive but not in progress even it's not that hard to implement. I'll take a look at it when I'm free. Thanks, Navis 2014-03-13 22:08 GMT+09:00 Clay McDonald

Re: Issue with Querying External Hive Table created on hbase

2014-03-19 Thread Navis류승우
You can check the exact reason from job log, but generally, it's caused by missing libs in auxlib conf. Thar includes hive-hbase-handler.jar, hbase-*.jar, guava-*.jar, zookeeper-*.jar, etc. ,varying the version of your hive and hbase. Thanks, Navis 2014-03-20 3:42 GMT+09:00 Sunil Ranka

Re: Hive vs Pig against number of files spawned

2014-04-01 Thread Navis류승우
try hive.hadoop.supports.splittable.combineinputformat=true; Thanks, Navis 2014-04-01 15:55 GMT+09:00 Sreenath sreenaths1...@gmail.com: Hi all, I have a partitioned table in hive where each partition will have 630 gzip compressed files each of average size 100kb. If I query over these files

Re: Reading query columns in ExecuteWithHookContext

2014-04-14 Thread Navis류승우
It's a bug in ColumnAccessAnalyzer. I've booked this on https://issues.apache.org/jira/browse/HIVE-6910. Thanks, Navis 2014-04-15 11:41 GMT+09:00 Adeel Qureshi adeelmahm...@gmail.com: I am trying to read the columns from hive queries being executed by implementing the ExecuteWithHookContext

Re: what is the meaning of Table.createTime()?

2014-05-21 Thread Navis류승우
It's seconds. new Date(time * 1000); 2014-05-22 14:19 GMT+09:00 Santhosh Thomas santhosh.tho...@yahoo.com: I am trying to find the creation time of a table using table.createTime() function. I was hoping that it returns the time in milli seconds, but looks like it is not. Any idea how to

Re: Hive 0.13/Hcatalog : Mapreduce Exception : java.lang.IncompatibleClassChangeError

2014-06-04 Thread Navis류승우
It's fixed in HIVE-6432. I think you should rebuild your own hcatalog from source with profile -Phadoop-1. 2014-06-05 9:08 GMT+09:00 Sundaramoorthy, Malliyanathan malliyanathan.sundaramoor...@citi.com: Hi, I am using Hadoop 2.4.0 with Hive 0.13 + included package of HCatalog . Wrote a

Re: desc database extended dbname doesn't print dbproperties?

2014-06-25 Thread Navis류승우
Seemed regression of HIVE-6386. Will be fixed in next version. 2014-06-26 7:58 GMT+09:00 Sumit Kumar skbrnwl-...@yahoo.com: Hey guys, I just discovered that this syntax doesn't print the dbproperties any more. I've two hive versions that i'm testing following query on: create database

Re: desc database extended dbname doesn't print dbproperties?

2014-06-25 Thread Navis류승우
Booked in https://issues.apache.org/jira/browse/HIVE-7298 Thanks, 2014-06-26 14:28 GMT+09:00 Navis류승우 navis@nexr.com: Seemed regression of HIVE-6386. Will be fixed in next version. 2014-06-26 7:58 GMT+09:00 Sumit Kumar skbrnwl-...@yahoo.com: Hey guys, I just discovered

Re: mapjoin error

2014-07-08 Thread Navis류승우
Looks like HIVE-6913(https://issues.apache.org/jira/browse/HIVE-6913) and will be fixed in hive-0.14.0. Thanks, Navis 2014-07-04 17:12 GMT+09:00 sunww spe...@outlook.com: Hi I'm using hive0.11 and hadoop2.2. When I use a large table join on two empty tables , it convert to mapjoin

Re: Possible memory leak with 0.13 and JDBC

2014-07-08 Thread Navis류승우
Could you try jmap -histo:live pid and check hive objects which seemed too many? Thanks, Navis 2014-07-07 22:22 GMT+09:00 jonas.partner jonas.part...@opencredo.com: Hi Benjamin, Unfortunately this was a really critical issue for us and I didn’t think we would find a fix in time so we

Re: Hive UDF performance issue

2014-07-09 Thread Navis류승우
It's cross producting. Not strange taking so much time even with small tables. Thanks, Navis 2014-07-09 2:53 GMT+09:00 Malligarjunan S malligarju...@gmail.com: Hello All, Can any one help me to answer to my question posted on Stackoverflow?

Re: Possible memory leak with 0.13 and JDBC

2014-07-09 Thread Navis류승우
java.lang.Long 50:34 7176 [Z On 8 July 2014 at 08:40:20, Navis류승우 (navis@nexr.com //navis@nexr.com) wrote: Could you try jmap -histo:live pid and check hive objects which seemed too many? Thanks, Navis 2014-07-07 22:22 GMT+09:00 jonas.partner jonas.part

Re: Hive UDF performance issue

2014-07-09 Thread Navis류승우
Yes, 2M x 1M makes 2T pairing in single reducer. Thanks, Navis 2014-07-10 1:50 GMT+09:00 Malligarjunan S malligarju...@gmail.com: Hello All, Is that the expected behavior from hive to take so much of time? Thanks and Regards, Sankar S On Tue, Jul 8, 2014 at 11:23 PM, Malligarjunan S

Re: pass new job name to tez

2014-07-09 Thread Navis류승우
In GenTezProcContext, you can find new TezWork(conf.getVar(HiveConf.ConfVars.HIVEQUERYID)) part. And the final name is name + : + (++counter); Thanks, Navis 2014-07-10 12:43 GMT+09:00 Grandl Robert rgra...@yahoo.com: Hi guys, I am trying to identify a DAG in Tez with a different id, based

Re: Hive can I contribute to Hive confluence wiki documents?

2014-07-13 Thread Navis류승우
For cache.expr.evaluation problem, it's fixed in https://issues.apache.org/jira/browse/HIVE-7314 (will be included in next release) But I agree that it can be a critical problem for users of 0.12.0 and 0.13.x version and need proper warning on that. Thanks, Navis 2014-07-13 16:48 GMT+09:00 郭士伟

Re: Custom HBaseKeyFactory and ColumnMapping

2014-07-16 Thread Navis류승우
My bad. Could you do that? Thanks, Navis 2014-07-17 9:15 GMT+09:00 Andrew Mains andrew.ma...@kontagent.com: Hi all, I'm currently experimenting with using the new HBaseKeyFactory interface (implemented in https://issues.apache.org/jira/browse/HIVE-6411) to do some custom serialization

Re: random NPE in HiveInputFormat.init() ??

2014-07-20 Thread Navis류승우
I thinks it's fixed in https://issues.apache.org/jira/browse/HIVE-7011, which will be included in hive-0.14.0. Sadly, there seemed no simple walkaround for this. Thanks, Navis 2014-07-19 15:03 GMT+09:00 Yang tedd...@gmail.com: we are getting a random (happening about 20% of the time, if

Re: exchange partition documentation

2014-07-20 Thread Navis류승우
HIVE-4095, originally intended to implement, alter table target_table_name exchange partition (partition spec) with table source_table_name; But in implementation, Dheeraj Kumar Singh, the original implementor, seemed confused and implemented this in inverted manner (target to source).

Re: Help on restricting users

2014-07-20 Thread Navis류승우
You can implement that in Hook and register in hive-site.xml. Thanks, Navis 2014-07-19 17:32 GMT+09:00 sai chaitanya tirumerla tirume...@gmail.com: Hi, I would like to restrict users doing select * from table; when accessed from any jdbc/odbc tools like sql workbench/excel etc..

Re: Help on restricting users

2014-07-21 Thread Navis류승우
to look at. Thanks, Sai On Sun, Jul 20, 2014 at 9:23 PM, Navis류승우 navis@nexr.com wrote: You can implement that in Hook and register in hive-site.xml. Thanks, Navis 2014-07-19 17:32 GMT+09:00 sai chaitanya tirumerla tirume...@gmail.com: Hi, I would like to restrict users doing

Re: Hive UDF gives duplicate result regardless of parameters, when nested in a subquery

2014-07-24 Thread Navis류승우
Looks like it's caused by HIVE-7314. Could you try that with hive.cache.expr.evaluation=false? Thanks, Navis 2014-07-24 14:34 GMT+09:00 丁桂涛(桂花) dinggui...@baixing.com: Yes. The output is correct: [tp,p,sp]. I developed the UDF using JAVA in eclipse and exported the jar file into the auxlib

  1   2   >