How to query data by page in Hive?
Hello, How to query data by page in Hive? hive select * from u_data a limit 1,2; FAILED: ParseException line 1:31 missing EOF at ',' near '1' r7raul1...@163.com
Re: [ANNOUNCE] Apache Hive 1.0.0 Released
+1 Updated https://en.wikipedia.org/wiki/Apache_Hive with the latest version info regards Devopam On Thu, Feb 5, 2015 at 11:46 PM, Thejas Nair thejas.n...@gmail.com wrote: Congrats to all the users and contributors in the Apache Hive community! It is great that we finally move away from the 0.x versioning scheme to the 1.x versioning scheme for new releases. This is a great way of honoring the work from hive community that has made hive the defacto standard for SQL on Hadoop! Thanks for the hard work of driving the release Vikram! On Wed, Feb 4, 2015 at 3:07 PM, Vikram Dixit K vikram.di...@gmail.com wrote: The Apache Hive team is proud to announce the the release of Apache Hive version 1.0.0. The Apache Hive (TM) data warehouse software facilitates querying and managing large datasets residing in distributed storage. Built on top of Apache Hadoop (TM), it provides: * Tools to enable easy data extract/transform/load (ETL) * A mechanism to impose structure on a variety of data formats * Access to files stored either directly in Apache HDFS (TM) or in other data storage systems such as Apache HBase (TM) * Query execution via Apache Hadoop MapReduce and Apache Tez frameworks. For Hive release details and downloads, please visit:https://hive.apache.org/downloads.html Hive 1.0.0 Release Notes are available here: https://issues.apache.org/jira/secure/ReleaseNote.jspa?version=12329278styleName=TextprojectId=12310843 We would like to thank the many contributors who made this release possible. Regards, The Apache Hive Team -- Devopam Mittra Life and Relations are not binary
Re: [ANNOUNCE] Apache Hive 1.0.0 Released
Congrats! Great job! Thanks, Zhuoluo (Clark) Yang 2015-02-05 6:07 GMT+08:00 Vikram Dixit K vikram.di...@gmail.com: The Apache Hive team is proud to announce the the release of Apache Hive version 1.0.0. The Apache Hive (TM) data warehouse software facilitates querying and managing large datasets residing in distributed storage. Built on top of Apache Hadoop (TM), it provides: * Tools to enable easy data extract/transform/load (ETL) * A mechanism to impose structure on a variety of data formats * Access to files stored either directly in Apache HDFS (TM) or in other data storage systems such as Apache HBase (TM) * Query execution via Apache Hadoop MapReduce and Apache Tez frameworks. For Hive release details and downloads, please visit:https://hive.apache.org/downloads.html Hive 1.0.0 Release Notes are available here: https://issues.apache.org/jira/secure/ReleaseNote.jspa?version=12329278styleName=TextprojectId=12310843 We would like to thank the many contributors who made this release possible. Regards, The Apache Hive Team
Re: [ANNOUNCE] Apache Hive 1.0.0 Released
It is mainly a collection of bug fix on 0.14.0, is it? Thanks, Will On Thu, Feb 5, 2015 at 8:38 AM, Clark Yang (杨卓荦) yangzhuo...@gmail.com wrote: Congrats! Great job! Thanks, Zhuoluo (Clark) Yang 2015-02-05 6:07 GMT+08:00 Vikram Dixit K vikram.di...@gmail.com: The Apache Hive team is proud to announce the the release of Apache Hive version 1.0.0. The Apache Hive (TM) data warehouse software facilitates querying and managing large datasets residing in distributed storage. Built on top of Apache Hadoop (TM), it provides: * Tools to enable easy data extract/transform/load (ETL) * A mechanism to impose structure on a variety of data formats * Access to files stored either directly in Apache HDFS (TM) or in other data storage systems such as Apache HBase (TM) * Query execution via Apache Hadoop MapReduce and Apache Tez frameworks. For Hive release details and downloads, please visit:https://hive.apache.org/downloads.html Hive 1.0.0 Release Notes are available here: https://issues.apache.org/jira/secure/ReleaseNote.jspa?version=12329278styleName=TextprojectId=12310843 We would like to thank the many contributors who made this release possible. Regards, The Apache Hive Team -- Thanks, Dayong
Re: [ANNOUNCE] Apache Hive 1.0.0 Released
I know. But comparing to 0.14.0 and 0.13.0, version 1.0.0 is expected to have much, is it? On Thu, Feb 5, 2015 at 9:44 AM, grimaldi.vince...@gmail.com grimaldi.vince...@gmail.com wrote: Please note that the Tez support is for version 0.5.2 now. 2015-02-05 14:33 GMT+00:00 DU DU will...@gmail.com: It is mainly a collection of bug fix on 0.14.0, is it? Thanks, Will On Thu, Feb 5, 2015 at 8:38 AM, Clark Yang (杨卓荦) yangzhuo...@gmail.com wrote: Congrats! Great job! Thanks, Zhuoluo (Clark) Yang 2015-02-05 6:07 GMT+08:00 Vikram Dixit K vikram.di...@gmail.com: The Apache Hive team is proud to announce the the release of Apache Hive version 1.0.0. The Apache Hive (TM) data warehouse software facilitates querying and managing large datasets residing in distributed storage. Built on top of Apache Hadoop (TM), it provides: * Tools to enable easy data extract/transform/load (ETL) * A mechanism to impose structure on a variety of data formats * Access to files stored either directly in Apache HDFS (TM) or in other data storage systems such as Apache HBase (TM) * Query execution via Apache Hadoop MapReduce and Apache Tez frameworks. For Hive release details and downloads, please visit:https://hive.apache.org/downloads.html Hive 1.0.0 Release Notes are available here: https://issues.apache.org/jira/secure/ReleaseNote.jspa?version=12329278styleName=TextprojectId=12310843 We would like to thank the many contributors who made this release possible. Regards, The Apache Hive Team -- Thanks, Dayong -- [image: photo] *Vincenzo Grimaldi* Senior Business Intelligence Consultant p:00 39 380 52 22 218 | m:00 353 851 69 84 58 | e: grimaldi.vince...@gmail.com | a: Aprt. 148A, Smithfield Market, Smithfield, Dublin 7, Ireland https://mail.google.com/mail/u/0/facebook.com/vgrimaldi2 https://mail.google.com/mail/u/0/ie.linkedin.com/pub/vincenzo-grimaldi/14/422/bb1/ -- Thanks, Dayong
Re: [ANNOUNCE] Apache Hive 1.0.0 Released
Please note that the Tez support is for version 0.5.2 now. 2015-02-05 14:33 GMT+00:00 DU DU will...@gmail.com: It is mainly a collection of bug fix on 0.14.0, is it? Thanks, Will On Thu, Feb 5, 2015 at 8:38 AM, Clark Yang (杨卓荦) yangzhuo...@gmail.com wrote: Congrats! Great job! Thanks, Zhuoluo (Clark) Yang 2015-02-05 6:07 GMT+08:00 Vikram Dixit K vikram.di...@gmail.com: The Apache Hive team is proud to announce the the release of Apache Hive version 1.0.0. The Apache Hive (TM) data warehouse software facilitates querying and managing large datasets residing in distributed storage. Built on top of Apache Hadoop (TM), it provides: * Tools to enable easy data extract/transform/load (ETL) * A mechanism to impose structure on a variety of data formats * Access to files stored either directly in Apache HDFS (TM) or in other data storage systems such as Apache HBase (TM) * Query execution via Apache Hadoop MapReduce and Apache Tez frameworks. For Hive release details and downloads, please visit:https://hive.apache.org/downloads.html Hive 1.0.0 Release Notes are available here: https://issues.apache.org/jira/secure/ReleaseNote.jspa?version=12329278styleName=TextprojectId=12310843 We would like to thank the many contributors who made this release possible. Regards, The Apache Hive Team -- Thanks, Dayong -- [image: photo] *Vincenzo Grimaldi* Senior Business Intelligence Consultant p:00 39 380 52 22 218 | m:00 353 851 69 84 58 | e: grimaldi.vince...@gmail.com | a: Aprt. 148A, Smithfield Market, Smithfield, Dublin 7, Ireland https://mail.google.com/mail/u/0/facebook.com/vgrimaldi2 https://mail.google.com/mail/u/0/ie.linkedin.com/pub/vincenzo-grimaldi/14/422/bb1/
Re: Re: How to query data by page in Hive?
ROW_NUMBER doc http://docs.oracle.com/cd/B28359_01/server.111/b28286/functions144.htm#SQLRF06100 On Thu, Feb 5, 2015 at 4:48 PM, r7raul1...@163.com r7raul1...@163.com wrote: *Table structure :* CREATE TABLE `u_data`( `userid` int, `movieid` int, `rating` int, `unixtime` string) ROW FORMAT DELIMITED FIELDS TERMINATED BY '\t' STORED AS INPUTFORMAT 'org.apache.hadoop.mapred.TextInputFormat' OUTPUTFORMAT 'org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat' LOCATION 'hdfs://localhost:8020/user/hive/warehouse/u_data' TBLPROPERTIES ( 'COLUMN_STATS_ACCURATE'='true', 'numFiles'='1', 'numRows'='0', 'rawDataSize'='0', 'totalSize'='1979173', 'transient_lastDdlTime'='1421076916') *columns :* movieid -- r7raul1...@163.com *From:* Devopam Mittra devo...@gmail.com *Date:* 2015-02-05 18:48 *To:* user@hive.apache.org *Subject:* Re: Re: How to query data by page in Hive? Please provide a valid table structure and the columns you wish to pick and I shall email you the query directly regards Devopam On Thu, Feb 5, 2015 at 3:20 PM, r7raul1...@163.com r7raul1...@163.com wrote: Thank you Devopam! Could you show me a example? -- r7raul1...@163.com *From:* Devopam Mittra devo...@gmail.com *Date:* 2015-02-05 18:05 *To:* user@hive.apache.org *Subject:* Re: How to query data by page in Hive? You may want to use a ROW_NUMBER OR RANK / DENSE RANK in the inner query and then select only a subset of it in the outer query to control pagination. Based on your need, you may want to order the records as well .. Alternatively you may want to use CTE( https://cwiki.apache.org/confluence/display/Hive/Common+Table+Expression) for selecting the data in one go and then use row number to select as in previous case. regards Devopam On Thu, Feb 5, 2015 at 1:31 PM, r7raul1...@163.com r7raul1...@163.com wrote: Hello, How to query data by page in Hive? hive select * from u_data a limit 1,2; FAILED: ParseException line 1:31 missing EOF at ',' near '1' -- r7raul1...@163.com -- Devopam Mittra Life and Relations are not binary -- Devopam Mittra Life and Relations are not binary
Re: How to query data by page in Hive?
You may want to use a ROW_NUMBER OR RANK / DENSE RANK in the inner query and then select only a subset of it in the outer query to control pagination. Based on your need, you may want to order the records as well .. Alternatively you may want to use CTE( https://cwiki.apache.org/confluence/display/Hive/Common+Table+Expression) for selecting the data in one go and then use row number to select as in previous case. regards Devopam On Thu, Feb 5, 2015 at 1:31 PM, r7raul1...@163.com r7raul1...@163.com wrote: Hello, How to query data by page in Hive? hive select * from u_data a limit 1,2; FAILED: ParseException line 1:31 missing EOF at ',' near '1' -- r7raul1...@163.com -- Devopam Mittra Life and Relations are not binary
Re: Re: How to query data by page in Hive?
Thank you Devopam! Could you show me a example? r7raul1...@163.com From: Devopam Mittra Date: 2015-02-05 18:05 To: user@hive.apache.org Subject: Re: How to query data by page in Hive? You may want to use a ROW_NUMBER OR RANK / DENSE RANK in the inner query and then select only a subset of it in the outer query to control pagination. Based on your need, you may want to order the records as well .. Alternatively you may want to use CTE(https://cwiki.apache.org/confluence/display/Hive/Common+Table+Expression) for selecting the data in one go and then use row number to select as in previous case. regards Devopam On Thu, Feb 5, 2015 at 1:31 PM, r7raul1...@163.com r7raul1...@163.com wrote: Hello, How to query data by page in Hive? hive select * from u_data a limit 1,2; FAILED: ParseException line 1:31 missing EOF at ',' near '1' r7raul1...@163.com -- Devopam Mittra Life and Relations are not binary
Re: Re: How to query data by page in Hive?
Please provide a valid table structure and the columns you wish to pick and I shall email you the query directly regards Devopam On Thu, Feb 5, 2015 at 3:20 PM, r7raul1...@163.com r7raul1...@163.com wrote: Thank you Devopam! Could you show me a example? -- r7raul1...@163.com *From:* Devopam Mittra devo...@gmail.com *Date:* 2015-02-05 18:05 *To:* user@hive.apache.org *Subject:* Re: How to query data by page in Hive? You may want to use a ROW_NUMBER OR RANK / DENSE RANK in the inner query and then select only a subset of it in the outer query to control pagination. Based on your need, you may want to order the records as well .. Alternatively you may want to use CTE( https://cwiki.apache.org/confluence/display/Hive/Common+Table+Expression) for selecting the data in one go and then use row number to select as in previous case. regards Devopam On Thu, Feb 5, 2015 at 1:31 PM, r7raul1...@163.com r7raul1...@163.com wrote: Hello, How to query data by page in Hive? hive select * from u_data a limit 1,2; FAILED: ParseException line 1:31 missing EOF at ',' near '1' -- r7raul1...@163.com -- Devopam Mittra Life and Relations are not binary -- Devopam Mittra Life and Relations are not binary
Re: Predicate push-down on nested types ?
ORC does not support at this point. There are plans to do so. On Feb 5, 2015, at 10:29 AM, The Watcher watche...@gmail.com wrote: I'm wondering if predicates are pushed down when they apply to elements of a nested struct. More specifically, imagine a table such as CREATE TABLE t ( c1 int, c2 STRUCTa:int,b:int ) STORED as XXX; Will SELECT * from t where c2.b=3 be pushed down to ORC ? Would it be pushed down to Parquet ? Thanks
hive-testbench install fails
hello all, i am trying to install the hive testbench( https://github.com/hortonworks/hive-testbench) on my cluster but it seems to fail with some maven error…below is the output for installing that tool…any help is appreciated… http://pastebin.com/MJecpGxh Funny thing is, i was able to install this on cloudera, but now when i am trying to do the same on hortonworks it does not work..
Re: Predicate push-down on nested types ?
Thanks for the quick reply. I suppose you are referring to https://issues.apache.org/jira/browse/HIVE-7214 ? Does Parquet really already support this ? That ticket says it does but I'd like confirmation Thanks 2015-02-05 19:47 GMT+01:00 Prasanth Jayachandran pjayachand...@hortonworks.com: ORC does not support at this point. There are plans to do so. On Feb 5, 2015, at 10:29 AM, The Watcher watche...@gmail.com wrote: I'm wondering if predicates are pushed down when they apply to elements of a nested struct. More specifically, imagine a table such as CREATE TABLE t ( c1 int, c2 STRUCTa:int,b:int ) STORED as XXX; Will SELECT * from t where c2.b=3 be pushed down to ORC ? Would it be pushed down to Parquet ? Thanks
