RE: how to get counts as a byproduct of a query

Didn’t get any response, so trying one more time. I cannot believe I am the only one facing the problem. From: Frank Luo Sent: Tuesday, December 01, 2015 10:40 PM To: user@hive.apache.org Subject: how to get counts as a byproduct of a query Very often I need to run a query against a table(s),

RE: how to get counts as a byproduct of a query

https://cwiki.apache.org/confluence/display/Hive/GettingStarted#GettingStarted-MULTITABLEINSERT From: Frank Luo [mailto:j...@merkleinc.com] Sent: Wednesday, December 02, 2015 1:26 PM To: user@hive.apache.org Subject: RE: how to get counts as a byproduct of a query Didn’t get any response, so

Re: how to get counts as a byproduct of a query

I am not sure if I understand, but why this should not be possible using SQL in hive? > On 02 Dec 2015, at 21:26, Frank Luo wrote: > > Didn’t get any response, so trying one more time. I cannot believe I am the > only one facing the problem. > > From: Frank Luo >

RE: how to get counts as a byproduct of a query

I might not illustrate the problem well. Let’s try on a sample. Here is what I have: Table_A, column X and Y Table B, column X and Z I want to do a join on both tables on column X, like “select a.X, a.Y, b.Z From A a Join B b on A.X=B.X” In the meaning time, I want to get counts for both Table

RE: how to get counts as a byproduct of a query

Personally, I'd do it this way... https://cwiki.apache.org/confluence/display/Hive/LanguageManual+WindowingAndAnalytics Select suba.X, suba.Y, suba.countA, subb.Z, subb.countB FROM (SELECT x, y, count(1) OVER (PARTITION BY X) as countA) suba JOIN (SELECT x, z, count(1) OVER (PARTITION BY X) as

How to register HCatlog Library as part of pig script file

Hi, We would like to make use of HCatlog table in our PIG Script file, Currently we are opening the PIG Command with -UseHCatlog option for registering or loading HCatlog library. Is there a way in PIG script to register HCatlog jar files and execute directly on pig command prompt directly

Hive on spark table caching

Hi, I have started using Hive on Spark recently and am exploring the benefits it offers. I was wondering if Hive on Spark has capabilities to cache table like Spark SQL. Or does it do any form of implicit caching in the long running job which it starts after running the first query? Thanks, Udit

Create table from ORC or Parquet file?

Hi Everyone Is it possible to create Hive table from ORC or Parquet file without specifying field names and their types. ORC or Parquet files contain field name and type information inside. Alex

RE: Hive on spark table caching

Hi, Which version of spark are you using please? Mich Talebzadeh Sybase ASE 15 Gold Medal Award 2008 A Winning Strategy: Running the most Critical Financial Data on ASE 15

RE: Hive on spark table caching

OK How did you build your Spark 1.3. Was that from the source code or pre-build for Hadoop 2.6 please? The one I have 1.Spark version 1.5.2 2.Hive version 1.2.1 3.Hadoop version 2.6 Does not work with Hive on Spark Mich Talebzadeh Sybase ASE 15 Gold Medal

Re: Hive on spark table caching

Im using Spark 1.3 with Hive 1.2.1. I dont mind using a version of Spark higher than that but I read somewhere that 1.3 is the version of Spark currently supported by Hive. Can I use Spark 1.4 or 1.5 with Hive 1.2.1? On Wed, Dec 2, 2015 at 3:19 PM, Mich Talebzadeh wrote: >

Re: Hive on spark table caching

Depending on the query, Hive on Spark does implicitly cache datasets (not necessarily the input tables) for performance benefits. Such queries include multi-insert, self-join, self-union, etc. However, no caching happens across queries at this time, which may be improved in the future. Thanks,