hive benchmark

Hi Folks, I need to insert 1 billion records into hive and here is my cluster details. 1. 6-node Hadoop cluster cluster. 2. 16GB RAM on each node. 3. 2TB Hard-disk on each node. Is this configuration suitable for storing 1 billion records? If not, what is that all we need to

Re: How to create new table like existing with an extra column in single query.

Thanks Adam. As mentioned we have a need that in a single query it should be done, since ours is an automated query generation and no manual interactions are done. We need a query like *Create table if not exists newTable like oldTable, newColumn datatype* *Without copying data from oldTable. *

Re: Request write access to the Hive wiki

I couldn't find your user id based on either your name or email address. You probably need to register there first. On Mon, Aug 10, 2015 at 12:41 PM, kulkarni.swar...@gmail.com < kulkarni.swar...@gmail.com> wrote: > @Xuefu While you are already at it, would you mind giving me this access > too? :

Re: Request write access to the Hive wiki

Swarnim Kulkarni, first you need a Confluence username (see How to get permission to edit ) -- the system finds several Kulkarnis but they don't seem to be you. -- Lefty On Mon, Aug 10, 2015 at

RE: Request write access to the Hive wiki

Thanks Xuefu. From: Xuefu Zhang [mailto:xzh...@cloudera.com] Sent: Tuesday, August 11, 2015 3:38 AM To: user@hive.apache.org Subject: Re: Request write access to the Hive wiki Done! On Mon, Aug 10, 2015 at 1:05 AM, Xu, Cheng A mailto:cheng.a...@intel.com>> wrote: Hi, I’d like to have write acce

jdbc.sql.ResultSetMetaData.getColumnName() - problem

The HIVE 1.2.1 driver returns a compound table_name.column_name when calling ResultSetMetaData.getColumnName(). As the method name implies it should return the column name only. There is a designated getTableName() for the purpose of returning the table name. With these two properly implemented

Re: Request write access to the Hive wiki

@Xuefu While you are already at it, would you mind giving me this access too? :) Thanks, On Mon, Aug 10, 2015 at 2:37 PM, Xuefu Zhang wrote: > Done! > > On Mon, Aug 10, 2015 at 1:05 AM, Xu, Cheng A wrote: > >> Hi, >> >> I’d like to have write access to the Hive wiki. My Confluence username is

Re: Request write access to the Hive wiki

Done! On Mon, Aug 10, 2015 at 1:05 AM, Xu, Cheng A wrote: > Hi, > > I’d like to have write access to the Hive wiki. My Confluence username is > cheng.a...@intel.com with Full Name “Ferdinand Xu”. Please help me deal > with it. Thank you! > > > > Regards, > > Ferdinand Xu > > >

Re: Running hive on tez locally

> How do I point Hive to tez libraries? Is it sufficient to populate >CLASSPATH environment variables with location of tez libraries ? The easiest option there is to populate the hive-config.sh with these two parameters https://github.com/t3rmin4t0r/tez-autobuild/blob/master/Makefile#L138 Once

Re: Running hive on tez locally

I have my HADOOP_CLASSPATH set to point to both $TEZ_HOME/* and $TEZ_HOME/lib/*. Also have TEZ_H0ME and TEZ_CONF_DIR set, though I'm not sure if these are required or not.? From: Raajay Sent: Friday, August 07, 2015 2:50 PM To: user@hive.apache.org Subject: Re:

Re: Persistent (and possibly asynchronous) Hive access from within Scala

We can use JDBC. But there’s also this new library http://hive.apache.org/javadocs/r1.1.1/api/org/apache/hive/jdbc/package-frame.html based on this Thrift interface (https://github.com/apache/hive/blob/0af6cb4

Hive CBO - Calcite Interface

For a research project, I am studying the performance gains in Hive queries achieved through cost based optimization. Rather than just study scenarios where CBO is switched on or off, I would like to study the performance across a “collection" of query plans considered by the CBO (Calcite); for exa

RE: How to create new table like existing with an extra column in single query.

How about: create table XXX like YYY; alter table XXX add columns (new_column int); From: venkatesh b [mailto:venkateshmailingl...@gmail.com] Sent: Monday, August 10, 2015 9:28 AM To: Wangwenli Cc: user Subject: Re: How to create new table like existing with an extra column in single query. Tha

Re: How to create new table like existing with an extra column in single query.

Thanks Wenli for quick reply. Yes, we have a requirement exactly the same which you mentioned. I have tried the query which you provided but got an error saying that "Invalid column reference for newColumn possible columns are .." Thanks Venkatesh On Monday, August 10, 2015, Wangwenli wr

Re: How to create new table like existing with an extra column in single query.

something like this? create table newTable as select oldTable.*, newCol1,newCol2 from oldTable; wenli Regards From: venkatesh b Date: 2015-08-10 20:54 To: user Subject: How to create new table lik

How to create new table like existing with an extra column in single query.

Hello, In hive we got a need that we need to create a new table like the old table with an extra column at run time. This must be done in a single query. No second query with alter table statement. Query like: Create table new_table like old_table , new_column datatype; Please help me out, sear

Request write access to the Hive wiki

Hi, I'd like to have write access to the Hive wiki. My Confluence username is cheng.a...@intel.com with Full Name "Ferdinand Xu". Please help me deal with it. Thank you! Regards, Ferdinand Xu