Behavior of Hive 2837: insert into external tables should not be allowed

2012-05-31 Thread Mark Grover
Hi folks, I have a question regarding HIVE 2837( https://issues.apache.org/jira/browse/HIVE-2837) that deals with disallowing external table from using insert into queries. >From looking at the JIRA, it seems like it applies to external tables on HDFS as well. Technically, insert into should be ok

Re: confused on different behavior of Bucketized tables do not support INSERT INTO

2012-05-31 Thread Mark Grover
Hi Bruce, It seems to me that your issue may be two-fold. 1) The JIRA that introduced "insert into " (https://issues.apache.org/jira/browse/HIVE-306) in Hive 0.8 only works for non-bucketed partitions or tables. This is because appending inside a table or (its partition, if it's partitioned) wo

Re: Error in Hive execution on cluster : Wrong FS: hdfs

2012-05-31 Thread Abhishek Pratap Singh
Can you check the configuration in your Hadoop's core-site.xml file. Also are you able to fire fs command from hadoop successfully by using the fully qualified directory structure? Seems a problem in your file system config. Regards, Abhishek On Thu, May 31, 2012 at 1:35 AM, Puneet Khatod wrote

Re: Hive in IntelliJ

2012-05-31 Thread Lars Francke
Hi Owen (& Edward), >> has anyone managed to get Hive properly set up in IntelliJ? I've tried >> but so far I've failed to get it running with Ivy and its >> dependencies. > > I managed, but it wasn't easy. I let it do the original import, but then had > to fix up the dependencies between modules

Re: Setup for contributing to Hive from MAC

2012-05-31 Thread Ashutosh Chauhan
https://cwiki.apache.org/confluence/display/Hive/HowToContribute will help you get started. Thanks, Ashutosh On Thu, May 31, 2012 at 8:55 AM, Mapred Learn wrote: > Hi, > Does anyone know how to set up apache hive on MAC to contribute and fix > JIRAs ? > > Thanks, > JJ > > Sent from my iPhone

Re: Hive in IntelliJ

2012-05-31 Thread Edward Capriolo
I agree, even if we had a decent ant script most IDEs could deal with it, but maven would be better. I understand why Hive's ant build infrastructure is the way it is. generating thrift, building a hive-exec.jar fat jar, getting hadoop in the old days before it was in maven, templating engine to g

Re: Hive in IntelliJ

2012-05-31 Thread Owen O'Malley
On Thu, May 31, 2012 at 9:35 AM, Edward Capriolo wrote: > Hive is in maven I meant that Hive is built with Ant. Intellij has better support for importing projects built by Maven. Hive's jars are published in Maven central, but that is a different thing. -- Owen

Re: Hive in IntelliJ

2012-05-31 Thread Edward Capriolo
Hive is in maven but it always forks a hadoop to run stuff. The approach I took is make fetching and extracting hadoop a 1 time install target. https://github.com/edwardcapriolo/hive_test/ I know its not the "right" way, but Runs unit tests lightning fast like 15 sec vs like 1:20 or however long

Re: Hive in IntelliJ

2012-05-31 Thread Owen O'Malley
On Thu, May 31, 2012 at 12:45 AM, Lars Francke wrote: > Hi, > > has anyone managed to get Hive properly set up in IntelliJ? I've tried > but so far I've failed to get it running with Ivy and its > dependencies. > I managed, but it wasn't easy. I let it do the original import, but then had to fix

Re: Parse Error with '-' in Hive

2012-05-31 Thread shrikanth shankar
I believe Hive column names cant have '-' in them . From what I know this JSON serde uses column names as JSON expressions. This means that if you renamed the column name you would end up with a null value for the column. You might want to try a different json serde (for e.g. the one Amazon use

Re: Parse Error with '-' in Hive

2012-05-31 Thread Edward Capriolo
I never use - for variable names or object names. Makes trouble for ORM tools package names etc etc. _ is your friend. I believe the correct escape here is backtick `. But you might be out of luck. On 5/31/12, Alex Barrera wrote: > Hi there, > > I'm looking for help with the following error: I ke

Setup for contributing to Hive from MAC

2012-05-31 Thread Mapred Learn
Hi, Does anyone know how to set up apache hive on MAC to contribute and fix JIRAs ? Thanks, JJ Sent from my iPhone

Re: Parse Error with '-' in Hive

2012-05-31 Thread alo alt
Hi, try to use _ instead - UA_Platform STRING, and so on. - Alex -- Alexander Alten-Lorenz http://mapredit.blogspot.com German Hadoop LinkedIn Group: http://goo.gl/N8pCF On May 31, 2012, at 5:35 PM, Alex Barrera wrote: > Hi there, > > I'm looking for help with the following error: I keep r

Parse Error with '-' in Hive

2012-05-31 Thread Alex Barrera
Hi there, I'm looking for help with the following error: I keep receiving an error 'Parse Error: line 40:4 cannot recognize input near '-' 'Platform' STRING column type. ' It looks to be a problem with my JSON deserializer OR Hive not recognizing the - in the field names. So far I've tried commen

Re: confused on different behavior of Bucketized tables do not support INSERT INTO

2012-05-31 Thread Bruce Bian
So I did another test on this. hive> create table test(foo int,bar string) clustered by(foo) sorted by (foo asc) into 2 buckets; OK Time taken: 0.097 seconds hive> create table test2 (foo int,bar string) clustered by(foo) sorted by (foo asc) into 2 buckets; OK hive> LOAD DATA LOCAL INPATH 'hive/exa

Re: confused on different behavior of Bucketized tables do not support INSERT INTO

2012-05-31 Thread Bruce Bian
I'm using hive 0.9.0 On Thursday, May 31, 2012, Bruce Bian wrote: > Hi, > I've got a table vt_new_data which is defined as follows: > CREATE TABLE VT_NEW_DATA > ( > V_ACCOUNT_NUM string > ,V_ACCOUNT_MODIFIER_NUM string > ,V_DEPOSIT_TYPE_CD string > ,V_DEPOSIT_TERM int > ,V_LE

Re: Hive partition problem

2012-05-31 Thread wd
It's too bad. I've file a issue at hive jira, https://issues.apache.org/jira/browse/HIVE-3069. On Thu, May 31, 2012 at 5:16 PM, Malcolm Tye wrote: > Hi, >        We used to use integer partitioning for 2 columns, "year_month_day" > and "hour" > > When it came to dropping 1 partition for an hour f

RE: Hive partition problem

2012-05-31 Thread Malcolm Tye
Hi, We used to use integer partitioning for 2 columns, "year_month_day" and "hour" When it came to dropping 1 partition for an hour for a particular day, it seemed to drop all of the hour partitions for the day. IMO it's a bug. We just moved to using text Malc -Original Message---

Hadoop Russia user group

2012-05-31 Thread Ruslan Al-Fakikh
Hi everyone, I've created a group on Linkedin for Russian-speaking folks. It is about Hadoop and its components (Hive, Pig, etc). http://www.linkedin.com/groups/Hadoop-Russia-4468740?gid=4468740 Thanks, Ruslan Al-Fakikh

Error in Hive execution on cluster : Wrong FS: hdfs

2012-05-31 Thread Puneet Khatod
Hi, I am facing below error when I am firing any query in Hive. My Hive setup is present on the master node in my cluster. Hadoop is configured using IP_addresses in configuration xmls and in master, slave files and it is running fine. The error only arises when hive query is executed which had

Re: Hive partition problem

2012-05-31 Thread wd
Is this a bug? Should I report a bug to hive? On Thu, May 31, 2012 at 3:56 PM, wd wrote: > Still no useful output. But we found the problem. > > When the partition col type is int, it can't be droped. After change > it to string, it can be droped. > > On Thu, May 31, 2012 at 3:48 PM, Aniket Mokas

Re: Hive partition problem

2012-05-31 Thread wd
Still no useful output. But we found the problem. When the partition col type is int, it can't be droped. After change it to string, it can be droped. On Thu, May 31, 2012 at 3:48 PM, Aniket Mokashi wrote: > -hiveconf hive.root.logger=DEBUG,FA \ >     -hiveconf hive.log.dir="$logDir" \ >     -hi

Re: Hive partition problem

2012-05-31 Thread Aniket Mokashi
-hiveconf hive.root.logger=DEBUG,FA \ -hiveconf hive.log.dir="$logDir" \ -hiveconf hive.log.file="$logFile" \ On Thu, May 31, 2012 at 12:41 AM, wd wrote: > Nothing output in hive history file, is there an other log file or an > option to output detail log ? > > On Thu, May 31, 2012 at 3:

Hive in IntelliJ

2012-05-31 Thread Lars Francke
Hi, has anyone managed to get Hive properly set up in IntelliJ? I've tried but so far I've failed to get it running with Ivy and its dependencies. Pointers would be very appreciated. Cheers, Lars

Re: Hive partition problem

2012-05-31 Thread wd
Nothing output in hive history file, is there an other log file or an option to output detail log ? On Thu, May 31, 2012 at 3:34 PM, Aniket Mokashi wrote: > You should look at hive log and find exact exception. That will give you a > hint. > > > On Thu, May 31, 2012 at 12:33 AM, wd wrote: >> >>

Re: Hive partition problem

2012-05-31 Thread Aniket Mokashi
You should look at hive log and find exact exception. That will give you a hint. On Thu, May 31, 2012 at 12:33 AM, wd wrote: > No problem, thanks for your reply. > I'm very curious why this didn't work, this sql come from hive wiki. > The metadata is store in postgres, does it matter? > > On Thu

Re: Hive partition problem

2012-05-31 Thread wd
No problem, thanks for your reply. I'm very curious why this didn't work, this sql come from hive wiki. The metadata is store in postgres, does it matter? On Thu, May 31, 2012 at 3:26 PM, Bhavesh Shah wrote: > Sorry for previous reply. Actually you had try that one, I didn't saw that. > Really So

Re: Hive partition problem

2012-05-31 Thread Bhavesh Shah
Sorry for previous reply. Actually you had try that one, I didn't saw that. Really Sorry. On Thu, May 31, 2012 at 12:53 PM, Bhavesh Shah wrote: > Hello wd, > Try this one... I am not sure about this > ALTER TABLE t1 DROP PARTITION(dt = '111') > > > -- > Regards, > Bhavesh Shah > > > On Thu, May 3

Re: Hive partition problem

2012-05-31 Thread Bhavesh Shah
Hello wd, Try this one... I am not sure about this ALTER TABLE t1 DROP PARTITION(dt = '111') -- Regards, Bhavesh Shah On Thu, May 31, 2012 at 12:49 PM, wd wrote: > hi, > > We setup a new hive 0.9 client, Found some sql did not work, for example > > hive> create table t1(a int) partitioned by

Hive partition problem

2012-05-31 Thread wd
hi, We setup a new hive 0.9 client, Found some sql did not work, for example hive> create table t1(a int) partitioned by ( dt int ); OK Time taken: 0.097 seconds hive> load data local inpath '/tmp/t' into table t1 partition (dt=111); Copying data from file:/tmp/t Copying file: file:/tmp/t Loading