Re: sqoop export query error out..

2015-07-16 Thread James Bond
What is the format of the data you are trying to export? The hive table - is it in textt/parquet or something else? On Fri, Jul 17, 2015 at 12:25 AM, Kumar Jayapal wrote: > Hi James, > > I tried with uppercase and it worked now I am getting different error. I > don't see the .metadata dir in the

Re: How to convert files in HIVE

2015-07-16 Thread Ted Yu
Looks like you may get more help from Hive mailing list: http://hive.apache.org/mailing_lists.html FYI On Thu, Jul 16, 2015 at 4:15 PM, Kumar Jayapal wrote: > Hi, > > How can we convert files stored in snappy compressed parquet format in > Hive to avro format. > > is it possible to do it. Can

What about Create Table As Select (CTAS)? RE: How to convert files in HIVE

2015-07-16 Thread Han-Cheol Cho
Hi, In a simple way, you can create a table in avro format and insert all data from the old table into the new one. Please check the "Create Table As Select" section in the following hive documentation. https://cwiki.apache.org/confluence/display/Hive/LanguageManual+DDL#LanguageManualDDL-

Re: FW: Is it valid to use userLimit in calculating maxApplicationsPerUser ?

2015-07-16 Thread Wangda Tan
I think it's not valid. Multiply it by ULF seems not reasonable, I think it should be: max(1, maxApplications * max(userlimit/100, 1/#activeUsers)) Assuming admin setups a very large ULF (e.g. 100), maxApplicationsPerUser can be much more than maxApplications of a queue. Also, multiply ULF to co

How to convert files in HIVE

2015-07-16 Thread Kumar Jayapal
Hi, How can we convert files stored in snappy compressed parquet format in Hive to avro format. is it possible to do it. Can you please guide me or give me the link which describe the procdure Thanks Jay

Fwd: Properties file not loaded with hadoop jar command

2015-07-16 Thread Nishanth S
Hello, I have built a jar file with maven as build tool which reads properties from a file.I am doing this like InputStream is = ClassLoader.getSystemResourceAsStream(("hadoop.properties")) on start iup.No qonce the jar is built I could see that the properties get loaded when I do java -jar

Re: sqoop export query error out..

2015-07-16 Thread Kumar Jayapal
Hi James, I tried with uppercase and it worked now I am getting different error. I don't see the .metadata dir in the path. 15/07/16 18:49:12 ERROR sqoop.Sqoop: Got exception running Sqoop: org.kitesdk.data.DatasetNotFoundException: Descriptor location does not exist: hdfs://name/user/hive/wareho

Properties file not loaded with hadoop jar command

2015-07-16 Thread Nishanth S
Hello, I have built a jar file with maven as build tool which reads properties from a file.I am doing this like InputStream is = ClassLoader.getSystemResourceAsStream(("hadoop.properties")) on start iup.No qonce the jar is built I could see that the properties get loaded when I do java -jar

FW: Is it valid to use userLimit in calculating maxApplicationsPerUser ?

2015-07-16 Thread Naganarasimha G R (Naga)
Hi Folks , Came across one scenario where in maxApplications @ cluster level(2 node) was set to a low value like 10 and based on capacity configuration for a particular queue it was coming to 2 as value, but further while calculating maxApplicationsPerUser formula used is : maxApplicationsPerUs