Re: Welcome new Hive committer, Zhihai Xu

Congrats Zhihai!! On Friday, May 5, 2017 9:52 AM, Xuefu Zhang wrote: Hi all, I'm very please to announce that Hive PMC has recently voted to offer Zhihai a committership which he accepted. Please join me in congratulating on this recognition and thanking him for his

Supporting special characters in nested column name

Hi,Looks like Hive supports special character (unicode) with "`" in first level column names since Hive 0.13 (HIVE-6013).Similarly does hive supports special characters in nested column names? For example, struct create table test1_mislam(a int, `$b_` int);OKTime taken: 0.036 secondshive>

Re: Schema evolution for parquet table

, 2015 at 7:23 PM, Mohammad Islam <misla...@yahoo.com.invalid> wrote: > > Any hive+parquet user/dev to address this? > > > Regards, > Mohammad > > On Monday, October 5, 2015 3:41 PM, Mohammad Islam <misla...@yahoo.com> > wrote: > > > > Hi, > Doe

Re: Schema evolution for parquet table

Any hive+parquet user/dev to address this? Regards, Mohammad On Monday, October 5, 2015 3:41 PM, Mohammad Islam <misla...@yahoo.com> wrote: Hi, Does the parquet table support auto casting to wider data types? For example, If I have a parquet table where some parquet data files whic

Schema evolution for parquet table

Hi,Does the parquet table support auto casting to wider data types? For example, If I have a parquet table where some parquet data files which have "int"  as data type and other files have "long" data type for the same field. The table schema has type "bigint" for the same field.Does hive can

Re: [ANNOUNCE] New Hive PMC Chair - Ashutosh Chauhan

Congratulations Asutosh! On Wednesday, September 16, 2015 4:51 PM, Bright Ling wrote: #yiv7221259285 #yiv7221259285 -- _filtered #yiv7221259285 {font-family:SimSun;panose-1:2 1 6 0 3 1 1 1 1 1;} _filtered #yiv7221259285 {font-family:PMingLiU;panose-1:2 2

Re: [VOTE] officially stop supporting hadoop 0.20.x in hive 0.14 ?

+1. On Tuesday, October 7, 2014 6:02 PM, Edward Capriolo edlinuxg...@gmail.com wrote: +1 On Tue, Oct 7, 2014 at 7:51 PM, Vikram Dixit vik...@hortonworks.com wrote: +1 On Tue, Oct 7, 2014 at 4:44 PM, Bill Busch bill.bu...@perficient.com wrote: +1 Follow me on  @BigData73

Re: Review Request 11925: Hive-3159 Update AvroSerde to determine schema of new tables

for testing multiple use-cases. Thanks, Mohammad Islam

Review Request 21353: Unable to deserialize AVRO data when file schema and record schema are different and nullable

/TestAvroDeserializer.java 198bd24 serde/src/test/org/apache/hadoop/hive/serde2/avro/TestAvroObjectInspectorGenerator.java 76c1940 Diff: https://reviews.apache.org/r/21353/diff/ Testing --- Thanks, Mohammad Islam

Re: Review Request 11925: Hive-3159 Update AvroSerde to determine schema of new tables

class for a new Java class. Added a new test case into existing java test class. In addition, there are 4 .q file for testing multiple use-cases. Thanks, Mohammad Islam

Re: Review Request 21026: Remove HDFS check in tez/DagUtils

://reviews.apache.org/r/21026/#comment75826 Just a proposal: in place of throwing exception, you could LOG info/debug something telling what type of FS is utilized. This might help to debug if someone used (intentionally or mistakenly) unsupported FS . - Mohammad Islam On May

Hive 0.12 with Hadoop 2.3

Hi , Does Hive 0.12 + Hadoop 2.3 work fine? We recently experienced the protobuf issue. Looks like hive includes protobuf 2.4.1 whereas Hadoop 2.3 needs protobuf 2.5.0. Should I rebuild Hive 0.12 code with protobuf 2.5.0? I found some discussion in JIRA but not sure if it is safe (  

Re: Review Request 19212: HIVE-6645: to_date()/to_unix_timestamp() fail with NPE if input is null

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/19212/#review37540 --- Ship it! +1 - Mohammad Islam On March 14, 2014, 9:23 p.m

Re: Review Request 19212: HIVE-6645: to_date()/to_unix_timestamp() fail with NPE if input is null

://reviews.apache.org/r/19212/#comment68594 what about test cases where 1st arg is not null but the second arg is null? - Mohammad Islam On March 14, 2014, 2:40 a.m., Jason Dere wrote: --- This is an automatically generated e-mail

Re: Review Request 19181: show grant ... on all fails with NPE

/hive/ql/exec/DDLTask.java https://reviews.apache.org/r/19181/#comment68426 Since you are adding a lot of Null checks, adding one more something like this privSubjectDesc == null || -- will be complete. I know it is unlikely this REF could be null. - Mohammad Islam On March 13

Re: Review Request 18065: HIVE-6024 Load data local inpath unnecessarily creates a copy task

/clientpositive/stats3.q.out a14e449 Diff: https://reviews.apache.org/r/18065/diff/ Testing --- Ran some existing q tests with LOAD DATA LOCAL INPATH. Thanks, Mohammad Islam

Re: Review Request 18992: Fix ctrl-C killing MR jobs

/hadoop/hive/ql/exec/mr/HadoopJobExecHelper.java https://reviews.apache.org/r/18992/#comment68074 It will be better to include a more informative error message including job id and exception. - Mohammad Islam On March 10, 2014, 11:39 p.m., Ashutosh Chauhan wrote

Re: Review Request 18065: HIVE-6024 Load data local inpath unnecessarily creates a copy task

PRE-CREATION ql/src/test/results/clientpositive/stats11.q.out ce1197e ql/src/test/results/clientpositive/stats3.q.out a14e449 Diff: https://reviews.apache.org/r/18065/diff/ Testing --- Ran some existing q tests with LOAD DATA LOCAL INPATH. Thanks, Mohammad Islam

Re: Review Request 18065: HIVE-6024 Load data local inpath unnecessarily creates a copy task

LOCAL INPATH. Thanks, Mohammad Islam

Re: Review Request 18592: HIVE-6137

/trunk/metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java https://reviews.apache.org/r/18592/#comment66655 nit: pls remove space. - Mohammad Islam On Feb. 28, 2014, 7:37 p.m., Hari Sankar Sivarama Subramaniyan wrote

Re: Review Request 18065: HIVE-6024 Load data local inpath unnecessarily creates a copy task

--- On Feb. 28, 2014, 8:03 a.m., Mohammad Islam wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/18065

Re: Review Request 18592: HIVE-6137

/apache/hadoop/hive/metastore/HiveMetaStore.java https://reviews.apache.org/r/18592/#comment66447 Can some more info be added in the exception message. such as : table name/path/URI? - Mohammad Islam On Feb. 27, 2014, 11:38 p.m., Hari Sankar Sivarama Subramaniyan wrote

Re: Review Request 18065: HIVE-6024 Load data local inpath unnecessarily creates a copy task

. - Mohammad --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/18065/#review35637 --- On Feb. 26, 2014, 8:20 a.m., Mohammad Islam wrote

Re: Review Request 18065: HIVE-6024 Load data local inpath unnecessarily creates a copy task

://reviews.apache.org/r/18065/diff/ Testing --- Ran some existing q tests with LOAD DATA LOCAL INPATH. Thanks, Mohammad Islam

Re: [ANNOUNCE] New Hive Committer - Remus Rusanu

Congratulations Remus! On Wednesday, February 26, 2014 11:38 AM, Prasanth Jayachandran pjayachand...@hortonworks.com wrote: Congratulations Remus! Thanks Prasanth Jayachandran On Feb 26, 2014, at 11:36 AM, Brock Noland br...@cloudera.com wrote: Congrats!! On Wed, Feb 26, 2014 at 1:10

Re: Review Request 18254: HIVE-6375 Implement CTAS and column rename for parquet

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/18254/#review34937 --- Ship it! - Mohammad Islam On Feb. 19, 2014, 9:56 p.m., Szehon Ho

Re: Review Request 18291: Add support for pluggable authentication modules (PAM) in Hive

4. These are similar to LDAPAuth...Impl.java - Mohammad Islam On Feb. 19, 2014, 11:44 p.m., Vaibhav Gumashta wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/18291

Re: Review Request 18224: HIVE-5976 - Decouple input formats from STORED as keywords

/StorageFormat.java https://reviews.apache.org/r/18224/#comment65026 Looks like this would be a very generic method useful for other places in future. Therefore, moving the method to some util class in common module would be beneficial for others. - Mohammad Islam On Feb. 18, 2014, 7:13 p.m., Brock

Review Request 18065: HIVE-6024 Load data local inpath unnecessarily creates a copy task

with LOAD DATA LOCAL INPATH. Thanks, Mohammad Islam

Re: Review Request 17959: HIVE-6383: Newly added tests in TestJdbcDriver2 from HIVE-4395 is not running

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/17959/#review34237 --- +1 - Mohammad Islam On Feb. 11, 2014, 8:13 p.m., Prasad

Re: Review Request 17930: HIVE-6402: Improve Hive behavior when deleting data with miss configured Trash

deletion succeeds? If that is possible, the message doesn't seem correct. - Mohammad Islam On Feb. 11, 2014, 1:33 a.m., Szehon Ho wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r

Re: Review Request 17939: authorize role ops

/SQLStdHiveAccessController.java https://reviews.apache.org/r/17939/#comment64190 Ditto trunk/ql/src/java/org/apache/hadoop/hive/ql/security/authorization/plugin/sqlstd/SQLStdHiveAccessController.java https://reviews.apache.org/r/17939/#comment64191 equalsIgnoreCase could be better. - Mohammad Islam On Feb. 11

Re: Review Request 17661: HIVE-6327: A few mathematic functions don't take decimal input

/UDFSin.java https://reviews.apache.org/r/17661/#comment62922 same here - Mohammad Islam On Feb. 3, 2014, 2:26 p.m., Xuefu Zhang wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/17661

Re: Review Request 11925: Hive-3159 Update AvroSerde to determine schema of new tables

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/11925/#review31584 --- On Jan. 10, 2014, 7:58 p.m., Mohammad Islam wrote

Re: Review Request 11925: Hive-3159 Update AvroSerde to determine schema of new tables

for testing multiple use-cases. Thanks, Mohammad Islam

Re: Review Request 13383: HIVE-4545 - HS2 should return describe table results without space padding

/formatting/JsonMetaDataFormatter.java https://reviews.apache.org/r/13383/#comment62698 No usage/meaning in this MDFormatter . right? - Mohammad Islam On Jan. 31, 2014, 1:29 a.m., Thejas Nair wrote: --- This is an automatically

Re: Review Request 17262: HIVE-6246: Sign(a) UDF is not supported for decimal type

On Jan. 23, 2014, 10:57 p.m., Mohammad Islam wrote: Overall looks good. Is it possible to add a .q test or append to an existing .q test? Xuefu Zhang wrote: Actually unit test such as that provided is preferrable. Additional .q test only prelongs the build process while

Re: Review Request 17217: HIVE-6288 MSCK can be slow when adding partitions

/DDLTask.java https://reviews.apache.org/r/17217/#comment61558 Do we need to cleanup anything for 'db' as well that might have partially changed in line '1841' before the exception? - Mohammad Islam On Jan. 23, 2014, 6:58 p.m., Sergey Shelukhin wrote

Re: Review Request 17262: HIVE-6246: Sign(a) UDF is not supported for decimal type

to an existing .q test? ql/src/test/org/apache/hadoop/hive/ql/udf/TestUDFSign.java https://reviews.apache.org/r/17262/#comment61610 Minor issue: does testByte() is a good name? - Mohammad Islam On Jan. 23, 2014, 8:57 p.m., Xuefu Zhang wrote

Re: Review Request 17302: HIVE-6012: restore backward compatibility of arithmetic operations

better if you can add a .q test something like decimal_udf.q? Basically the same q file after setting new config to backward compat mode. the result could be different. - Mohammad Islam On Jan. 24, 2014, 2:54 a.m., Jason Dere wrote

Re: Review Request 17217: HIVE-???? MSCK can be slow when adding partitions

https://reviews.apache.org/r/17217/#comment61496 Is it the same 'db' object that might be populated in the caller method? - Mohammad Islam On Jan. 23, 2014, 12:49 a.m., Sergey Shelukhin wrote: --- This is an automatically

Re: Review Request 16938: HIVE-6209 'LOAD DATA INPATH ... OVERWRITE ..' doesn't overwrite current data

://reviews.apache.org/r/16938/#comment61105 Can you please add a test case (.q file ) around these condition? This is in an important execution code path. - Mohammad Islam On Jan. 16, 2014, 1:45 a.m., Szehon Ho wrote

Re: Review Request 16807: HIVE-6174: Beeline set varible doesn't show the value of the variable as Hive CLI

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/16807/#review31593 --- Ship it! Ship It! - Mohammad Islam On Jan. 12, 2014, 1:58 a.m

Re: Review Request 11925: Hive-3159 Update AvroSerde to determine schema of new tables

, there are 4 .q file for testing multiple use-cases. Thanks, Mohammad Islam

Re: Review Request 15654: Rewrite Trim and Pad UDFs based on GenericUDF

/GenericUDFBaseTrim.java https://reviews.apache.org/r/15654/#comment58764 Would you please further explain on this? preferably with an example. - Mohammad Islam On Dec. 18, 2013, 3:16 a.m., Mohammad Islam wrote: --- This is an automatically

Re: Review Request 15654: Rewrite Trim and Pad UDFs based on GenericUDF

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/15654/#review30604 --- On Dec. 18, 2013, 3:16 a.m., Mohammad Islam wrote

Re: Review Request 15654: Rewrite Trim and Pad UDFs based on GenericUDF

., Mohammad Islam wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/15654/ --- (Updated Dec. 18, 2013, 3:16 a.m.) Review request

Re: Review Request 16329: HIVE-6039: Round, AVG and SUM functions reject char/varch input while accepting string input

/16329/#comment58770 same here. ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFRound.java https://reviews.apache.org/r/16329/#comment58771 Same. more informative message could be helpful. - Mohammad Islam On Dec. 17, 2013, 9:26 p.m., Xuefu Zhang wrote

Re: Review Request 15654: Rewrite Trim and Pad UDFs based on GenericUDF

ql/src/test/org/apache/hadoop/hive/ql/udf/generic/TestGenericUDFRpad.java PRE-CREATION ql/src/test/org/apache/hadoop/hive/ql/udf/generic/TestGenericUDFTrim.java PRE-CREATION Diff: https://reviews.apache.org/r/15654/diff/ Testing --- Thanks, Mohammad Islam

Re: Review Request 15654: Rewrite Trim and Pad UDFs based on GenericUDF

-CREATION ql/src/test/org/apache/hadoop/hive/ql/udf/TestGenericUDFTrim.java PRE-CREATION Diff: https://reviews.apache.org/r/15654/diff/ Testing --- Thanks, Mohammad Islam

Review Request 15654: Rewrite Trim and Pad UDFs based on GenericUDF

-CREATION Diff: https://reviews.apache.org/r/15654/diff/ Testing --- Thanks, Mohammad Islam

Re: Review Request 15649: HIVE-5842 - Fix issues with new paths to jar in hcatalog

://reviews.apache.org/r/15649/#comment56209 Same as above: Create a new variable and use it. hcatalog/bin/hcat.py https://reviews.apache.org/r/15649/#comment56208 If possible, creating a variable and use the variable would make it much better.| - Mohammad Islam On Nov. 18, 2013, 4:49 p.m., Brock

Re: Review Request 11925: Hive-3159 Update AvroSerde to determine schema of new tables

On July 29, 2013, 5:41 p.m., Jakob Homan wrote: There is still no text covering a map-reduce job on an already existing, non-Avro table into an avro table. ie, create a text table, populate it, run a CTAS to manipulate the data into an Avro table. Mohammad Islam wrote

Re: Review Request 15213: HIVE-5731: Use new GenericUDF instead of basic UDF for UDFDate* classes

/hive/ql/udf/TestUDFDateDiff.java 8a6dbc3 ql/src/test/org/apache/hadoop/hive/ql/udf/TestUDFDateSub.java fa722a9 ql/src/test/results/clientpositive/udf_to_date.q.out 6ff5ee8 Diff: https://reviews.apache.org/r/15213/diff/ Testing --- Thanks, Mohammad Islam

Re: Review Request 15213: HIVE-5731: Use new GenericUDF instead of basic UDF for UDFDate* classes

://reviews.apache.org/r/15213/#review28624 --- On Nov. 11, 2013, 9:03 p.m., Mohammad Islam wrote: --- This is an automatically generated e-mail. To reply, visit: https

Re: Review Request 15213: HIVE-5731: Use new GenericUDF instead of basic UDF for UDFDate* classes

/TestUDFDateDiff.java 8a6dbc3 ql/src/test/org/apache/hadoop/hive/ql/udf/TestUDFDateSub.java fa722a9 Diff: https://reviews.apache.org/r/15213/diff/ Testing --- Thanks, Mohammad Islam

Review Request 15213: HIVE-5731: Use new GenericUDF instead of basic UDF for UDFDate* classes

, Mohammad Islam

Re: Hive compilation error

Thanks Carl for the reply. Yes these were my own .q files. Since there are no such current files, we can skip it for the time being. Need to make sure no future q filenames would contain multiple dots.   Regards, Mohammad  On Sunday, November 3, 2013 4:59 PM, Carl Steinbach

Re: [ANNOUNCE] New Hive Committer - Xuefu Zhang

Congrats Xuefu! --Mohammad On Sunday, November 3, 2013 9:07 PM, hsubramani...@hortonworks.com hsubramani...@hortonworks.com wrote: Congrats Xuefu! Thanks, Hari On Nov 3, 2013, at 8:28 PM, Gunther Hagleitner ghagleit...@hortonworks.com wrote: Congrats Xuefu! Gunther. On Sun, Nov 3,

Hive compilation error

Hi, I ran mvn clean install -DskipTests; cd itests; mvn clean install  -DskipTests and got the following error. Error === [ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:3.1:testCompile (default-testCompile) on project hive-it-qfile: Compilation failure:

Re: Review Request 14121: Base64 decoding for binary type

, Mohammad Islam

Review Request 14121: Base64 decoding for binary type

/LazyBinaryObjectInspector.java dbd60f7 serde/src/java/org/apache/hadoop/hive/serde2/lazy/objectinspector/primitive/LazyPrimitiveObjectInspectorFactory.java afbf454 Diff: https://reviews.apache.org/r/14121/diff/ Testing --- WIP: working on writing new test cases. Thanks, Mohammad Islam

Re: Review Request 12480: HIVE-4732 Reduce or eliminate the expensive Schema equals() check for AvroSerde

: https://reviews.apache.org/r/12480/diff/ Testing --- Thanks, Mohammad Islam

Re: Review Request 12480: HIVE-4732 Reduce or eliminate the expensive Schema equals() check for AvroSerde

--- On Aug. 30, 2013, 6:49 p.m., Mohammad Islam wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/12480

Hive trunk Jenkins build failed

Hi, I found the last Jenkins build failed for hadoop2. The console output link and error message are pasted below. I also experienced the same issue yesterday. Running ant very-clean fixed my issue (Thanks to Ashutosh!). If someone can run the same on build machine, the problem might go

Re: Review Request 11925: Hive-3159 Update AvroSerde to determine schema of new tables

class. In addition, there are 4 .q file for testing multiple use-cases. Thanks, Mohammad Islam

Re: Review Request 12480: HIVE-4732 Reduce or eliminate the expensive Schema equals() check for AvroSerde

/Utils.java 2b948eb Diff: https://reviews.apache.org/r/12480/diff/ Testing --- Thanks, Mohammad Islam

Re: Review Request 11925: Hive-3159 Update AvroSerde to determine schema of new tables

., Mohammad Islam wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/11925/ --- (Updated July 23, 2013, 9:51 a.m.) Review

Re: [Discuss] project chop up

Yes hive build and test cases got convoluted as the project scope gradually  increased. This is the time to take action! Based on my other Apache experiences, I prefer the option #3 Breakup the projects within our own source tree. Make multiple modules or sub-projects. By default, only key

Re: Review Request 11925: Hive-3159 Update AvroSerde to determine schema of new tables

.q file for testing multiple use-cases. Thanks, Mohammad Islam

Reminder: Bay Area Hive user meetup at LinkedIn tomorrow (7/24)

Please join us at LinkedIn (2025 Stierlin Court, Mountain View, CA) tomorrow at 6 PM for this month's Hive user meet up.We have a packed agenda : * Hive at LinkedIn, Mohammad Islam, Mark Wagner, and Karthik Ramasamy * Hive Server 2 at Yahoo!, Chris Drome * Hive on Tez

Re: Review Request 11925: Hive-3159 Update AvroSerde to determine schema of new tables

JIRA. - Mohammad --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/11925/#review23102 --- On July 12, 2013, 6:49 p.m., Mohammad Islam wrote

Re: Review Request 12480: HIVE-4732 Reduce or eliminate the expensive Schema equals() check for AvroSerde

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/12480/#review23113 --- On July 11, 2013, 10:31 p.m., Mohammad Islam wrote

Re: Review Request 12480: HIVE-4732 Reduce or eliminate the expensive Schema equals() check for AvroSerde

: https://reviews.apache.org/r/12480/diff/ Testing --- Thanks, Mohammad Islam

Re: Review Request 11925: Hive-3159 Update AvroSerde to determine schema of new tables

, visit: https://reviews.apache.org/r/11925/#review22571 --- On June 18, 2013, 3:26 a.m., Mohammad Islam wrote: --- This is an automatically generated e-mail. To reply, visit: https

Re: Review Request 11925: Hive-3159 Update AvroSerde to determine schema of new tables

, there are 4 .q file for testing multiple use-cases. Thanks, Mohammad Islam

Review Request 12480: HIVE-4732 Reduce or eliminate the expensive Schema equals() check for AvroSerde

/TestSchemaReEncoder.java 9af751b serde/src/test/org/apache/hadoop/hive/serde2/avro/Utils.java 2b948eb Diff: https://reviews.apache.org/r/12480/diff/ Testing --- Thanks, Mohammad Islam

Review Request: Hive-3159 Update AvroSerde to determine schema of new tables

Diff: https://reviews.apache.org/r/11925/diff/ Testing --- Wrote a new java Test class for a new Java class. Added a new test case into existing java test class. In addition, there are 4 .q file for testing multiple use-cases. Thanks, Mohammad Islam