[jira] [Created] (HIVE-15431) Round(1234567891.1234567891,50) returns null, result is not consistent with Mysql.

2016-12-14 Thread Chinna Rao Lalam (JIRA) 
Chinna Rao Lalam created HIVE-15431:
---

 Summary: Round(1234567891.1234567891,50) returns null, result is 
not consistent with Mysql.
 Key: HIVE-15431
 URL: https://issues.apache.org/jira/browse/HIVE-15431
 Project: Hive
  Issue Type: Bug
  Components: UDF
Reporter: Chinna Rao Lalam
Assignee: Chinna Rao Lalam


round(1234567891.1234567891,50) returns null. Result is not consistent with 
Mysql.

Mysql output:
{quote}
  select round(1234567891.1234567891,50);

  '1234567891.1234567891'
{quote}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: Review Request 54711: HIVE-1555 [currently support only reading primitive types from oracle db]

2016-12-14 Thread Illya Yalovyy 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/54711/#review159187
---




jdbc-handler/pom.xml (lines 82 - 86)


Could you please clarify why jdbc connector depends on hbase?



jdbc-handler/src/java/org/apache/hive/storagehandler/JDBCStorageHandler.java 
(line 27)


Style: extra line is not required.



jdbc-handler/src/java/org/apache/hive/storagehandler/db/HiveJDBCTypeBridgeUtil.java
 (line 10)


It is better to make this class *final*.



jdbc-handler/src/java/org/apache/hive/storagehandler/db/HiveJDBCTypeBridgeUtil.java
 (line 11)


Please add a private constructor to avoid accidental creation of an 
instance of this utility class.



jdbc-handler/src/java/org/apache/hive/storagehandler/db/HiveJDBCTypeBridgeUtil.java
 (line 21)


Should we take into account version of the DB?



jdbc-handler/src/java/org/apache/hive/storagehandler/db/HiveJDBCTypeBridgeUtil.java
 (line 24)


It is better to use enumeration instead of plain string.



jdbc-handler/src/java/org/apache/hive/storagehandler/db/HiveJDBCTypeBridgeUtil.java
 (line 33)


I think it will be more user friendly and informative, to say " 
database is not supported"

If you have an enumeration of supported DBs, than message might be 
"Currently supported databases: {}, but found {}".



jdbc-handler/src/java/org/apache/hive/storagehandler/db/HiveJDBCTypeBridgeUtil.java
 (line 38)


Extra line is not required.



jdbc-handler/src/java/org/apache/hive/storagehandler/db/HiveJDBCVendorBridge.java
 (line 19)


Returns...



jdbc-handler/src/java/org/apache/hive/storagehandler/db/HiveJDBCVendorBridge.java
 (line 26)


@return is missing



jdbc-handler/src/java/org/apache/hive/storagehandler/db/HiveJDBCVendorBridge.java
 (line 28)


Is it necessary  to pass DBConfig every time? It could be used once when 
the instance is created. I think a factory will work well here.



jdbc-handler/src/java/org/apache/hive/storagehandler/db/HiveJDBCVendorBridge.java
 (line 31)


Returns...



jdbc-handler/src/java/org/apache/hive/storagehandler/db/HiveJDBCVendorBridge.java
 (line 34)


@return is missing



jdbc-handler/src/java/org/apache/hive/storagehandler/db/HiveJDBCVendorBridge.java
 (line 39)


@return is missing



jdbc-handler/src/java/org/apache/hive/storagehandler/db/HiveJDBCVendorBridge.java
 (line 44)


Provides ...



jdbc-handler/src/java/org/apache/hive/storagehandler/db/HiveJDBCVendorBridge.java
 (line 56)


@return is missing



jdbc-handler/src/java/org/apache/hive/storagehandler/db/HiveJDBCVendorBridge.java
 (line 68)


@return is missing



jdbc-handler/src/java/org/apache/hive/storagehandler/db/HiveJDBCVendorBridge.java
 (line 73)


@return is missing



jdbc-handler/src/java/org/apache/hive/storagehandler/db/HiveJDBCVendorBridge.java
 (line 78)


Provides... in the table.



jdbc-handler/src/java/org/apache/hive/storagehandler/db/HiveJDBCVendorBridge.java
 (line 85)


@return is missing



jdbc-handler/src/java/org/apache/hive/storagehandler/db/oracle/HiveOracleJDBCDBWritable.java
 (line 31)


Should HiveJDBCTypeBridgeUtil be used to create an instance of the bridge?



jdbc-handler/src/java/org/apache/hive/storagehandler/db/oracle/HiveOracleJDBCDBWritable.java
 (line 37)


It is better to do

for (Map.Entry> entry : columnTypeMapping.entrySet()) {
...
}



jdbc-handler/src/java/org/apache/hive/storagehandler/db/oracle/HiveOracleTypeBridge.java
 (line 8)


Style:
It seems like in Hive code base annotations are on a separate line. Please 
make sure your files pass checkstyle.



jdbc-handler/src/java/org/apache/hive/stor

Re: [ANNOUNCE] New Hive Committer - Rajesh Balamohan

2016-12-14 Thread Rui Li 
Congratulations :)

On Thu, Dec 15, 2016 at 6:50 AM, Gunther Hagleitner <
ghagleit...@hortonworks.com> wrote:

> Congrats Rajesh!
> 
> From: Jimmy Xiang 
> Sent: Wednesday, December 14, 2016 11:38 AM
> To: u...@hive.apache.org
> Cc: dev@hive.apache.org; rbalamo...@apache.org
> Subject: Re: [ANNOUNCE] New Hive Committer - Rajesh Balamohan
>
> Congrats, Rajesh!!
>
> On Wed, Dec 14, 2016 at 11:32 AM, Sergey Shelukhin
>  wrote:
> > Congratulations!
> >
> > From: Chao Sun 
> > Reply-To: "u...@hive.apache.org" 
> > Date: Wednesday, December 14, 2016 at 10:52
> > To: "dev@hive.apache.org" 
> > Cc: "u...@hive.apache.org" , "
> rbalamo...@apache.org"
> > 
> > Subject: Re: [ANNOUNCE] New Hive Committer - Rajesh Balamohan
> >
> > Congrats Rajesh!
> >
> > On Wed, Dec 14, 2016 at 9:26 AM, Vihang Karajgaonkar <
> vih...@cloudera.com>
> > wrote:
> >>
> >> Congrats Rajesh!
> >>
> >> On Wed, Dec 14, 2016 at 1:54 AM, Jesus Camacho Rodriguez <
> >> jcamachorodrig...@hortonworks.com> wrote:
> >>
> >> > Congrats Rajesh, well deserved! :)
> >> >
> >> > --
> >> > Jesús
> >> >
> >> >
> >> >
> >> >
> >> > On 12/14/16, 8:41 AM, "Lefty Leverenz" 
> wrote:
> >> >
> >> > >Congratulations Rajesh!
> >> > >
> >> > >-- Lefty
> >> > >
> >> > >
> >> > >On Tue, Dec 13, 2016 at 11:58 PM, Rajesh Balamohan
> >> > >  >> > >
> >> > >wrote:
> >> > >
> >> > >> Thanks a lot for providing this opportunity and to all for their
> >> > messages.
> >> > >> :)
> >> > >>
> >> > >> ~Rajesh.B
> >> > >>
> >> > >> On Wed, Dec 14, 2016 at 11:33 AM, Dharmesh Kakadia
> >> > >>  >> > >
> >> > >> wrote:
> >> > >>
> >> > >> > Congrats Rajesh !
> >> > >> >
> >> > >> > Thanks,
> >> > >> > Dharmesh
> >> > >> >
> >> > >> > On Tue, Dec 13, 2016 at 7:37 PM, Vikram Dixit K <
> >> > vikram.di...@gmail.com>
> >> > >> > wrote:
> >> > >> >
> >> > >> >> Congrats Rajesh! :)
> >> > >> >>
> >> > >> >> On Tue, Dec 13, 2016 at 9:36 PM, Pengcheng Xiong
> >> > >> >> 
> >> > >> >> wrote:
> >> > >> >>
> >> > >> >>> Congrats Rajesh! :)
> >> > >> >>>
> >> > >> >>> On Tue, Dec 13, 2016 at 6:51 PM, Prasanth Jayachandran <
> >> > >> >>> prasan...@apache.org
> >> > >> >>> > wrote:
> >> > >> >>>
> >> > >> >>> > The Apache Hive PMC has voted to make Rajesh Balamohan a
> >> > committer on
> >> > >> >>> the
> >> > >> >>> > Apache Hive Project. Please join me in congratulating Rajesh.
> >> > >> >>> >
> >> > >> >>> > Congratulations Rajesh!
> >> > >> >>> >
> >> > >> >>> > Thanks
> >> > >> >>> > Prasanth
> >> > >> >>>
> >> > >> >>
> >> > >> >>
> >> > >> >>
> >> > >> >> --
> >> > >> >> Nothing better than when appreciated for hard work.
> >> > >> >> -Mark
> >> > >> >>
> >> > >> >
> >> > >> >
> >> > >>
> >> >
> >
> >
>
>


-- 
Best regards!
Rui Li
Cell: (+86) 13564950210

Review Request 54771: Capture additional metadata to replicate a simple insert at destination

2016-12-14 Thread Vaibhav Gumashta 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/54771/
---

Review request for hive, Sushanth Sowmyan and Thejas Nair.


Bugs: HIVE-15294
https://issues.apache.org/jira/browse/HIVE-15294


Repository: hive-git


Description
---

https://issues.apache.org/jira/browse/HIVE-15294


Diffs
-

  
hcatalog/server-extensions/src/main/java/org/apache/hive/hcatalog/listener/DbNotificationListener.java
 119801f 
  
itests/hcatalog-unit/src/test/java/org/apache/hive/hcatalog/listener/TestDbNotificationListener.java
 690616d 
  metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java 
f7b2ed7 
  metastore/src/java/org/apache/hadoop/hive/metastore/events/InsertEvent.java 
102754e 
  
metastore/src/java/org/apache/hadoop/hive/metastore/messaging/MessageFactory.java
 adf2fd8 
  
metastore/src/java/org/apache/hadoop/hive/metastore/messaging/json/JSONInsertMessage.java
 ef89b17 
  
metastore/src/java/org/apache/hadoop/hive/metastore/messaging/json/JSONMessageFactory.java
 0407210 
  ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java 750fdef 

Diff: https://reviews.apache.org/r/54771/diff/


Testing
---


Thanks,

Vaibhav Gumashta

Review Request 54765: HIVE-15411: ADD PARTITION should support setting FILEFORMAT and SERDEPROPERTIES

2016-12-14 Thread Anthony Hsu 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/54765/
---

Review request for hive.


Bugs: HIVE-15411
https://issues.apache.org/jira/browse/HIVE-15411


Repository: hive-git


Description
---

HIVE-15411: ADD PARTITION should support setting FILEFORMAT and SERDEPROPERTIES


Diffs
-

  
hcatalog/core/src/main/java/org/apache/hive/hcatalog/cli/SemanticAnalysis/HCatSemanticAnalyzer.java
 18bf172116828439751ca4d0e99c83912f2b3915 
  ql/src/java/org/apache/hadoop/hive/ql/parse/DDLSemanticAnalyzer.java 
3f5813018b9305734e66dcff76064d6e3e6061f1 
  ql/src/java/org/apache/hadoop/hive/ql/parse/HiveParser.g 
55915a63be916b79dae022d76a4252ab1a18c64b 
  ql/src/java/org/apache/hadoop/hive/ql/parse/ImportSemanticAnalyzer.java 
ce952c5ee4d54b4c2a092f9ee15197ec0337fb4c 
  ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java 
79e55b2de07983c7b799ff382b9c71ef14d25b43 
  ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzerFactory.java 
520d3de9a7cc07b728b5d3ad3845622ddbec22fb 
  
ql/src/test/queries/clientpositive/add_part_fileformat_serdeproperties_location.q
 PRE-CREATION 
  
ql/src/test/results/clientpositive/add_part_fileformat_serdeproperties_location.q.out
 PRE-CREATION 

Diff: https://reviews.apache.org/r/54765/diff/


Testing
---

Added qtest.


Thanks,

Anthony Hsu

Re: [ANNOUNCE] New Hive Committer - Rajesh Balamohan

2016-12-14 Thread Gunther Hagleitner 
Congrats Rajesh!

From: Jimmy Xiang 
Sent: Wednesday, December 14, 2016 11:38 AM
To: u...@hive.apache.org
Cc: dev@hive.apache.org; rbalamo...@apache.org
Subject: Re: [ANNOUNCE] New Hive Committer - Rajesh Balamohan

Congrats, Rajesh!!

On Wed, Dec 14, 2016 at 11:32 AM, Sergey Shelukhin
 wrote:
> Congratulations!
>
> From: Chao Sun 
> Reply-To: "u...@hive.apache.org" 
> Date: Wednesday, December 14, 2016 at 10:52
> To: "dev@hive.apache.org" 
> Cc: "u...@hive.apache.org" , "rbalamo...@apache.org"
> 
> Subject: Re: [ANNOUNCE] New Hive Committer - Rajesh Balamohan
>
> Congrats Rajesh!
>
> On Wed, Dec 14, 2016 at 9:26 AM, Vihang Karajgaonkar 
> wrote:
>>
>> Congrats Rajesh!
>>
>> On Wed, Dec 14, 2016 at 1:54 AM, Jesus Camacho Rodriguez <
>> jcamachorodrig...@hortonworks.com> wrote:
>>
>> > Congrats Rajesh, well deserved! :)
>> >
>> > --
>> > Jesús
>> >
>> >
>> >
>> >
>> > On 12/14/16, 8:41 AM, "Lefty Leverenz"  wrote:
>> >
>> > >Congratulations Rajesh!
>> > >
>> > >-- Lefty
>> > >
>> > >
>> > >On Tue, Dec 13, 2016 at 11:58 PM, Rajesh Balamohan
>> > > > > >
>> > >wrote:
>> > >
>> > >> Thanks a lot for providing this opportunity and to all for their
>> > messages.
>> > >> :)
>> > >>
>> > >> ~Rajesh.B
>> > >>
>> > >> On Wed, Dec 14, 2016 at 11:33 AM, Dharmesh Kakadia
>> > >> > > >
>> > >> wrote:
>> > >>
>> > >> > Congrats Rajesh !
>> > >> >
>> > >> > Thanks,
>> > >> > Dharmesh
>> > >> >
>> > >> > On Tue, Dec 13, 2016 at 7:37 PM, Vikram Dixit K <
>> > vikram.di...@gmail.com>
>> > >> > wrote:
>> > >> >
>> > >> >> Congrats Rajesh! :)
>> > >> >>
>> > >> >> On Tue, Dec 13, 2016 at 9:36 PM, Pengcheng Xiong
>> > >> >> 
>> > >> >> wrote:
>> > >> >>
>> > >> >>> Congrats Rajesh! :)
>> > >> >>>
>> > >> >>> On Tue, Dec 13, 2016 at 6:51 PM, Prasanth Jayachandran <
>> > >> >>> prasan...@apache.org
>> > >> >>> > wrote:
>> > >> >>>
>> > >> >>> > The Apache Hive PMC has voted to make Rajesh Balamohan a
>> > committer on
>> > >> >>> the
>> > >> >>> > Apache Hive Project. Please join me in congratulating Rajesh.
>> > >> >>> >
>> > >> >>> > Congratulations Rajesh!
>> > >> >>> >
>> > >> >>> > Thanks
>> > >> >>> > Prasanth
>> > >> >>>
>> > >> >>
>> > >> >>
>> > >> >>
>> > >> >> --
>> > >> >> Nothing better than when appreciated for hard work.
>> > >> >> -Mark
>> > >> >>
>> > >> >
>> > >> >
>> > >>
>> >
>
>

Re: [ANNOUNCE] New Hive Committer - Rajesh Balamohan

2016-12-14 Thread Jimmy Xiang 
Congrats, Rajesh!!

On Wed, Dec 14, 2016 at 11:32 AM, Sergey Shelukhin
 wrote:
> Congratulations!
>
> From: Chao Sun 
> Reply-To: "u...@hive.apache.org" 
> Date: Wednesday, December 14, 2016 at 10:52
> To: "dev@hive.apache.org" 
> Cc: "u...@hive.apache.org" , "rbalamo...@apache.org"
> 
> Subject: Re: [ANNOUNCE] New Hive Committer - Rajesh Balamohan
>
> Congrats Rajesh!
>
> On Wed, Dec 14, 2016 at 9:26 AM, Vihang Karajgaonkar 
> wrote:
>>
>> Congrats Rajesh!
>>
>> On Wed, Dec 14, 2016 at 1:54 AM, Jesus Camacho Rodriguez <
>> jcamachorodrig...@hortonworks.com> wrote:
>>
>> > Congrats Rajesh, well deserved! :)
>> >
>> > --
>> > Jesús
>> >
>> >
>> >
>> >
>> > On 12/14/16, 8:41 AM, "Lefty Leverenz"  wrote:
>> >
>> > >Congratulations Rajesh!
>> > >
>> > >-- Lefty
>> > >
>> > >
>> > >On Tue, Dec 13, 2016 at 11:58 PM, Rajesh Balamohan
>> > > > > >
>> > >wrote:
>> > >
>> > >> Thanks a lot for providing this opportunity and to all for their
>> > messages.
>> > >> :)
>> > >>
>> > >> ~Rajesh.B
>> > >>
>> > >> On Wed, Dec 14, 2016 at 11:33 AM, Dharmesh Kakadia
>> > >> > > >
>> > >> wrote:
>> > >>
>> > >> > Congrats Rajesh !
>> > >> >
>> > >> > Thanks,
>> > >> > Dharmesh
>> > >> >
>> > >> > On Tue, Dec 13, 2016 at 7:37 PM, Vikram Dixit K <
>> > vikram.di...@gmail.com>
>> > >> > wrote:
>> > >> >
>> > >> >> Congrats Rajesh! :)
>> > >> >>
>> > >> >> On Tue, Dec 13, 2016 at 9:36 PM, Pengcheng Xiong
>> > >> >> 
>> > >> >> wrote:
>> > >> >>
>> > >> >>> Congrats Rajesh! :)
>> > >> >>>
>> > >> >>> On Tue, Dec 13, 2016 at 6:51 PM, Prasanth Jayachandran <
>> > >> >>> prasan...@apache.org
>> > >> >>> > wrote:
>> > >> >>>
>> > >> >>> > The Apache Hive PMC has voted to make Rajesh Balamohan a
>> > committer on
>> > >> >>> the
>> > >> >>> > Apache Hive Project. Please join me in congratulating Rajesh.
>> > >> >>> >
>> > >> >>> > Congratulations Rajesh!
>> > >> >>> >
>> > >> >>> > Thanks
>> > >> >>> > Prasanth
>> > >> >>>
>> > >> >>
>> > >> >>
>> > >> >>
>> > >> >> --
>> > >> >> Nothing better than when appreciated for hard work.
>> > >> >> -Mark
>> > >> >>
>> > >> >
>> > >> >
>> > >>
>> >
>
>

Re: [ANNOUNCE] New Hive Committer - Rajesh Balamohan

2016-12-14 Thread Sergey Shelukhin 
Congratulations!

From: Chao Sun mailto:sunc...@apache.org>>
Reply-To: "u...@hive.apache.org" 
mailto:u...@hive.apache.org>>
Date: Wednesday, December 14, 2016 at 10:52
To: "dev@hive.apache.org" 
mailto:dev@hive.apache.org>>
Cc: "u...@hive.apache.org" 
mailto:u...@hive.apache.org>>, 
"rbalamo...@apache.org" 
mailto:rbalamo...@apache.org>>
Subject: Re: [ANNOUNCE] New Hive Committer - Rajesh Balamohan

Congrats Rajesh!

On Wed, Dec 14, 2016 at 9:26 AM, Vihang Karajgaonkar 
mailto:vih...@cloudera.com>> wrote:
Congrats Rajesh!

On Wed, Dec 14, 2016 at 1:54 AM, Jesus Camacho Rodriguez <
jcamachorodrig...@hortonworks.com> 
wrote:

> Congrats Rajesh, well deserved! :)
>
> --
> Jesús
>
>
>
>
> On 12/14/16, 8:41 AM, "Lefty Leverenz" 
> mailto:leftylever...@gmail.com>> wrote:
>
> >Congratulations Rajesh!
> >
> >-- Lefty
> >
> >
> >On Tue, Dec 13, 2016 at 11:58 PM, Rajesh Balamohan 
> >mailto:rbalamo...@apache.org>
> >
> >wrote:
> >
> >> Thanks a lot for providing this opportunity and to all for their
> messages.
> >> :)
> >>
> >> ~Rajesh.B
> >>
> >> On Wed, Dec 14, 2016 at 11:33 AM, Dharmesh Kakadia 
> >> mailto:dhkaka...@gmail.com>
> >
> >> wrote:
> >>
> >> > Congrats Rajesh !
> >> >
> >> > Thanks,
> >> > Dharmesh
> >> >
> >> > On Tue, Dec 13, 2016 at 7:37 PM, Vikram Dixit K <
> vikram.di...@gmail.com>
> >> > wrote:
> >> >
> >> >> Congrats Rajesh! :)
> >> >>
> >> >> On Tue, Dec 13, 2016 at 9:36 PM, Pengcheng Xiong 
> >> >> mailto:pxi...@apache.org>>
> >> >> wrote:
> >> >>
> >> >>> Congrats Rajesh! :)
> >> >>>
> >> >>> On Tue, Dec 13, 2016 at 6:51 PM, Prasanth Jayachandran <
> >> >>> prasan...@apache.org
> >> >>> > wrote:
> >> >>>
> >> >>> > The Apache Hive PMC has voted to make Rajesh Balamohan a
> committer on
> >> >>> the
> >> >>> > Apache Hive Project. Please join me in congratulating Rajesh.
> >> >>> >
> >> >>> > Congratulations Rajesh!
> >> >>> >
> >> >>> > Thanks
> >> >>> > Prasanth
> >> >>>
> >> >>
> >> >>
> >> >>
> >> >> --
> >> >> Nothing better than when appreciated for hard work.
> >> >> -Mark
> >> >>
> >> >
> >> >
> >>
>

Re: Review Request 54517: HIVE-15192: Subquery support with Calcite

2016-12-14 Thread Vineet Garg 


> On Dec. 13, 2016, 1:51 a.m., Ashutosh Chauhan wrote:
> > ql/src/java/org/apache/hadoop/hive/ql/lib/SubQueryWalker.java, line 27
> > 
> >
> > Better name: Expression walker?

Updated class name to ExpressionWalker


> On Dec. 13, 2016, 1:51 a.m., Ashutosh Chauhan wrote:
> > ql/src/java/org/apache/hadoop/hive/ql/lib/SubQueryWalker.java, line 40
> > 
> >
> > When is later? 
> > SubQueryExprProcessor already handles TOK_SUBQUERY_OP so why do we need 
> > to bypass subquery processing?

Later is when we generate plan for subquery. (Shouldn't have said later since 
we have already processed and created logical plan for subquery at this point). 
SubQueryWalker bypasses subquery since we are seperately generating logical 
plan for it. SubQueryExprProcessor then uses this generated pland and creates 
appropriate ExprNodeSubQueryDesc. I'll update comments to make it more clear.


> On Dec. 13, 2016, 1:51 a.m., Ashutosh Chauhan wrote:
> > ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/reloperators/HiveFilter.java,
> >  line 82
> > 
> >
> > Each HiveFilter will be visited by RelShuttle. Why do we need to 
> > traverse the tree here?

RelShuttle is used by decorrelation logic but SubqueryRemoveRule uses 
getVariableSet function which needs to traverse the filter to get set of all 
corr vars.


> On Dec. 13, 2016, 1:51 a.m., Ashutosh Chauhan wrote:
> > ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/rules/HiveRelDecorrelator.java,
> >  line 283
> > 
> >
> > can be called before Decorrelator?

This particular rule is defined in HiveRelDecorrelator and is called just 
before decorrelation.
Edit: BTW we plan to use Calcite's RelDecorrelator soon and will have all these 
rules in decorrelation anyway. Not sure if it is worth updating it now.


> On Dec. 13, 2016, 1:51 a.m., Ashutosh Chauhan wrote:
> > ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/rules/HiveRelDecorrelator.java,
> >  line 419
> > 
> >
> > This can be removed since we alreafy have overloaded version of this 
> > with HiveAggregate.

We can not remove this since at this point we have a plan with mix of 
LogicalAggregate and HiveAggregate. Decorrelator uses reflection to dynamically 
figure out the type and call appropriate method.


> On Dec. 13, 2016, 1:51 a.m., Ashutosh Chauhan wrote:
> > ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/rules/HiveRelDecorrelator.java,
> >  lines 1053-1054
> > 
> >
> > Can this be removed? If so, what above comment above?

Yes removed.


> On Dec. 13, 2016, 1:51 a.m., Ashutosh Chauhan wrote:
> > ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/rules/HiveRelDecorrelator.java,
> >  line 860
> > 
> >
> > Can be removed.

Can't remove (same as above)


> On Dec. 13, 2016, 1:51 a.m., Ashutosh Chauhan wrote:
> > ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/rules/HiveRelDecorrelator.java,
> >  line 1144
> > 
> >
> > Can be removed.

Can not (same as above)


> On Dec. 13, 2016, 1:51 a.m., Ashutosh Chauhan wrote:
> > ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/rules/HiveRelDecorrelator.java,
> >  line 1355
> > 
> >
> > can be removed.

Cannot (reason is same as above)


> On Dec. 13, 2016, 1:51 a.m., Ashutosh Chauhan wrote:
> > ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/rules/HiveSubQueryRemoveRule.java,
> >  line 77
> > 
> >
> > This assert can fail if there is no subquery, no?

this rule is called only if there is subquery


> On Dec. 13, 2016, 1:51 a.m., Ashutosh Chauhan wrote:
> > ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/translator/RexNodeConverter.java,
> >  line 513
> > 
> >
> > this should be RexCorrelVariable, not RexFieldAccess? No?

No it consists of RexCorrelVariable (corExpr here)


> On Dec. 13, 2016, 1:51 a.m., Ashutosh Chauhan wrote:
> > ql/src/java/org/apache/hadoop/hive/ql/plan/ExprNodeSubQueryDesc.java, line 
> > 29
> > 
> >
> > I am not sure about ExprNodeSubQueryDesc class since it encapsulates 
> > both calcite as well as Hive data structure. Till Now, ExprNodeDesc is 
> > ex

Re: [ANNOUNCE] New Hive Committer - Rajesh Balamohan

2016-12-14 Thread Sergio Pena 
Congratulations Rajesh !!!

On Wed, Dec 14, 2016 at 12:52 PM, Chao Sun  wrote:

> Congrats Rajesh!
>
> On Wed, Dec 14, 2016 at 9:26 AM, Vihang Karajgaonkar 
> wrote:
>
> > Congrats Rajesh!
> >
> > On Wed, Dec 14, 2016 at 1:54 AM, Jesus Camacho Rodriguez <
> > jcamachorodrig...@hortonworks.com> wrote:
> >
> > > Congrats Rajesh, well deserved! :)
> > >
> > > --
> > > Jesús
> > >
> > >
> > >
> > >
> > > On 12/14/16, 8:41 AM, "Lefty Leverenz" 
> wrote:
> > >
> > > >Congratulations Rajesh!
> > > >
> > > >-- Lefty
> > > >
> > > >
> > > >On Tue, Dec 13, 2016 at 11:58 PM, Rajesh Balamohan <
> > rbalamo...@apache.org
> > > >
> > > >wrote:
> > > >
> > > >> Thanks a lot for providing this opportunity and to all for their
> > > messages.
> > > >> :)
> > > >>
> > > >> ~Rajesh.B
> > > >>
> > > >> On Wed, Dec 14, 2016 at 11:33 AM, Dharmesh Kakadia <
> > dhkaka...@gmail.com
> > > >
> > > >> wrote:
> > > >>
> > > >> > Congrats Rajesh !
> > > >> >
> > > >> > Thanks,
> > > >> > Dharmesh
> > > >> >
> > > >> > On Tue, Dec 13, 2016 at 7:37 PM, Vikram Dixit K <
> > > vikram.di...@gmail.com>
> > > >> > wrote:
> > > >> >
> > > >> >> Congrats Rajesh! :)
> > > >> >>
> > > >> >> On Tue, Dec 13, 2016 at 9:36 PM, Pengcheng Xiong <
> > pxi...@apache.org>
> > > >> >> wrote:
> > > >> >>
> > > >> >>> Congrats Rajesh! :)
> > > >> >>>
> > > >> >>> On Tue, Dec 13, 2016 at 6:51 PM, Prasanth Jayachandran <
> > > >> >>> prasan...@apache.org
> > > >> >>> > wrote:
> > > >> >>>
> > > >> >>> > The Apache Hive PMC has voted to make Rajesh Balamohan a
> > > committer on
> > > >> >>> the
> > > >> >>> > Apache Hive Project. Please join me in congratulating Rajesh.
> > > >> >>> >
> > > >> >>> > Congratulations Rajesh!
> > > >> >>> >
> > > >> >>> > Thanks
> > > >> >>> > Prasanth
> > > >> >>>
> > > >> >>
> > > >> >>
> > > >> >>
> > > >> >> --
> > > >> >> Nothing better than when appreciated for hard work.
> > > >> >> -Mark
> > > >> >>
> > > >> >
> > > >> >
> > > >>
> > >
> >
>

Re: [ANNOUNCE] New Hive Committer - Rajesh Balamohan

2016-12-14 Thread Chao Sun 
Congrats Rajesh!

On Wed, Dec 14, 2016 at 9:26 AM, Vihang Karajgaonkar 
wrote:

> Congrats Rajesh!
>
> On Wed, Dec 14, 2016 at 1:54 AM, Jesus Camacho Rodriguez <
> jcamachorodrig...@hortonworks.com> wrote:
>
> > Congrats Rajesh, well deserved! :)
> >
> > --
> > Jesús
> >
> >
> >
> >
> > On 12/14/16, 8:41 AM, "Lefty Leverenz"  wrote:
> >
> > >Congratulations Rajesh!
> > >
> > >-- Lefty
> > >
> > >
> > >On Tue, Dec 13, 2016 at 11:58 PM, Rajesh Balamohan <
> rbalamo...@apache.org
> > >
> > >wrote:
> > >
> > >> Thanks a lot for providing this opportunity and to all for their
> > messages.
> > >> :)
> > >>
> > >> ~Rajesh.B
> > >>
> > >> On Wed, Dec 14, 2016 at 11:33 AM, Dharmesh Kakadia <
> dhkaka...@gmail.com
> > >
> > >> wrote:
> > >>
> > >> > Congrats Rajesh !
> > >> >
> > >> > Thanks,
> > >> > Dharmesh
> > >> >
> > >> > On Tue, Dec 13, 2016 at 7:37 PM, Vikram Dixit K <
> > vikram.di...@gmail.com>
> > >> > wrote:
> > >> >
> > >> >> Congrats Rajesh! :)
> > >> >>
> > >> >> On Tue, Dec 13, 2016 at 9:36 PM, Pengcheng Xiong <
> pxi...@apache.org>
> > >> >> wrote:
> > >> >>
> > >> >>> Congrats Rajesh! :)
> > >> >>>
> > >> >>> On Tue, Dec 13, 2016 at 6:51 PM, Prasanth Jayachandran <
> > >> >>> prasan...@apache.org
> > >> >>> > wrote:
> > >> >>>
> > >> >>> > The Apache Hive PMC has voted to make Rajesh Balamohan a
> > committer on
> > >> >>> the
> > >> >>> > Apache Hive Project. Please join me in congratulating Rajesh.
> > >> >>> >
> > >> >>> > Congratulations Rajesh!
> > >> >>> >
> > >> >>> > Thanks
> > >> >>> > Prasanth
> > >> >>>
> > >> >>
> > >> >>
> > >> >>
> > >> >> --
> > >> >> Nothing better than when appreciated for hard work.
> > >> >> -Mark
> > >> >>
> > >> >
> > >> >
> > >>
> >
>

Re: [ANNOUNCE] New Hive Committer - Rajesh Balamohan

2016-12-14 Thread Vihang Karajgaonkar 
Congrats Rajesh!

On Wed, Dec 14, 2016 at 1:54 AM, Jesus Camacho Rodriguez <
jcamachorodrig...@hortonworks.com> wrote:

> Congrats Rajesh, well deserved! :)
>
> --
> Jesús
>
>
>
>
> On 12/14/16, 8:41 AM, "Lefty Leverenz"  wrote:
>
> >Congratulations Rajesh!
> >
> >-- Lefty
> >
> >
> >On Tue, Dec 13, 2016 at 11:58 PM, Rajesh Balamohan  >
> >wrote:
> >
> >> Thanks a lot for providing this opportunity and to all for their
> messages.
> >> :)
> >>
> >> ~Rajesh.B
> >>
> >> On Wed, Dec 14, 2016 at 11:33 AM, Dharmesh Kakadia  >
> >> wrote:
> >>
> >> > Congrats Rajesh !
> >> >
> >> > Thanks,
> >> > Dharmesh
> >> >
> >> > On Tue, Dec 13, 2016 at 7:37 PM, Vikram Dixit K <
> vikram.di...@gmail.com>
> >> > wrote:
> >> >
> >> >> Congrats Rajesh! :)
> >> >>
> >> >> On Tue, Dec 13, 2016 at 9:36 PM, Pengcheng Xiong 
> >> >> wrote:
> >> >>
> >> >>> Congrats Rajesh! :)
> >> >>>
> >> >>> On Tue, Dec 13, 2016 at 6:51 PM, Prasanth Jayachandran <
> >> >>> prasan...@apache.org
> >> >>> > wrote:
> >> >>>
> >> >>> > The Apache Hive PMC has voted to make Rajesh Balamohan a
> committer on
> >> >>> the
> >> >>> > Apache Hive Project. Please join me in congratulating Rajesh.
> >> >>> >
> >> >>> > Congratulations Rajesh!
> >> >>> >
> >> >>> > Thanks
> >> >>> > Prasanth
> >> >>>
> >> >>
> >> >>
> >> >>
> >> >> --
> >> >> Nothing better than when appreciated for hard work.
> >> >> -Mark
> >> >>
> >> >
> >> >
> >>
>

Re: Review Request 54517: HIVE-15192: Subquery support with Calcite

2016-12-14 Thread Ashutosh Chauhan 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/54517/#review159181
---



Also, Hive uses Sun's style guide. 
http://www.oracle.com/technetwork/java/codeconventions-150003.pdf Particularly, 
{ are on same line where the conditional and not after.

- Ashutosh Chauhan


On Dec. 12, 2016, 6:17 p.m., Vineet Garg wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/54517/
> ---
> 
> (Updated Dec. 12, 2016, 6:17 p.m.)
> 
> 
> Review request for hive and Ashutosh Chauhan.
> 
> 
> Bugs: HIVE-15192
> https://issues.apache.org/jira/browse/HIVE-15192
> 
> 
> Repository: hive-git
> 
> 
> Description
> ---
> 
> This patch is 1st phase of getting rid of Hive's subquery transformation and 
> de-corelation logic and leverage Calcite's functionality to plan sub-queries.
> 
> Known issues with this patch
> * Few return path tests are failing and are disabled with this patch.
> * Semi-join optimization (HIVE-15227) is disabled currently as it doesn't 
> work with this patch.
> 
> 
> Diffs
> -
> 
>   common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 9064e49 
>   itests/src/test/resources/testconfiguration.properties aa3d72d 
>   ql/src/java/org/apache/hadoop/hive/ql/lib/SubQueryWalker.java PRE-CREATION 
>   ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/HiveRelShuttle.java 
> PRE-CREATION 
>   
> ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/HiveRelShuttleImpl.java
>  PRE-CREATION 
>   
> ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/HiveReplicatedRelBuilder.java
>  PRE-CREATION 
>   
> ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/reloperators/HiveFilter.java
>  0410c91 
>   
> ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/reloperators/HiveJoin.java
>  ba9483e 
>   
> ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/reloperators/HiveProject.java
>  3e0a9a6 
>   
> ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/rules/HiveRelDecorrelator.java
>  PRE-CREATION 
>   
> ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/rules/HiveSubQueryRemoveRule.java
>  PRE-CREATION 
>   
> ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/translator/HiveOpConverter.java
>  d494c9f 
>   
> ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/translator/RexNodeConverter.java
>  f8fb475 
>   ql/src/java/org/apache/hadoop/hive/ql/parse/CalcitePlanner.java f1f3bf9 
>   ql/src/java/org/apache/hadoop/hive/ql/parse/QBSubQuery.java 3458fb6 
>   ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java 79e55b2 
>   ql/src/java/org/apache/hadoop/hive/ql/parse/TypeCheckCtx.java 02896ff 
>   ql/src/java/org/apache/hadoop/hive/ql/parse/TypeCheckProcFactory.java 
> ace3eaf 
>   ql/src/java/org/apache/hadoop/hive/ql/plan/ExprNodeSubQueryDesc.java 
> PRE-CREATION 
>   ql/src/test/queries/clientnegative/subquery_nested_subquery.q  
>   ql/src/test/queries/clientnegative/subquery_restrictions.q PRE-CREATION 
>   ql/src/test/queries/clientnegative/subquery_shared_alias.q  
>   ql/src/test/queries/clientpositive/cbo_rp_auto_join1.q cbfb5d5 
>   ql/src/test/queries/clientpositive/join31.q c79105f 
>   ql/src/test/queries/clientpositive/multiMapJoin2.q cf5dbb0 
>   ql/src/test/queries/clientpositive/semijoin5.q 3e7c20a 
>   ql/src/test/queries/clientpositive/subquery_in.q c01ae70 
>   ql/src/test/queries/clientpositive/subquery_notin.q 3f4fb7f 
>   ql/src/test/queries/clientpositive/subquery_notin_having.q 05148df 
>   ql/src/test/results/clientnegative/subquery_in_groupby.q.out 809bb0a 
>   ql/src/test/results/clientnegative/subquery_in_select.q.out 3d74132 
>   ql/src/test/results/clientnegative/subquery_nested_subquery.q.out 140b093 
>   ql/src/test/results/clientnegative/subquery_restrictions.q.out PRE-CREATION 
>   ql/src/test/results/clientpositive/constant_prop_3.q.out 58f1065 
>   ql/src/test/results/clientpositive/constprog_partitioner.q.out d1016ad 
>   ql/src/test/results/clientpositive/llap/cbo_rp_subq_in.q.out f6bfad2 
>   ql/src/test/results/clientpositive/llap/dynamic_partition_pruning.q.out 
> f993cf0 
>   ql/src/test/results/clientpositive/llap/explainuser_1.q.out 70ec02f 
>   ql/src/test/results/clientpositive/llap/lineage3.q.out 1a532da 
>   ql/src/test/results/clientpositive/llap/subquery_exists.q.out 1a006d8 
>   ql/src/test/results/clientpositive/llap/subquery_in.q.out 321f1cc 
>   ql/src/test/results/clientpositive/llap/subquery_nested_subquery.q.out 
> PRE-CREATION 
>   ql/src/test/results/clientpositive/llap/subquery_notin.q.out 3da1acb 
>   ql/src/test/results/clientpositive/llap/subquery_shared_alias.q.out 
> PRE-CREATION 
>   ql/src/test/results/clientpositive/llap/subquery_views.q.out 35e80ae 
>   ql/src/test/re

[jira] [Created] (HIVE-15430) Change SchemaTool table validator to test based on the dbType

2016-12-14 Thread Naveen Gangam (JIRA) 
Naveen Gangam created HIVE-15430:


 Summary: Change SchemaTool table validator to test based on the 
dbType
 Key: HIVE-15430
 URL: https://issues.apache.org/jira/browse/HIVE-15430
 Project: Hive
  Issue Type: Sub-task
  Components: Hive
Affects Versions: 2.2.0
Reporter: Naveen Gangam
Assignee: Naveen Gangam
Priority: Minor


Currently the validator parses the "oracle" schema file to determine what 
tables are expected in the database.  (mostly because of ease of parsing the 
schema file compared to other syntax). We have learnt from HIVE-15118, that not 
all schema files have the same amount of tables. For example, derby has an old 
table that is never used that other DBs do not contain).



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (HIVE-15429) Alter column type from bigint to double got errors

2016-12-14 Thread LongShangRen (JIRA) 
LongShangRen created HIVE-15429:
---

 Summary: Alter column type from bigint to double got errors
 Key: HIVE-15429
 URL: https://issues.apache.org/jira/browse/HIVE-15429
 Project: Hive
  Issue Type: Bug
Affects Versions: 1.2.1
Reporter: LongShangRen


we try to alter a column for bigint to double. 
Altering was succeed.
when we query from the same table,wo got errors as follow

{color:red}
 2016-12-14 17:53:53.660 [main] ERROR com.mogujie.hdata.core.HData - 
java.util.concurrent.ExecutionException: 
java.lang.ArrayIndexOutOfBoundsException: 32
at java.util.concurrent.FutureTask.report(FutureTask.java:122)
at java.util.concurrent.FutureTask.get(FutureTask.java:192)
at com.mogujie.hdata.core.HData.start(HData.java:189)
at com.mogujie.hdata.CliDriver.main(CliDriver.java:175)
Caused by: java.lang.ArrayIndexOutOfBoundsException: 32
at 
org.apache.hadoop.hive.serde2.lazybinary.LazyBinaryUtils.byteArrayToLong(LazyBinaryUtils.java:78)
at 
org.apache.hadoop.hive.serde2.lazybinary.LazyBinaryDouble.init(LazyBinaryDouble.java:43)
at 
org.apache.hadoop.hive.serde2.columnar.ColumnarStructBase$FieldInfo.uncheckedGetField(ColumnarStructBase.java:111)
at 
org.apache.hadoop.hive.serde2.columnar.ColumnarStructBase.getField(ColumnarStructBase.java:172)
at 
org.apache.hadoop.hive.serde2.objectinspector.ColumnarStructObjectInspector.getStructFieldData(ColumnarStructObjectInspector.java:67)
at 
org.apache.hive.hcatalog.data.LazyHCatRecord.get(LazyHCatRecord.java:54)
at 
org.apache.hive.hcatalog.data.LazyHCatRecord.get(LazyHCatRecord.java:97)
at 
org.apache.hive.hcatalog.mapreduce.HCatRecordReader.nextKeyValue(HCatRecordReader.java:204)
at 
org.apache.hive.hcatalog.data.transfer.impl.HCatInputFormatReader$HCatRecordItr.hasNext(HCatInputFormatReader.java:107)
at 
com.mogujie.hdata.plugin.reader.hive.HiveReader.execute(HiveReader.java:91)
at com.mogujie.hdata.core.ReaderWorker.call(ReaderWorker.java:34)
at com.mogujie.hdata.core.ReaderWorker.call(ReaderWorker.java:16)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at 
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
{color}






--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: [ANNOUNCE] New Hive Committer - Rajesh Balamohan

2016-12-14 Thread Jesus Camacho Rodriguez 
Congrats Rajesh, well deserved! :)

--
Jesús




On 12/14/16, 8:41 AM, "Lefty Leverenz"  wrote:

>Congratulations Rajesh!
>
>-- Lefty
>
>
>On Tue, Dec 13, 2016 at 11:58 PM, Rajesh Balamohan 
>wrote:
>
>> Thanks a lot for providing this opportunity and to all for their messages.
>> :)
>>
>> ~Rajesh.B
>>
>> On Wed, Dec 14, 2016 at 11:33 AM, Dharmesh Kakadia 
>> wrote:
>>
>> > Congrats Rajesh !
>> >
>> > Thanks,
>> > Dharmesh
>> >
>> > On Tue, Dec 13, 2016 at 7:37 PM, Vikram Dixit K 
>> > wrote:
>> >
>> >> Congrats Rajesh! :)
>> >>
>> >> On Tue, Dec 13, 2016 at 9:36 PM, Pengcheng Xiong 
>> >> wrote:
>> >>
>> >>> Congrats Rajesh! :)
>> >>>
>> >>> On Tue, Dec 13, 2016 at 6:51 PM, Prasanth Jayachandran <
>> >>> prasan...@apache.org
>> >>> > wrote:
>> >>>
>> >>> > The Apache Hive PMC has voted to make Rajesh Balamohan a committer on
>> >>> the
>> >>> > Apache Hive Project. Please join me in congratulating Rajesh.
>> >>> >
>> >>> > Congratulations Rajesh!
>> >>> >
>> >>> > Thanks
>> >>> > Prasanth
>> >>>
>> >>
>> >>
>> >>
>> >> --
>> >> Nothing better than when appreciated for hard work.
>> >> -Mark
>> >>
>> >
>> >
>>

Re: [ANNOUNCE] New Hive Committer - Rajesh Balamohan

2016-12-14 Thread Lefty Leverenz 
Congratulations Rajesh!

-- Lefty


On Tue, Dec 13, 2016 at 11:58 PM, Rajesh Balamohan 
wrote:

> Thanks a lot for providing this opportunity and to all for their messages.
> :)
>
> ~Rajesh.B
>
> On Wed, Dec 14, 2016 at 11:33 AM, Dharmesh Kakadia 
> wrote:
>
> > Congrats Rajesh !
> >
> > Thanks,
> > Dharmesh
> >
> > On Tue, Dec 13, 2016 at 7:37 PM, Vikram Dixit K 
> > wrote:
> >
> >> Congrats Rajesh! :)
> >>
> >> On Tue, Dec 13, 2016 at 9:36 PM, Pengcheng Xiong 
> >> wrote:
> >>
> >>> Congrats Rajesh! :)
> >>>
> >>> On Tue, Dec 13, 2016 at 6:51 PM, Prasanth Jayachandran <
> >>> prasan...@apache.org
> >>> > wrote:
> >>>
> >>> > The Apache Hive PMC has voted to make Rajesh Balamohan a committer on
> >>> the
> >>> > Apache Hive Project. Please join me in congratulating Rajesh.
> >>> >
> >>> > Congratulations Rajesh!
> >>> >
> >>> > Thanks
> >>> > Prasanth
> >>>
> >>
> >>
> >>
> >> --
> >> Nothing better than when appreciated for hard work.
> >> -Mark
> >>
> >
> >
>