Re: Review Request 50787: Add a timezone-aware timestamp

[Rui Li]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/50787/
---

(Updated May 10, 2017, 4:19 a.m.)


Review request for hive, Ashutosh Chauhan, pengcheng xiong, and Xuefu Zhang.


Bugs: HIVE-14412
https://issues.apache.org/jira/browse/HIVE-14412


Repository: hive-git


Description
---

The 1st patch to add timezone-aware timestamp.


Diffs (updated)
-

  common/src/java/org/apache/hadoop/hive/common/type/TimestampTZ.java 
PRE-CREATION 
  common/src/test/org/apache/hadoop/hive/common/type/TestTimestampTZ.java 
PRE-CREATION 
  contrib/src/test/queries/clientnegative/serde_regex.q a676338 
  contrib/src/test/queries/clientpositive/serde_regex.q d75d607 
  contrib/src/test/results/clientnegative/serde_regex.q.out 58b1c02 
  contrib/src/test/results/clientpositive/serde_regex.q.out 2984293 
  hbase-handler/src/test/queries/positive/hbase_timestamp.q 0350afe 
  hbase-handler/src/test/results/positive/hbase_timestamp.q.out 3918121 
  itests/hive-blobstore/src/test/queries/clientpositive/orc_format_part.q 
358eccd 
  
itests/hive-blobstore/src/test/queries/clientpositive/orc_nonstd_partitions_loc.q
 c462538 
  itests/hive-blobstore/src/test/queries/clientpositive/rcfile_format_part.q 
c563d3a 
  
itests/hive-blobstore/src/test/queries/clientpositive/rcfile_nonstd_partitions_loc.q
 d17c281 
  itests/hive-blobstore/src/test/results/clientpositive/orc_format_part.q.out 
5d1319f 
  
itests/hive-blobstore/src/test/results/clientpositive/orc_nonstd_partitions_loc.q.out
 70e72f7 
  
itests/hive-blobstore/src/test/results/clientpositive/rcfile_format_part.q.out 
bed10ab 
  
itests/hive-blobstore/src/test/results/clientpositive/rcfile_nonstd_partitions_loc.q.out
 c6442f9 
  jdbc/src/java/org/apache/hive/jdbc/HiveBaseResultSet.java ade1900 
  jdbc/src/java/org/apache/hive/jdbc/JdbcColumn.java 38918f0 
  ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionRegistry.java bf18a8d 
  ql/src/java/org/apache/hadoop/hive/ql/exec/GroupByOperator.java f8b55da 
  ql/src/java/org/apache/hadoop/hive/ql/exec/SerializationUtilities.java 
01a652d 
  
ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/translator/TypeConverter.java
 38308c9 
  ql/src/java/org/apache/hadoop/hive/ql/parse/DDLSemanticAnalyzer.java 0cf9205 
  ql/src/java/org/apache/hadoop/hive/ql/parse/HiveLexer.g 190b66b 
  ql/src/java/org/apache/hadoop/hive/ql/parse/HiveParser.g 3136c93 
  ql/src/java/org/apache/hadoop/hive/ql/parse/IdentifiersParser.g 1c78c1a 
  ql/src/java/org/apache/hadoop/hive/ql/parse/TypeCheckProcFactory.java 82141be 
  ql/src/java/org/apache/hadoop/hive/ql/stats/StatsUtils.java bda2050 
  ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToBoolean.java 7cdf2c3 
  ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToString.java 5cacd59 
  ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDF.java 68d98f5 
  ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFDate.java 5a31e61 
  
ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFToTimestampTZ.java 
PRE-CREATION 
  
ql/src/test/org/apache/hadoop/hive/ql/parse/TestSQL11ReservedKeyWordsNegative.java
 0dc6b19 
  ql/src/test/queries/clientnegative/serde_regex.q c9cfc7d 
  ql/src/test/queries/clientnegative/serde_regex2.q a29bb9c 
  ql/src/test/queries/clientnegative/serde_regex3.q 4e91f06 
  ql/src/test/queries/clientpositive/create_like.q bd39731 
  ql/src/test/queries/clientpositive/join43.q 12c45a6 
  ql/src/test/queries/clientpositive/serde_regex.q e21c6e1 
  ql/src/test/queries/clientpositive/timestamptz.q PRE-CREATION 
  ql/src/test/queries/clientpositive/timestamptz_1.q PRE-CREATION 
  ql/src/test/queries/clientpositive/timestamptz_2.q PRE-CREATION 
  ql/src/test/results/clientnegative/serde_regex.q.out a1ec5ca 
  ql/src/test/results/clientnegative/serde_regex2.q.out 374675d 
  ql/src/test/results/clientnegative/serde_regex3.q.out dc0a0e2 
  ql/src/test/results/clientpositive/create_like.q.out ff2e752 
  ql/src/test/results/clientpositive/join43.q.out e8c7278 
  ql/src/test/results/clientpositive/serde_regex.q.out 7bebb0c 
  ql/src/test/results/clientpositive/timestamptz.q.out PRE-CREATION 
  ql/src/test/results/clientpositive/timestamptz_1.q.out PRE-CREATION 
  ql/src/test/results/clientpositive/timestamptz_2.q.out PRE-CREATION 
  serde/if/serde.thrift 1d40d5a 
  serde/src/gen/thrift/gen-cpp/serde_constants.h 8785bd2 
  serde/src/gen/thrift/gen-cpp/serde_constants.cpp 907acf2 
  
serde/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/serde/serdeConstants.java
 2578d3e 
  serde/src/gen/thrift/gen-php/org/apache/hadoop/hive/serde/Types.php ea2dbbe 
  serde/src/gen/thrift/gen-py/org_apache_hadoop_hive_serde/constants.py e3b24eb 
  serde/src/gen/thrift/gen-rb/serde_constants.rb 15efaea 
  serde/src/java/org/apache/hadoop/hive/serde2/SerDeUtils.java 5ecfbca 
  
serde/src/java/org/apache/hadoop/hive/serde2/binarysortable/Bin

Re: Review Request 50787: Add a timezone-aware timestamp

[Rui Li]

> On May 9, 2017, 11:05 p.m., Ashutosh Chauhan wrote:
> > ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/translator/TypeConverter.java
> > Lines 204 (patched)
> > 
> >
> > Can you file a bug in Calcite that it should have sql type to represent 
> > TS w TZ?

Filed CALCITE-1784 for it


> On May 9, 2017, 11:05 p.m., Ashutosh Chauhan wrote:
> > ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToString.java
> > Lines 160 (patched)
> > 
> >
> > Add a comment that string reperesentation will return TS in UTC zone 
> > and not in original TZ.

Here we convert timestamptz to string, which means TZ is already in UTC. I will 
add the comment when we convert string to timestamptz.


- Rui


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/50787/#review174383
---


On May 8, 2017, 3:17 p.m., Rui Li wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/50787/
> ---
> 
> (Updated May 8, 2017, 3:17 p.m.)
> 
> 
> Review request for hive, pengcheng xiong and Xuefu Zhang.
> 
> 
> Bugs: HIVE-14412
> https://issues.apache.org/jira/browse/HIVE-14412
> 
> 
> Repository: hive-git
> 
> 
> Description
> ---
> 
> The 1st patch to add timezone-aware timestamp.
> 
> 
> Diffs
> -
> 
>   common/src/java/org/apache/hadoop/hive/common/type/TimestampTZ.java 
> PRE-CREATION 
>   common/src/test/org/apache/hadoop/hive/common/type/TestTimestampTZ.java 
> PRE-CREATION 
>   contrib/src/test/queries/clientnegative/serde_regex.q a676338 
>   contrib/src/test/queries/clientpositive/serde_regex.q d75d607 
>   contrib/src/test/results/clientnegative/serde_regex.q.out 58b1c02 
>   contrib/src/test/results/clientpositive/serde_regex.q.out 2984293 
>   hbase-handler/src/test/queries/positive/hbase_timestamp.q 0350afe 
>   hbase-handler/src/test/results/positive/hbase_timestamp.q.out 3918121 
>   itests/hive-blobstore/src/test/queries/clientpositive/orc_format_part.q 
> 358eccd 
>   
> itests/hive-blobstore/src/test/queries/clientpositive/orc_nonstd_partitions_loc.q
>  c462538 
>   itests/hive-blobstore/src/test/queries/clientpositive/rcfile_format_part.q 
> c563d3a 
>   
> itests/hive-blobstore/src/test/queries/clientpositive/rcfile_nonstd_partitions_loc.q
>  d17c281 
>   itests/hive-blobstore/src/test/results/clientpositive/orc_format_part.q.out 
> 5d1319f 
>   
> itests/hive-blobstore/src/test/results/clientpositive/orc_nonstd_partitions_loc.q.out
>  70e72f7 
>   
> itests/hive-blobstore/src/test/results/clientpositive/rcfile_format_part.q.out
>  bed10ab 
>   
> itests/hive-blobstore/src/test/results/clientpositive/rcfile_nonstd_partitions_loc.q.out
>  c6442f9 
>   jdbc/src/java/org/apache/hive/jdbc/HiveBaseResultSet.java ade1900 
>   jdbc/src/java/org/apache/hive/jdbc/JdbcColumn.java 38918f0 
>   ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionRegistry.java 1b556ac 
>   ql/src/java/org/apache/hadoop/hive/ql/exec/GroupByOperator.java f8b55da 
>   ql/src/java/org/apache/hadoop/hive/ql/exec/SerializationUtilities.java 
> 01a652d 
>   
> ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/translator/TypeConverter.java
>  38308c9 
>   ql/src/java/org/apache/hadoop/hive/ql/parse/DDLSemanticAnalyzer.java 
> 0cf9205 
>   ql/src/java/org/apache/hadoop/hive/ql/parse/HiveLexer.g 190b66b 
>   ql/src/java/org/apache/hadoop/hive/ql/parse/HiveParser.g ca639d3 
>   ql/src/java/org/apache/hadoop/hive/ql/parse/IdentifiersParser.g 645ced9 
>   ql/src/java/org/apache/hadoop/hive/ql/parse/TypeCheckProcFactory.java 
> c3227c9 
>   ql/src/java/org/apache/hadoop/hive/ql/stats/StatsUtils.java bda2050 
>   ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToBoolean.java 7cdf2c3 
>   ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToString.java 5cacd59 
>   ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDF.java 68d98f5 
>   ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFDate.java 
> 5a31e61 
>   
> ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFToTimestampTZ.java
>  PRE-CREATION 
>   
> ql/src/test/org/apache/hadoop/hive/ql/parse/TestSQL11ReservedKeyWordsNegative.java
>  0dc6b19 
>   ql/src/test/queries/clientnegative/serde_regex.q c9cfc7d 
>   ql/src/test/queries/clientnegative/serde_regex2.q a29bb9c 
>   ql/src/test/queries/clientnegative/serde_regex3.q 4e91f06 
>   ql/src/test/queries/clientpositive/create_like.q bd39731 
>   ql/src/test/queries/clientpositive/join43.q 12c45a6 
>   ql/src/test/queries/clientpositive/serde_regex.q e21c6e1 
>   ql/src/test/queries/clientpositive/timestamptz.q PRE-CREATION 
>   ql/src/test/queries/clientpositive/

Re: Review Request 50787: Add a timezone-aware timestamp

[Ashutosh Chauhan]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/50787/#review174383
---




common/src/java/org/apache/hadoop/hive/common/type/TimestampTZ.java
Lines 48 (patched)


Worth adding a comment on how value is represented when zone info is 
absent. From code it seems, like it will be parsed as system's default zone and 
then normalized and stored as UTC.



common/src/java/org/apache/hadoop/hive/common/type/TimestampTZ.java
Lines 158 (patched)


May want to add a TODO that currently we assume zone as system default, in 
future this may come from user specificed zone (via set time zone) command.



common/src/java/org/apache/hadoop/hive/common/type/TimestampTZ.java
Lines 190 (patched)


TODO here to get zone from user provided set value in future.



ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/translator/TypeConverter.java
Lines 204 (patched)


Can you file a bug in Calcite that it should have sql type to represent TS 
w TZ?



ql/src/java/org/apache/hadoop/hive/ql/parse/HiveLexer.g
Lines 133 (patched)


Lets not use Timestamptz. Non-standard, with no advantage.



ql/src/java/org/apache/hadoop/hive/ql/parse/HiveLexer.g
Lines 134 (patched)


Add in release note as incompatiable change that time is now a reserved 
word. Should be ok, since standard also has it as reserved.



ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToString.java
Lines 160 (patched)


Add a comment that string reperesentation will return TS in UTC zone and 
not in original TZ.



ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFToTimestampTZ.java
Lines 35 (patched)


Comment about what happens if string has no zone info.



ql/src/test/queries/clientpositive/timestamptz_1.q
Lines 16 (patched)


Can you add a test which adds string literal in TSTZ column which has no 
zone info?

insert into tstz1 values (timestamp '2016-01-03 12:26:34.1');
insert into tstz1 values ('2016-01-03 12:26:34.1');
insert into tstz1 values (cast ('2016-01-03 12:26:34.1' as timestamp with 
time zone));

create table t (ts timestamp);
insert into t values ();
insert into tstz1 select ts from t;



ql/src/test/queries/clientpositive/timestamptz_1.q
Lines 21 (patched)


Can you add a test which inserts TS column in TSTZ column?



serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/PrimitiveObjectInspectorUtils.java
Lines 1078 (patched)


Instead of throwing exception, return null ?



serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/PrimitiveObjectInspectorUtils.java
Lines 1154 (patched)


Instead of throwing exception, return null ?


- Ashutosh Chauhan


On May 8, 2017, 3:17 p.m., Rui Li wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/50787/
> ---
> 
> (Updated May 8, 2017, 3:17 p.m.)
> 
> 
> Review request for hive, pengcheng xiong and Xuefu Zhang.
> 
> 
> Bugs: HIVE-14412
> https://issues.apache.org/jira/browse/HIVE-14412
> 
> 
> Repository: hive-git
> 
> 
> Description
> ---
> 
> The 1st patch to add timezone-aware timestamp.
> 
> 
> Diffs
> -
> 
>   common/src/java/org/apache/hadoop/hive/common/type/TimestampTZ.java 
> PRE-CREATION 
>   common/src/test/org/apache/hadoop/hive/common/type/TestTimestampTZ.java 
> PRE-CREATION 
>   contrib/src/test/queries/clientnegative/serde_regex.q a676338 
>   contrib/src/test/queries/clientpositive/serde_regex.q d75d607 
>   contrib/src/test/results/clientnegative/serde_regex.q.out 58b1c02 
>   contrib/src/test/results/clientpositive/serde_regex.q.out 2984293 
>   hbase-handler/src/test/queries/positive/hbase_timestamp.q 0350afe 
>   hbase-handler/src/test/results/positive/hbase_timestamp.q.out 3918121 
>   itests/hive-blobstore/src/test/queries/clientpositive/orc_format_part.q 
> 358eccd 
>   
> itests/hive-blobstore/src/test/queries/clientpositive/orc_nonstd_partitions_loc.q
>  c462538 
>   itests/hive-blobstore/src/test/queries/clientpositive/rcfile_format_part.q 
> c563d3a 
>   
> itests/hive-blobstore/src/test/queries/clientpositive/rcfile_nons

Re: Review Request 50787: Add a timezone-aware timestamp

[pengcheng xiong]

> On May 3, 2017, 9:57 p.m., pengcheng xiong wrote:
> > ql/src/java/org/apache/hadoop/hive/ql/parse/HiveLexer.g
> > Lines 132 (patched)
> > 
> >
> > I think Identifier["timestamptz"] and Identifier["zone"] may be 
> > sufficient. It is not necessary to make them as key words and then add them 
> > back as identifiers. You can have a try and see if it works. Thanks..
> 
> Rui Li wrote:
> Hi Pengcheng, sorry I'm quite ignorant about antlr. Could you please be 
> more specific how to add the Identifiers? Let me explain what I intend to do. 
> The new data type is named "timestamp with time zone", and "timestamptz" is 
> added as a type alias. I thought it's required to add key words for type 
> names. And according to the PostgreSQL doc we referenced 
> (https://www.postgresql.org/docs/9.5/static/sql-keywords-appendix.html), 
> "zone" is a non-reserved SQL key word and "timestamptz" is not a key word. So 
> I added them in IdentifierParser.g as nonReserved.

I have a second thought and I think you are right about this in your current 
patch. Thanks.


- pengcheng


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/50787/#review173833
---


On May 8, 2017, 3:17 p.m., Rui Li wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/50787/
> ---
> 
> (Updated May 8, 2017, 3:17 p.m.)
> 
> 
> Review request for hive, pengcheng xiong and Xuefu Zhang.
> 
> 
> Bugs: HIVE-14412
> https://issues.apache.org/jira/browse/HIVE-14412
> 
> 
> Repository: hive-git
> 
> 
> Description
> ---
> 
> The 1st patch to add timezone-aware timestamp.
> 
> 
> Diffs
> -
> 
>   common/src/java/org/apache/hadoop/hive/common/type/TimestampTZ.java 
> PRE-CREATION 
>   common/src/test/org/apache/hadoop/hive/common/type/TestTimestampTZ.java 
> PRE-CREATION 
>   contrib/src/test/queries/clientnegative/serde_regex.q a676338 
>   contrib/src/test/queries/clientpositive/serde_regex.q d75d607 
>   contrib/src/test/results/clientnegative/serde_regex.q.out 58b1c02 
>   contrib/src/test/results/clientpositive/serde_regex.q.out 2984293 
>   hbase-handler/src/test/queries/positive/hbase_timestamp.q 0350afe 
>   hbase-handler/src/test/results/positive/hbase_timestamp.q.out 3918121 
>   itests/hive-blobstore/src/test/queries/clientpositive/orc_format_part.q 
> 358eccd 
>   
> itests/hive-blobstore/src/test/queries/clientpositive/orc_nonstd_partitions_loc.q
>  c462538 
>   itests/hive-blobstore/src/test/queries/clientpositive/rcfile_format_part.q 
> c563d3a 
>   
> itests/hive-blobstore/src/test/queries/clientpositive/rcfile_nonstd_partitions_loc.q
>  d17c281 
>   itests/hive-blobstore/src/test/results/clientpositive/orc_format_part.q.out 
> 5d1319f 
>   
> itests/hive-blobstore/src/test/results/clientpositive/orc_nonstd_partitions_loc.q.out
>  70e72f7 
>   
> itests/hive-blobstore/src/test/results/clientpositive/rcfile_format_part.q.out
>  bed10ab 
>   
> itests/hive-blobstore/src/test/results/clientpositive/rcfile_nonstd_partitions_loc.q.out
>  c6442f9 
>   jdbc/src/java/org/apache/hive/jdbc/HiveBaseResultSet.java ade1900 
>   jdbc/src/java/org/apache/hive/jdbc/JdbcColumn.java 38918f0 
>   ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionRegistry.java 1b556ac 
>   ql/src/java/org/apache/hadoop/hive/ql/exec/GroupByOperator.java f8b55da 
>   ql/src/java/org/apache/hadoop/hive/ql/exec/SerializationUtilities.java 
> 01a652d 
>   
> ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/translator/TypeConverter.java
>  38308c9 
>   ql/src/java/org/apache/hadoop/hive/ql/parse/DDLSemanticAnalyzer.java 
> 0cf9205 
>   ql/src/java/org/apache/hadoop/hive/ql/parse/HiveLexer.g 190b66b 
>   ql/src/java/org/apache/hadoop/hive/ql/parse/HiveParser.g ca639d3 
>   ql/src/java/org/apache/hadoop/hive/ql/parse/IdentifiersParser.g 645ced9 
>   ql/src/java/org/apache/hadoop/hive/ql/parse/TypeCheckProcFactory.java 
> c3227c9 
>   ql/src/java/org/apache/hadoop/hive/ql/stats/StatsUtils.java bda2050 
>   ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToBoolean.java 7cdf2c3 
>   ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToString.java 5cacd59 
>   ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDF.java 68d98f5 
>   ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFDate.java 
> 5a31e61 
>   
> ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFToTimestampTZ.java
>  PRE-CREATION 
>   
> ql/src/test/org/apache/hadoop/hive/ql/parse/TestSQL11ReservedKeyWordsNegative.java
>  0dc6b19 
>   ql/src/test/queries/clientnegative/serde_regex.q c9cfc7d 
>   ql/src/test/queries/clientnegative/serde_regex2.q a29bb9c 
>   ql/src/test/queries/clientnegative/ser

Re: Review Request 50787: Add a timezone-aware timestamp

[Xuefu Zhang]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/50787/#review174202
---


Ship it!




Ship It!

- Xuefu Zhang


On May 8, 2017, 3:17 p.m., Rui Li wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/50787/
> ---
> 
> (Updated May 8, 2017, 3:17 p.m.)
> 
> 
> Review request for hive, pengcheng xiong and Xuefu Zhang.
> 
> 
> Bugs: HIVE-14412
> https://issues.apache.org/jira/browse/HIVE-14412
> 
> 
> Repository: hive-git
> 
> 
> Description
> ---
> 
> The 1st patch to add timezone-aware timestamp.
> 
> 
> Diffs
> -
> 
>   common/src/java/org/apache/hadoop/hive/common/type/TimestampTZ.java 
> PRE-CREATION 
>   common/src/test/org/apache/hadoop/hive/common/type/TestTimestampTZ.java 
> PRE-CREATION 
>   contrib/src/test/queries/clientnegative/serde_regex.q a676338 
>   contrib/src/test/queries/clientpositive/serde_regex.q d75d607 
>   contrib/src/test/results/clientnegative/serde_regex.q.out 58b1c02 
>   contrib/src/test/results/clientpositive/serde_regex.q.out 2984293 
>   hbase-handler/src/test/queries/positive/hbase_timestamp.q 0350afe 
>   hbase-handler/src/test/results/positive/hbase_timestamp.q.out 3918121 
>   itests/hive-blobstore/src/test/queries/clientpositive/orc_format_part.q 
> 358eccd 
>   
> itests/hive-blobstore/src/test/queries/clientpositive/orc_nonstd_partitions_loc.q
>  c462538 
>   itests/hive-blobstore/src/test/queries/clientpositive/rcfile_format_part.q 
> c563d3a 
>   
> itests/hive-blobstore/src/test/queries/clientpositive/rcfile_nonstd_partitions_loc.q
>  d17c281 
>   itests/hive-blobstore/src/test/results/clientpositive/orc_format_part.q.out 
> 5d1319f 
>   
> itests/hive-blobstore/src/test/results/clientpositive/orc_nonstd_partitions_loc.q.out
>  70e72f7 
>   
> itests/hive-blobstore/src/test/results/clientpositive/rcfile_format_part.q.out
>  bed10ab 
>   
> itests/hive-blobstore/src/test/results/clientpositive/rcfile_nonstd_partitions_loc.q.out
>  c6442f9 
>   jdbc/src/java/org/apache/hive/jdbc/HiveBaseResultSet.java ade1900 
>   jdbc/src/java/org/apache/hive/jdbc/JdbcColumn.java 38918f0 
>   ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionRegistry.java 1b556ac 
>   ql/src/java/org/apache/hadoop/hive/ql/exec/GroupByOperator.java f8b55da 
>   ql/src/java/org/apache/hadoop/hive/ql/exec/SerializationUtilities.java 
> 01a652d 
>   
> ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/translator/TypeConverter.java
>  38308c9 
>   ql/src/java/org/apache/hadoop/hive/ql/parse/DDLSemanticAnalyzer.java 
> 0cf9205 
>   ql/src/java/org/apache/hadoop/hive/ql/parse/HiveLexer.g 190b66b 
>   ql/src/java/org/apache/hadoop/hive/ql/parse/HiveParser.g ca639d3 
>   ql/src/java/org/apache/hadoop/hive/ql/parse/IdentifiersParser.g 645ced9 
>   ql/src/java/org/apache/hadoop/hive/ql/parse/TypeCheckProcFactory.java 
> c3227c9 
>   ql/src/java/org/apache/hadoop/hive/ql/stats/StatsUtils.java bda2050 
>   ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToBoolean.java 7cdf2c3 
>   ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToString.java 5cacd59 
>   ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDF.java 68d98f5 
>   ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFDate.java 
> 5a31e61 
>   
> ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFToTimestampTZ.java
>  PRE-CREATION 
>   
> ql/src/test/org/apache/hadoop/hive/ql/parse/TestSQL11ReservedKeyWordsNegative.java
>  0dc6b19 
>   ql/src/test/queries/clientnegative/serde_regex.q c9cfc7d 
>   ql/src/test/queries/clientnegative/serde_regex2.q a29bb9c 
>   ql/src/test/queries/clientnegative/serde_regex3.q 4e91f06 
>   ql/src/test/queries/clientpositive/create_like.q bd39731 
>   ql/src/test/queries/clientpositive/join43.q 12c45a6 
>   ql/src/test/queries/clientpositive/serde_regex.q e21c6e1 
>   ql/src/test/queries/clientpositive/timestamptz.q PRE-CREATION 
>   ql/src/test/queries/clientpositive/timestamptz_1.q PRE-CREATION 
>   ql/src/test/queries/clientpositive/timestamptz_2.q PRE-CREATION 
>   ql/src/test/results/clientnegative/serde_regex.q.out a1ec5ca 
>   ql/src/test/results/clientnegative/serde_regex2.q.out 374675d 
>   ql/src/test/results/clientnegative/serde_regex3.q.out dc0a0e2 
>   ql/src/test/results/clientpositive/create_like.q.out ff2e752 
>   ql/src/test/results/clientpositive/join43.q.out e8c7278 
>   ql/src/test/results/clientpositive/serde_regex.q.out 7bebb0c 
>   ql/src/test/results/clientpositive/timestamptz.q.out PRE-CREATION 
>   ql/src/test/results/clientpositive/timestamptz_1.q.out PRE-CREATION 
>   ql/src/test/results/clientpositive/timestamptz_2.q.out PRE-CREATION 
>   serde/if/serde.thrift 1d40d5a 
>   serde/src/gen/thrift/gen-cpp/serde_constants.h 8785bd2 
>   serde/src

Re: Review Request 50787: Add a timezone-aware timestamp

[Rui Li]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/50787/
---

(Updated May 8, 2017, 3:17 p.m.)


Review request for hive, pengcheng xiong and Xuefu Zhang.


Bugs: HIVE-14412
https://issues.apache.org/jira/browse/HIVE-14412


Repository: hive-git


Description
---

The 1st patch to add timezone-aware timestamp.


Diffs (updated)
-

  common/src/java/org/apache/hadoop/hive/common/type/TimestampTZ.java 
PRE-CREATION 
  common/src/test/org/apache/hadoop/hive/common/type/TestTimestampTZ.java 
PRE-CREATION 
  contrib/src/test/queries/clientnegative/serde_regex.q a676338 
  contrib/src/test/queries/clientpositive/serde_regex.q d75d607 
  contrib/src/test/results/clientnegative/serde_regex.q.out 58b1c02 
  contrib/src/test/results/clientpositive/serde_regex.q.out 2984293 
  hbase-handler/src/test/queries/positive/hbase_timestamp.q 0350afe 
  hbase-handler/src/test/results/positive/hbase_timestamp.q.out 3918121 
  itests/hive-blobstore/src/test/queries/clientpositive/orc_format_part.q 
358eccd 
  
itests/hive-blobstore/src/test/queries/clientpositive/orc_nonstd_partitions_loc.q
 c462538 
  itests/hive-blobstore/src/test/queries/clientpositive/rcfile_format_part.q 
c563d3a 
  
itests/hive-blobstore/src/test/queries/clientpositive/rcfile_nonstd_partitions_loc.q
 d17c281 
  itests/hive-blobstore/src/test/results/clientpositive/orc_format_part.q.out 
5d1319f 
  
itests/hive-blobstore/src/test/results/clientpositive/orc_nonstd_partitions_loc.q.out
 70e72f7 
  
itests/hive-blobstore/src/test/results/clientpositive/rcfile_format_part.q.out 
bed10ab 
  
itests/hive-blobstore/src/test/results/clientpositive/rcfile_nonstd_partitions_loc.q.out
 c6442f9 
  jdbc/src/java/org/apache/hive/jdbc/HiveBaseResultSet.java ade1900 
  jdbc/src/java/org/apache/hive/jdbc/JdbcColumn.java 38918f0 
  ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionRegistry.java 1b556ac 
  ql/src/java/org/apache/hadoop/hive/ql/exec/GroupByOperator.java f8b55da 
  ql/src/java/org/apache/hadoop/hive/ql/exec/SerializationUtilities.java 
01a652d 
  
ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/translator/TypeConverter.java
 38308c9 
  ql/src/java/org/apache/hadoop/hive/ql/parse/DDLSemanticAnalyzer.java 0cf9205 
  ql/src/java/org/apache/hadoop/hive/ql/parse/HiveLexer.g 190b66b 
  ql/src/java/org/apache/hadoop/hive/ql/parse/HiveParser.g ca639d3 
  ql/src/java/org/apache/hadoop/hive/ql/parse/IdentifiersParser.g 645ced9 
  ql/src/java/org/apache/hadoop/hive/ql/parse/TypeCheckProcFactory.java c3227c9 
  ql/src/java/org/apache/hadoop/hive/ql/stats/StatsUtils.java bda2050 
  ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToBoolean.java 7cdf2c3 
  ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToString.java 5cacd59 
  ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDF.java 68d98f5 
  ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFDate.java 5a31e61 
  
ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFToTimestampTZ.java 
PRE-CREATION 
  
ql/src/test/org/apache/hadoop/hive/ql/parse/TestSQL11ReservedKeyWordsNegative.java
 0dc6b19 
  ql/src/test/queries/clientnegative/serde_regex.q c9cfc7d 
  ql/src/test/queries/clientnegative/serde_regex2.q a29bb9c 
  ql/src/test/queries/clientnegative/serde_regex3.q 4e91f06 
  ql/src/test/queries/clientpositive/create_like.q bd39731 
  ql/src/test/queries/clientpositive/join43.q 12c45a6 
  ql/src/test/queries/clientpositive/serde_regex.q e21c6e1 
  ql/src/test/queries/clientpositive/timestamptz.q PRE-CREATION 
  ql/src/test/queries/clientpositive/timestamptz_1.q PRE-CREATION 
  ql/src/test/queries/clientpositive/timestamptz_2.q PRE-CREATION 
  ql/src/test/results/clientnegative/serde_regex.q.out a1ec5ca 
  ql/src/test/results/clientnegative/serde_regex2.q.out 374675d 
  ql/src/test/results/clientnegative/serde_regex3.q.out dc0a0e2 
  ql/src/test/results/clientpositive/create_like.q.out ff2e752 
  ql/src/test/results/clientpositive/join43.q.out e8c7278 
  ql/src/test/results/clientpositive/serde_regex.q.out 7bebb0c 
  ql/src/test/results/clientpositive/timestamptz.q.out PRE-CREATION 
  ql/src/test/results/clientpositive/timestamptz_1.q.out PRE-CREATION 
  ql/src/test/results/clientpositive/timestamptz_2.q.out PRE-CREATION 
  serde/if/serde.thrift 1d40d5a 
  serde/src/gen/thrift/gen-cpp/serde_constants.h 8785bd2 
  serde/src/gen/thrift/gen-cpp/serde_constants.cpp 907acf2 
  
serde/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/serde/serdeConstants.java
 2578d3e 
  serde/src/gen/thrift/gen-php/org/apache/hadoop/hive/serde/Types.php ea2dbbe 
  serde/src/gen/thrift/gen-py/org_apache_hadoop_hive_serde/constants.py e3b24eb 
  serde/src/gen/thrift/gen-rb/serde_constants.rb 15efaea 
  serde/src/java/org/apache/hadoop/hive/serde2/SerDeUtils.java 5ecfbca 
  
serde/src/java/org/apache/hadoop/hive/serde2/binarysortable/BinarySortableSerDe.jav

Re: Review Request 50787: Add a timezone-aware timestamp

[Rui Li]

> On May 7, 2017, 11:22 p.m., Xuefu Zhang wrote:
> > common/src/java/org/apache/hadoop/hive/common/type/TimestampTZ.java
> > Lines 138 (patched)
> > 
> >
> > Not sure if I understand this, but why cannot we get seconds/nanos from 
> > date/timestamp and then convert to TimestapTZ? I assume this is a faster 
> > way.

Hi Xuefu, the reason why I did this:

1. As Ashutosh suggested, we will use LocalDate and LocalDateTime for Date and 
Timestamp in the future. When that happens, date/timestamp won't have 
seconds/nanos part, instead they're only descriptions of time. So the 
conversion should be done based on text format.
2. At the moment, the seconds/nanos of date/timestamp is computed using system 
timezone. So the conversion can have different results in different systems.

I noted Carter also suggested that SQL standard requires session zone should be 
taken into consideration in the conversion.
Consolidating your suggestions with Carter's, I think we can: make the 
conversion text-wise, and append the system zone (Hive currently doesn't have 
session zone). For example, a date of '2017-01-01' in LA will be converted to 
timestamptz as '2017-01-01 00:00:00 America/Los_Angeles', which in turn 
converted to '2017-01-01 08:00:00.0 Z'. Does this make sense?


> On May 7, 2017, 11:22 p.m., Xuefu Zhang wrote:
> > ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/translator/TypeConverter.java
> > Lines 204 (patched)
> > 
> >
> > What does this imply?

The method converts our primitive type to a SqlTypeName in calcite. But 
SqlTypeName currently doesn't have timestamp with time zone. This will have 
some impact when calcite does the optimization, e.g. computing average value 
sizes. But I think we have to live with it untile SqlTypeName supports 
timestamp with time zone.


> On May 7, 2017, 11:22 p.m., Xuefu Zhang wrote:
> > serde/src/java/org/apache/hadoop/hive/serde2/lazy/LazyTimestampTZ.java
> > Lines 32 (patched)
> > 
> >
> > Can you also make a note about the source of the code, like 
> > TimeStampTZWritable?

sure, will do


- Rui


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/50787/#review174136
---


On May 8, 2017, 6:51 a.m., Rui Li wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/50787/
> ---
> 
> (Updated May 8, 2017, 6:51 a.m.)
> 
> 
> Review request for hive, pengcheng xiong and Xuefu Zhang.
> 
> 
> Bugs: HIVE-14412
> https://issues.apache.org/jira/browse/HIVE-14412
> 
> 
> Repository: hive-git
> 
> 
> Description
> ---
> 
> The 1st patch to add timezone-aware timestamp.
> 
> 
> Diffs
> -
> 
>   common/src/java/org/apache/hadoop/hive/common/type/TimestampTZ.java 
> PRE-CREATION 
>   common/src/test/org/apache/hadoop/hive/common/type/TestTimestampTZ.java 
> PRE-CREATION 
>   contrib/src/test/queries/clientnegative/serde_regex.q a676338 
>   contrib/src/test/queries/clientpositive/serde_regex.q d75d607 
>   contrib/src/test/results/clientnegative/serde_regex.q.out 58b1c02 
>   contrib/src/test/results/clientpositive/serde_regex.q.out 2984293 
>   hbase-handler/src/test/queries/positive/hbase_timestamp.q 0350afe 
>   hbase-handler/src/test/results/positive/hbase_timestamp.q.out 3918121 
>   itests/hive-blobstore/src/test/queries/clientpositive/orc_format_part.q 
> 358eccd 
>   
> itests/hive-blobstore/src/test/queries/clientpositive/orc_nonstd_partitions_loc.q
>  c462538 
>   itests/hive-blobstore/src/test/queries/clientpositive/rcfile_format_part.q 
> c563d3a 
>   
> itests/hive-blobstore/src/test/queries/clientpositive/rcfile_nonstd_partitions_loc.q
>  d17c281 
>   itests/hive-blobstore/src/test/results/clientpositive/orc_format_part.q.out 
> 5d1319f 
>   
> itests/hive-blobstore/src/test/results/clientpositive/orc_nonstd_partitions_loc.q.out
>  70e72f7 
>   
> itests/hive-blobstore/src/test/results/clientpositive/rcfile_format_part.q.out
>  bed10ab 
>   
> itests/hive-blobstore/src/test/results/clientpositive/rcfile_nonstd_partitions_loc.q.out
>  c6442f9 
>   jdbc/src/java/org/apache/hive/jdbc/HiveBaseResultSet.java ade1900 
>   jdbc/src/java/org/apache/hive/jdbc/JdbcColumn.java 38918f0 
>   ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionRegistry.java 8dc5f2e 
>   ql/src/java/org/apache/hadoop/hive/ql/exec/GroupByOperator.java f8b55da 
>   ql/src/java/org/apache/hadoop/hive/ql/exec/SerializationUtilities.java 
> 01a652d 
>   
> ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/translator/TypeConverter.java
>  

Re: Review Request 50787: Add a timezone-aware timestamp

[Xuefu Zhang]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/50787/#review174136
---




common/src/java/org/apache/hadoop/hive/common/type/TimestampTZ.java
Lines 138 (patched)


Not sure if I understand this, but why cannot we get seconds/nanos from 
date/timestamp and then convert to TimestapTZ? I assume this is a faster way.



ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/translator/TypeConverter.java
Lines 204 (patched)


What does this imply?



serde/src/java/org/apache/hadoop/hive/serde2/lazy/LazyTimestampTZ.java
Lines 32 (patched)


Can you also make a note about the source of the code, like 
TimeStampTZWritable?


- Xuefu Zhang


On May 3, 2017, 6:34 a.m., Rui Li wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/50787/
> ---
> 
> (Updated May 3, 2017, 6:34 a.m.)
> 
> 
> Review request for hive.
> 
> 
> Bugs: HIVE-14412
> https://issues.apache.org/jira/browse/HIVE-14412
> 
> 
> Repository: hive-git
> 
> 
> Description
> ---
> 
> The 1st patch to add timezone-aware timestamp.
> 
> 
> Diffs
> -
> 
>   common/src/java/org/apache/hadoop/hive/common/type/TimestampTZ.java 
> PRE-CREATION 
>   common/src/test/org/apache/hadoop/hive/common/type/TestTimestampTZ.java 
> PRE-CREATION 
>   contrib/src/test/queries/clientnegative/serde_regex.q a676338 
>   contrib/src/test/queries/clientpositive/serde_regex.q d75d607 
>   contrib/src/test/results/clientnegative/serde_regex.q.out 58b1c02 
>   contrib/src/test/results/clientpositive/serde_regex.q.out 2984293 
>   hbase-handler/src/test/queries/positive/hbase_timestamp.q 0350afe 
>   hbase-handler/src/test/results/positive/hbase_timestamp.q.out 3918121 
>   itests/hive-blobstore/src/test/queries/clientpositive/orc_format_part.q 
> 358eccd 
>   
> itests/hive-blobstore/src/test/queries/clientpositive/orc_nonstd_partitions_loc.q
>  c462538 
>   itests/hive-blobstore/src/test/queries/clientpositive/rcfile_format_part.q 
> c563d3a 
>   
> itests/hive-blobstore/src/test/queries/clientpositive/rcfile_nonstd_partitions_loc.q
>  d17c281 
>   itests/hive-blobstore/src/test/results/clientpositive/orc_format_part.q.out 
> 5d1319f 
>   
> itests/hive-blobstore/src/test/results/clientpositive/orc_nonstd_partitions_loc.q.out
>  70e72f7 
>   
> itests/hive-blobstore/src/test/results/clientpositive/rcfile_format_part.q.out
>  bed10ab 
>   
> itests/hive-blobstore/src/test/results/clientpositive/rcfile_nonstd_partitions_loc.q.out
>  c6442f9 
>   jdbc/src/java/org/apache/hive/jdbc/HiveBaseResultSet.java ade1900 
>   jdbc/src/java/org/apache/hive/jdbc/JdbcColumn.java 38918f0 
>   ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionRegistry.java 8dc5f2e 
>   ql/src/java/org/apache/hadoop/hive/ql/exec/GroupByOperator.java f8b55da 
>   ql/src/java/org/apache/hadoop/hive/ql/exec/SerializationUtilities.java 
> 01a652d 
>   
> ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/translator/TypeConverter.java
>  38308c9 
>   ql/src/java/org/apache/hadoop/hive/ql/parse/DDLSemanticAnalyzer.java 
> 0cf9205 
>   ql/src/java/org/apache/hadoop/hive/ql/parse/HiveLexer.g 0721b92 
>   ql/src/java/org/apache/hadoop/hive/ql/parse/HiveParser.g d98a663 
>   ql/src/java/org/apache/hadoop/hive/ql/parse/IdentifiersParser.g 8598fae 
>   ql/src/java/org/apache/hadoop/hive/ql/parse/TypeCheckProcFactory.java 
> 8f8eab0 
>   ql/src/java/org/apache/hadoop/hive/ql/stats/StatsUtils.java bda2050 
>   ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToBoolean.java 7cdf2c3 
>   ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToString.java 5cacd59 
>   ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDF.java 68d98f5 
>   ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFDate.java 
> 5a31e61 
>   
> ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFToTimestampTZ.java
>  PRE-CREATION 
>   
> ql/src/test/org/apache/hadoop/hive/ql/parse/TestSQL11ReservedKeyWordsNegative.java
>  0dc6b19 
>   ql/src/test/queries/clientnegative/serde_regex.q c9cfc7d 
>   ql/src/test/queries/clientnegative/serde_regex2.q a29bb9c 
>   ql/src/test/queries/clientnegative/serde_regex3.q 4e91f06 
>   ql/src/test/queries/clientpositive/create_like.q bd39731 
>   ql/src/test/queries/clientpositive/join43.q 12c45a6 
>   ql/src/test/queries/clientpositive/serde_regex.q e21c6e1 
>   ql/src/test/queries/clientpositive/timestamptz.q PRE-CREATION 
>   ql/src/test/queries/clientpositive/timestamptz_1.q PRE-CREATION 
>   ql/src/test/queries/clientpositive/timestamptz_2.q PRE-CREATION 
>   ql/src/test/results/clientnegative/serde_regex.q.o

Re: Review Request 50787: Add a timezone-aware timestamp

[Rui Li]

> On May 3, 2017, 9:57 p.m., pengcheng xiong wrote:
> > ql/src/java/org/apache/hadoop/hive/ql/parse/HiveLexer.g
> > Lines 132 (patched)
> > 
> >
> > I think Identifier["timestamptz"] and Identifier["zone"] may be 
> > sufficient. It is not necessary to make them as key words and then add them 
> > back as identifiers. You can have a try and see if it works. Thanks..

Hi Pengcheng, sorry I'm quite ignorant about antlr. Could you please be more 
specific how to add the Identifiers? Let me explain what I intend to do. The 
new data type is named "timestamp with time zone", and "timestamptz" is added 
as a type alias. I thought it's required to add key words for type names. And 
according to the PostgreSQL doc we referenced 
(https://www.postgresql.org/docs/9.5/static/sql-keywords-appendix.html), "zone" 
is a non-reserved SQL key word and "timestamptz" is not a key word. So I added 
them in IdentifierParser.g as nonReserved.


- Rui


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/50787/#review173833
---


On May 3, 2017, 6:34 a.m., Rui Li wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/50787/
> ---
> 
> (Updated May 3, 2017, 6:34 a.m.)
> 
> 
> Review request for hive.
> 
> 
> Bugs: HIVE-14412
> https://issues.apache.org/jira/browse/HIVE-14412
> 
> 
> Repository: hive-git
> 
> 
> Description
> ---
> 
> The 1st patch to add timezone-aware timestamp.
> 
> 
> Diffs
> -
> 
>   common/src/java/org/apache/hadoop/hive/common/type/TimestampTZ.java 
> PRE-CREATION 
>   common/src/test/org/apache/hadoop/hive/common/type/TestTimestampTZ.java 
> PRE-CREATION 
>   contrib/src/test/queries/clientnegative/serde_regex.q a676338 
>   contrib/src/test/queries/clientpositive/serde_regex.q d75d607 
>   contrib/src/test/results/clientnegative/serde_regex.q.out 58b1c02 
>   contrib/src/test/results/clientpositive/serde_regex.q.out 2984293 
>   hbase-handler/src/test/queries/positive/hbase_timestamp.q 0350afe 
>   hbase-handler/src/test/results/positive/hbase_timestamp.q.out 3918121 
>   itests/hive-blobstore/src/test/queries/clientpositive/orc_format_part.q 
> 358eccd 
>   
> itests/hive-blobstore/src/test/queries/clientpositive/orc_nonstd_partitions_loc.q
>  c462538 
>   itests/hive-blobstore/src/test/queries/clientpositive/rcfile_format_part.q 
> c563d3a 
>   
> itests/hive-blobstore/src/test/queries/clientpositive/rcfile_nonstd_partitions_loc.q
>  d17c281 
>   itests/hive-blobstore/src/test/results/clientpositive/orc_format_part.q.out 
> 5d1319f 
>   
> itests/hive-blobstore/src/test/results/clientpositive/orc_nonstd_partitions_loc.q.out
>  70e72f7 
>   
> itests/hive-blobstore/src/test/results/clientpositive/rcfile_format_part.q.out
>  bed10ab 
>   
> itests/hive-blobstore/src/test/results/clientpositive/rcfile_nonstd_partitions_loc.q.out
>  c6442f9 
>   jdbc/src/java/org/apache/hive/jdbc/HiveBaseResultSet.java ade1900 
>   jdbc/src/java/org/apache/hive/jdbc/JdbcColumn.java 38918f0 
>   ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionRegistry.java 8dc5f2e 
>   ql/src/java/org/apache/hadoop/hive/ql/exec/GroupByOperator.java f8b55da 
>   ql/src/java/org/apache/hadoop/hive/ql/exec/SerializationUtilities.java 
> 01a652d 
>   
> ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/translator/TypeConverter.java
>  38308c9 
>   ql/src/java/org/apache/hadoop/hive/ql/parse/DDLSemanticAnalyzer.java 
> 0cf9205 
>   ql/src/java/org/apache/hadoop/hive/ql/parse/HiveLexer.g 0721b92 
>   ql/src/java/org/apache/hadoop/hive/ql/parse/HiveParser.g d98a663 
>   ql/src/java/org/apache/hadoop/hive/ql/parse/IdentifiersParser.g 8598fae 
>   ql/src/java/org/apache/hadoop/hive/ql/parse/TypeCheckProcFactory.java 
> 8f8eab0 
>   ql/src/java/org/apache/hadoop/hive/ql/stats/StatsUtils.java bda2050 
>   ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToBoolean.java 7cdf2c3 
>   ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToString.java 5cacd59 
>   ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDF.java 68d98f5 
>   ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFDate.java 
> 5a31e61 
>   
> ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFToTimestampTZ.java
>  PRE-CREATION 
>   
> ql/src/test/org/apache/hadoop/hive/ql/parse/TestSQL11ReservedKeyWordsNegative.java
>  0dc6b19 
>   ql/src/test/queries/clientnegative/serde_regex.q c9cfc7d 
>   ql/src/test/queries/clientnegative/serde_regex2.q a29bb9c 
>   ql/src/test/queries/clientnegative/serde_regex3.q 4e91f06 
>   ql/src/test/queries/clientpositive/create_like.q bd39731 
>   ql/src/test/queries/clientpositive/join43.q 12c45a6 
>   ql/src/test/queries/clientp

Re: Review Request 50787: Add a timezone-aware timestamp

[pengcheng xiong]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/50787/#review173833
---




ql/src/java/org/apache/hadoop/hive/ql/parse/HiveLexer.g
Lines 132 (patched)


I think Identifier["timestamptz"] and Identifier["zone"] may be sufficient. 
It is not necessary to make them as key words and then add them back as 
identifiers. You can have a try and see if it works. Thanks..


- pengcheng xiong


On May 3, 2017, 6:34 a.m., Rui Li wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/50787/
> ---
> 
> (Updated May 3, 2017, 6:34 a.m.)
> 
> 
> Review request for hive.
> 
> 
> Bugs: HIVE-14412
> https://issues.apache.org/jira/browse/HIVE-14412
> 
> 
> Repository: hive-git
> 
> 
> Description
> ---
> 
> The 1st patch to add timezone-aware timestamp.
> 
> 
> Diffs
> -
> 
>   common/src/java/org/apache/hadoop/hive/common/type/TimestampTZ.java 
> PRE-CREATION 
>   common/src/test/org/apache/hadoop/hive/common/type/TestTimestampTZ.java 
> PRE-CREATION 
>   contrib/src/test/queries/clientnegative/serde_regex.q a676338 
>   contrib/src/test/queries/clientpositive/serde_regex.q d75d607 
>   contrib/src/test/results/clientnegative/serde_regex.q.out 58b1c02 
>   contrib/src/test/results/clientpositive/serde_regex.q.out 2984293 
>   hbase-handler/src/test/queries/positive/hbase_timestamp.q 0350afe 
>   hbase-handler/src/test/results/positive/hbase_timestamp.q.out 3918121 
>   itests/hive-blobstore/src/test/queries/clientpositive/orc_format_part.q 
> 358eccd 
>   
> itests/hive-blobstore/src/test/queries/clientpositive/orc_nonstd_partitions_loc.q
>  c462538 
>   itests/hive-blobstore/src/test/queries/clientpositive/rcfile_format_part.q 
> c563d3a 
>   
> itests/hive-blobstore/src/test/queries/clientpositive/rcfile_nonstd_partitions_loc.q
>  d17c281 
>   itests/hive-blobstore/src/test/results/clientpositive/orc_format_part.q.out 
> 5d1319f 
>   
> itests/hive-blobstore/src/test/results/clientpositive/orc_nonstd_partitions_loc.q.out
>  70e72f7 
>   
> itests/hive-blobstore/src/test/results/clientpositive/rcfile_format_part.q.out
>  bed10ab 
>   
> itests/hive-blobstore/src/test/results/clientpositive/rcfile_nonstd_partitions_loc.q.out
>  c6442f9 
>   jdbc/src/java/org/apache/hive/jdbc/HiveBaseResultSet.java ade1900 
>   jdbc/src/java/org/apache/hive/jdbc/JdbcColumn.java 38918f0 
>   ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionRegistry.java 8dc5f2e 
>   ql/src/java/org/apache/hadoop/hive/ql/exec/GroupByOperator.java f8b55da 
>   ql/src/java/org/apache/hadoop/hive/ql/exec/SerializationUtilities.java 
> 01a652d 
>   
> ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/translator/TypeConverter.java
>  38308c9 
>   ql/src/java/org/apache/hadoop/hive/ql/parse/DDLSemanticAnalyzer.java 
> 0cf9205 
>   ql/src/java/org/apache/hadoop/hive/ql/parse/HiveLexer.g 0721b92 
>   ql/src/java/org/apache/hadoop/hive/ql/parse/HiveParser.g d98a663 
>   ql/src/java/org/apache/hadoop/hive/ql/parse/IdentifiersParser.g 8598fae 
>   ql/src/java/org/apache/hadoop/hive/ql/parse/TypeCheckProcFactory.java 
> 8f8eab0 
>   ql/src/java/org/apache/hadoop/hive/ql/stats/StatsUtils.java bda2050 
>   ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToBoolean.java 7cdf2c3 
>   ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToString.java 5cacd59 
>   ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDF.java 68d98f5 
>   ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFDate.java 
> 5a31e61 
>   
> ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFToTimestampTZ.java
>  PRE-CREATION 
>   
> ql/src/test/org/apache/hadoop/hive/ql/parse/TestSQL11ReservedKeyWordsNegative.java
>  0dc6b19 
>   ql/src/test/queries/clientnegative/serde_regex.q c9cfc7d 
>   ql/src/test/queries/clientnegative/serde_regex2.q a29bb9c 
>   ql/src/test/queries/clientnegative/serde_regex3.q 4e91f06 
>   ql/src/test/queries/clientpositive/create_like.q bd39731 
>   ql/src/test/queries/clientpositive/join43.q 12c45a6 
>   ql/src/test/queries/clientpositive/serde_regex.q e21c6e1 
>   ql/src/test/queries/clientpositive/timestamptz.q PRE-CREATION 
>   ql/src/test/queries/clientpositive/timestamptz_1.q PRE-CREATION 
>   ql/src/test/queries/clientpositive/timestamptz_2.q PRE-CREATION 
>   ql/src/test/results/clientnegative/serde_regex.q.out a1ec5ca 
>   ql/src/test/results/clientnegative/serde_regex2.q.out 374675d 
>   ql/src/test/results/clientnegative/serde_regex3.q.out dc0a0e2 
>   ql/src/test/results/clientpositive/create_like.q.out ff2e752 
>   ql/src/test/results/clientpositive/join43.q.out e8c7278 
>   ql/src/test/results/clientpositive/serde_regex.q.out 7bebb0c 
>   ql/src/test/results/clie

Re: Review Request 50787: Add a timezone-aware timestamp

[Rui Li]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/50787/
---

(Updated May 3, 2017, 6:34 a.m.)


Review request for hive.


Bugs: HIVE-14412
https://issues.apache.org/jira/browse/HIVE-14412


Repository: hive-git


Description
---

The 1st patch to add timezone-aware timestamp.


Diffs (updated)
-

  common/src/java/org/apache/hadoop/hive/common/type/TimestampTZ.java 
PRE-CREATION 
  common/src/test/org/apache/hadoop/hive/common/type/TestTimestampTZ.java 
PRE-CREATION 
  contrib/src/test/queries/clientnegative/serde_regex.q a676338 
  contrib/src/test/queries/clientpositive/serde_regex.q d75d607 
  contrib/src/test/results/clientnegative/serde_regex.q.out 58b1c02 
  contrib/src/test/results/clientpositive/serde_regex.q.out 2984293 
  hbase-handler/src/test/queries/positive/hbase_timestamp.q 0350afe 
  hbase-handler/src/test/results/positive/hbase_timestamp.q.out 3918121 
  itests/hive-blobstore/src/test/queries/clientpositive/orc_format_part.q 
358eccd 
  
itests/hive-blobstore/src/test/queries/clientpositive/orc_nonstd_partitions_loc.q
 c462538 
  itests/hive-blobstore/src/test/queries/clientpositive/rcfile_format_part.q 
c563d3a 
  
itests/hive-blobstore/src/test/queries/clientpositive/rcfile_nonstd_partitions_loc.q
 d17c281 
  itests/hive-blobstore/src/test/results/clientpositive/orc_format_part.q.out 
5d1319f 
  
itests/hive-blobstore/src/test/results/clientpositive/orc_nonstd_partitions_loc.q.out
 70e72f7 
  
itests/hive-blobstore/src/test/results/clientpositive/rcfile_format_part.q.out 
bed10ab 
  
itests/hive-blobstore/src/test/results/clientpositive/rcfile_nonstd_partitions_loc.q.out
 c6442f9 
  jdbc/src/java/org/apache/hive/jdbc/HiveBaseResultSet.java ade1900 
  jdbc/src/java/org/apache/hive/jdbc/JdbcColumn.java 38918f0 
  ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionRegistry.java 8dc5f2e 
  ql/src/java/org/apache/hadoop/hive/ql/exec/GroupByOperator.java f8b55da 
  ql/src/java/org/apache/hadoop/hive/ql/exec/SerializationUtilities.java 
01a652d 
  
ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/translator/TypeConverter.java
 38308c9 
  ql/src/java/org/apache/hadoop/hive/ql/parse/DDLSemanticAnalyzer.java 0cf9205 
  ql/src/java/org/apache/hadoop/hive/ql/parse/HiveLexer.g 0721b92 
  ql/src/java/org/apache/hadoop/hive/ql/parse/HiveParser.g d98a663 
  ql/src/java/org/apache/hadoop/hive/ql/parse/IdentifiersParser.g 8598fae 
  ql/src/java/org/apache/hadoop/hive/ql/parse/TypeCheckProcFactory.java 8f8eab0 
  ql/src/java/org/apache/hadoop/hive/ql/stats/StatsUtils.java bda2050 
  ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToBoolean.java 7cdf2c3 
  ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToString.java 5cacd59 
  ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDF.java 68d98f5 
  ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFDate.java 5a31e61 
  
ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFToTimestampTZ.java 
PRE-CREATION 
  
ql/src/test/org/apache/hadoop/hive/ql/parse/TestSQL11ReservedKeyWordsNegative.java
 0dc6b19 
  ql/src/test/queries/clientnegative/serde_regex.q c9cfc7d 
  ql/src/test/queries/clientnegative/serde_regex2.q a29bb9c 
  ql/src/test/queries/clientnegative/serde_regex3.q 4e91f06 
  ql/src/test/queries/clientpositive/create_like.q bd39731 
  ql/src/test/queries/clientpositive/join43.q 12c45a6 
  ql/src/test/queries/clientpositive/serde_regex.q e21c6e1 
  ql/src/test/queries/clientpositive/timestamptz.q PRE-CREATION 
  ql/src/test/queries/clientpositive/timestamptz_1.q PRE-CREATION 
  ql/src/test/queries/clientpositive/timestamptz_2.q PRE-CREATION 
  ql/src/test/results/clientnegative/serde_regex.q.out a1ec5ca 
  ql/src/test/results/clientnegative/serde_regex2.q.out 374675d 
  ql/src/test/results/clientnegative/serde_regex3.q.out dc0a0e2 
  ql/src/test/results/clientpositive/create_like.q.out ff2e752 
  ql/src/test/results/clientpositive/join43.q.out e8c7278 
  ql/src/test/results/clientpositive/serde_regex.q.out 7bebb0c 
  ql/src/test/results/clientpositive/timestamptz.q.out PRE-CREATION 
  ql/src/test/results/clientpositive/timestamptz_1.q.out PRE-CREATION 
  ql/src/test/results/clientpositive/timestamptz_2.q.out PRE-CREATION 
  serde/if/serde.thrift 1d40d5a 
  serde/src/gen/thrift/gen-cpp/serde_constants.h 8785bd2 
  serde/src/gen/thrift/gen-cpp/serde_constants.cpp 907acf2 
  
serde/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/serde/serdeConstants.java
 2578d3e 
  serde/src/gen/thrift/gen-php/org/apache/hadoop/hive/serde/Types.php ea2dbbe 
  serde/src/gen/thrift/gen-py/org_apache_hadoop_hive_serde/constants.py e3b24eb 
  serde/src/gen/thrift/gen-rb/serde_constants.rb 15efaea 
  serde/src/java/org/apache/hadoop/hive/serde2/SerDeUtils.java 5ecfbca 
  
serde/src/java/org/apache/hadoop/hive/serde2/binarysortable/BinarySortableSerDe.java
 89e15c3 
  serde/src/java/org/

Re: Review Request 50787: Add a timezone-aware timestamp

[Rui Li]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/50787/
---

(Updated April 27, 2017, 2:32 a.m.)


Review request for hive.


Bugs: HIVE-14412
https://issues.apache.org/jira/browse/HIVE-14412


Repository: hive-git


Description
---

The 1st patch to add timezone-aware timestamp.


Diffs (updated)
-

  common/src/java/org/apache/hadoop/hive/common/type/TimestampTZ.java 
PRE-CREATION 
  common/src/test/org/apache/hadoop/hive/common/type/TestTimestampTZ.java 
PRE-CREATION 
  contrib/src/test/queries/clientnegative/serde_regex.q a676338 
  contrib/src/test/queries/clientpositive/serde_regex.q d75d607 
  contrib/src/test/results/clientnegative/serde_regex.q.out 58b1c02 
  contrib/src/test/results/clientpositive/serde_regex.q.out 2984293 
  hbase-handler/src/test/queries/positive/hbase_timestamp.q 0350afe 
  hbase-handler/src/test/results/positive/hbase_timestamp.q.out 3918121 
  itests/hive-blobstore/src/test/queries/clientpositive/orc_format_part.q 
358eccd 
  
itests/hive-blobstore/src/test/queries/clientpositive/orc_nonstd_partitions_loc.q
 c462538 
  itests/hive-blobstore/src/test/queries/clientpositive/rcfile_format_part.q 
c563d3a 
  
itests/hive-blobstore/src/test/queries/clientpositive/rcfile_nonstd_partitions_loc.q
 d17c281 
  itests/hive-blobstore/src/test/results/clientpositive/orc_format_part.q.out 
5d1319f 
  
itests/hive-blobstore/src/test/results/clientpositive/orc_nonstd_partitions_loc.q.out
 70e72f7 
  
itests/hive-blobstore/src/test/results/clientpositive/rcfile_format_part.q.out 
bed10ab 
  
itests/hive-blobstore/src/test/results/clientpositive/rcfile_nonstd_partitions_loc.q.out
 c6442f9 
  jdbc/src/java/org/apache/hive/jdbc/HiveBaseResultSet.java ade1900 
  jdbc/src/java/org/apache/hive/jdbc/JdbcColumn.java 38918f0 
  ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionRegistry.java 8dc5f2e 
  ql/src/java/org/apache/hadoop/hive/ql/exec/GroupByOperator.java f8b55da 
  ql/src/java/org/apache/hadoop/hive/ql/exec/SerializationUtilities.java 
247d589 
  
ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/translator/TypeConverter.java
 38308c9 
  ql/src/java/org/apache/hadoop/hive/ql/parse/DDLSemanticAnalyzer.java 0cf9205 
  ql/src/java/org/apache/hadoop/hive/ql/parse/HiveLexer.g 0721b92 
  ql/src/java/org/apache/hadoop/hive/ql/parse/HiveParser.g d98a663 
  ql/src/java/org/apache/hadoop/hive/ql/parse/IdentifiersParser.g 8598fae 
  ql/src/java/org/apache/hadoop/hive/ql/parse/TypeCheckProcFactory.java 8f8eab0 
  ql/src/java/org/apache/hadoop/hive/ql/stats/StatsUtils.java bda2050 
  ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToBoolean.java 7cdf2c3 
  ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToString.java 5cacd59 
  ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDF.java 68d98f5 
  ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFDate.java 5a31e61 
  
ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFToTimestampTZ.java 
PRE-CREATION 
  
ql/src/test/org/apache/hadoop/hive/ql/parse/TestSQL11ReservedKeyWordsNegative.java
 0dc6b19 
  ql/src/test/queries/clientnegative/serde_regex.q c9cfc7d 
  ql/src/test/queries/clientnegative/serde_regex2.q a29bb9c 
  ql/src/test/queries/clientnegative/serde_regex3.q 4e91f06 
  ql/src/test/queries/clientpositive/create_like.q bd39731 
  ql/src/test/queries/clientpositive/join43.q 12c45a6 
  ql/src/test/queries/clientpositive/serde_regex.q e21c6e1 
  ql/src/test/queries/clientpositive/timestamptz.q PRE-CREATION 
  ql/src/test/queries/clientpositive/timestamptz_1.q PRE-CREATION 
  ql/src/test/queries/clientpositive/timestamptz_2.q PRE-CREATION 
  ql/src/test/results/clientnegative/serde_regex.q.out a1ec5ca 
  ql/src/test/results/clientnegative/serde_regex2.q.out 374675d 
  ql/src/test/results/clientnegative/serde_regex3.q.out dc0a0e2 
  ql/src/test/results/clientpositive/create_like.q.out ff2e752 
  ql/src/test/results/clientpositive/join43.q.out e8c7278 
  ql/src/test/results/clientpositive/serde_regex.q.out 7bebb0c 
  ql/src/test/results/clientpositive/timestamptz.q.out PRE-CREATION 
  ql/src/test/results/clientpositive/timestamptz_1.q.out PRE-CREATION 
  ql/src/test/results/clientpositive/timestamptz_2.q.out PRE-CREATION 
  serde/if/serde.thrift 1d40d5a 
  serde/src/gen/thrift/gen-cpp/serde_constants.h 8785bd2 
  serde/src/gen/thrift/gen-cpp/serde_constants.cpp 907acf2 
  
serde/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/serde/serdeConstants.java
 2578d3e 
  serde/src/gen/thrift/gen-php/org/apache/hadoop/hive/serde/Types.php ea2dbbe 
  serde/src/gen/thrift/gen-py/org_apache_hadoop_hive_serde/constants.py e3b24eb 
  serde/src/gen/thrift/gen-rb/serde_constants.rb 15efaea 
  serde/src/java/org/apache/hadoop/hive/serde2/SerDeUtils.java 5ecfbca 
  
serde/src/java/org/apache/hadoop/hive/serde2/binarysortable/BinarySortableSerDe.java
 89e15c3 
  serde/src/java/o

Re: Review Request 50787: Add a timezone-aware timestamp

[Rui Li]

> On Sept. 22, 2016, 11:20 a.m., Jason Dere wrote:
> > - How about compatbility with the various date functions 
> > (year()/month()/day()/etc)?

For most of the functions, TIMESTAMPTZ is implicitly converted to text. 
Therefore I think we can get correct results. I added some special handle in 
HOUR because some hour may be unavailable due to DST.
So far I've verified the following funcsions work:

to_date
year
quarter
month
day
dayofmonth
hour
minute
second
weekofyear

Is it OK we leave others in follow-on tasks? I'd like to keep the patch small.


> On Sept. 22, 2016, 11:20 a.m., Jason Dere wrote:
> > ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFToTimestampTZ.java,
> >  line 58
> > 
> >
> > No conversions to/from DATE/TIMESTAMP?

Added conversion from date/timestamp to timestamptz. Default timezone is used 
for the converted timestamptz.
We can add convertion from numeric types in follow-on task.


> On Sept. 22, 2016, 11:20 a.m., Jason Dere wrote:
> > serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/PrimitiveObjectInspectorUtils.java,
> >  line 1109
> > 
> >
> > If the local timezone is different from the timezone in the 
> > TimestampTZ, is it possible that the year/month/day of the DATE might be 
> > different from the year/month/day of the TimestampTZ?

Good catch! It makes more sense to convert from the text representation than 
the time/nanos. So I convert the timestamptz to string first, and use that 
string to create the date. Same applies when converting to timestamp.


- Rui


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/50787/#review149983
---


On Sept. 22, 2016, 4:05 a.m., Rui Li wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/50787/
> ---
> 
> (Updated Sept. 22, 2016, 4:05 a.m.)
> 
> 
> Review request for hive.
> 
> 
> Bugs: HIVE-14412
> https://issues.apache.org/jira/browse/HIVE-14412
> 
> 
> Repository: hive-git
> 
> 
> Description
> ---
> 
> The 1st patch to add timezone-aware timestamp.
> 
> 
> Diffs
> -
> 
>   common/src/test/org/apache/hadoop/hive/common/type/TestTimestampTZ.java 
> PRE-CREATION 
>   contrib/src/test/queries/clientnegative/serde_regex.q a676338 
>   contrib/src/test/queries/clientpositive/serde_regex.q d75d607 
>   contrib/src/test/results/clientnegative/serde_regex.q.out 0f9b036 
>   contrib/src/test/results/clientpositive/serde_regex.q.out 2984293 
>   hbase-handler/src/test/queries/positive/hbase_timestamp.q 0350afe 
>   hbase-handler/src/test/results/positive/hbase_timestamp.q.out 3918121 
>   jdbc/src/java/org/apache/hive/jdbc/HiveBaseResultSet.java 93f093f 
>   jdbc/src/java/org/apache/hive/jdbc/JdbcColumn.java 38918f0 
>   ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionRegistry.java de74c3e 
>   ql/src/java/org/apache/hadoop/hive/ql/exec/GroupByOperator.java f28d33e 
>   ql/src/java/org/apache/hadoop/hive/ql/exec/SerializationUtilities.java 
> 7be628e 
>   
> ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/translator/TypeConverter.java
>  ba41518 
>   ql/src/java/org/apache/hadoop/hive/ql/parse/DDLSemanticAnalyzer.java 
> 8b0db4a 
>   ql/src/java/org/apache/hadoop/hive/ql/parse/HiveLexer.g 7ceb005 
>   ql/src/java/org/apache/hadoop/hive/ql/parse/HiveParser.g 62bbcc6 
>   ql/src/java/org/apache/hadoop/hive/ql/parse/IdentifiersParser.g 9ba1865 
>   ql/src/java/org/apache/hadoop/hive/ql/parse/TypeCheckProcFactory.java 
> 82080eb 
>   ql/src/java/org/apache/hadoop/hive/ql/stats/StatsUtils.java a718264 
>   ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToBoolean.java 17b892c 
>   ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToByte.java efae82d 
>   ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToDouble.java 9cbc114 
>   ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToFloat.java 5808c90 
>   ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToInteger.java a7551cb 
>   ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToLong.java c961d14 
>   ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToShort.java 570408a 
>   ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToString.java 5cacd59 
>   ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDF.java 259fde8 
>   
> ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFToTimestampTZ.java
>  PRE-CREATION 
>   ql/src/test/queries/clientnegative/serde_regex.q c9cfc7d 
>   ql/src/test/queries/clientnegative/serde_regex2.q a29bb9c 
>   ql/src/test/queries/clientnegative/serde_regex3.q 4e91f06 
>   ql/src/test/queries/clientpositive/create_like.q bd39731 
>   ql/src/test/queries/clientpo

Re: Review Request 50787: Add a timezone-aware timestamp

[Rui Li]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/50787/
---

(Updated Sept. 23, 2016, 8:58 a.m.)


Review request for hive.


Bugs: HIVE-14412
https://issues.apache.org/jira/browse/HIVE-14412


Repository: hive-git


Description
---

The 1st patch to add timezone-aware timestamp.


Diffs (updated)
-

  common/src/test/org/apache/hadoop/hive/common/type/TestTimestampTZ.java 
PRE-CREATION 
  contrib/src/test/queries/clientnegative/serde_regex.q a676338 
  contrib/src/test/queries/clientpositive/serde_regex.q d75d607 
  contrib/src/test/results/clientnegative/serde_regex.q.out 0f9b036 
  contrib/src/test/results/clientpositive/serde_regex.q.out 2984293 
  hbase-handler/src/test/queries/positive/hbase_timestamp.q 0350afe 
  hbase-handler/src/test/results/positive/hbase_timestamp.q.out 3918121 
  jdbc/src/java/org/apache/hive/jdbc/HiveBaseResultSet.java 93f093f 
  jdbc/src/java/org/apache/hive/jdbc/JdbcColumn.java 38918f0 
  ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionRegistry.java de74c3e 
  ql/src/java/org/apache/hadoop/hive/ql/exec/GroupByOperator.java f28d33e 
  ql/src/java/org/apache/hadoop/hive/ql/exec/SerializationUtilities.java 
7be628e 
  
ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/translator/TypeConverter.java
 ba41518 
  ql/src/java/org/apache/hadoop/hive/ql/parse/DDLSemanticAnalyzer.java 8b0db4a 
  ql/src/java/org/apache/hadoop/hive/ql/parse/HiveLexer.g 7ceb005 
  ql/src/java/org/apache/hadoop/hive/ql/parse/HiveParser.g 62bbcc6 
  ql/src/java/org/apache/hadoop/hive/ql/parse/IdentifiersParser.g 9ba1865 
  ql/src/java/org/apache/hadoop/hive/ql/parse/TypeCheckProcFactory.java 82080eb 
  ql/src/java/org/apache/hadoop/hive/ql/stats/StatsUtils.java a718264 
  ql/src/java/org/apache/hadoop/hive/ql/udf/UDFHour.java bccf5a6 
  ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToBoolean.java 17b892c 
  ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToByte.java efae82d 
  ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToDouble.java 9cbc114 
  ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToFloat.java 5808c90 
  ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToInteger.java a7551cb 
  ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToLong.java c961d14 
  ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToShort.java 570408a 
  ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToString.java 5cacd59 
  ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDF.java 259fde8 
  ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFDate.java 5a31e61 
  
ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFToTimestampTZ.java 
PRE-CREATION 
  ql/src/test/queries/clientnegative/serde_regex.q c9cfc7d 
  ql/src/test/queries/clientnegative/serde_regex2.q a29bb9c 
  ql/src/test/queries/clientnegative/serde_regex3.q 4e91f06 
  ql/src/test/queries/clientpositive/create_like.q bd39731 
  ql/src/test/queries/clientpositive/join43.q 12c45a6 
  ql/src/test/queries/clientpositive/serde_regex.q e21c6e1 
  ql/src/test/queries/clientpositive/timestamptz.q PRE-CREATION 
  ql/src/test/queries/clientpositive/timestamptz_1.q PRE-CREATION 
  ql/src/test/queries/clientpositive/timestamptz_2.q PRE-CREATION 
  ql/src/test/queries/clientpositive/timestamptz_3.q PRE-CREATION 
  ql/src/test/results/clientnegative/invalid_cast_from_binary_1.q.out acecbae 
  ql/src/test/results/clientnegative/invalid_cast_from_binary_2.q.out 41e1c80 
  ql/src/test/results/clientnegative/invalid_cast_from_binary_3.q.out 23e3403 
  ql/src/test/results/clientnegative/invalid_cast_from_binary_4.q.out 3541ef6 
  ql/src/test/results/clientnegative/invalid_cast_from_binary_5.q.out 177039c 
  ql/src/test/results/clientnegative/invalid_cast_from_binary_6.q.out 668380f 
  ql/src/test/results/clientnegative/serde_regex.q.out 7892bb2 
  ql/src/test/results/clientnegative/serde_regex2.q.out 1ceb387 
  ql/src/test/results/clientnegative/serde_regex3.q.out 028a24f 
  ql/src/test/results/clientnegative/wrong_column_type.q.out 6ff90ea 
  ql/src/test/results/clientpositive/create_like.q.out 0111c94 
  ql/src/test/results/clientpositive/join43.q.out 127d5d0 
  ql/src/test/results/clientpositive/serde_regex.q.out 7bebb0c 
  ql/src/test/results/clientpositive/timestamptz.q.out PRE-CREATION 
  ql/src/test/results/clientpositive/timestamptz_1.q.out PRE-CREATION 
  ql/src/test/results/clientpositive/timestamptz_2.q.out PRE-CREATION 
  ql/src/test/results/clientpositive/timestamptz_3.q.out PRE-CREATION 
  serde/if/serde.thrift 6caad36 
  serde/src/gen/thrift/gen-cpp/serde_constants.h a5f33fb 
  serde/src/gen/thrift/gen-cpp/serde_constants.cpp 3a675bf 
  
serde/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/serde/serdeConstants.java
 04ed8f5 
  serde/src/gen/thrift/gen-php/org/apache/hadoop/hive/serde/Types.php 18c3991 
  serde/src/gen/thrift/gen-py/org_apache_hadoop_hive_serde/constants.py fafdc24 
  serde/s

Re: Review Request 50787: Add a timezone-aware timestamp

[Jason Dere]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/50787/#review149983
---



- How about compatbility with the various date functions 
(year()/month()/day()/etc)?


ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFToTimestampTZ.java 
(line 58)


No conversions to/from DATE/TIMESTAMP?



serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/PrimitiveObjectInspectorUtils.java
 (line 1075)


If the local timezone is different from the timezone in the TimestampTZ, is 
it possible that the year/month/day of the DATE might be different from the 
year/month/day of the TimestampTZ?


- Jason Dere


On Sept. 22, 2016, 4:05 a.m., Rui Li wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/50787/
> ---
> 
> (Updated Sept. 22, 2016, 4:05 a.m.)
> 
> 
> Review request for hive.
> 
> 
> Bugs: HIVE-14412
> https://issues.apache.org/jira/browse/HIVE-14412
> 
> 
> Repository: hive-git
> 
> 
> Description
> ---
> 
> The 1st patch to add timezone-aware timestamp.
> 
> 
> Diffs
> -
> 
>   common/src/test/org/apache/hadoop/hive/common/type/TestTimestampTZ.java 
> PRE-CREATION 
>   contrib/src/test/queries/clientnegative/serde_regex.q a676338 
>   contrib/src/test/queries/clientpositive/serde_regex.q d75d607 
>   contrib/src/test/results/clientnegative/serde_regex.q.out 0f9b036 
>   contrib/src/test/results/clientpositive/serde_regex.q.out 2984293 
>   hbase-handler/src/test/queries/positive/hbase_timestamp.q 0350afe 
>   hbase-handler/src/test/results/positive/hbase_timestamp.q.out 3918121 
>   jdbc/src/java/org/apache/hive/jdbc/HiveBaseResultSet.java 93f093f 
>   jdbc/src/java/org/apache/hive/jdbc/JdbcColumn.java 38918f0 
>   ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionRegistry.java de74c3e 
>   ql/src/java/org/apache/hadoop/hive/ql/exec/GroupByOperator.java f28d33e 
>   ql/src/java/org/apache/hadoop/hive/ql/exec/SerializationUtilities.java 
> 7be628e 
>   
> ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/translator/TypeConverter.java
>  ba41518 
>   ql/src/java/org/apache/hadoop/hive/ql/parse/DDLSemanticAnalyzer.java 
> 8b0db4a 
>   ql/src/java/org/apache/hadoop/hive/ql/parse/HiveLexer.g 7ceb005 
>   ql/src/java/org/apache/hadoop/hive/ql/parse/HiveParser.g 62bbcc6 
>   ql/src/java/org/apache/hadoop/hive/ql/parse/IdentifiersParser.g 9ba1865 
>   ql/src/java/org/apache/hadoop/hive/ql/parse/TypeCheckProcFactory.java 
> 82080eb 
>   ql/src/java/org/apache/hadoop/hive/ql/stats/StatsUtils.java a718264 
>   ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToBoolean.java 17b892c 
>   ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToByte.java efae82d 
>   ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToDouble.java 9cbc114 
>   ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToFloat.java 5808c90 
>   ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToInteger.java a7551cb 
>   ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToLong.java c961d14 
>   ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToShort.java 570408a 
>   ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToString.java 5cacd59 
>   ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDF.java 259fde8 
>   
> ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFToTimestampTZ.java
>  PRE-CREATION 
>   ql/src/test/queries/clientnegative/serde_regex.q c9cfc7d 
>   ql/src/test/queries/clientnegative/serde_regex2.q a29bb9c 
>   ql/src/test/queries/clientnegative/serde_regex3.q 4e91f06 
>   ql/src/test/queries/clientpositive/create_like.q bd39731 
>   ql/src/test/queries/clientpositive/join43.q 12c45a6 
>   ql/src/test/queries/clientpositive/serde_regex.q e21c6e1 
>   ql/src/test/queries/clientpositive/timestamptz.q PRE-CREATION 
>   ql/src/test/queries/clientpositive/timestamptz_1.q PRE-CREATION 
>   ql/src/test/queries/clientpositive/timestamptz_2.q PRE-CREATION 
>   ql/src/test/results/clientnegative/invalid_cast_from_binary_1.q.out acecbae 
>   ql/src/test/results/clientnegative/invalid_cast_from_binary_2.q.out 41e1c80 
>   ql/src/test/results/clientnegative/invalid_cast_from_binary_3.q.out 23e3403 
>   ql/src/test/results/clientnegative/invalid_cast_from_binary_4.q.out 3541ef6 
>   ql/src/test/results/clientnegative/invalid_cast_from_binary_5.q.out 177039c 
>   ql/src/test/results/clientnegative/invalid_cast_from_binary_6.q.out 668380f 
>   ql/src/test/results/clientnegative/serde_regex.q.out 7892bb2 
>   ql/src/test/results/clientnegative/serde_regex2.q.out 1ceb387 
>   ql/src/test/results/clientnegative/serde_regex3.q.out 028a24f 
>   ql/src/test/results/clientnegative/wrong_column_type.q.out 6ff90ea 
>  

Re: Review Request 50787: Add a timezone-aware timestamp

[Xuefu Zhang]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/50787/#review149942
---


Ship it!




Ship It!

- Xuefu Zhang


On Sept. 22, 2016, 4:05 a.m., Rui Li wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/50787/
> ---
> 
> (Updated Sept. 22, 2016, 4:05 a.m.)
> 
> 
> Review request for hive.
> 
> 
> Bugs: HIVE-14412
> https://issues.apache.org/jira/browse/HIVE-14412
> 
> 
> Repository: hive-git
> 
> 
> Description
> ---
> 
> The 1st patch to add timezone-aware timestamp.
> 
> 
> Diffs
> -
> 
>   common/src/test/org/apache/hadoop/hive/common/type/TestTimestampTZ.java 
> PRE-CREATION 
>   contrib/src/test/queries/clientnegative/serde_regex.q a676338 
>   contrib/src/test/queries/clientpositive/serde_regex.q d75d607 
>   contrib/src/test/results/clientnegative/serde_regex.q.out 0f9b036 
>   contrib/src/test/results/clientpositive/serde_regex.q.out 2984293 
>   hbase-handler/src/test/queries/positive/hbase_timestamp.q 0350afe 
>   hbase-handler/src/test/results/positive/hbase_timestamp.q.out 3918121 
>   jdbc/src/java/org/apache/hive/jdbc/HiveBaseResultSet.java 93f093f 
>   jdbc/src/java/org/apache/hive/jdbc/JdbcColumn.java 38918f0 
>   ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionRegistry.java de74c3e 
>   ql/src/java/org/apache/hadoop/hive/ql/exec/GroupByOperator.java f28d33e 
>   ql/src/java/org/apache/hadoop/hive/ql/exec/SerializationUtilities.java 
> 7be628e 
>   
> ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/translator/TypeConverter.java
>  ba41518 
>   ql/src/java/org/apache/hadoop/hive/ql/parse/DDLSemanticAnalyzer.java 
> 8b0db4a 
>   ql/src/java/org/apache/hadoop/hive/ql/parse/HiveLexer.g 7ceb005 
>   ql/src/java/org/apache/hadoop/hive/ql/parse/HiveParser.g 62bbcc6 
>   ql/src/java/org/apache/hadoop/hive/ql/parse/IdentifiersParser.g 9ba1865 
>   ql/src/java/org/apache/hadoop/hive/ql/parse/TypeCheckProcFactory.java 
> 82080eb 
>   ql/src/java/org/apache/hadoop/hive/ql/stats/StatsUtils.java a718264 
>   ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToBoolean.java 17b892c 
>   ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToByte.java efae82d 
>   ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToDouble.java 9cbc114 
>   ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToFloat.java 5808c90 
>   ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToInteger.java a7551cb 
>   ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToLong.java c961d14 
>   ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToShort.java 570408a 
>   ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToString.java 5cacd59 
>   ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDF.java 259fde8 
>   
> ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFToTimestampTZ.java
>  PRE-CREATION 
>   ql/src/test/queries/clientnegative/serde_regex.q c9cfc7d 
>   ql/src/test/queries/clientnegative/serde_regex2.q a29bb9c 
>   ql/src/test/queries/clientnegative/serde_regex3.q 4e91f06 
>   ql/src/test/queries/clientpositive/create_like.q bd39731 
>   ql/src/test/queries/clientpositive/join43.q 12c45a6 
>   ql/src/test/queries/clientpositive/serde_regex.q e21c6e1 
>   ql/src/test/queries/clientpositive/timestamptz.q PRE-CREATION 
>   ql/src/test/queries/clientpositive/timestamptz_1.q PRE-CREATION 
>   ql/src/test/queries/clientpositive/timestamptz_2.q PRE-CREATION 
>   ql/src/test/results/clientnegative/invalid_cast_from_binary_1.q.out acecbae 
>   ql/src/test/results/clientnegative/invalid_cast_from_binary_2.q.out 41e1c80 
>   ql/src/test/results/clientnegative/invalid_cast_from_binary_3.q.out 23e3403 
>   ql/src/test/results/clientnegative/invalid_cast_from_binary_4.q.out 3541ef6 
>   ql/src/test/results/clientnegative/invalid_cast_from_binary_5.q.out 177039c 
>   ql/src/test/results/clientnegative/invalid_cast_from_binary_6.q.out 668380f 
>   ql/src/test/results/clientnegative/serde_regex.q.out 7892bb2 
>   ql/src/test/results/clientnegative/serde_regex2.q.out 1ceb387 
>   ql/src/test/results/clientnegative/serde_regex3.q.out 028a24f 
>   ql/src/test/results/clientnegative/wrong_column_type.q.out 6ff90ea 
>   ql/src/test/results/clientpositive/create_like.q.out 0111c94 
>   ql/src/test/results/clientpositive/join43.q.out 127d5d0 
>   ql/src/test/results/clientpositive/serde_regex.q.out 7bebb0c 
>   ql/src/test/results/clientpositive/timestamptz.q.out PRE-CREATION 
>   ql/src/test/results/clientpositive/timestamptz_1.q.out PRE-CREATION 
>   ql/src/test/results/clientpositive/timestamptz_2.q.out PRE-CREATION 
>   serde/if/serde.thrift 6caad36 
>   serde/src/gen/thrift/gen-cpp/serde_constants.h a5f33fb 
>   serde/src/gen/thrift/gen-cpp/serde_constants.cpp 3a675bf 
>   
> serde/src/gen/thrift/gen-javabean/org/apache/hadoo

Re: Review Request 50787: Add a timezone-aware timestamp

[Rui Li]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/50787/
---

(Updated Sept. 22, 2016, 4:05 a.m.)


Review request for hive.


Bugs: HIVE-14412
https://issues.apache.org/jira/browse/HIVE-14412


Repository: hive-git


Description
---

The 1st patch to add timezone-aware timestamp.


Diffs (updated)
-

  common/src/test/org/apache/hadoop/hive/common/type/TestTimestampTZ.java 
PRE-CREATION 
  contrib/src/test/queries/clientnegative/serde_regex.q a676338 
  contrib/src/test/queries/clientpositive/serde_regex.q d75d607 
  contrib/src/test/results/clientnegative/serde_regex.q.out 0f9b036 
  contrib/src/test/results/clientpositive/serde_regex.q.out 2984293 
  hbase-handler/src/test/queries/positive/hbase_timestamp.q 0350afe 
  hbase-handler/src/test/results/positive/hbase_timestamp.q.out 3918121 
  jdbc/src/java/org/apache/hive/jdbc/HiveBaseResultSet.java 93f093f 
  jdbc/src/java/org/apache/hive/jdbc/JdbcColumn.java 38918f0 
  ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionRegistry.java de74c3e 
  ql/src/java/org/apache/hadoop/hive/ql/exec/GroupByOperator.java f28d33e 
  ql/src/java/org/apache/hadoop/hive/ql/exec/SerializationUtilities.java 
7be628e 
  
ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/translator/TypeConverter.java
 ba41518 
  ql/src/java/org/apache/hadoop/hive/ql/parse/DDLSemanticAnalyzer.java 8b0db4a 
  ql/src/java/org/apache/hadoop/hive/ql/parse/HiveLexer.g 7ceb005 
  ql/src/java/org/apache/hadoop/hive/ql/parse/HiveParser.g 62bbcc6 
  ql/src/java/org/apache/hadoop/hive/ql/parse/IdentifiersParser.g 9ba1865 
  ql/src/java/org/apache/hadoop/hive/ql/parse/TypeCheckProcFactory.java 82080eb 
  ql/src/java/org/apache/hadoop/hive/ql/stats/StatsUtils.java a718264 
  ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToBoolean.java 17b892c 
  ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToByte.java efae82d 
  ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToDouble.java 9cbc114 
  ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToFloat.java 5808c90 
  ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToInteger.java a7551cb 
  ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToLong.java c961d14 
  ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToShort.java 570408a 
  ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToString.java 5cacd59 
  ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDF.java 259fde8 
  
ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFToTimestampTZ.java 
PRE-CREATION 
  ql/src/test/queries/clientnegative/serde_regex.q c9cfc7d 
  ql/src/test/queries/clientnegative/serde_regex2.q a29bb9c 
  ql/src/test/queries/clientnegative/serde_regex3.q 4e91f06 
  ql/src/test/queries/clientpositive/create_like.q bd39731 
  ql/src/test/queries/clientpositive/join43.q 12c45a6 
  ql/src/test/queries/clientpositive/serde_regex.q e21c6e1 
  ql/src/test/queries/clientpositive/timestamptz.q PRE-CREATION 
  ql/src/test/queries/clientpositive/timestamptz_1.q PRE-CREATION 
  ql/src/test/queries/clientpositive/timestamptz_2.q PRE-CREATION 
  ql/src/test/results/clientnegative/invalid_cast_from_binary_1.q.out acecbae 
  ql/src/test/results/clientnegative/invalid_cast_from_binary_2.q.out 41e1c80 
  ql/src/test/results/clientnegative/invalid_cast_from_binary_3.q.out 23e3403 
  ql/src/test/results/clientnegative/invalid_cast_from_binary_4.q.out 3541ef6 
  ql/src/test/results/clientnegative/invalid_cast_from_binary_5.q.out 177039c 
  ql/src/test/results/clientnegative/invalid_cast_from_binary_6.q.out 668380f 
  ql/src/test/results/clientnegative/serde_regex.q.out 7892bb2 
  ql/src/test/results/clientnegative/serde_regex2.q.out 1ceb387 
  ql/src/test/results/clientnegative/serde_regex3.q.out 028a24f 
  ql/src/test/results/clientnegative/wrong_column_type.q.out 6ff90ea 
  ql/src/test/results/clientpositive/create_like.q.out 0111c94 
  ql/src/test/results/clientpositive/join43.q.out 127d5d0 
  ql/src/test/results/clientpositive/serde_regex.q.out 7bebb0c 
  ql/src/test/results/clientpositive/timestamptz.q.out PRE-CREATION 
  ql/src/test/results/clientpositive/timestamptz_1.q.out PRE-CREATION 
  ql/src/test/results/clientpositive/timestamptz_2.q.out PRE-CREATION 
  serde/if/serde.thrift 6caad36 
  serde/src/gen/thrift/gen-cpp/serde_constants.h a5f33fb 
  serde/src/gen/thrift/gen-cpp/serde_constants.cpp 3a675bf 
  
serde/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/serde/serdeConstants.java
 04ed8f5 
  serde/src/gen/thrift/gen-php/org/apache/hadoop/hive/serde/Types.php 18c3991 
  serde/src/gen/thrift/gen-py/org_apache_hadoop_hive_serde/constants.py fafdc24 
  serde/src/gen/thrift/gen-rb/serde_constants.rb 0ce9f27 
  serde/src/java/org/apache/hadoop/hive/serde2/SerDeUtils.java 7ffc964 
  
serde/src/java/org/apache/hadoop/hive/serde2/binarysortable/BinarySortableSerDe.java
 5e119d7 
  
serde/src/java/org/apache/hadoop/hive/serde2/binarysortable/fas

Re: Review Request 50787: Add a timezone-aware timestamp

[Xuefu Zhang]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/50787/#review149940
---




ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDF.java (line 406)


Nit: might be better to put timestamptz right after timestamp for visual 
ease, but not a big deal. Same below.



serde/src/java/org/apache/hadoop/hive/serde2/io/TimestampTZWritable.java (line 
1)


License header please.


- Xuefu Zhang


On Sept. 22, 2016, 1:08 a.m., Rui Li wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/50787/
> ---
> 
> (Updated Sept. 22, 2016, 1:08 a.m.)
> 
> 
> Review request for hive.
> 
> 
> Bugs: HIVE-14412
> https://issues.apache.org/jira/browse/HIVE-14412
> 
> 
> Repository: hive-git
> 
> 
> Description
> ---
> 
> The 1st patch to add timezone-aware timestamp.
> 
> 
> Diffs
> -
> 
>   common/src/test/org/apache/hadoop/hive/common/type/TestTimestampTZ.java 
> PRE-CREATION 
>   contrib/src/test/queries/clientnegative/serde_regex.q a676338 
>   contrib/src/test/queries/clientpositive/serde_regex.q d75d607 
>   contrib/src/test/results/clientnegative/serde_regex.q.out 0f9b036 
>   contrib/src/test/results/clientpositive/serde_regex.q.out 2984293 
>   hbase-handler/src/test/queries/positive/hbase_timestamp.q 0350afe 
>   hbase-handler/src/test/results/positive/hbase_timestamp.q.out 3918121 
>   jdbc/src/java/org/apache/hive/jdbc/HiveBaseResultSet.java 93f093f 
>   jdbc/src/java/org/apache/hive/jdbc/JdbcColumn.java 38918f0 
>   ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionRegistry.java de74c3e 
>   ql/src/java/org/apache/hadoop/hive/ql/exec/GroupByOperator.java f28d33e 
>   ql/src/java/org/apache/hadoop/hive/ql/exec/SerializationUtilities.java 
> 7be628e 
>   
> ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/translator/TypeConverter.java
>  ba41518 
>   ql/src/java/org/apache/hadoop/hive/ql/parse/DDLSemanticAnalyzer.java 
> 8b0db4a 
>   ql/src/java/org/apache/hadoop/hive/ql/parse/HiveLexer.g 7ceb005 
>   ql/src/java/org/apache/hadoop/hive/ql/parse/HiveParser.g 62bbcc6 
>   ql/src/java/org/apache/hadoop/hive/ql/parse/IdentifiersParser.g 9ba1865 
>   ql/src/java/org/apache/hadoop/hive/ql/parse/TypeCheckProcFactory.java 
> 82080eb 
>   ql/src/java/org/apache/hadoop/hive/ql/stats/StatsUtils.java a718264 
>   ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToBoolean.java 17b892c 
>   ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToByte.java efae82d 
>   ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToDouble.java 9cbc114 
>   ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToFloat.java 5808c90 
>   ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToInteger.java a7551cb 
>   ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToLong.java c961d14 
>   ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToShort.java 570408a 
>   ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToString.java 5cacd59 
>   ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDF.java 259fde8 
>   
> ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFToTimestampTZ.java
>  PRE-CREATION 
>   ql/src/test/queries/clientnegative/serde_regex.q c9cfc7d 
>   ql/src/test/queries/clientnegative/serde_regex2.q a29bb9c 
>   ql/src/test/queries/clientnegative/serde_regex3.q 4e91f06 
>   ql/src/test/queries/clientpositive/create_like.q bd39731 
>   ql/src/test/queries/clientpositive/join43.q 12c45a6 
>   ql/src/test/queries/clientpositive/serde_regex.q e21c6e1 
>   ql/src/test/queries/clientpositive/timestamptz.q PRE-CREATION 
>   ql/src/test/queries/clientpositive/timestamptz_1.q PRE-CREATION 
>   ql/src/test/queries/clientpositive/timestamptz_2.q PRE-CREATION 
>   ql/src/test/results/clientnegative/invalid_cast_from_binary_1.q.out acecbae 
>   ql/src/test/results/clientnegative/invalid_cast_from_binary_2.q.out 41e1c80 
>   ql/src/test/results/clientnegative/invalid_cast_from_binary_3.q.out 23e3403 
>   ql/src/test/results/clientnegative/invalid_cast_from_binary_4.q.out 3541ef6 
>   ql/src/test/results/clientnegative/invalid_cast_from_binary_5.q.out 177039c 
>   ql/src/test/results/clientnegative/invalid_cast_from_binary_6.q.out 668380f 
>   ql/src/test/results/clientnegative/serde_regex.q.out 7892bb2 
>   ql/src/test/results/clientnegative/serde_regex2.q.out 1ceb387 
>   ql/src/test/results/clientnegative/serde_regex3.q.out 028a24f 
>   ql/src/test/results/clientnegative/wrong_column_type.q.out 6ff90ea 
>   ql/src/test/results/clientpositive/create_like.q.out 0111c94 
>   ql/src/test/results/clientpositive/join43.q.out 127d5d0 
>   ql/src/test/results/clientpositive/serde_regex.q.out 7bebb0c 
>   ql/src/test/results/clientpositive

Re: Review Request 50787: Add a timezone-aware timestamp

[Rui Li]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/50787/
---

(Updated Sept. 22, 2016, 1:08 a.m.)


Review request for hive.


Bugs: HIVE-14412
https://issues.apache.org/jira/browse/HIVE-14412


Repository: hive-git


Description
---

The 1st patch to add timezone-aware timestamp.


Diffs (updated)
-

  common/src/test/org/apache/hadoop/hive/common/type/TestTimestampTZ.java 
PRE-CREATION 
  contrib/src/test/queries/clientnegative/serde_regex.q a676338 
  contrib/src/test/queries/clientpositive/serde_regex.q d75d607 
  contrib/src/test/results/clientnegative/serde_regex.q.out 0f9b036 
  contrib/src/test/results/clientpositive/serde_regex.q.out 2984293 
  hbase-handler/src/test/queries/positive/hbase_timestamp.q 0350afe 
  hbase-handler/src/test/results/positive/hbase_timestamp.q.out 3918121 
  jdbc/src/java/org/apache/hive/jdbc/HiveBaseResultSet.java 93f093f 
  jdbc/src/java/org/apache/hive/jdbc/JdbcColumn.java 38918f0 
  ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionRegistry.java de74c3e 
  ql/src/java/org/apache/hadoop/hive/ql/exec/GroupByOperator.java f28d33e 
  ql/src/java/org/apache/hadoop/hive/ql/exec/SerializationUtilities.java 
7be628e 
  
ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/translator/TypeConverter.java
 ba41518 
  ql/src/java/org/apache/hadoop/hive/ql/parse/DDLSemanticAnalyzer.java 8b0db4a 
  ql/src/java/org/apache/hadoop/hive/ql/parse/HiveLexer.g 7ceb005 
  ql/src/java/org/apache/hadoop/hive/ql/parse/HiveParser.g 62bbcc6 
  ql/src/java/org/apache/hadoop/hive/ql/parse/IdentifiersParser.g 9ba1865 
  ql/src/java/org/apache/hadoop/hive/ql/parse/TypeCheckProcFactory.java 82080eb 
  ql/src/java/org/apache/hadoop/hive/ql/stats/StatsUtils.java a718264 
  ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToBoolean.java 17b892c 
  ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToByte.java efae82d 
  ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToDouble.java 9cbc114 
  ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToFloat.java 5808c90 
  ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToInteger.java a7551cb 
  ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToLong.java c961d14 
  ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToShort.java 570408a 
  ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToString.java 5cacd59 
  ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDF.java 259fde8 
  
ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFToTimestampTZ.java 
PRE-CREATION 
  ql/src/test/queries/clientnegative/serde_regex.q c9cfc7d 
  ql/src/test/queries/clientnegative/serde_regex2.q a29bb9c 
  ql/src/test/queries/clientnegative/serde_regex3.q 4e91f06 
  ql/src/test/queries/clientpositive/create_like.q bd39731 
  ql/src/test/queries/clientpositive/join43.q 12c45a6 
  ql/src/test/queries/clientpositive/serde_regex.q e21c6e1 
  ql/src/test/queries/clientpositive/timestamptz.q PRE-CREATION 
  ql/src/test/queries/clientpositive/timestamptz_1.q PRE-CREATION 
  ql/src/test/queries/clientpositive/timestamptz_2.q PRE-CREATION 
  ql/src/test/results/clientnegative/invalid_cast_from_binary_1.q.out acecbae 
  ql/src/test/results/clientnegative/invalid_cast_from_binary_2.q.out 41e1c80 
  ql/src/test/results/clientnegative/invalid_cast_from_binary_3.q.out 23e3403 
  ql/src/test/results/clientnegative/invalid_cast_from_binary_4.q.out 3541ef6 
  ql/src/test/results/clientnegative/invalid_cast_from_binary_5.q.out 177039c 
  ql/src/test/results/clientnegative/invalid_cast_from_binary_6.q.out 668380f 
  ql/src/test/results/clientnegative/serde_regex.q.out 7892bb2 
  ql/src/test/results/clientnegative/serde_regex2.q.out 1ceb387 
  ql/src/test/results/clientnegative/serde_regex3.q.out 028a24f 
  ql/src/test/results/clientnegative/wrong_column_type.q.out 6ff90ea 
  ql/src/test/results/clientpositive/create_like.q.out 0111c94 
  ql/src/test/results/clientpositive/join43.q.out 127d5d0 
  ql/src/test/results/clientpositive/serde_regex.q.out 7bebb0c 
  ql/src/test/results/clientpositive/timestamptz.q.out PRE-CREATION 
  ql/src/test/results/clientpositive/timestamptz_1.q.out PRE-CREATION 
  ql/src/test/results/clientpositive/timestamptz_2.q.out PRE-CREATION 
  serde/if/serde.thrift 6caad36 
  serde/src/gen/thrift/gen-cpp/serde_constants.h a5f33fb 
  serde/src/gen/thrift/gen-cpp/serde_constants.cpp 3a675bf 
  
serde/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/serde/serdeConstants.java
 04ed8f5 
  serde/src/gen/thrift/gen-php/org/apache/hadoop/hive/serde/Types.php 18c3991 
  serde/src/gen/thrift/gen-py/org_apache_hadoop_hive_serde/constants.py fafdc24 
  serde/src/gen/thrift/gen-rb/serde_constants.rb 0ce9f27 
  serde/src/java/org/apache/hadoop/hive/serde2/SerDeUtils.java 7ffc964 
  
serde/src/java/org/apache/hadoop/hive/serde2/binarysortable/BinarySortableSerDe.java
 5e119d7 
  
serde/src/java/org/apache/hadoop/hive/serde2/binarysortable/fas

Re: Review Request 50787: Add a timezone-aware timestamp

[Rui Li]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/50787/
---

(Updated Sept. 12, 2016, 3:06 a.m.)


Review request for hive.


Bugs: HIVE-14412
https://issues.apache.org/jira/browse/HIVE-14412


Repository: hive-git


Description
---

The 1st patch to add timezone-aware timestamp.


Diffs (updated)
-

  common/src/test/org/apache/hadoop/hive/common/type/TestHiveTimestamp.java 
PRE-CREATION 
  jdbc/src/java/org/apache/hive/jdbc/HiveBaseResultSet.java 93f093f 
  jdbc/src/java/org/apache/hive/jdbc/JdbcColumn.java 38918f0 
  ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionRegistry.java 60646ba 
  ql/src/java/org/apache/hadoop/hive/ql/exec/GroupByOperator.java f28d33e 
  ql/src/java/org/apache/hadoop/hive/ql/exec/SerializationUtilities.java 
7be628e 
  
ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/translator/TypeConverter.java
 ba41518 
  ql/src/java/org/apache/hadoop/hive/ql/parse/DDLSemanticAnalyzer.java 9329e00 
  ql/src/java/org/apache/hadoop/hive/ql/parse/HiveLexer.g 7ceb005 
  ql/src/java/org/apache/hadoop/hive/ql/parse/HiveParser.g df596ff 
  ql/src/java/org/apache/hadoop/hive/ql/parse/TypeCheckProcFactory.java 82080eb 
  ql/src/java/org/apache/hadoop/hive/ql/stats/StatsUtils.java a718264 
  ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToBoolean.java 17b892c 
  ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToByte.java efae82d 
  ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToDouble.java 9cbc114 
  ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToFloat.java 5808c90 
  ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToInteger.java a7551cb 
  ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToLong.java c961d14 
  ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToShort.java 570408a 
  ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToString.java 5cacd59 
  ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDF.java 259fde8 
  
ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFToTimestampTZ.java 
PRE-CREATION 
  ql/src/test/queries/clientpositive/timestamptz.q PRE-CREATION 
  ql/src/test/queries/clientpositive/timestamptz_1.q PRE-CREATION 
  ql/src/test/queries/clientpositive/timestamptz_2.q PRE-CREATION 
  ql/src/test/results/clientnegative/invalid_cast_from_binary_1.q.out acecbae 
  ql/src/test/results/clientnegative/invalid_cast_from_binary_2.q.out 41e1c80 
  ql/src/test/results/clientnegative/invalid_cast_from_binary_3.q.out 23e3403 
  ql/src/test/results/clientnegative/invalid_cast_from_binary_4.q.out 3541ef6 
  ql/src/test/results/clientnegative/invalid_cast_from_binary_5.q.out 177039c 
  ql/src/test/results/clientnegative/invalid_cast_from_binary_6.q.out 668380f 
  ql/src/test/results/clientnegative/wrong_column_type.q.out 6ff90ea 
  ql/src/test/results/clientpositive/timestamptz.q.out PRE-CREATION 
  ql/src/test/results/clientpositive/timestamptz_1.q.out PRE-CREATION 
  ql/src/test/results/clientpositive/timestamptz_2.q.out PRE-CREATION 
  serde/if/serde.thrift 6caad36 
  serde/src/gen/thrift/gen-cpp/serde_constants.h a5f33fb 
  serde/src/gen/thrift/gen-cpp/serde_constants.cpp 3a675bf 
  
serde/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/serde/serdeConstants.java
 04ed8f5 
  serde/src/gen/thrift/gen-php/org/apache/hadoop/hive/serde/Types.php 18c3991 
  serde/src/gen/thrift/gen-py/org_apache_hadoop_hive_serde/constants.py fafdc24 
  serde/src/gen/thrift/gen-rb/serde_constants.rb 0ce9f27 
  serde/src/java/org/apache/hadoop/hive/serde2/SerDeUtils.java 6e08dfd 
  
serde/src/java/org/apache/hadoop/hive/serde2/binarysortable/BinarySortableSerDe.java
 5e119d7 
  
serde/src/java/org/apache/hadoop/hive/serde2/binarysortable/fast/BinarySortableDeserializeRead.java
 0cbc8d0 
  serde/src/java/org/apache/hadoop/hive/serde2/io/HiveTimestampWritable.java 
PRE-CREATION 
  serde/src/java/org/apache/hadoop/hive/serde2/io/TimestampWritable.java 
bbccc7f 
  serde/src/java/org/apache/hadoop/hive/serde2/io/TimestampWritableBase.java 
PRE-CREATION 
  serde/src/java/org/apache/hadoop/hive/serde2/lazy/LazyFactory.java 23dbe6a 
  serde/src/java/org/apache/hadoop/hive/serde2/lazy/LazyHiveTimestamp.java 
PRE-CREATION 
  serde/src/java/org/apache/hadoop/hive/serde2/lazy/LazyTimestamp.java 56945d1 
  serde/src/java/org/apache/hadoop/hive/serde2/lazy/LazyUtils.java 73c72e1 
  
serde/src/java/org/apache/hadoop/hive/serde2/lazy/objectinspector/primitive/LazyHiveTimestampObjectInspector.java
 PRE-CREATION 
  
serde/src/java/org/apache/hadoop/hive/serde2/lazy/objectinspector/primitive/LazyPrimitiveObjectInspectorFactory.java
 5601734 
  
serde/src/java/org/apache/hadoop/hive/serde2/lazybinary/LazyBinaryFactory.java 
52f3527 
  
serde/src/java/org/apache/hadoop/hive/serde2/lazybinary/LazyBinaryHiveTimestamp.java
 PRE-CREATION 
  serde/src/java/org/apache/hadoop/hive/serde2/lazybinary/LazyBinarySerDe.java 
54bfd2d 
  serde/src/java/org/apache/hadoop/hi