[
https://issues.apache.org/jira/browse/TAJO-1925?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14993251#comment-14993251
]
ASF GitHub Bot commented on TAJO-1925:
--------------------------------------
GitHub user blrunner opened a pull request:
https://github.com/apache/tajo/pull/853
TAJO-1925: Improve hive compatibility with TIMESTAMP partition column.
This patch contains following modifications:
* When making partition directories, use partition name which are accepted
in the format ```YYYY-MM-DD HH:MM:SS.MS```.
* When pruning partitions, convert UTC of EvalNode to customer timezone.
* When building partition filter in catalog, convert UTC of algebra
expression to customer timezone.
* Add timestamp cast operation with the number of milliseconds.
And I found that the patch ran successfully with MySQLStore and
HiveCatalogStore.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/blrunner/tajo TAJO-1925
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/tajo/pull/853.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #853
----
commit e98cf2c7c7bb3b69e871dec45dd53d9ae567b42b
Author: JaeHwa Jung <[email protected]>
Date: 2015-11-05T07:46:48Z
TAJO-1925: Improve hive compatibility with TIMESTAMP partition column.
commit 36e9ec498b7486cb63c78b54d63e438834a51a6a
Author: JaeHwa Jung <[email protected]>
Date: 2015-11-05T07:49:54Z
Remove debug logs
commit 88d1335b36df19d58e2d68b7671e6405df686eec
Author: JaeHwa Jung <[email protected]>
Date: 2015-11-05T08:08:07Z
Add more descriptions for added codes
commit 9856ddae8eb1022af36a8999169a5a429b652377
Author: JaeHwa Jung <[email protected]>
Date: 2015-11-05T08:16:17Z
Add more description for
PartitionFilterAlgebraVisitor::visitTimestampLiteral
commit 2743bfbf63855a80bf081e83adc80b4e1fbbbd22
Author: JaeHwa Jung <[email protected]>
Date: 2015-11-05T08:18:00Z
Remove unncessary codes
commit 359d9886b61a33744ee9748a79e077e5b05a77ff
Author: JaeHwa Jung <[email protected]>
Date: 2015-11-06T03:30:01Z
Implement ColPartitionStoreExec::encodeTimestamp
commit 2e09493e91bccc44c2680408da81843df4b4191d
Author: JaeHwa Jung <[email protected]>
Date: 2015-11-06T03:33:43Z
Escape the path name of timestamp partition
commit 84c016426a817159c3b2dc40675d5951236b40a3
Author: JaeHwa Jung <[email protected]>
Date: 2015-11-06T03:45:59Z
Apply users timezone to partition pruning.
commit 0dad0601edbb710453899b8c9a68350f78bb4670
Author: JaeHwa Jung <[email protected]>
Date: 2015-11-06T05:20:59Z
Apply UTC timezone for casting operation.
commit 491d685cbe43066082e55f4ab36507451d4697b8
Author: JaeHwa Jung <[email protected]>
Date: 2015-11-06T06:56:28Z
Escape partition and unescase partition values
----
> Improve hive compatibility with TIMESTAMP partition column.
> -----------------------------------------------------------
>
> Key: TAJO-1925
> URL: https://issues.apache.org/jira/browse/TAJO-1925
> Project: Tajo
> Issue Type: Improvement
> Components: Catalog, Physical Operator
> Affects Versions: 0.11.0, 0.12.0
> Reporter: Jaehwa Jung
> Assignee: Jaehwa Jung
> Fix For: 0.12.0, 0.11.1
>
>
> -Currently, Tajo allow to use TIMESTAMP column as a partition key. But if
> users use TIMESTAMP partition column, Tajo doesn't keep the original
> TIMESTAMP values. Actually, Tajo automatically converts TIMESTAMP values to
> STRING literals which are accepted in the format YYYY-MM-DD HH:MM:SS. As a
> result, Tajo can't keep mills of second and can't provide right hive
> compatibility-
> We need to support hive compatibility for all partition column types. But
> when using TIMESTAMP partition column, users might not get correct partitions
> occasionally because tajo partition name is different from hive partition
> name. Actually, Tajo automatically converts TIMESTAMP values to STRING
> literals which are accepted in the format YYYY-MM-DD HH:MM:SS. But Hive
> automatically converts it to STRING literals which are accepted in the format
> YYYY-MM-DD HH:MM:SS.MS.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
