[jira] [Commented] (HIVE-8102) Partitions of type 'date' behave incorrectly with daylight saving time.
[ https://issues.apache.org/jira/browse/HIVE-8102?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14171879#comment-14171879 ] Thejas M Nair commented on HIVE-8102: - +1 Partitions of type 'date' behave incorrectly with daylight saving time. --- Key: HIVE-8102 URL: https://issues.apache.org/jira/browse/HIVE-8102 Project: Hive Issue Type: Bug Components: Database/Schema, Serializers/Deserializers Affects Versions: 0.13.0 Reporter: Eli Acherkan Assignee: Jason Dere Priority: Critical Fix For: 0.14.0 Attachments: HIVE-8102.1.patch, HIVE-8102.2.patch, HIVE-8102.3.patch On 2AM on March 28th 2014, Israel went from standard time (GMT+2) to daylight saving time (GMT+3). The server's timezone is Asia/Jerusalem. When creating a partition whose key is 2014-03-28, Hive creates a partition for 2013-03-27 instead: hive (default) create table test (a int) partitioned by (`b_prt` date); OK Time taken: 0.092 seconds hive (default) alter table test add partition (b_prt='2014-03-28'); OK Time taken: 0.187 seconds hive (default) show partitions test; OK partition b_prt=2014-03-27 Time taken: 0.134 seconds, Fetched: 1 row(s) It seems that the root cause is the behavior of DateWritable.daysToMillis/dateToDays. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-8102) Partitions of type 'date' behave incorrectly with daylight saving time.
[ https://issues.apache.org/jira/browse/HIVE-8102?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14171883#comment-14171883 ] Thejas M Nair commented on HIVE-8102: - Since this just a test update in the patch, I think you can commit if you have verified that the test passes. Partitions of type 'date' behave incorrectly with daylight saving time. --- Key: HIVE-8102 URL: https://issues.apache.org/jira/browse/HIVE-8102 Project: Hive Issue Type: Bug Components: Database/Schema, Serializers/Deserializers Affects Versions: 0.13.0 Reporter: Eli Acherkan Assignee: Jason Dere Priority: Critical Fix For: 0.14.0 Attachments: HIVE-8102.1.patch, HIVE-8102.2.patch, HIVE-8102.3.patch On 2AM on March 28th 2014, Israel went from standard time (GMT+2) to daylight saving time (GMT+3). The server's timezone is Asia/Jerusalem. When creating a partition whose key is 2014-03-28, Hive creates a partition for 2013-03-27 instead: hive (default) create table test (a int) partitioned by (`b_prt` date); OK Time taken: 0.092 seconds hive (default) alter table test add partition (b_prt='2014-03-28'); OK Time taken: 0.187 seconds hive (default) show partitions test; OK partition b_prt=2014-03-27 Time taken: 0.134 seconds, Fetched: 1 row(s) It seems that the root cause is the behavior of DateWritable.daysToMillis/dateToDays. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-8102) Partitions of type 'date' behave incorrectly with daylight saving time.
[ https://issues.apache.org/jira/browse/HIVE-8102?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14171917#comment-14171917 ] Lefty Leverenz commented on HIVE-8102: -- No doc needed? Partitions of type 'date' behave incorrectly with daylight saving time. --- Key: HIVE-8102 URL: https://issues.apache.org/jira/browse/HIVE-8102 Project: Hive Issue Type: Bug Components: Database/Schema, Serializers/Deserializers Affects Versions: 0.13.0 Reporter: Eli Acherkan Assignee: Jason Dere Priority: Critical Fix For: 0.14.0 Attachments: HIVE-8102.1.patch, HIVE-8102.2.patch, HIVE-8102.3.patch On 2AM on March 28th 2014, Israel went from standard time (GMT+2) to daylight saving time (GMT+3). The server's timezone is Asia/Jerusalem. When creating a partition whose key is 2014-03-28, Hive creates a partition for 2013-03-27 instead: hive (default) create table test (a int) partitioned by (`b_prt` date); OK Time taken: 0.092 seconds hive (default) alter table test add partition (b_prt='2014-03-28'); OK Time taken: 0.187 seconds hive (default) show partitions test; OK partition b_prt=2014-03-27 Time taken: 0.134 seconds, Fetched: 1 row(s) It seems that the root cause is the behavior of DateWritable.daysToMillis/dateToDays. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-8102) Partitions of type 'date' behave incorrectly with daylight saving time.
[ https://issues.apache.org/jira/browse/HIVE-8102?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14171923#comment-14171923 ] Jason Dere commented on HIVE-8102: -- nope, just a bad bug Partitions of type 'date' behave incorrectly with daylight saving time. --- Key: HIVE-8102 URL: https://issues.apache.org/jira/browse/HIVE-8102 Project: Hive Issue Type: Bug Components: Database/Schema, Serializers/Deserializers Affects Versions: 0.13.0 Reporter: Eli Acherkan Assignee: Jason Dere Priority: Critical Fix For: 0.14.0 Attachments: HIVE-8102.1.patch, HIVE-8102.2.patch, HIVE-8102.3.patch On 2AM on March 28th 2014, Israel went from standard time (GMT+2) to daylight saving time (GMT+3). The server's timezone is Asia/Jerusalem. When creating a partition whose key is 2014-03-28, Hive creates a partition for 2013-03-27 instead: hive (default) create table test (a int) partitioned by (`b_prt` date); OK Time taken: 0.092 seconds hive (default) alter table test add partition (b_prt='2014-03-28'); OK Time taken: 0.187 seconds hive (default) show partitions test; OK partition b_prt=2014-03-27 Time taken: 0.134 seconds, Fetched: 1 row(s) It seems that the root cause is the behavior of DateWritable.daysToMillis/dateToDays. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-8102) Partitions of type 'date' behave incorrectly with daylight saving time.
[ https://issues.apache.org/jira/browse/HIVE-8102?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14166118#comment-14166118 ] Thejas M Nair commented on HIVE-8102: - +1 Partitions of type 'date' behave incorrectly with daylight saving time. --- Key: HIVE-8102 URL: https://issues.apache.org/jira/browse/HIVE-8102 Project: Hive Issue Type: Bug Components: Database/Schema, Serializers/Deserializers Affects Versions: 0.13.0 Reporter: Eli Acherkan Assignee: Jason Dere Priority: Critical Fix For: 0.14.0 Attachments: HIVE-8102.1.patch, HIVE-8102.2.patch On 2AM on March 28th 2014, Israel went from standard time (GMT+2) to daylight saving time (GMT+3). The server's timezone is Asia/Jerusalem. When creating a partition whose key is 2014-03-28, Hive creates a partition for 2013-03-27 instead: hive (default) create table test (a int) partitioned by (`b_prt` date); OK Time taken: 0.092 seconds hive (default) alter table test add partition (b_prt='2014-03-28'); OK Time taken: 0.187 seconds hive (default) show partitions test; OK partition b_prt=2014-03-27 Time taken: 0.134 seconds, Fetched: 1 row(s) It seems that the root cause is the behavior of DateWritable.daysToMillis/dateToDays. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-8102) Partitions of type 'date' behave incorrectly with daylight saving time.
[ https://issues.apache.org/jira/browse/HIVE-8102?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14135083#comment-14135083 ] Eli Acherkan commented on HIVE-8102: Thanks [~jdere]! The patch appears to work well for us. (Haven't tested on other timezones.) Partitions of type 'date' behave incorrectly with daylight saving time. --- Key: HIVE-8102 URL: https://issues.apache.org/jira/browse/HIVE-8102 Project: Hive Issue Type: Bug Components: Database/Schema, Serializers/Deserializers Affects Versions: 0.13.0 Reporter: Eli Acherkan Attachments: HIVE-8102.1.patch On 2AM on March 28th 2014, Israel went from standard time (GMT+2) to daylight saving time (GMT+3). The server's timezone is Asia/Jerusalem. When creating a partition whose key is 2014-03-28, Hive creates a partition for 2013-03-27 instead: hive (default) create table test (a int) partitioned by (`b_prt` date); OK Time taken: 0.092 seconds hive (default) alter table test add partition (b_prt='2014-03-28'); OK Time taken: 0.187 seconds hive (default) show partitions test; OK partition b_prt=2014-03-27 Time taken: 0.134 seconds, Fetched: 1 row(s) It seems that the root cause is the behavior of DateWritable.daysToMillis/dateToDays. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-8102) Partitions of type 'date' behave incorrectly with daylight saving time.
[
https://issues.apache.org/jira/browse/HIVE-8102?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14136369#comment-14136369
]
Hive QA commented on HIVE-8102:
---
{color:green}Overall{color}: +1 all checks pass
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12669189/HIVE-8102.2.patch
{color:green}SUCCESS:{color} +1 6279 tests passed
Test results:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/827/testReport
Console output:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/827/console
Test logs:
http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-827/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12669189
Partitions of type 'date' behave incorrectly with daylight saving time.
---
Key: HIVE-8102
URL: https://issues.apache.org/jira/browse/HIVE-8102
Project: Hive
Issue Type: Bug
Components: Database/Schema, Serializers/Deserializers
Affects Versions: 0.13.0
Reporter: Eli Acherkan
Attachments: HIVE-8102.1.patch, HIVE-8102.2.patch
On 2AM on March 28th 2014, Israel went from standard time (GMT+2) to daylight
saving time (GMT+3).
The server's timezone is Asia/Jerusalem. When creating a partition whose key
is 2014-03-28, Hive creates a partition for 2013-03-27 instead:
hive (default) create table test (a int) partitioned by (`b_prt` date);
OK
Time taken: 0.092 seconds
hive (default) alter table test add partition (b_prt='2014-03-28');
OK
Time taken: 0.187 seconds
hive (default) show partitions test;
OK
partition
b_prt=2014-03-27
Time taken: 0.134 seconds, Fetched: 1 row(s)
It seems that the root cause is the behavior of
DateWritable.daysToMillis/dateToDays.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-8102) Partitions of type 'date' behave incorrectly with daylight saving time.
[
https://issues.apache.org/jira/browse/HIVE-8102?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14133985#comment-14133985
]
Eli Acherkan commented on HIVE-8102:
The following test fails when running in Asia/Jerusalem timezone:
Date originalDate = new Date(114, 2, 28); // March 28th 2014 - DST
begins on this day at 02:00.
DateWritable dateWritable = new DateWritable(originalDate);
assertEquals(originalDate, dateWritable.get()); // Assertion fails
because dateWritable.get() returns 2014-03-27 23:00:00 IST.
In order to be able to run this unit test in any timezone, we explicitly set
the timezone and run it in a separate thread, so that the thread local member
DataWritable.LOCAL_TIMEZONE is initialized with the correct one:
public void testDaylightSavingsTime() throws InterruptedException,
ExecutionException {
TimeZone previousDefault = TimeZone.getDefault();
TimeZone.setDefault(TimeZone.getTimeZone(Asia/Jerusalem));
ExecutorService threadPool = Executors.newFixedThreadPool(1);
try {
FutureBoolean future = threadPool.submit(new
CallableBoolean() {
@Override
public Boolean call() throws Exception {
Date originalDate = new Date(114, 2,
28); // March 28th 2014 - DST begins on this day at 02:00.
DateWritable dateWritable = new
DateWritable(originalDate);
return
originalDate.equals(dateWritable.get());
}
});
assertTrue(future.get());
} finally {
threadPool.shutdown();
TimeZone.setDefault(previousDefault);
}
}
Partitions of type 'date' behave incorrectly with daylight saving time.
---
Key: HIVE-8102
URL: https://issues.apache.org/jira/browse/HIVE-8102
Project: Hive
Issue Type: Bug
Components: Database/Schema, Serializers/Deserializers
Affects Versions: 0.13.0
Reporter: Eli Acherkan
On 2AM on March 28th 2014, Israel went from standard time (GMT+2) to daylight
saving time (GMT+3).
The server's timezone is Asia/Jerusalem. When creating a partition whose key
is 2014-03-28, Hive creates a partition for 2013-03-27 instead:
hive (default) create table test (a int) partitioned by (`b_prt` date);
OK
Time taken: 0.092 seconds
hive (default) alter table test add partition (b_prt='2014-03-28');
OK
Time taken: 0.187 seconds
hive (default) show partitions test;
OK
partition
b_prt=2014-03-27
Time taken: 0.134 seconds, Fetched: 1 row(s)
It seems that the root cause is the behavior of
DateWritable.daysToMillis/dateToDays.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
