[jira] (DRILL-5034) Select timestamp from hive generated parquet always return in UTC

[ASF GitHub Bot (JIRA)]

Title: Message Title

ASF GitHub Bot commented on DRILL-5034   Re: Select timestamp from hive generated parquet always return in UTC Github user vdiravka commented on a diff in the pull request: https://github.com/apache/drill/pull/656#discussion_r98358344 — Diff: exec/java-exec/src/main/java/org/apache/drill/exec/store/parquet/ParquetReaderUtility.java — @@ -323,18 +323,28 @@ public static DateCorruptionStatus checkForCorruptDateValuesInStatistics(Parquet @param binaryTimeStampValue hive, impala timestamp values with nanoseconds precision are stored in parquet Binary as INT96 (12 constant bytes) * + * @param retainLocalTimezone + * parquet files don't keep local timeZone according to the + * <a href="" spec</a>, + * but some tools (hive, for example) retain local timezone for parquet files by default + * Note: Impala doesn't retain local timezone by default @return Unix Timestamp - the number of milliseconds since January 1, 1970, 00:00:00 GMT represented by @param binaryTimeStampValue . */ public static long getDateTimeValueFromBinary(Binary binaryTimeStampValue) { + public static long getDateTimeValueFromBinary(Binary binaryTimeStampValue, boolean retainLocalTimezone) { // This method represents binaryTimeStampValue as ByteBuffer, where timestamp is stored as sum of // julian day number (32-bit) and nanos of day (64-bit) NanoTime nt = NanoTime.fromBinary(binaryTimeStampValue); int julianDay = nt.getJulianDay(); long nanosOfDay = nt.getTimeOfDayNanos(); return (julianDay - JULIAN_DAY_NUMBER_FOR_UNIX_EPOCH) * DateTimeConstants.MILLIS_PER_DAY + long dateTime = (julianDay - JULIAN_DAY_NUMBER_FOR_UNIX_EPOCH) * DateTimeConstants.MILLIS_PER_DAY + nanosOfDay / NANOS_PER_MILLISECOND; + if (retainLocalTimezone) { + return new org.joda.time.DateTime(dateTime, org.joda.time.chrono.JulianChronology.getInstance()) + .withZoneRetainFields(org.joda.time.DateTimeZone.UTC).getMillis(); End diff – `withZoneRetainFields` method calculates the difference between local timezone and UTC (parameter of that method) and returns original dateTime with a shift of that difference. This approach is used frequently in drill code. But thinking a little more on this I decided that it is possible to use more simpler statement, without creating DateTime object. `DateTimeZone.getDefault().convertUTCToLocal(dateTime)`. I think it's more clear. Add Comment   This message was sent by Atlassian JIRA (v6.3.15#6346-sha1:dbc023d)