[GitHub] drill pull request #732: DRILL-5230: Translation of millisecond duration int...

[kkhatua]

GitHub user kkhatua opened a pull request:

https://github.com/apache/drill/pull/732

DRILL-5230: Translation of millisecond duration into hours is incorrect

Fixed invalid representation of readable elapsed time using `TimeUnit` 
class in JDK.
e.g. 4545 sec is now correctly translated as `1h15m` instead of `17h15m`

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/kkhatua/drill DRILL-5230

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/drill/pull/732.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #732


commit 1ced57de095c5e7f19f4213002f6a9eb4418f899
Author: Kunal Khatua 
Date:   2017-01-30T07:08:12Z

DRILL-5230: Translation of millisecond duration into hours is incorrect
Fixed invalid representation of readable elapsed time using `TimeUnit` 
class in JDK.
e.g. 4545 sec is now correctly translated as `1h15m` instead of `17h15m`




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] drill pull request #656: DRILL-5034: Select timestamp from hive generated pa...

[vdiravka]

Github user vdiravka commented on a diff in the pull request:

https://github.com/apache/drill/pull/656#discussion_r98358344
  
--- Diff: 
exec/java-exec/src/main/java/org/apache/drill/exec/store/parquet/ParquetReaderUtility.java
 ---
@@ -323,18 +323,28 @@ public static DateCorruptionStatus 
checkForCorruptDateValuesInStatistics(Parquet
* @param binaryTimeStampValue
*  hive, impala timestamp values with nanoseconds precision
*  are stored in parquet Binary as INT96 (12 constant bytes)
-   *
+   * @param retainLocalTimezone
+   *  parquet files don't keep local timeZone according to the
+   *  https://github.com/Parquet/parquet-format/blob/master/LogicalTypes.md#timestamp;>Parquet
 spec,
+   *  but some tools (hive, for example) retain local timezone for 
parquet files by default
+   *  Note: Impala doesn't retain local timezone by default
* @return  Unix Timestamp - the number of milliseconds since January 1, 
1970, 00:00:00 GMT
*  represented by @param binaryTimeStampValue .
*/
-public static long getDateTimeValueFromBinary(Binary 
binaryTimeStampValue) {
+public static long getDateTimeValueFromBinary(Binary 
binaryTimeStampValue, boolean retainLocalTimezone) {
   // This method represents binaryTimeStampValue as ByteBuffer, where 
timestamp is stored as sum of
   // julian day number (32-bit) and nanos of day (64-bit)
   NanoTime nt = NanoTime.fromBinary(binaryTimeStampValue);
   int julianDay = nt.getJulianDay();
   long nanosOfDay = nt.getTimeOfDayNanos();
-  return (julianDay - JULIAN_DAY_NUMBER_FOR_UNIX_EPOCH) * 
DateTimeConstants.MILLIS_PER_DAY
+  long dateTime = (julianDay - JULIAN_DAY_NUMBER_FOR_UNIX_EPOCH) * 
DateTimeConstants.MILLIS_PER_DAY
   + nanosOfDay / NANOS_PER_MILLISECOND;
+  if (retainLocalTimezone) {
+return new org.joda.time.DateTime(dateTime, 
org.joda.time.chrono.JulianChronology.getInstance())
+
.withZoneRetainFields(org.joda.time.DateTimeZone.UTC).getMillis();
--- End diff --

`withZoneRetainFields` method calculates the difference between local 
timezone and UTC (parameter of that method) and returns original dateTime with 
a shift of that difference. This approach is used frequently in drill code.
But thinking a little more on this I decided that it is possible to use 
more simpler statement, without creating DateTime object. 
`DateTimeZone.getDefault().convertUTCToLocal(dateTime)`. I think it's more 
clear.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] drill pull request #685: Drill 5043: Function that returns a unique id per s...

[nagarajanchinnasamy]

Github user nagarajanchinnasamy commented on a diff in the pull request:

https://github.com/apache/drill/pull/685#discussion_r98355138
  
--- Diff: 
exec/java-exec/src/main/java/org/apache/drill/exec/expr/fn/impl/ContextFunctions.java
 ---
@@ -64,17 +65,45 @@ public void eval() {
 @Inject DrillBuf buffer;
 @Workspace int currentSchemaBytesLength;
 
+@Override
 public void setup() {
   final byte[] currentSchemaBytes = 
contextInfo.getCurrentDefaultSchema().getBytes();
   buffer = buffer.reallocIfNeeded(currentSchemaBytes.length);
   currentSchemaBytesLength= currentSchemaBytes.length;
   buffer.setBytes(0, currentSchemaBytes);
 }
 
+@Override
 public void eval() {
   out.start = 0;
   out.end = currentSchemaBytesLength;
   out.buffer = buffer;
 }
   }
+
+  /**
+   * Implement "session_id" function. Returns the unique id of the current 
session.
+   */
+  @FunctionTemplate(name = "session_id", scope = 
FunctionTemplate.FunctionScope.SIMPLE, isNiladic = true)
--- End diff --

@jinfengni @arina-ielchiieva am in the process of adding `isNiladic=true` 
to the standard functions. I am referring to 
[SqlStdOperatorTable.java](https://github.com/apache/calcite/blob/master/core/src/main/java/org/apache/calcite/sql/fun/SqlStdOperatorTable.java)
 in Calcite and tried to find the equivalent functions in Drill.

I have marked following functions from 
[DateTypeFunctions.java](https://github.com/apache/drill/blob/master/exec/java-exec/src/main/java/org/apache/drill/exec/expr/fn/impl/DateTypeFunctions.java)
 and 
[ContextFunctions.java](https://github.com/apache/drill/blob/master/exec/java-exec/src/main/java/org/apache/drill/exec/expr/fn/impl/ContextFunctions.java)
 as isNiladic=true.

1. "current_date"
2. "timeofday" (**is this a niladic function? Not found in Calcite**)
3. {"localtimestamp", "current_timestamp", "now", "statement_timestamp", 
"transaction_timestamp"}
4. {"current_time", "localtime"}
5. "unix_timestamp" (**is this a niladic function? Not found in Calcite**)
6. {"user", "session_user", "system_user"}
7. "current_schema"
8. "session_id"

Please check the above list... and let me know if I should add more. Also, 
I should not be marking something niladic if it is not as that will break the 
existing queries.



---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---