[jira] [Work logged] (HIVE-22170) from_unixtime and unix_timestamp should use user session time zone
[ https://issues.apache.org/jira/browse/HIVE-22170?focusedWorklogId=313853=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-313853 ] ASF GitHub Bot logged work on HIVE-22170: - Author: ASF GitHub Bot Created on: 17/Sep/19 18:45 Start Date: 17/Sep/19 18:45 Worklog Time Spent: 10m Work Description: asfgit commented on pull request #764: HIVE-22170 URL: https://github.com/apache/hive/pull/764 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 313853) Time Spent: 1h (was: 50m) > from_unixtime and unix_timestamp should use user session time zone > -- > > Key: HIVE-22170 > URL: https://issues.apache.org/jira/browse/HIVE-22170 > Project: Hive > Issue Type: Bug >Affects Versions: 3.1.0, 4.0.0, 3.2.0, 3.1.1, 3.1.2 >Reporter: Riju Trivedi >Assignee: Jesus Camacho Rodriguez >Priority: Major > Labels: pull-request-available > Fix For: 4.0.0 > > Attachments: HIVE-22170.01.patch, HIVE-22170.02.patch, > HIVE-22170.03.patch, HIVE-22170.04.patch, HIVE-22170.05.patch, > HIVE-22170.06.patch > > Time Spent: 1h > Remaining Estimate: 0h > > According to documentation, that is the expected behavior (since session time > zone was not present, system time zone was being used previously). This was > incorrectly changed by HIVE-12192 / HIVE-20007. This JIRA should fix this > issue. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Work logged] (HIVE-22170) from_unixtime and unix_timestamp should use user session time zone
[
https://issues.apache.org/jira/browse/HIVE-22170?focusedWorklogId=312479=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-312479
]
ASF GitHub Bot logged work on HIVE-22170:
-
Author: ASF GitHub Bot
Created on: 14/Sep/19 02:42
Start Date: 14/Sep/19 02:42
Worklog Time Spent: 10m
Work Description: jcamachor commented on pull request #764: HIVE-22170
URL: https://github.com/apache/hive/pull/764#discussion_r324409418
##
File path:
ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFFromUnixTime.java
##
@@ -0,0 +1,173 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hadoop.hive.ql.udf.generic;
+
+import java.text.SimpleDateFormat;
+import java.time.ZoneId;
+import java.util.Date;
+import java.util.TimeZone;
+import org.apache.commons.lang.StringUtils;
+import org.apache.hadoop.hive.common.type.TimestampTZUtil;
+import org.apache.hadoop.hive.conf.HiveConf;
+import org.apache.hadoop.hive.ql.exec.Description;
+import org.apache.hadoop.hive.ql.exec.MapredContext;
+import org.apache.hadoop.hive.ql.exec.UDFArgumentException;
+import org.apache.hadoop.hive.ql.exec.UDFArgumentLengthException;
+import org.apache.hadoop.hive.ql.metadata.HiveException;
+import org.apache.hadoop.hive.ql.session.SessionState;
+import org.apache.hadoop.hive.serde2.objectinspector.ObjectInspector;
+import org.apache.hadoop.hive.serde2.objectinspector.ObjectInspector.Category;
+import org.apache.hadoop.hive.serde2.objectinspector.ObjectInspectorConverters;
+import
org.apache.hadoop.hive.serde2.objectinspector.ObjectInspectorConverters.Converter;
+import org.apache.hadoop.hive.serde2.objectinspector.PrimitiveObjectInspector;
+import
org.apache.hadoop.hive.serde2.objectinspector.primitive.IntObjectInspector;
+import
org.apache.hadoop.hive.serde2.objectinspector.primitive.LongObjectInspector;
+import
org.apache.hadoop.hive.serde2.objectinspector.primitive.PrimitiveObjectInspectorFactory;
+import org.apache.hadoop.io.Text;
+
+/**
+ * GenericUDFFromUnixTime.
+ *
+ */
+@Description(name = "from_unixtime",
+value = "_FUNC_(unix_time, format) - returns unix_time in the specified
format",
+extended = "Example:\n"
++ " > SELECT _FUNC_(0, '-MM-dd HH:mm:ss') FROM src LIMIT 1;\n"
++ " '1970-01-01 00:00:00'")
+public class GenericUDFFromUnixTime extends GenericUDF {
+
+ private transient IntObjectInspector inputIntOI;
+ private transient LongObjectInspector inputLongOI;
+ private transient Converter inputTextConverter;
+ private transient ZoneId timeZone;
+ private transient final Text result = new Text();
+
+ private transient SimpleDateFormat formatter = new
SimpleDateFormat("-MM-dd HH:mm:ss");
+ private transient String lastFormat = null;
+
+
+ @Override
+ public ObjectInspector initialize(ObjectInspector[] arguments) throws
UDFArgumentException {
+if (arguments.length < 1) {
+ throw new UDFArgumentLengthException("The function " +
getName().toUpperCase() +
+ "requires at least one argument");
+}
+if (arguments.length > 2) {
+ throw new UDFArgumentLengthException("Too many arguments for the
function " + getName().toUpperCase());
+}
+for (ObjectInspector argument : arguments) {
+ if (argument.getCategory() != Category.PRIMITIVE) {
+throw new UDFArgumentException(getName().toUpperCase() +
+" only takes primitive types, got " + argument.getTypeName());
+ }
+}
+
+PrimitiveObjectInspector arg0OI = (PrimitiveObjectInspector) arguments[0];
+switch (arg0OI.getPrimitiveCategory()) {
+ case INT:
+inputIntOI = (IntObjectInspector) arguments[0];
+break;
+ case LONG:
+inputLongOI = (LongObjectInspector) arguments[0];
+break;
+ default:
+throw new UDFArgumentException("The function " +
getName().toUpperCase()
++ " takes only int/long types for first argument. Got Type:" +
arg0OI.getPrimitiveCategory().name());
+}
+
+if (arguments.length == 2) {
+ PrimitiveObjectInspector arg1OI = (PrimitiveObjectInspector)
[jira] [Work logged] (HIVE-22170) from_unixtime and unix_timestamp should use user session time zone
[
https://issues.apache.org/jira/browse/HIVE-22170?focusedWorklogId=312477=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-312477
]
ASF GitHub Bot logged work on HIVE-22170:
-
Author: ASF GitHub Bot
Created on: 14/Sep/19 02:37
Start Date: 14/Sep/19 02:37
Worklog Time Spent: 10m
Work Description: jcamachor commented on pull request #764: HIVE-22170
URL: https://github.com/apache/hive/pull/764#discussion_r324409305
##
File path:
ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFToUnixTimeStamp.java
##
@@ -117,6 +121,19 @@ protected void initializeInput(ObjectInspector[]
arguments) throws UDFArgumentEx
+ " takes only string/date/timestamp/timestampwltz types. Got
Type:" + arg1OI
.getPrimitiveCategory().name());
}
+
+if (timeZone == null) {
+ timeZone = SessionState.get().getConf().getLocalTimeZone();
+ formatter.setTimeZone(TimeZone.getTimeZone(timeZone));
+}
+ }
+
+ public void configure(MapredContext context) {
Review comment:
No, this is the comment of the super method that it overrides (I added the
override notation since it was missing):
```
/**
* Additionally setup GenericUDF with MapredContext before initializing.
* This is only called in runtime of MapRedTask.
*
* @param context context
*/
```
Basically this is needed because we need to set up the timezone from the
configuration when we are about to execute the udf in each of the nodes, but we
only have access to the configuration object through this context. This is
called before calling the initialize method.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 312477)
Time Spent: 40m (was: 0.5h)
> from_unixtime and unix_timestamp should use user session time zone
> --
>
> Key: HIVE-22170
> URL: https://issues.apache.org/jira/browse/HIVE-22170
> Project: Hive
> Issue Type: Bug
>Affects Versions: 3.1.0, 4.0.0, 3.2.0, 3.1.1, 3.1.2
>Reporter: Riju Trivedi
>Assignee: Jesus Camacho Rodriguez
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-22170.01.patch, HIVE-22170.02.patch,
> HIVE-22170.03.patch, HIVE-22170.04.patch
>
> Time Spent: 40m
> Remaining Estimate: 0h
>
> According to documentation, that is the expected behavior (since session time
> zone was not present, system time zone was being used previously). This was
> incorrectly changed by HIVE-12192 / HIVE-20007. This JIRA should fix this
> issue.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Work logged] (HIVE-22170) from_unixtime and unix_timestamp should use user session time zone
[
https://issues.apache.org/jira/browse/HIVE-22170?focusedWorklogId=312475=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-312475
]
ASF GitHub Bot logged work on HIVE-22170:
-
Author: ASF GitHub Bot
Created on: 14/Sep/19 02:22
Start Date: 14/Sep/19 02:22
Worklog Time Spent: 10m
Work Description: jcamachor commented on pull request #764: HIVE-22170
URL: https://github.com/apache/hive/pull/764#discussion_r324408973
##
File path: ql/src/java/org/apache/hadoop/hive/ql/udf/UDFFromUnixTime.java
##
@@ -31,12 +30,14 @@
/**
* UDFFromUnixTime.
*
+ * @deprecated Replaced by {@link
org.apache.hadoop.hive.ql.udf.generic.GenericUDFFromUnixTime}.
Review comment:
Yes, that's a good point. I will remove it from the patch for master and I
will keep it for the backport to 3.x branch in case users are referencing the
UDF directly in their queries.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 312475)
Time Spent: 0.5h (was: 20m)
> from_unixtime and unix_timestamp should use user session time zone
> --
>
> Key: HIVE-22170
> URL: https://issues.apache.org/jira/browse/HIVE-22170
> Project: Hive
> Issue Type: Bug
>Affects Versions: 3.1.0, 4.0.0, 3.2.0, 3.1.1, 3.1.2
>Reporter: Riju Trivedi
>Assignee: Jesus Camacho Rodriguez
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-22170.01.patch, HIVE-22170.02.patch,
> HIVE-22170.03.patch, HIVE-22170.04.patch
>
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> According to documentation, that is the expected behavior (since session time
> zone was not present, system time zone was being used previously). This was
> incorrectly changed by HIVE-12192 / HIVE-20007. This JIRA should fix this
> issue.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Work logged] (HIVE-22170) from_unixtime and unix_timestamp should use user session time zone
[
https://issues.apache.org/jira/browse/HIVE-22170?focusedWorklogId=311845=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-311845
]
ASF GitHub Bot logged work on HIVE-22170:
-
Author: ASF GitHub Bot
Created on: 13/Sep/19 01:58
Start Date: 13/Sep/19 01:58
Worklog Time Spent: 10m
Work Description: vineetgarg02 commented on pull request #764: HIVE-22170
URL: https://github.com/apache/hive/pull/764#discussion_r323992223
##
File path: ql/src/java/org/apache/hadoop/hive/ql/udf/UDFFromUnixTime.java
##
@@ -31,12 +30,14 @@
/**
* UDFFromUnixTime.
*
+ * @deprecated Replaced by {@link
org.apache.hadoop.hive.ql.udf.generic.GenericUDFFromUnixTime}.
Review comment:
Can this class be removed altogether now that we have
`GenericUDFFromUnixTime`?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 311845)
Time Spent: 20m (was: 10m)
> from_unixtime and unix_timestamp should use user session time zone
> --
>
> Key: HIVE-22170
> URL: https://issues.apache.org/jira/browse/HIVE-22170
> Project: Hive
> Issue Type: Bug
>Affects Versions: 3.1.0, 4.0.0, 3.2.0, 3.1.1, 3.1.2
>Reporter: Riju Trivedi
>Assignee: Jesus Camacho Rodriguez
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-22170.01.patch, HIVE-22170.02.patch,
> HIVE-22170.03.patch, HIVE-22170.04.patch
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> According to documentation, that is the expected behavior (since session time
> zone was not present, system time zone was being used previously). This was
> incorrectly changed by HIVE-12192 / HIVE-20007. This JIRA should fix this
> issue.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Work logged] (HIVE-22170) from_unixtime and unix_timestamp should use user session time zone
[
https://issues.apache.org/jira/browse/HIVE-22170?focusedWorklogId=311846=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-311846
]
ASF GitHub Bot logged work on HIVE-22170:
-
Author: ASF GitHub Bot
Created on: 13/Sep/19 01:58
Start Date: 13/Sep/19 01:58
Worklog Time Spent: 10m
Work Description: vineetgarg02 commented on pull request #764: HIVE-22170
URL: https://github.com/apache/hive/pull/764#discussion_r323995778
##
File path:
ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFToUnixTimeStamp.java
##
@@ -117,6 +121,19 @@ protected void initializeInput(ObjectInspector[]
arguments) throws UDFArgumentEx
+ " takes only string/date/timestamp/timestampwltz types. Got
Type:" + arg1OI
.getPrimitiveCategory().name());
}
+
+if (timeZone == null) {
+ timeZone = SessionState.get().getConf().getLocalTimeZone();
+ formatter.setTimeZone(TimeZone.getTimeZone(timeZone));
+}
+ }
+
+ public void configure(MapredContext context) {
Review comment:
Is this for testing purpose only? Can't find who is calling this beside
tests.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 311846)
> from_unixtime and unix_timestamp should use user session time zone
> --
>
> Key: HIVE-22170
> URL: https://issues.apache.org/jira/browse/HIVE-22170
> Project: Hive
> Issue Type: Bug
>Affects Versions: 3.1.0, 4.0.0, 3.2.0, 3.1.1, 3.1.2
>Reporter: Riju Trivedi
>Assignee: Jesus Camacho Rodriguez
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-22170.01.patch, HIVE-22170.02.patch,
> HIVE-22170.03.patch, HIVE-22170.04.patch
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> According to documentation, that is the expected behavior (since session time
> zone was not present, system time zone was being used previously). This was
> incorrectly changed by HIVE-12192 / HIVE-20007. This JIRA should fix this
> issue.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Work logged] (HIVE-22170) from_unixtime and unix_timestamp should use user session time zone
[
https://issues.apache.org/jira/browse/HIVE-22170?focusedWorklogId=311844=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-311844
]
ASF GitHub Bot logged work on HIVE-22170:
-
Author: ASF GitHub Bot
Created on: 13/Sep/19 01:58
Start Date: 13/Sep/19 01:58
Worklog Time Spent: 10m
Work Description: vineetgarg02 commented on pull request #764: HIVE-22170
URL: https://github.com/apache/hive/pull/764#discussion_r324011653
##
File path:
ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFFromUnixTime.java
##
@@ -0,0 +1,173 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hadoop.hive.ql.udf.generic;
+
+import java.text.SimpleDateFormat;
+import java.time.ZoneId;
+import java.util.Date;
+import java.util.TimeZone;
+import org.apache.commons.lang.StringUtils;
+import org.apache.hadoop.hive.common.type.TimestampTZUtil;
+import org.apache.hadoop.hive.conf.HiveConf;
+import org.apache.hadoop.hive.ql.exec.Description;
+import org.apache.hadoop.hive.ql.exec.MapredContext;
+import org.apache.hadoop.hive.ql.exec.UDFArgumentException;
+import org.apache.hadoop.hive.ql.exec.UDFArgumentLengthException;
+import org.apache.hadoop.hive.ql.metadata.HiveException;
+import org.apache.hadoop.hive.ql.session.SessionState;
+import org.apache.hadoop.hive.serde2.objectinspector.ObjectInspector;
+import org.apache.hadoop.hive.serde2.objectinspector.ObjectInspector.Category;
+import org.apache.hadoop.hive.serde2.objectinspector.ObjectInspectorConverters;
+import
org.apache.hadoop.hive.serde2.objectinspector.ObjectInspectorConverters.Converter;
+import org.apache.hadoop.hive.serde2.objectinspector.PrimitiveObjectInspector;
+import
org.apache.hadoop.hive.serde2.objectinspector.primitive.IntObjectInspector;
+import
org.apache.hadoop.hive.serde2.objectinspector.primitive.LongObjectInspector;
+import
org.apache.hadoop.hive.serde2.objectinspector.primitive.PrimitiveObjectInspectorFactory;
+import org.apache.hadoop.io.Text;
+
+/**
+ * GenericUDFFromUnixTime.
+ *
+ */
+@Description(name = "from_unixtime",
+value = "_FUNC_(unix_time, format) - returns unix_time in the specified
format",
+extended = "Example:\n"
++ " > SELECT _FUNC_(0, '-MM-dd HH:mm:ss') FROM src LIMIT 1;\n"
++ " '1970-01-01 00:00:00'")
+public class GenericUDFFromUnixTime extends GenericUDF {
+
+ private transient IntObjectInspector inputIntOI;
+ private transient LongObjectInspector inputLongOI;
+ private transient Converter inputTextConverter;
+ private transient ZoneId timeZone;
+ private transient final Text result = new Text();
+
+ private transient SimpleDateFormat formatter = new
SimpleDateFormat("-MM-dd HH:mm:ss");
+ private transient String lastFormat = null;
+
+
+ @Override
+ public ObjectInspector initialize(ObjectInspector[] arguments) throws
UDFArgumentException {
+if (arguments.length < 1) {
+ throw new UDFArgumentLengthException("The function " +
getName().toUpperCase() +
+ "requires at least one argument");
+}
+if (arguments.length > 2) {
+ throw new UDFArgumentLengthException("Too many arguments for the
function " + getName().toUpperCase());
+}
+for (ObjectInspector argument : arguments) {
+ if (argument.getCategory() != Category.PRIMITIVE) {
+throw new UDFArgumentException(getName().toUpperCase() +
+" only takes primitive types, got " + argument.getTypeName());
+ }
+}
+
+PrimitiveObjectInspector arg0OI = (PrimitiveObjectInspector) arguments[0];
+switch (arg0OI.getPrimitiveCategory()) {
+ case INT:
+inputIntOI = (IntObjectInspector) arguments[0];
+break;
+ case LONG:
+inputLongOI = (LongObjectInspector) arguments[0];
+break;
+ default:
+throw new UDFArgumentException("The function " +
getName().toUpperCase()
++ " takes only int/long types for first argument. Got Type:" +
arg0OI.getPrimitiveCategory().name());
+}
+
+if (arguments.length == 2) {
+ PrimitiveObjectInspector arg1OI = (PrimitiveObjectInspector)
[jira] [Work logged] (HIVE-22170) from_unixtime and unix_timestamp should use user session time zone
[ https://issues.apache.org/jira/browse/HIVE-22170?focusedWorklogId=310166=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-310166 ] ASF GitHub Bot logged work on HIVE-22170: - Author: ASF GitHub Bot Created on: 10/Sep/19 22:55 Start Date: 10/Sep/19 22:55 Worklog Time Spent: 10m Work Description: jcamachor commented on pull request #764: HIVE-22170 URL: https://github.com/apache/hive/pull/764 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 310166) Remaining Estimate: 0h Time Spent: 10m > from_unixtime and unix_timestamp should use user session time zone > -- > > Key: HIVE-22170 > URL: https://issues.apache.org/jira/browse/HIVE-22170 > Project: Hive > Issue Type: Bug >Affects Versions: 3.1.0, 4.0.0, 3.2.0, 3.1.1, 3.1.2 >Reporter: Riju Trivedi >Assignee: Jesus Camacho Rodriguez >Priority: Major > Labels: pull-request-available > Attachments: HIVE-22170.01.patch, HIVE-22170.02.patch, > HIVE-22170.03.patch > > Time Spent: 10m > Remaining Estimate: 0h > > According to documentation, that is the expected behavior (since session time > zone was not present, system time zone was being used previously). This was > incorrectly changed by HIVE-12192 / HIVE-20007. This JIRA should fix this > issue. -- This message was sent by Atlassian Jira (v8.3.2#803003)
