[jira] [Work logged] (HIVE-25314) Implement SHOW CREATE TABLE command for Iceberg tables
[
https://issues.apache.org/jira/browse/HIVE-25314?focusedWorklogId=620865=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-620865
]
ASF GitHub Bot logged work on HIVE-25314:
-
Author: ASF GitHub Bot
Created on: 09/Jul/21 05:22
Start Date: 09/Jul/21 05:22
Worklog Time Spent: 10m
Work Description: lcspinter commented on a change in pull request #2458:
URL: https://github.com/apache/hive/pull/2458#discussion_r78521
##
File path:
ql/src/java/org/apache/hadoop/hive/ql/metadata/HiveStorageHandler.java
##
@@ -257,6 +258,16 @@ default boolean supportsPartitionTransform() {
return false;
}
+ /**
+ * Return the partition transform specification table property key together
with the specification value in json
Review comment:
Correct. Fixed it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 620865)
Time Spent: 1h 20m (was: 1h 10m)
> Implement SHOW CREATE TABLE command for Iceberg tables
> --
>
> Key: HIVE-25314
> URL: https://issues.apache.org/jira/browse/HIVE-25314
> Project: Hive
> Issue Type: Improvement
>Reporter: László Pintér
>Assignee: László Pintér
>Priority: Major
> Labels: pull-request-available
> Time Spent: 1h 20m
> Remaining Estimate: 0h
>
> Implement SHOW CREATE TABLE, including the partitioning info as well in the
> output (even though the HMS table is unpartitioned). That would probably
> require loading the Iceberg table to acquire the partitioning information.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (HIVE-25314) Implement SHOW CREATE TABLE command for Iceberg tables
[
https://issues.apache.org/jira/browse/HIVE-25314?focusedWorklogId=620864=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-620864
]
ASF GitHub Bot logged work on HIVE-25314:
-
Author: ASF GitHub Bot
Created on: 09/Jul/21 05:22
Start Date: 09/Jul/21 05:22
Worklog Time Spent: 10m
Work Description: lcspinter commented on a change in pull request #2458:
URL: https://github.com/apache/hive/pull/2458#discussion_r78405
##
File path:
iceberg/iceberg-handler/src/main/java/org/apache/iceberg/mr/hive/HiveIcebergStorageHandler.java
##
@@ -270,6 +270,15 @@ public boolean supportsPartitionTransform() {
return true;
}
+ @Override
+ public Map
getPartitionTransformSpecProperty(org.apache.hadoop.hive.ql.metadata.Table
hmsTable) {
+Map property = new HashMap<>();
+TableDesc tableDesc = Utilities.getTableDesc(hmsTable);
+Table table = Catalogs.loadTable(conf, tableDesc.getProperties());
Review comment:
The iceberg table is only loaded once, so changing the call to
`IcebergTableUtil#getTable()` doesn't add much value at the moment. But I
guess, we should consider that this might change in the future.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 620864)
Time Spent: 1h 10m (was: 1h)
> Implement SHOW CREATE TABLE command for Iceberg tables
> --
>
> Key: HIVE-25314
> URL: https://issues.apache.org/jira/browse/HIVE-25314
> Project: Hive
> Issue Type: Improvement
>Reporter: László Pintér
>Assignee: László Pintér
>Priority: Major
> Labels: pull-request-available
> Time Spent: 1h 10m
> Remaining Estimate: 0h
>
> Implement SHOW CREATE TABLE, including the partitioning info as well in the
> output (even though the HMS table is unpartitioned). That would probably
> require loading the Iceberg table to acquire the partitioning information.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (HIVE-25314) Implement SHOW CREATE TABLE command for Iceberg tables
[
https://issues.apache.org/jira/browse/HIVE-25314?focusedWorklogId=620862=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-620862
]
ASF GitHub Bot logged work on HIVE-25314:
-
Author: ASF GitHub Bot
Created on: 09/Jul/21 05:10
Start Date: 09/Jul/21 05:10
Worklog Time Spent: 10m
Work Description: lcspinter commented on a change in pull request #2458:
URL: https://github.com/apache/hive/pull/2458#discussion_r74565
##
File path:
ql/src/java/org/apache/hadoop/hive/ql/metadata/HiveStorageHandler.java
##
@@ -257,6 +258,16 @@ default boolean supportsPartitionTransform() {
return false;
}
+ /**
+ * Return the partition transform specification table property key together
with the specification value in json
Review comment:
Good point will add that.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 620862)
Time Spent: 1h (was: 50m)
> Implement SHOW CREATE TABLE command for Iceberg tables
> --
>
> Key: HIVE-25314
> URL: https://issues.apache.org/jira/browse/HIVE-25314
> Project: Hive
> Issue Type: Improvement
>Reporter: László Pintér
>Assignee: László Pintér
>Priority: Major
> Labels: pull-request-available
> Time Spent: 1h
> Remaining Estimate: 0h
>
> Implement SHOW CREATE TABLE, including the partitioning info as well in the
> output (even though the HMS table is unpartitioned). That would probably
> require loading the Iceberg table to acquire the partitioning information.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Resolved] (HIVE-25131) PreAlterPartitionEvent should have table owner details that can authorized in ranger/sentry
[ https://issues.apache.org/jira/browse/HIVE-25131?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Naveen Gangam resolved HIVE-25131. -- Fix Version/s: 4.0.0 Resolution: Fixed Fix has been committed to master. Thank you for the contribution [~hemanth619] > PreAlterPartitionEvent should have table owner details that can authorized in > ranger/sentry > > > Key: HIVE-25131 > URL: https://issues.apache.org/jira/browse/HIVE-25131 > Project: Hive > Issue Type: Bug >Reporter: Sai Hemanth Gantasala >Assignee: Sai Hemanth Gantasala >Priority: Major > Labels: pull-request-available > Fix For: 4.0.0 > > Time Spent: 40m > Remaining Estimate: 0h > > PreAlterPartition event should have a table object, so that the call can be > authorized in ranger/sentry using the owner details of the table object. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (HIVE-25131) PreAlterPartitionEvent should have table owner details that can authorized in ranger/sentry
[ https://issues.apache.org/jira/browse/HIVE-25131?focusedWorklogId=620856=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-620856 ] ASF GitHub Bot logged work on HIVE-25131: - Author: ASF GitHub Bot Created on: 09/Jul/21 04:49 Start Date: 09/Jul/21 04:49 Worklog Time Spent: 10m Work Description: nrg4878 commented on pull request #2449: URL: https://github.com/apache/hive/pull/2449#issuecomment-876910750 fix has been pushed to master. Please close the PR. Thanks -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 620856) Time Spent: 40m (was: 0.5h) > PreAlterPartitionEvent should have table owner details that can authorized in > ranger/sentry > > > Key: HIVE-25131 > URL: https://issues.apache.org/jira/browse/HIVE-25131 > Project: Hive > Issue Type: Bug >Reporter: Sai Hemanth Gantasala >Assignee: Sai Hemanth Gantasala >Priority: Major > Labels: pull-request-available > Time Spent: 40m > Remaining Estimate: 0h > > PreAlterPartition event should have a table object, so that the call can be > authorized in ranger/sentry using the owner details of the table object. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Comment Edited] (HIVE-19551) Enable TestBeeLineWithArgs#testQueryProgress and TestBeeLineWithArgs#testQueryProgressParallel
[ https://issues.apache.org/jira/browse/HIVE-19551?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17377766#comment-17377766 ] Zhihua Deng edited comment on HIVE-19551 at 7/9/21, 3:46 AM: - Resolved via HIVE-24901 was (Author: dengzh): Resoved by HIVE-24901 > Enable TestBeeLineWithArgs#testQueryProgress and > TestBeeLineWithArgs#testQueryProgressParallel > -- > > Key: HIVE-19551 > URL: https://issues.apache.org/jira/browse/HIVE-19551 > Project: Hive > Issue Type: Test > Components: Test >Affects Versions: 3.1.0 >Reporter: Jesus Camacho Rodriguez >Assignee: Zhihua Deng >Priority: Critical > Fix For: 4.0.0 > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (HIVE-19551) Enable TestBeeLineWithArgs#testQueryProgress and TestBeeLineWithArgs#testQueryProgressParallel
[ https://issues.apache.org/jira/browse/HIVE-19551?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhihua Deng resolved HIVE-19551. Fix Version/s: 4.0.0 Resolution: Fixed Resoved by HIVE-24901 > Enable TestBeeLineWithArgs#testQueryProgress and > TestBeeLineWithArgs#testQueryProgressParallel > -- > > Key: HIVE-19551 > URL: https://issues.apache.org/jira/browse/HIVE-19551 > Project: Hive > Issue Type: Test > Components: Test >Affects Versions: 3.1.0 >Reporter: Jesus Camacho Rodriguez >Assignee: Zhihua Deng >Priority: Critical > Fix For: 4.0.0 > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Assigned] (HIVE-19551) Enable TestBeeLineWithArgs#testQueryProgress and TestBeeLineWithArgs#testQueryProgressParallel
[ https://issues.apache.org/jira/browse/HIVE-19551?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhihua Deng reassigned HIVE-19551: -- Assignee: Zhihua Deng > Enable TestBeeLineWithArgs#testQueryProgress and > TestBeeLineWithArgs#testQueryProgressParallel > -- > > Key: HIVE-19551 > URL: https://issues.apache.org/jira/browse/HIVE-19551 > Project: Hive > Issue Type: Test > Components: Test >Affects Versions: 3.1.0 >Reporter: Jesus Camacho Rodriguez >Assignee: Zhihua Deng >Priority: Critical > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (HIVE-25294) Optimise the metadata count queries for local mode
[ https://issues.apache.org/jira/browse/HIVE-25294?focusedWorklogId=620802=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-620802 ] ASF GitHub Bot logged work on HIVE-25294: - Author: ASF GitHub Bot Created on: 09/Jul/21 00:11 Start Date: 09/Jul/21 00:11 Worklog Time Spent: 10m Work Description: dengzhhu653 opened a new pull request #2435: URL: https://github.com/apache/hive/pull/2435 ### What changes were proposed in this pull request? ### Why are the changes needed? ### Does this PR introduce _any_ user-facing change? ### How was this patch tested? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 620802) Time Spent: 40m (was: 0.5h) > Optimise the metadata count queries for local mode > -- > > Key: HIVE-25294 > URL: https://issues.apache.org/jira/browse/HIVE-25294 > Project: Hive > Issue Type: Improvement > Components: Standalone Metastore >Reporter: Zhihua Deng >Assignee: Zhihua Deng >Priority: Major > Labels: pull-request-available > Time Spent: 40m > Remaining Estimate: 0h > > When Metastore is in local mode, the client uses his own private HMSHandler > to get the meta data, the HMSHandler should be initialized before being > ready to serve. When the metrics is enabled, HMSHandler will count the number > of db, table, partitions, which cloud lead to some problems. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (HIVE-25294) Optimise the metadata count queries for local mode
[ https://issues.apache.org/jira/browse/HIVE-25294?focusedWorklogId=620801=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-620801 ] ASF GitHub Bot logged work on HIVE-25294: - Author: ASF GitHub Bot Created on: 09/Jul/21 00:09 Start Date: 09/Jul/21 00:09 Worklog Time Spent: 10m Work Description: dengzhhu653 closed pull request #2435: URL: https://github.com/apache/hive/pull/2435 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 620801) Time Spent: 0.5h (was: 20m) > Optimise the metadata count queries for local mode > -- > > Key: HIVE-25294 > URL: https://issues.apache.org/jira/browse/HIVE-25294 > Project: Hive > Issue Type: Improvement > Components: Standalone Metastore >Reporter: Zhihua Deng >Assignee: Zhihua Deng >Priority: Major > Labels: pull-request-available > Time Spent: 0.5h > Remaining Estimate: 0h > > When Metastore is in local mode, the client uses his own private HMSHandler > to get the meta data, the HMSHandler should be initialized before being > ready to serve. When the metrics is enabled, HMSHandler will count the number > of db, table, partitions, which cloud lead to some problems. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (HIVE-24483) Bump protobuf version to 3.12.0
[ https://issues.apache.org/jira/browse/HIVE-24483?focusedWorklogId=620799=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-620799 ] ASF GitHub Bot logged work on HIVE-24483: - Author: ASF GitHub Bot Created on: 09/Jul/21 00:08 Start Date: 09/Jul/21 00:08 Worklog Time Spent: 10m Work Description: github-actions[bot] closed pull request #1975: URL: https://github.com/apache/hive/pull/1975 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 620799) Time Spent: 2h 20m (was: 2h 10m) > Bump protobuf version to 3.12.0 > --- > > Key: HIVE-24483 > URL: https://issues.apache.org/jira/browse/HIVE-24483 > Project: Hive > Issue Type: Improvement >Reporter: Cameron Moberg >Priority: Minor > Labels: pull-request-available > Time Spent: 2h 20m > Remaining Estimate: 0h > > The following protoc version's used in hive is very old i.e. 2.5.0 > [https://repo.maven.apache.org/maven2/com/google/protobuf/protoc/] . The > v2.5.0 does not have aarch64 support. But the AArch64 support started from > v3.5.0 on-words in google's protobuf project release. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (HIVE-25317) Relocate dependencies in shaded hive-exec module
[ https://issues.apache.org/jira/browse/HIVE-25317?focusedWorklogId=620674=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-620674 ] ASF GitHub Bot logged work on HIVE-25317: - Author: ASF GitHub Bot Created on: 08/Jul/21 19:31 Start Date: 08/Jul/21 19:31 Worklog Time Spent: 10m Work Description: viirya opened a new pull request #2459: URL: https://github.com/apache/hive/pull/2459 ### What changes were proposed in this pull request? Trying to relocate dependencies which could conflict Spark. ### Why are the changes needed? When we want to use shaded version of hive-exec (i.e., w/o classifier), more dependencies conflict with Spark. We need to relocate these dependencies too. ### Does this PR introduce _any_ user-facing change? If previously downstream projects rely on included dependencies in shaded release, they might need to explicitly include these dependencies after the relocation here. ### How was this patch tested? CI -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 620674) Remaining Estimate: 0h Time Spent: 10m > Relocate dependencies in shaded hive-exec module > > > Key: HIVE-25317 > URL: https://issues.apache.org/jira/browse/HIVE-25317 > Project: Hive > Issue Type: Improvement >Affects Versions: 2.3.8 >Reporter: L. C. Hsieh >Priority: Major > Time Spent: 10m > Remaining Estimate: 0h > > When we want to use shaded version of hive-exec (i.e., w/o classifier), more > dependencies conflict with Spark. We need to relocate these dependencies too. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-25317) Relocate dependencies in shaded hive-exec module
[ https://issues.apache.org/jira/browse/HIVE-25317?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HIVE-25317: -- Labels: pull-request-available (was: ) > Relocate dependencies in shaded hive-exec module > > > Key: HIVE-25317 > URL: https://issues.apache.org/jira/browse/HIVE-25317 > Project: Hive > Issue Type: Improvement >Affects Versions: 2.3.8 >Reporter: L. C. Hsieh >Priority: Major > Labels: pull-request-available > Time Spent: 10m > Remaining Estimate: 0h > > When we want to use shaded version of hive-exec (i.e., w/o classifier), more > dependencies conflict with Spark. We need to relocate these dependencies too. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (HIVE-25303) CTAS hive.create.as.external.legacy tries to place data files in managed WH path
[
https://issues.apache.org/jira/browse/HIVE-25303?focusedWorklogId=620669=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-620669
]
ASF GitHub Bot logged work on HIVE-25303:
-
Author: ASF GitHub Bot
Created on: 08/Jul/21 19:22
Start Date: 08/Jul/21 19:22
Worklog Time Spent: 10m
Work Description: saihemanth-cloudera commented on a change in pull
request #2442:
URL: https://github.com/apache/hive/pull/2442#discussion_r666459224
##
File path: ql/src/java/org/apache/hadoop/hive/ql/parse/TaskCompiler.java
##
@@ -472,6 +473,25 @@ private void setLoadFileLocation(
loc = cmv.getLocation();
}
Path location = (loc == null) ? getDefaultCtasLocation(pCtx) : new
Path(loc);
+boolean isExternal = false;
+boolean isAcid = false;
+if (pCtx.getQueryProperties().isCTAS()) {
+ isExternal = pCtx.getCreateTable().isExternal();
+ isAcid = pCtx.getCreateTable().getTblProps().getOrDefault(
+ hive_metastoreConstants.TABLE_IS_TRANSACTIONAL,
"false").equalsIgnoreCase("true") ||
+
pCtx.getCreateTable().getTblProps().containsKey(hive_metastoreConstants.TABLE_TRANSACTIONAL_PROPERTIES);
+ if(location != null && (HiveConf.getBoolVar(conf,
HiveConf.ConfVars.CREATE_TABLE_AS_EXTERNAL) || (isExternal || !isAcid))){
+CreateTableDesc ctd = pCtx.getCreateTable();
+ctd.setLocation(location.toString());
+
if(ctd.getSerdeProps().containsKey(hive_metastoreConstants.TABLE_TRANSACTIONAL_PROPERTIES)){
+
ctd.getSerdeProps().remove(hive_metastoreConstants.TABLE_TRANSACTIONAL_PROPERTIES);
Review comment:
Yes this is being set in the SemanticAnalyzer#L13187. In the compilation
phase if it determines that this is a managed table, ACID properties are set in
the serde. I'll do the ctd.getSerdeProps().remove().
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 620669)
Time Spent: 1h (was: 50m)
> CTAS hive.create.as.external.legacy tries to place data files in managed WH
> path
>
>
> Key: HIVE-25303
> URL: https://issues.apache.org/jira/browse/HIVE-25303
> Project: Hive
> Issue Type: Bug
> Components: HiveServer2, Standalone Metastore
>Reporter: Sai Hemanth Gantasala
>Assignee: Sai Hemanth Gantasala
>Priority: Major
> Labels: pull-request-available
> Time Spent: 1h
> Remaining Estimate: 0h
>
> Under legacy table creation mode (hive.create.as.external.legacy=true), when
> a database has been created in a specific LOCATION, in a session where that
> database is USEd, tables created using
> CREATE TABLE AS SELECT
> should inherit the HDFS path from the database's location.
> Instead, Hive is trying to write the table data into
> /warehouse/tablespace/managed/hive//
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (HIVE-25130) alter table concat gives NullPointerException, when data is inserted from Spark
[ https://issues.apache.org/jira/browse/HIVE-25130?focusedWorklogId=620663=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-620663 ] ASF GitHub Bot logged work on HIVE-25130: - Author: ASF GitHub Bot Created on: 08/Jul/21 18:53 Start Date: 08/Jul/21 18:53 Worklog Time Spent: 10m Work Description: kishendas closed pull request #2285: URL: https://github.com/apache/hive/pull/2285 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 620663) Time Spent: 1h 10m (was: 1h) > alter table concat gives NullPointerException, when data is inserted from > Spark > --- > > Key: HIVE-25130 > URL: https://issues.apache.org/jira/browse/HIVE-25130 > Project: Hive > Issue Type: Bug >Reporter: Kishen Das >Assignee: Kishen Das >Priority: Major > Labels: pull-request-available > Time Spent: 1h 10m > Remaining Estimate: 0h > > This is the complete stack trace of the NullPointerException > 2021-03-01 14:50:32,201 ERROR org.apache.hadoop.hive.ql.exec.Task: > [HiveServer2-Background-Pool: Thread-76760]: Job Commit failed with exception > 'java.lang.NullPointerException(null)' > java.lang.NullPointerException > at > org.apache.hadoop.hive.ql.exec.Utilities.getAttemptIdFromFilename(Utilities.java:1333) > at > org.apache.hadoop.hive.ql.exec.Utilities.compareTempOrDuplicateFiles(Utilities.java:1966) > at > org.apache.hadoop.hive.ql.exec.Utilities.ponderRemovingTempOrDuplicateFile(Utilities.java:1907) > at > org.apache.hadoop.hive.ql.exec.Utilities.removeTempOrDuplicateFilesNonMm(Utilities.java:1892) > at > org.apache.hadoop.hive.ql.exec.Utilities.removeTempOrDuplicateFiles(Utilities.java:1797) > at > org.apache.hadoop.hive.ql.exec.Utilities.removeTempOrDuplicateFiles(Utilities.java:1674) > at > org.apache.hadoop.hive.ql.exec.Utilities.mvFileToFinalPath(Utilities.java:1544) > at > org.apache.hadoop.hive.ql.exec.AbstractFileMergeOperator.jobCloseOp(AbstractFileMergeOperator.java:304) > at org.apache.hadoop.hive.ql.exec.Operator.jobClose(Operator.java:798) > at org.apache.hadoop.hive.ql.exec.tez.TezTask.close(TezTask.java:637) > at org.apache.hadoop.hive.ql.exec.tez.TezTask.execute(TezTask.java:335) > at > org.apache.hadoop.hive.ql.ddl.table.storage.concatenate.AlterTableConcatenateOperation.executeTask(AlterTableConcatenateOperation.java:129) > at > org.apache.hadoop.hive.ql.ddl.table.storage.concatenate.AlterTableConcatenateOperation.execute(AlterTableConcatenateOperation.java:63) > at org.apache.hadoop.hive.ql.ddl.DDLTask.execute(DDLTask.java:80) > at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:213) > at > org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:105) > at org.apache.hadoop.hive.ql.Executor.launchTask(Executor.java:357) > at org.apache.hadoop.hive.ql.Executor.launchTasks(Executor.java:330) > at org.apache.hadoop.hive.ql.Executor.runTasks(Executor.java:246) > at org.apache.hadoop.hive.ql.Executor.execute(Executor.java:109) > at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:740) > at org.apache.hadoop.hive.ql.Driver.run(Driver.java:495) > at org.apache.hadoop.hive.ql.Driver.run(Driver.java:489) > at org.apache.hadoop.hive.ql.reexec.ReExecDriver.run(ReExecDriver.java:166) > at > org.apache.hive.service.cli.operation.SQLOperation.runQuery(SQLOperation.java:225) > at > org.apache.hive.service.cli.operation.SQLOperation.access$700(SQLOperation.java:87) > at > org.apache.hive.service.cli.operation.SQLOperation$BackgroundWork$1.run(SQLOperation.java:322) > at java.security.AccessController.doPrivileged(Native Method) > at javax.security.auth.Subject.doAs(Subject.java:422) > at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1898) > at > org.apache.hive.service.cli.operation.SQLOperation$BackgroundWork.run(SQLOperation.java:340) > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) > at java.util.concurrent.FutureTask.run(FutureTask.java:266) > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) > at java.util.concurrent.FutureTask.run(FutureTask.java:266) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) > at java.lang.Thread.run(Thread.java:748) -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (HIVE-25282) Drop/Alter table in REMOTE db should fail
[
https://issues.apache.org/jira/browse/HIVE-25282?focusedWorklogId=620633=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-620633
]
ASF GitHub Bot logged work on HIVE-25282:
-
Author: ASF GitHub Bot
Created on: 08/Jul/21 17:48
Start Date: 08/Jul/21 17:48
Worklog Time Spent: 10m
Work Description: nrg4878 commented on a change in pull request #2450:
URL: https://github.com/apache/hive/pull/2450#discussion_r666401686
##
File path:
standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/HMSHandler.java
##
@@ -5934,6 +5936,12 @@ private void alter_table_core(String catName, String
dbname, String name, Table
catName = MetaStoreUtils.getDefaultCatalog(conf);
}
+// HIVE-25282: Drop/Alter table in REMOTE db should fail
+Database db = get_database_core(catName, dbname);
Review comment:
per the comment above, can you wrap this code in a try/catch and
re-throw the NoSuchObjectException as an InvalidOperationException? Ideally
this should throw back a NoSuchObjectException, but I am concerned about
backward compatibilty with older clients.
##
File path:
standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/HMSHandler.java
##
@@ -5882,7 +5884,7 @@ public String getVersion() throws TException {
@Override
public void alter_table(final String dbname, final String name,
final Table newTable)
- throws InvalidOperationException, MetaException {
+ throws InvalidOperationException, MetaException, NoSuchObjectException {
Review comment:
so I am assuming the get_database_core() call now throws a
NoSuchObjectException that we now have to account for.
I realize the current code is a bit inconsistent when it comes to exception
handling. so for drop_table(), if the table does not exist, we throw a
NoSuchObjectException. But for alter_table() if the table does not exist, we
catch the NoSuchObjectException and rethrow it as an InvalidOperationException.
Ideally they should be consistent.
But I am a bit concerned about throwing a new exception from HMS. It might
be backwards-incompatible with the current code. Could you instead catch this
and re-throw as InvalidOperationException as well.
##
File path:
standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/HMSHandler.java
##
@@ -5892,7 +5894,7 @@ public void alter_table(final String dbname, final String
name,
@Override
public void alter_table_with_cascade(final String dbname, final String name,
final Table newTable, final boolean
cascade)
- throws InvalidOperationException, MetaException {
+ throws InvalidOperationException, MetaException, NoSuchObjectException {
EnvironmentContext envContext = null;
Review comment:
same as above comment
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 620633)
Time Spent: 20m (was: 10m)
> Drop/Alter table in REMOTE db should fail
> -
>
> Key: HIVE-25282
> URL: https://issues.apache.org/jira/browse/HIVE-25282
> Project: Hive
> Issue Type: Sub-task
>Reporter: Dantong Dong
>Assignee: Dantong Dong
>Priority: Major
> Labels: pull-request-available
> Fix For: 4.0.0
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> Drop/Alter table statement should be explicitly rejected in REMOTE database.
> In consistency with HIVE-24425: Create table in REMOTE db should fail.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (HIVE-25209) SELECT query with SUM function producing unexpected result
[
https://issues.apache.org/jira/browse/HIVE-25209?focusedWorklogId=620593=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-620593
]
ASF GitHub Bot logged work on HIVE-25209:
-
Author: ASF GitHub Bot
Created on: 08/Jul/21 16:45
Start Date: 08/Jul/21 16:45
Worklog Time Spent: 10m
Work Description: kgyrtkirk commented on a change in pull request #2360:
URL: https://github.com/apache/hive/pull/2360#discussion_r666359063
##
File path: ql/src/java/org/apache/hadoop/hive/ql/optimizer/StatsOptimizer.java
##
@@ -397,6 +397,11 @@ else if (getGbyKeyType(cgbyOp) == GbyKeyType.CONSTANT &&
rowCnt == 0) {
if (udaf instanceof GenericUDAFSum) {
// long/double/decimal
ExprNodeDesc desc = aggr.getParameters().get(0);
+// add null for SUM(1), when the table is empty. Without this,
category = LONG, and the result is 0
+// instead of NULL.
+if (desc instanceof ExprNodeConstantDesc && rowCnt == 0) {
+ oneRow.add(null);
+}
Review comment:
down below it will add a second value to the row (`oneRow`) ; try a test
with multiple sum-s like 2
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 620593)
Time Spent: 1h 20m (was: 1h 10m)
> SELECT query with SUM function producing unexpected result
> --
>
> Key: HIVE-25209
> URL: https://issues.apache.org/jira/browse/HIVE-25209
> Project: Hive
> Issue Type: Bug
>Reporter: Soumyakanti Das
>Assignee: Soumyakanti Das
>Priority: Major
> Labels: pull-request-available
> Time Spent: 1h 20m
> Remaining Estimate: 0h
>
> Hive: SELECT query with SUM function producing unexpected result
> Problem Statement:
> {noformat}
> SELECT SUM(1) FROM t1;
> result: 0
> SELECT SUM(agg0) FROM (
> SELECT SUM(1) as agg0 FROM t1 WHERE t1.c0 UNION ALL
> SELECT SUM(1) as agg0 FROM t1 WHERE NOT (t1.c0) UNION ALL
> SELECT SUM(1) as agg0 FROM t1 WHERE (t1.c0) IS NULL
> ) as asdf;
> result: null {noformat}
> Steps to reproduce:
> {noformat}
> DROP DATABASE IF EXISTS db5 CASCADE;
> CREATE DATABASE db5;
> use db5;
> CREATE TABLE IF NOT EXISTS t1(c0 boolean, c1 boolean);
> SELECT SUM(1) FROM t1;
> -- result: 0
> SELECT SUM(agg0) FROM (
> SELECT SUM(1) as agg0 FROM t1 WHERE t1.c0 UNION ALL
> SELECT SUM(1) as agg0 FROM t1 WHERE NOT (t1.c0) UNION ALL
> SELECT SUM(1) as agg0 FROM t1 WHERE (t1.c0) IS NULL
> ) as asdf;
> -- result: null {noformat}
> Observations:
> SELECT SUM(1) as agg0 FROM t1 WHERE t1.c0 = t1.c1; – will result in null
> Similarity with postgres,
> both the queries result in null
> Similarity with Impala,
> both the queries result in null
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-25316) Query with window function over external JDBC table and filter fails at runtime
[
https://issues.apache.org/jira/browse/HIVE-25316?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17377487#comment-17377487
]
Stamatis Zampetakis commented on HIVE-25316:
The problem seems to be related with the TopN key optimization since disabling
it ({{set hive.optimize.topnkey=false}}) makes the problem disappear.
> Query with window function over external JDBC table and filter fails at
> runtime
> ---
>
> Key: HIVE-25316
> URL: https://issues.apache.org/jira/browse/HIVE-25316
> Project: Hive
> Issue Type: Bug
> Components: JDBC storage handler, Query Processor
>Affects Versions: 4.0.0
>Reporter: Stamatis Zampetakis
>Priority: Major
> Attachments: external_jdbc_table_perf2.q
>
>
> The following TPC-DS query fails at runtime when the table {{store_sales}} is
> an external JDBC table.
> {code:sql}
> SELECT ranking
> FROM
> (SELECT rank() OVER (PARTITION BY ss_store_sk
> ORDER BY sum(ss_net_profit)) AS ranking
> FROM store_sales
> GROUP BY ss_store_sk) tmp1
> WHERE ranking <= 5
> {code}
> The stacktrace below shows that problem occurs while trying to initialize the
> {{TopNKeyOperator}}.
> {noformat}
> 2021-07-08T09:04:37,444 ERROR [TezTR-270335_1_3_0_0_0] tez.TezProcessor:
> Failed initializeAndRunProcessor
> java.lang.RuntimeException: Map operator initialization failed
> at
> org.apache.hadoop.hive.ql.exec.tez.MapRecordProcessor.init(MapRecordProcessor.java:351)
> ~[hive-exec-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
> at
> org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:310)
> [hive-exec-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
> at
> org.apache.hadoop.hive.ql.exec.tez.TezProcessor.run(TezProcessor.java:277)
> [hive-exec-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
> at
> org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:381)
> [tez-runtime-internals-0.10.0.jar:0.10.0]
> at
> org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:75)
> [tez-runtime-internals-0.10.0.jar:0.10.0]
> at
> org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:62)
> [tez-runtime-internals-0.10.0.jar:0.10.0]
> at java.security.AccessController.doPrivileged(Native Method)
> ~[?:1.8.0_261]
> at javax.security.auth.Subject.doAs(Subject.java:422) [?:1.8.0_261]
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1682)
> [hadoop-common-3.1.0.jar:?]
> at
> org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:62)
> [tez-runtime-internals-0.10.0.jar:0.10.0]
> at
> org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:38)
> [tez-runtime-internals-0.10.0.jar:0.10.0]
> at
> org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
> [tez-common-0.10.0.jar:0.10.0]
> at
> org.apache.hadoop.hive.llap.daemon.impl.StatsRecordingThreadPool$WrappedCallable.call(StatsRecordingThreadPool.java:118)
> [hive-llap-server-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> [?:1.8.0_261]
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> [?:1.8.0_261]
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> [?:1.8.0_261]
> at java.lang.Thread.run(Thread.java:748) [?:1.8.0_261]
> Caused by: java.lang.RuntimeException: cannot find field _col0 from
> [0:ss_store_sk, 1:$f1]
> at
> org.apache.hadoop.hive.serde2.objectinspector.ObjectInspectorUtils.getStandardStructFieldRef(ObjectInspectorUtils.java:550)
> ~[hive-serde-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
> at
> org.apache.hadoop.hive.serde2.objectinspector.StandardStructObjectInspector.getStructFieldRef(StandardStructObjectInspector.java:153)
> ~[hive-serde-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
> at
> org.apache.hadoop.hive.ql.exec.ExprNodeColumnEvaluator.initialize(ExprNodeColumnEvaluator.java:56)
> ~[hive-exec-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
> at
> org.apache.hadoop.hive.ql.exec.TopNKeyOperator.initObjectInspectors(TopNKeyOperator.java:101)
> ~[hive-exec-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
> at
> org.apache.hadoop.hive.ql.exec.TopNKeyOperator.initializeOp(TopNKeyOperator.java:82)
> ~[hive-exec-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
> at
> org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:360)
> ~[hive-exec-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
> at
> org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:549)
>
[jira] [Commented] (HIVE-25316) Query with window function over external JDBC table and filter fails at runtime
[
https://issues.apache.org/jira/browse/HIVE-25316?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17377486#comment-17377486
]
Stamatis Zampetakis commented on HIVE-25316:
The problem can be reproduced by running [^external_jdbc_table_perf2.q] with
{{TestMiniLlapLocalCliDriver}}
{code:sh}
mvn test -Dtest=TestMiniLlapLocalCliDriver -Dqfile=external_jdbc_table_perf2.q
-Dtest.output.overwrite
{code}
> Query with window function over external JDBC table and filter fails at
> runtime
> ---
>
> Key: HIVE-25316
> URL: https://issues.apache.org/jira/browse/HIVE-25316
> Project: Hive
> Issue Type: Bug
> Components: JDBC storage handler, Query Processor
>Affects Versions: 4.0.0
>Reporter: Stamatis Zampetakis
>Priority: Major
> Attachments: external_jdbc_table_perf2.q
>
>
> The following TPC-DS query fails at runtime when the table {{store_sales}} is
> an external JDBC table.
> {code:sql}
> SELECT ranking
> FROM
> (SELECT rank() OVER (PARTITION BY ss_store_sk
> ORDER BY sum(ss_net_profit)) AS ranking
> FROM store_sales
> GROUP BY ss_store_sk) tmp1
> WHERE ranking <= 5
> {code}
> The stacktrace below shows that problem occurs while trying to initialize the
> {{TopNKeyOperator}}.
> {noformat}
> 2021-07-08T09:04:37,444 ERROR [TezTR-270335_1_3_0_0_0] tez.TezProcessor:
> Failed initializeAndRunProcessor
> java.lang.RuntimeException: Map operator initialization failed
> at
> org.apache.hadoop.hive.ql.exec.tez.MapRecordProcessor.init(MapRecordProcessor.java:351)
> ~[hive-exec-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
> at
> org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:310)
> [hive-exec-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
> at
> org.apache.hadoop.hive.ql.exec.tez.TezProcessor.run(TezProcessor.java:277)
> [hive-exec-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
> at
> org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:381)
> [tez-runtime-internals-0.10.0.jar:0.10.0]
> at
> org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:75)
> [tez-runtime-internals-0.10.0.jar:0.10.0]
> at
> org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:62)
> [tez-runtime-internals-0.10.0.jar:0.10.0]
> at java.security.AccessController.doPrivileged(Native Method)
> ~[?:1.8.0_261]
> at javax.security.auth.Subject.doAs(Subject.java:422) [?:1.8.0_261]
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1682)
> [hadoop-common-3.1.0.jar:?]
> at
> org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:62)
> [tez-runtime-internals-0.10.0.jar:0.10.0]
> at
> org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:38)
> [tez-runtime-internals-0.10.0.jar:0.10.0]
> at
> org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
> [tez-common-0.10.0.jar:0.10.0]
> at
> org.apache.hadoop.hive.llap.daemon.impl.StatsRecordingThreadPool$WrappedCallable.call(StatsRecordingThreadPool.java:118)
> [hive-llap-server-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> [?:1.8.0_261]
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> [?:1.8.0_261]
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> [?:1.8.0_261]
> at java.lang.Thread.run(Thread.java:748) [?:1.8.0_261]
> Caused by: java.lang.RuntimeException: cannot find field _col0 from
> [0:ss_store_sk, 1:$f1]
> at
> org.apache.hadoop.hive.serde2.objectinspector.ObjectInspectorUtils.getStandardStructFieldRef(ObjectInspectorUtils.java:550)
> ~[hive-serde-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
> at
> org.apache.hadoop.hive.serde2.objectinspector.StandardStructObjectInspector.getStructFieldRef(StandardStructObjectInspector.java:153)
> ~[hive-serde-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
> at
> org.apache.hadoop.hive.ql.exec.ExprNodeColumnEvaluator.initialize(ExprNodeColumnEvaluator.java:56)
> ~[hive-exec-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
> at
> org.apache.hadoop.hive.ql.exec.TopNKeyOperator.initObjectInspectors(TopNKeyOperator.java:101)
> ~[hive-exec-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
> at
> org.apache.hadoop.hive.ql.exec.TopNKeyOperator.initializeOp(TopNKeyOperator.java:82)
> ~[hive-exec-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
> at
> org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:360)
> ~[hive-exec-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
> at
>
[jira] [Updated] (HIVE-25316) Query with window function over external JDBC table and filter fails at runtime
[
https://issues.apache.org/jira/browse/HIVE-25316?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Stamatis Zampetakis updated HIVE-25316:
---
Attachment: external_jdbc_table_perf2.q
> Query with window function over external JDBC table and filter fails at
> runtime
> ---
>
> Key: HIVE-25316
> URL: https://issues.apache.org/jira/browse/HIVE-25316
> Project: Hive
> Issue Type: Bug
> Components: JDBC storage handler, Query Processor
>Affects Versions: 4.0.0
>Reporter: Stamatis Zampetakis
>Priority: Major
> Attachments: external_jdbc_table_perf2.q
>
>
> The following TPC-DS query fails at runtime when the table {{store_sales}} is
> an external JDBC table.
> {code:sql}
> SELECT ranking
> FROM
> (SELECT rank() OVER (PARTITION BY ss_store_sk
> ORDER BY sum(ss_net_profit)) AS ranking
> FROM store_sales
> GROUP BY ss_store_sk) tmp1
> WHERE ranking <= 5
> {code}
> The stacktrace below shows that problem occurs while trying to initialize the
> {{TopNKeyOperator}}.
> {noformat}
> 2021-07-08T09:04:37,444 ERROR [TezTR-270335_1_3_0_0_0] tez.TezProcessor:
> Failed initializeAndRunProcessor
> java.lang.RuntimeException: Map operator initialization failed
> at
> org.apache.hadoop.hive.ql.exec.tez.MapRecordProcessor.init(MapRecordProcessor.java:351)
> ~[hive-exec-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
> at
> org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:310)
> [hive-exec-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
> at
> org.apache.hadoop.hive.ql.exec.tez.TezProcessor.run(TezProcessor.java:277)
> [hive-exec-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
> at
> org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:381)
> [tez-runtime-internals-0.10.0.jar:0.10.0]
> at
> org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:75)
> [tez-runtime-internals-0.10.0.jar:0.10.0]
> at
> org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:62)
> [tez-runtime-internals-0.10.0.jar:0.10.0]
> at java.security.AccessController.doPrivileged(Native Method)
> ~[?:1.8.0_261]
> at javax.security.auth.Subject.doAs(Subject.java:422) [?:1.8.0_261]
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1682)
> [hadoop-common-3.1.0.jar:?]
> at
> org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:62)
> [tez-runtime-internals-0.10.0.jar:0.10.0]
> at
> org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:38)
> [tez-runtime-internals-0.10.0.jar:0.10.0]
> at
> org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
> [tez-common-0.10.0.jar:0.10.0]
> at
> org.apache.hadoop.hive.llap.daemon.impl.StatsRecordingThreadPool$WrappedCallable.call(StatsRecordingThreadPool.java:118)
> [hive-llap-server-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> [?:1.8.0_261]
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> [?:1.8.0_261]
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> [?:1.8.0_261]
> at java.lang.Thread.run(Thread.java:748) [?:1.8.0_261]
> Caused by: java.lang.RuntimeException: cannot find field _col0 from
> [0:ss_store_sk, 1:$f1]
> at
> org.apache.hadoop.hive.serde2.objectinspector.ObjectInspectorUtils.getStandardStructFieldRef(ObjectInspectorUtils.java:550)
> ~[hive-serde-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
> at
> org.apache.hadoop.hive.serde2.objectinspector.StandardStructObjectInspector.getStructFieldRef(StandardStructObjectInspector.java:153)
> ~[hive-serde-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
> at
> org.apache.hadoop.hive.ql.exec.ExprNodeColumnEvaluator.initialize(ExprNodeColumnEvaluator.java:56)
> ~[hive-exec-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
> at
> org.apache.hadoop.hive.ql.exec.TopNKeyOperator.initObjectInspectors(TopNKeyOperator.java:101)
> ~[hive-exec-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
> at
> org.apache.hadoop.hive.ql.exec.TopNKeyOperator.initializeOp(TopNKeyOperator.java:82)
> ~[hive-exec-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
> at
> org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:360)
> ~[hive-exec-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
> at
> org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:549)
> ~[hive-exec-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
> at
> org.apache.hadoop.hive.ql.exec.Operator.initializeChildren(Operator.java:503)
>
[jira] [Updated] (HIVE-25307) Hive Server 2 crashes when Thrift library encounters particular security protocol issue
[ https://issues.apache.org/jira/browse/HIVE-25307?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt McCline updated HIVE-25307: Attachment: (was: hive-thrift-fix-01-3_1.patch) > Hive Server 2 crashes when Thrift library encounters particular security > protocol issue > --- > > Key: HIVE-25307 > URL: https://issues.apache.org/jira/browse/HIVE-25307 > Project: Hive > Issue Type: Bug >Reporter: Matt McCline >Assignee: Matt McCline >Priority: Critical > Labels: pull-request-available > Attachments: hive-thrift-fix-02-3_1.patch > > Time Spent: 10m > Remaining Estimate: 0h > > A RuntimeException is thrown by the Thrift library that causes Hive Server 2 > to crash on our customer's machine. If you Google this the exception has been > reported a couple of times over the years but not fixed. A blog (see > references below) says it is an occasional security protocol issue between > Hive Server 2 and a proxy like a Gateway. > One challenge is the Thrift TTransportFactory getTransport method declaration > throws no Exceptions hence the likely choice of RuntimeException. But that > Exception is fatal to Hive Server 2. > The proposed fix is a work around that catches RuntimeException in Hive > Server 2, saves the Exception cause in a dummy TTransport object, and throws > the cause when TTransport's open method is called later. > > ExceptionClassName: > java.lang.RuntimeException > ExceptionStackTrace: > java.lang.RuntimeException: > org.apache.thrift.transport.TSaslTransportException: No data or no sasl data > in the stream > at > org.apache.thrift.transport.TSaslServerTransport$Factory.getTransport(TSaslServerTransport.java:219) > at > org.apache.hadoop.hive.metastore.security.HadoopThriftAuthBridge$Server$TUGIAssumingTransportFactory$1.run(HadoopThriftAuthBridge.java:694) > at > org.apache.hadoop.hive.metastore.security.HadoopThriftAuthBridge$Server$TUGIAssumingTransportFactory$1.run(HadoopThriftAuthBridge.java:691) > at java.security.AccessController.doPrivileged(Native Method) > at javax.security.auth.Subject.doAs(Subject.java:360) > at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1710) > at > org.apache.hadoop.hive.metastore.security.HadoopThriftAuthBridge$Server$TUGIAssumingTransportFactory.getTransport(HadoopThriftAuthBridge.java:691) > at > org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:269) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) > at java.lang.Thread.run(Thread.java:748) > Caused by: org.apache.thrift.transport.TSaslTransportException: No data or no > sasl data in the stream > at > org.apache.thrift.transport.TSaslTransport.open(TSaslTransport.java:326) > at > org.apache.thrift.transport.TSaslServerTransport.open(TSaslServerTransport.java:41) > at > org.apache.thrift.transport.TSaslServerTransport$Factory.getTransport(TSaslServerTransport.java:216) > ... 10 more > > References: > [Hive server 2 thrift error - Cloudera Community - > 34293|https://community.cloudera.com/t5/Support-Questions/Hive-server-2-thrift-error/td-p/34293] > Eric Lin blog "“NO DATA OR NO SASL DATA IN THE STREAM” ERROR IN HIVESERVER2 > LOG" > [HIVE-12754] AuthTypes.NONE cause exception after HS2 start - ASF JIRA > (apache.org) > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-25307) Hive Server 2 crashes when Thrift library encounters particular security protocol issue
[ https://issues.apache.org/jira/browse/HIVE-25307?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt McCline updated HIVE-25307: Attachment: hive-thrift-fix-02-3_1.patch > Hive Server 2 crashes when Thrift library encounters particular security > protocol issue > --- > > Key: HIVE-25307 > URL: https://issues.apache.org/jira/browse/HIVE-25307 > Project: Hive > Issue Type: Bug >Reporter: Matt McCline >Assignee: Matt McCline >Priority: Critical > Labels: pull-request-available > Attachments: hive-thrift-fix-01-3_1.patch, > hive-thrift-fix-02-3_1.patch > > Time Spent: 10m > Remaining Estimate: 0h > > A RuntimeException is thrown by the Thrift library that causes Hive Server 2 > to crash on our customer's machine. If you Google this the exception has been > reported a couple of times over the years but not fixed. A blog (see > references below) says it is an occasional security protocol issue between > Hive Server 2 and a proxy like a Gateway. > One challenge is the Thrift TTransportFactory getTransport method declaration > throws no Exceptions hence the likely choice of RuntimeException. But that > Exception is fatal to Hive Server 2. > The proposed fix is a work around that catches RuntimeException in Hive > Server 2, saves the Exception cause in a dummy TTransport object, and throws > the cause when TTransport's open method is called later. > > ExceptionClassName: > java.lang.RuntimeException > ExceptionStackTrace: > java.lang.RuntimeException: > org.apache.thrift.transport.TSaslTransportException: No data or no sasl data > in the stream > at > org.apache.thrift.transport.TSaslServerTransport$Factory.getTransport(TSaslServerTransport.java:219) > at > org.apache.hadoop.hive.metastore.security.HadoopThriftAuthBridge$Server$TUGIAssumingTransportFactory$1.run(HadoopThriftAuthBridge.java:694) > at > org.apache.hadoop.hive.metastore.security.HadoopThriftAuthBridge$Server$TUGIAssumingTransportFactory$1.run(HadoopThriftAuthBridge.java:691) > at java.security.AccessController.doPrivileged(Native Method) > at javax.security.auth.Subject.doAs(Subject.java:360) > at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1710) > at > org.apache.hadoop.hive.metastore.security.HadoopThriftAuthBridge$Server$TUGIAssumingTransportFactory.getTransport(HadoopThriftAuthBridge.java:691) > at > org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:269) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) > at java.lang.Thread.run(Thread.java:748) > Caused by: org.apache.thrift.transport.TSaslTransportException: No data or no > sasl data in the stream > at > org.apache.thrift.transport.TSaslTransport.open(TSaslTransport.java:326) > at > org.apache.thrift.transport.TSaslServerTransport.open(TSaslServerTransport.java:41) > at > org.apache.thrift.transport.TSaslServerTransport$Factory.getTransport(TSaslServerTransport.java:216) > ... 10 more > > References: > [Hive server 2 thrift error - Cloudera Community - > 34293|https://community.cloudera.com/t5/Support-Questions/Hive-server-2-thrift-error/td-p/34293] > Eric Lin blog "“NO DATA OR NO SASL DATA IN THE STREAM” ERROR IN HIVESERVER2 > LOG" > [HIVE-12754] AuthTypes.NONE cause exception after HS2 start - ASF JIRA > (apache.org) > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (HIVE-25255) Support ALTER TABLE REPLACE COLUMNS for Iceberg
[ https://issues.apache.org/jira/browse/HIVE-25255?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ádám Szita resolved HIVE-25255. --- Fix Version/s: 4.0.0 Resolution: Fixed +1, committed to master. Thanks for the contribution [~mbod]! > Support ALTER TABLE REPLACE COLUMNS for Iceberg > --- > > Key: HIVE-25255 > URL: https://issues.apache.org/jira/browse/HIVE-25255 > Project: Hive > Issue Type: New Feature >Reporter: Marton Bod >Assignee: Marton Bod >Priority: Major > Labels: pull-request-available > Fix For: 4.0.0 > > Time Spent: 3h > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (HIVE-25255) Support ALTER TABLE REPLACE COLUMNS for Iceberg
[ https://issues.apache.org/jira/browse/HIVE-25255?focusedWorklogId=620549=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-620549 ] ASF GitHub Bot logged work on HIVE-25255: - Author: ASF GitHub Bot Created on: 08/Jul/21 14:55 Start Date: 08/Jul/21 14:55 Worklog Time Spent: 10m Work Description: szlta merged pull request #2418: URL: https://github.com/apache/hive/pull/2418 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 620549) Time Spent: 3h (was: 2h 50m) > Support ALTER TABLE REPLACE COLUMNS for Iceberg > --- > > Key: HIVE-25255 > URL: https://issues.apache.org/jira/browse/HIVE-25255 > Project: Hive > Issue Type: New Feature >Reporter: Marton Bod >Assignee: Marton Bod >Priority: Major > Labels: pull-request-available > Time Spent: 3h > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-25315) SQL executed hiveserver is killed by the HiveServer2-Handler-Pool thread, very occasionally
[
https://issues.apache.org/jira/browse/HIVE-25315?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
GuangMing Lu updated HIVE-25315:
Component/s: HiveServer2
> SQL executed hiveserver is killed by the HiveServer2-Handler-Pool thread,
> very occasionally
> ---
>
> Key: HIVE-25315
> URL: https://issues.apache.org/jira/browse/HIVE-25315
> Project: Hive
> Issue Type: Bug
> Components: HiveServer2
>Affects Versions: 3.1.0
>Reporter: GuangMing Lu
>Priority: Major
>
> {code:java}
> 2021-07-05 15:23:17,376 | INFO | HiveServer2-Handler-Pool: Thread-226765745
> | Shutting down HiveServer2 |
> org.apache.hive.service.server.HiveServer2.stop(HiveServer2.java:1090)
> 2021-07-05 15:23:17,376 | INFO | HiveServer2-Handler-Pool: Thread-226765745
> | Thrift server has stopped |
> org.apache.hive.service.cli.thrift.ThriftBinaryCLIService.stopServer(ThriftBinaryCLIService.java:225)
> 2021-07-05 15:23:17,376 | INFO | HiveServer2-Handler-Pool: Thread-226765745
> | Service:ThriftBinaryCLIService is stopped. |
> org.apache.hive.service.AbstractService.stop(AbstractService.java:130)
> 2021-07-05 15:23:17,376 | INFO | HiveServer2-Handler-Pool: Thread-226765745
> | Service:OperationManager is stopped. |
> org.apache.hive.service.AbstractService.stop(AbstractService.java:130)
> 2021-07-05 15:23:17,376 | INFO | HiveServer2-Handler-Pool: Thread-226765745
> | Service:SessionManager is stopped. |
> org.apache.hive.service.AbstractService.stop(AbstractService.java:130)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (HIVE-25314) Implement SHOW CREATE TABLE command for Iceberg tables
[
https://issues.apache.org/jira/browse/HIVE-25314?focusedWorklogId=620516=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-620516
]
ASF GitHub Bot logged work on HIVE-25314:
-
Author: ASF GitHub Bot
Created on: 08/Jul/21 14:17
Start Date: 08/Jul/21 14:17
Worklog Time Spent: 10m
Work Description: marton-bod commented on a change in pull request #2458:
URL: https://github.com/apache/hive/pull/2458#discussion_r666236787
##
File path:
ql/src/java/org/apache/hadoop/hive/ql/metadata/HiveStorageHandler.java
##
@@ -257,6 +258,16 @@ default boolean supportsPartitionTransform() {
return false;
}
+ /**
+ * Return the partition transform specification table property key together
with the specification value in json
Review comment:
nit: I don't think we need to mandate that the value is in json format.
Some other handlers could choose a different format in the future. Since the
return type is just a string that'll be plugged into the output, there's no
reason to have this restriction I think.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 620516)
Time Spent: 50m (was: 40m)
> Implement SHOW CREATE TABLE command for Iceberg tables
> --
>
> Key: HIVE-25314
> URL: https://issues.apache.org/jira/browse/HIVE-25314
> Project: Hive
> Issue Type: Improvement
>Reporter: László Pintér
>Assignee: László Pintér
>Priority: Major
> Labels: pull-request-available
> Time Spent: 50m
> Remaining Estimate: 0h
>
> Implement SHOW CREATE TABLE, including the partitioning info as well in the
> output (even though the HMS table is unpartitioned). That would probably
> require loading the Iceberg table to acquire the partitioning information.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (HIVE-25314) Implement SHOW CREATE TABLE command for Iceberg tables
[
https://issues.apache.org/jira/browse/HIVE-25314?focusedWorklogId=620514=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-620514
]
ASF GitHub Bot logged work on HIVE-25314:
-
Author: ASF GitHub Bot
Created on: 08/Jul/21 14:14
Start Date: 08/Jul/21 14:14
Worklog Time Spent: 10m
Work Description: marton-bod commented on a change in pull request #2458:
URL: https://github.com/apache/hive/pull/2458#discussion_r666234150
##
File path:
iceberg/iceberg-handler/src/main/java/org/apache/iceberg/mr/hive/HiveIcebergStorageHandler.java
##
@@ -270,6 +270,15 @@ public boolean supportsPartitionTransform() {
return true;
}
+ @Override
+ public Map
getPartitionTransformSpecProperty(org.apache.hadoop.hive.ql.metadata.Table
hmsTable) {
+Map property = new HashMap<>();
+TableDesc tableDesc = Utilities.getTableDesc(hmsTable);
+Table table = Catalogs.loadTable(conf, tableDesc.getProperties());
Review comment:
Would it make sense to use the `IcebergTableUtil` to load the table
here? Not sure if there are other places too during `show create table` where
we load the table, maybe there's some serde initialization at some point?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 620514)
Time Spent: 40m (was: 0.5h)
> Implement SHOW CREATE TABLE command for Iceberg tables
> --
>
> Key: HIVE-25314
> URL: https://issues.apache.org/jira/browse/HIVE-25314
> Project: Hive
> Issue Type: Improvement
>Reporter: László Pintér
>Assignee: László Pintér
>Priority: Major
> Labels: pull-request-available
> Time Spent: 40m
> Remaining Estimate: 0h
>
> Implement SHOW CREATE TABLE, including the partitioning info as well in the
> output (even though the HMS table is unpartitioned). That would probably
> require loading the Iceberg table to acquire the partitioning information.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (HIVE-25314) Implement SHOW CREATE TABLE command for Iceberg tables
[
https://issues.apache.org/jira/browse/HIVE-25314?focusedWorklogId=620511=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-620511
]
ASF GitHub Bot logged work on HIVE-25314:
-
Author: ASF GitHub Bot
Created on: 08/Jul/21 14:11
Start Date: 08/Jul/21 14:11
Worklog Time Spent: 10m
Work Description: marton-bod commented on a change in pull request #2458:
URL: https://github.com/apache/hive/pull/2458#discussion_r666230993
##
File path:
ql/src/java/org/apache/hadoop/hive/ql/metadata/HiveStorageHandler.java
##
@@ -257,6 +258,16 @@ default boolean supportsPartitionTransform() {
return false;
}
+ /**
+ * Return the partition transform specification table property key together
with the specification value in json
Review comment:
Maybe mention in the docs that this should be overridden only if the
`supportsPartitionTransform()` method is true?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 620511)
Time Spent: 0.5h (was: 20m)
> Implement SHOW CREATE TABLE command for Iceberg tables
> --
>
> Key: HIVE-25314
> URL: https://issues.apache.org/jira/browse/HIVE-25314
> Project: Hive
> Issue Type: Improvement
>Reporter: László Pintér
>Assignee: László Pintér
>Priority: Major
> Labels: pull-request-available
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> Implement SHOW CREATE TABLE, including the partitioning info as well in the
> output (even though the HMS table is unpartitioned). That would probably
> require loading the Iceberg table to acquire the partitioning information.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (HIVE-25314) Implement SHOW CREATE TABLE command for Iceberg tables
[ https://issues.apache.org/jira/browse/HIVE-25314?focusedWorklogId=620501=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-620501 ] ASF GitHub Bot logged work on HIVE-25314: - Author: ASF GitHub Bot Created on: 08/Jul/21 13:57 Start Date: 08/Jul/21 13:57 Worklog Time Spent: 10m Work Description: marton-bod commented on pull request #2458: URL: https://github.com/apache/hive/pull/2458#issuecomment-876463024 Just to check my understanding: the show create table will produce an output like below? `CREATE TABLE tbl (date timestamp, name string) STORED BY '...' TBLPROPERTIES (..., 'iceberg.mr.table.partition.spec'='...')` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 620501) Time Spent: 20m (was: 10m) > Implement SHOW CREATE TABLE command for Iceberg tables > -- > > Key: HIVE-25314 > URL: https://issues.apache.org/jira/browse/HIVE-25314 > Project: Hive > Issue Type: Improvement >Reporter: László Pintér >Assignee: László Pintér >Priority: Major > Labels: pull-request-available > Time Spent: 20m > Remaining Estimate: 0h > > Implement SHOW CREATE TABLE, including the partitioning info as well in the > output (even though the HMS table is unpartitioned). That would probably > require loading the Iceberg table to acquire the partitioning information. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (HIVE-25308) Use new Tez API to get JobID for Iceberg commits
[
https://issues.apache.org/jira/browse/HIVE-25308?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Marton Bod resolved HIVE-25308.
---
Resolution: Fixed
> Use new Tez API to get JobID for Iceberg commits
>
>
> Key: HIVE-25308
> URL: https://issues.apache.org/jira/browse/HIVE-25308
> Project: Hive
> Issue Type: Improvement
>Reporter: Marton Bod
>Assignee: Marton Bod
>Priority: Major
> Labels: pull-request-available
> Time Spent: 10m
> Remaining Estimate: 0h
>
> When committing Iceberg writes, currently we only have the JobID without the
> vertexID, therefore we have to list the folder {{/temp}}
> first, and parse out the full JobIDs (incl. vertexID) from the resulting
> folder names.
> With Tez 0.10.1 released, now we have a new API we can call to acquire the
> full JobID, making the file listing unnecessary.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (HIVE-25313) Upgrade commons-codec to 1.15
[ https://issues.apache.org/jira/browse/HIVE-25313?focusedWorklogId=620471=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-620471 ] ASF GitHub Bot logged work on HIVE-25313: - Author: ASF GitHub Bot Created on: 08/Jul/21 12:30 Start Date: 08/Jul/21 12:30 Worklog Time Spent: 10m Work Description: kgyrtkirk merged pull request #2454: URL: https://github.com/apache/hive/pull/2454 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 620471) Time Spent: 20m (was: 10m) > Upgrade commons-codec to 1.15 > - > > Key: HIVE-25313 > URL: https://issues.apache.org/jira/browse/HIVE-25313 > Project: Hive > Issue Type: Improvement >Reporter: Zoltan Haindrich >Assignee: Zoltan Haindrich >Priority: Major > Labels: pull-request-available > Time Spent: 20m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (HIVE-25313) Upgrade commons-codec to 1.15
[ https://issues.apache.org/jira/browse/HIVE-25313?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zoltan Haindrich resolved HIVE-25313. - Fix Version/s: 4.0.0 Resolution: Fixed merged into master. Thank you Panos for taking a look! > Upgrade commons-codec to 1.15 > - > > Key: HIVE-25313 > URL: https://issues.apache.org/jira/browse/HIVE-25313 > Project: Hive > Issue Type: Improvement >Reporter: Zoltan Haindrich >Assignee: Zoltan Haindrich >Priority: Major > Labels: pull-request-available > Fix For: 4.0.0 > > Time Spent: 20m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (HIVE-25312) Upgrade netty to 4.1.65.Final
[ https://issues.apache.org/jira/browse/HIVE-25312?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zoltan Haindrich resolved HIVE-25312. - Fix Version/s: 4.0.0 Resolution: Fixed merged into master. Thank you Panos for taking a look! > Upgrade netty to 4.1.65.Final > - > > Key: HIVE-25312 > URL: https://issues.apache.org/jira/browse/HIVE-25312 > Project: Hive > Issue Type: Improvement >Reporter: Zoltan Haindrich >Assignee: Zoltan Haindrich >Priority: Major > Labels: pull-request-available > Fix For: 4.0.0 > > Time Spent: 20m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (HIVE-25312) Upgrade netty to 4.1.65.Final
[ https://issues.apache.org/jira/browse/HIVE-25312?focusedWorklogId=620470=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-620470 ] ASF GitHub Bot logged work on HIVE-25312: - Author: ASF GitHub Bot Created on: 08/Jul/21 12:29 Start Date: 08/Jul/21 12:29 Worklog Time Spent: 10m Work Description: kgyrtkirk merged pull request #2455: URL: https://github.com/apache/hive/pull/2455 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 620470) Time Spent: 20m (was: 10m) > Upgrade netty to 4.1.65.Final > - > > Key: HIVE-25312 > URL: https://issues.apache.org/jira/browse/HIVE-25312 > Project: Hive > Issue Type: Improvement >Reporter: Zoltan Haindrich >Assignee: Zoltan Haindrich >Priority: Major > Labels: pull-request-available > Time Spent: 20m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-25314) Implement SHOW CREATE TABLE command for Iceberg tables
[ https://issues.apache.org/jira/browse/HIVE-25314?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HIVE-25314: -- Labels: pull-request-available (was: ) > Implement SHOW CREATE TABLE command for Iceberg tables > -- > > Key: HIVE-25314 > URL: https://issues.apache.org/jira/browse/HIVE-25314 > Project: Hive > Issue Type: Improvement >Reporter: László Pintér >Assignee: László Pintér >Priority: Major > Labels: pull-request-available > Time Spent: 10m > Remaining Estimate: 0h > > Implement SHOW CREATE TABLE, including the partitioning info as well in the > output (even though the HMS table is unpartitioned). That would probably > require loading the Iceberg table to acquire the partitioning information. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (HIVE-25314) Implement SHOW CREATE TABLE command for Iceberg tables
[ https://issues.apache.org/jira/browse/HIVE-25314?focusedWorklogId=620463=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-620463 ] ASF GitHub Bot logged work on HIVE-25314: - Author: ASF GitHub Bot Created on: 08/Jul/21 11:28 Start Date: 08/Jul/21 11:28 Worklog Time Spent: 10m Work Description: lcspinter opened a new pull request #2458: URL: https://github.com/apache/hive/pull/2458 Include partition specification of the Iceberg table in the output of the `SHOW CREATE TABLE` command. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 620463) Remaining Estimate: 0h Time Spent: 10m > Implement SHOW CREATE TABLE command for Iceberg tables > -- > > Key: HIVE-25314 > URL: https://issues.apache.org/jira/browse/HIVE-25314 > Project: Hive > Issue Type: Improvement >Reporter: László Pintér >Assignee: László Pintér >Priority: Major > Time Spent: 10m > Remaining Estimate: 0h > > Implement SHOW CREATE TABLE, including the partitioning info as well in the > output (even though the HMS table is unpartitioned). That would probably > require loading the Iceberg table to acquire the partitioning information. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (HIVE-23633) Metastore some JDO query objects do not close properly
[ https://issues.apache.org/jira/browse/HIVE-23633?focusedWorklogId=620462=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-620462 ] ASF GitHub Bot logged work on HIVE-23633: - Author: ASF GitHub Bot Created on: 08/Jul/21 11:27 Start Date: 08/Jul/21 11:27 Worklog Time Spent: 10m Work Description: dengzhhu653 removed a comment on pull request #2344: URL: https://github.com/apache/hive/pull/2344#issuecomment-866884966 Tests passed. Hey @belugabehr @kgyrtkirk could you please take a look? :) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 620462) Time Spent: 7h 20m (was: 7h 10m) > Metastore some JDO query objects do not close properly > -- > > Key: HIVE-23633 > URL: https://issues.apache.org/jira/browse/HIVE-23633 > Project: Hive > Issue Type: Bug >Reporter: Zhihua Deng >Assignee: Zhihua Deng >Priority: Major > Labels: pull-request-available > Fix For: 4.0.0 > > Attachments: HIVE-23633.01.patch > > Time Spent: 7h 20m > Remaining Estimate: 0h > > After patched [HIVE-10895|https://issues.apache.org/jira/browse/HIVE-10895], > The metastore still has seen a memory leak on db resources: many > StatementImpls left unclosed. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (HIVE-23633) Metastore some JDO query objects do not close properly
[ https://issues.apache.org/jira/browse/HIVE-23633?focusedWorklogId=620461=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-620461 ] ASF GitHub Bot logged work on HIVE-23633: - Author: ASF GitHub Bot Created on: 08/Jul/21 11:26 Start Date: 08/Jul/21 11:26 Worklog Time Spent: 10m Work Description: dengzhhu653 commented on pull request #2344: URL: https://github.com/apache/hive/pull/2344#issuecomment-876358994 Hi @kgyrtkirk could you please take a look if have secs? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 620461) Time Spent: 7h 10m (was: 7h) > Metastore some JDO query objects do not close properly > -- > > Key: HIVE-23633 > URL: https://issues.apache.org/jira/browse/HIVE-23633 > Project: Hive > Issue Type: Bug >Reporter: Zhihua Deng >Assignee: Zhihua Deng >Priority: Major > Labels: pull-request-available > Fix For: 4.0.0 > > Attachments: HIVE-23633.01.patch > > Time Spent: 7h 10m > Remaining Estimate: 0h > > After patched [HIVE-10895|https://issues.apache.org/jira/browse/HIVE-10895], > The metastore still has seen a memory leak on db resources: many > StatementImpls left unclosed. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Assigned] (HIVE-25314) Implement SHOW CREATE TABLE command for Iceberg tables
[ https://issues.apache.org/jira/browse/HIVE-25314?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] László Pintér reassigned HIVE-25314: > Implement SHOW CREATE TABLE command for Iceberg tables > -- > > Key: HIVE-25314 > URL: https://issues.apache.org/jira/browse/HIVE-25314 > Project: Hive > Issue Type: Improvement >Reporter: László Pintér >Assignee: László Pintér >Priority: Major > > Implement SHOW CREATE TABLE, including the partitioning info as well in the > output (even though the HMS table is unpartitioned). That would probably > require loading the Iceberg table to acquire the partitioning information. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (HIVE-25294) Optimise the metadata count queries for local mode
[ https://issues.apache.org/jira/browse/HIVE-25294?focusedWorklogId=620457=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-620457 ] ASF GitHub Bot logged work on HIVE-25294: - Author: ASF GitHub Bot Created on: 08/Jul/21 11:14 Start Date: 08/Jul/21 11:14 Worklog Time Spent: 10m Work Description: dengzhhu653 commented on pull request #2435: URL: https://github.com/apache/hive/pull/2435#issuecomment-876351528 @nrg4878 cloud you please take a look if have secs? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 620457) Time Spent: 20m (was: 10m) > Optimise the metadata count queries for local mode > -- > > Key: HIVE-25294 > URL: https://issues.apache.org/jira/browse/HIVE-25294 > Project: Hive > Issue Type: Improvement > Components: Standalone Metastore >Reporter: Zhihua Deng >Assignee: Zhihua Deng >Priority: Major > Labels: pull-request-available > Time Spent: 20m > Remaining Estimate: 0h > > When Metastore is in local mode, the client uses his own private HMSHandler > to get the meta data, the HMSHandler should be initialized before being > ready to serve. When the metrics is enabled, HMSHandler will count the number > of db, table, partitions, which cloud lead to some problems. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (HIVE-25299) Casting timestamp to numeric data types is incorrect for non-UTC timezones
[
https://issues.apache.org/jira/browse/HIVE-25299?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sankar Hariappan resolved HIVE-25299.
-
Resolution: Fixed
Merged to master! Thanks [~adeshrao] for the contribution!
> Casting timestamp to numeric data types is incorrect for non-UTC timezones
> --
>
> Key: HIVE-25299
> URL: https://issues.apache.org/jira/browse/HIVE-25299
> Project: Hive
> Issue Type: Bug
> Components: UDF
>Affects Versions: 3.1.2, 4.0.0
>Reporter: Adesh Kumar Rao
>Assignee: Adesh Kumar Rao
>Priority: Major
> Labels: pull-request-available
> Fix For: 4.0.0
>
> Time Spent: 2.5h
> Remaining Estimate: 0h
>
> *Hive 1.2.1*
> {noformat}
> Connected to: Apache Hive (version 1.2.1000.2.6.5.3033-1)
> Driver: Hive JDBC (version 1.2.1000.2.6.5.3033-1)
> Transaction isolation: TRANSACTION_REPEATABLE_READ
> Beeline version 1.2.1000.2.6.5.3033-1 by Apache Hive
> 0: jdbc:hive2://zk0-nikhil.ae4yqb3genuuvaozdf> select cast ( cast
> ("2021-03-14 01:00:00" as timestamp) as int);
> +-+--+
> | _c0 |
> +-+--+
> | 1615658400 |
> +-+--+
> 1 row selected (0.387 seconds)
> 0: jdbc:hive2://zk0-nikhil.ae4yqb3genuuvaozdf> select cast ( cast
> ("2021-03-14 01:00:00" as timestamp) as bigint);
> +-+--+
> | _c0 |
> +-+--+
> | 1615658400 |
> +-+--+
> 1 row selected (0.369 seconds)
> 0: jdbc:hive2://zk0-nikhil.ae4yqb3genuuvaozdf> select cast ( cast
> ("2021-03-14 01:00:00" as timestamp) as double);
> +--+--+
> | _c0 |
> +--+--+
> | 1.6156584E9 |
> +--+--+
> {noformat}
> *Hive 3.1, 4.0*
> {noformat}
> Connected to: Apache Hive (version 3.1.0.3.1.6.1-6)
> Driver: Hive JDBC (version 3.1.4.4.1.4.8)
> Transaction isolation: TRANSACTION_REPEATABLE_READ
> Beeline version 3.1.4.4.1.4.8 by Apache Hive
> 0: jdbc:hive2://zk0-nikhil.usmltwlt0ncuxmbost> select cast ( cast
> ("2021-03-14 01:00:00" as timestamp) as int);
> +-+
> | _c0 |
> +-+
> | 1615683600 |
> +-+
> 1 row selected (0.666 seconds)
> 0: jdbc:hive2://zk0-nikhil.usmltwlt0ncuxmbost> select cast ( cast
> ("2021-03-14 01:00:00" as timestamp) as bigint);
> +-+
> | _c0 |
> +-+
> | 1615683600 |
> +-+
> 1 row selected (0.536 seconds)
> 0: jdbc:hive2://zk0-nikhil.usmltwlt0ncuxmbost> select cast ( cast
> ("2021-03-14 01:00:00" as timestamp) as double);
> +--+
> | _c0 |
> +--+
> | 1.6156836E9 |
> +--+
> 1 row selected (0.696 seconds)
> {noformat}
>
> The issue occurs for non-UTC timezone (VM timezone is set to 'Asia/Bangkok').
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (HIVE-25299) Casting timestamp to numeric data types is incorrect for non-UTC timezones
[
https://issues.apache.org/jira/browse/HIVE-25299?focusedWorklogId=620433=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-620433
]
ASF GitHub Bot logged work on HIVE-25299:
-
Author: ASF GitHub Bot
Created on: 08/Jul/21 09:41
Start Date: 08/Jul/21 09:41
Worklog Time Spent: 10m
Work Description: sankarh merged pull request #2447:
URL: https://github.com/apache/hive/pull/2447
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 620433)
Time Spent: 2.5h (was: 2h 20m)
> Casting timestamp to numeric data types is incorrect for non-UTC timezones
> --
>
> Key: HIVE-25299
> URL: https://issues.apache.org/jira/browse/HIVE-25299
> Project: Hive
> Issue Type: Bug
> Components: UDF
>Affects Versions: 3.1.2, 4.0.0
>Reporter: Adesh Kumar Rao
>Assignee: Adesh Kumar Rao
>Priority: Major
> Labels: pull-request-available
> Fix For: 4.0.0
>
> Time Spent: 2.5h
> Remaining Estimate: 0h
>
> *Hive 1.2.1*
> {noformat}
> Connected to: Apache Hive (version 1.2.1000.2.6.5.3033-1)
> Driver: Hive JDBC (version 1.2.1000.2.6.5.3033-1)
> Transaction isolation: TRANSACTION_REPEATABLE_READ
> Beeline version 1.2.1000.2.6.5.3033-1 by Apache Hive
> 0: jdbc:hive2://zk0-nikhil.ae4yqb3genuuvaozdf> select cast ( cast
> ("2021-03-14 01:00:00" as timestamp) as int);
> +-+--+
> | _c0 |
> +-+--+
> | 1615658400 |
> +-+--+
> 1 row selected (0.387 seconds)
> 0: jdbc:hive2://zk0-nikhil.ae4yqb3genuuvaozdf> select cast ( cast
> ("2021-03-14 01:00:00" as timestamp) as bigint);
> +-+--+
> | _c0 |
> +-+--+
> | 1615658400 |
> +-+--+
> 1 row selected (0.369 seconds)
> 0: jdbc:hive2://zk0-nikhil.ae4yqb3genuuvaozdf> select cast ( cast
> ("2021-03-14 01:00:00" as timestamp) as double);
> +--+--+
> | _c0 |
> +--+--+
> | 1.6156584E9 |
> +--+--+
> {noformat}
> *Hive 3.1, 4.0*
> {noformat}
> Connected to: Apache Hive (version 3.1.0.3.1.6.1-6)
> Driver: Hive JDBC (version 3.1.4.4.1.4.8)
> Transaction isolation: TRANSACTION_REPEATABLE_READ
> Beeline version 3.1.4.4.1.4.8 by Apache Hive
> 0: jdbc:hive2://zk0-nikhil.usmltwlt0ncuxmbost> select cast ( cast
> ("2021-03-14 01:00:00" as timestamp) as int);
> +-+
> | _c0 |
> +-+
> | 1615683600 |
> +-+
> 1 row selected (0.666 seconds)
> 0: jdbc:hive2://zk0-nikhil.usmltwlt0ncuxmbost> select cast ( cast
> ("2021-03-14 01:00:00" as timestamp) as bigint);
> +-+
> | _c0 |
> +-+
> | 1615683600 |
> +-+
> 1 row selected (0.536 seconds)
> 0: jdbc:hive2://zk0-nikhil.usmltwlt0ncuxmbost> select cast ( cast
> ("2021-03-14 01:00:00" as timestamp) as double);
> +--+
> | _c0 |
> +--+
> | 1.6156836E9 |
> +--+
> 1 row selected (0.696 seconds)
> {noformat}
>
> The issue occurs for non-UTC timezone (VM timezone is set to 'Asia/Bangkok').
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (HIVE-25306) Move Date and Timestamp parsing from ResolverStyle.LENIENT to ResolverStyle.STRICT
[ https://issues.apache.org/jira/browse/HIVE-25306?focusedWorklogId=620425=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-620425 ] ASF GitHub Bot logged work on HIVE-25306: - Author: ASF GitHub Bot Created on: 08/Jul/21 09:08 Start Date: 08/Jul/21 09:08 Worklog Time Spent: 10m Work Description: adesh-rao commented on pull request #2445: URL: https://github.com/apache/hive/pull/2445#issuecomment-876270108 This is going to change the behavior for customers. Previously, the queries which used to work fine, will start throwing exception now. Maybe we should put it behind a flag ? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 620425) Time Spent: 50m (was: 40m) > Move Date and Timestamp parsing from ResolverStyle.LENIENT to > ResolverStyle.STRICT > -- > > Key: HIVE-25306 > URL: https://issues.apache.org/jira/browse/HIVE-25306 > Project: Hive > Issue Type: Bug > Components: Query Planning, UDF >Reporter: Ashish Sharma >Assignee: Ashish Sharma >Priority: Major > Labels: pull-request-available > Time Spent: 50m > Remaining Estimate: 0h > > Description - > Currently Date.java and Timestamp.java use DateTimeFormatter for parsing to > convert the date/timpstamp from int,string,char etc to Date or Timestamp. > Default DateTimeFormatter which use ResolverStyle.LENIENT which mean date > like "1992-13-12" is converted to "2000-01-12", > Moving DateTimeFormatter which use ResolverStyle.STRICT which mean date like > "1992-13-12" is not be converted instead NULL is return. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-25308) Use new Tez API to get JobID for Iceberg commits
[
https://issues.apache.org/jira/browse/HIVE-25308?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17377216#comment-17377216
]
László Pintér commented on HIVE-25308:
--
Merged into master. Thanks, [~Marton Bod] for the patch and [~kuczoram] for the
review!
> Use new Tez API to get JobID for Iceberg commits
>
>
> Key: HIVE-25308
> URL: https://issues.apache.org/jira/browse/HIVE-25308
> Project: Hive
> Issue Type: Improvement
>Reporter: Marton Bod
>Assignee: Marton Bod
>Priority: Major
> Labels: pull-request-available
> Time Spent: 10m
> Remaining Estimate: 0h
>
> When committing Iceberg writes, currently we only have the JobID without the
> vertexID, therefore we have to list the folder {{/temp}}
> first, and parse out the full JobIDs (incl. vertexID) from the resulting
> folder names.
> With Tez 0.10.1 released, now we have a new API we can call to acquire the
> full JobID, making the file listing unnecessary.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-25308) Use new Tez API to get JobID for Iceberg commits
[
https://issues.apache.org/jira/browse/HIVE-25308?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated HIVE-25308:
--
Labels: pull-request-available (was: )
> Use new Tez API to get JobID for Iceberg commits
>
>
> Key: HIVE-25308
> URL: https://issues.apache.org/jira/browse/HIVE-25308
> Project: Hive
> Issue Type: Improvement
>Reporter: Marton Bod
>Assignee: Marton Bod
>Priority: Major
> Labels: pull-request-available
> Time Spent: 10m
> Remaining Estimate: 0h
>
> When committing Iceberg writes, currently we only have the JobID without the
> vertexID, therefore we have to list the folder {{/temp}}
> first, and parse out the full JobIDs (incl. vertexID) from the resulting
> folder names.
> With Tez 0.10.1 released, now we have a new API we can call to acquire the
> full JobID, making the file listing unnecessary.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (HIVE-25308) Use new Tez API to get JobID for Iceberg commits
[
https://issues.apache.org/jira/browse/HIVE-25308?focusedWorklogId=620412=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-620412
]
ASF GitHub Bot logged work on HIVE-25308:
-
Author: ASF GitHub Bot
Created on: 08/Jul/21 08:46
Start Date: 08/Jul/21 08:46
Worklog Time Spent: 10m
Work Description: lcspinter merged pull request #2446:
URL: https://github.com/apache/hive/pull/2446
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 620412)
Remaining Estimate: 0h
Time Spent: 10m
> Use new Tez API to get JobID for Iceberg commits
>
>
> Key: HIVE-25308
> URL: https://issues.apache.org/jira/browse/HIVE-25308
> Project: Hive
> Issue Type: Improvement
>Reporter: Marton Bod
>Assignee: Marton Bod
>Priority: Major
> Time Spent: 10m
> Remaining Estimate: 0h
>
> When committing Iceberg writes, currently we only have the JobID without the
> vertexID, therefore we have to list the folder {{/temp}}
> first, and parse out the full JobIDs (incl. vertexID) from the resulting
> folder names.
> With Tez 0.10.1 released, now we have a new API we can call to acquire the
> full JobID, making the file listing unnecessary.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Resolved] (HIVE-25310) Fix local test run problems with Iceberg tests: Socket closed by peer
[ https://issues.apache.org/jira/browse/HIVE-25310?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marta Kuczora resolved HIVE-25310. -- Resolution: Fixed Pushed to master. Thanks a lot [~szita] for the review! > Fix local test run problems with Iceberg tests: Socket closed by peer > - > > Key: HIVE-25310 > URL: https://issues.apache.org/jira/browse/HIVE-25310 > Project: Hive > Issue Type: Test >Reporter: Marta Kuczora >Assignee: Marta Kuczora >Priority: Major > Labels: pull-request-available > Time Spent: 20m > Remaining Estimate: 0h > > When running the tests in the iceberg-catalog and iceberg-handler module, > locally using mvn, we often get errors like this: > [ERROR] org.apache.iceberg.hive.TestHiveTableConcurrency Time elapsed: 5.022 > s <<< ERROR![ERROR] org.apache.iceberg.hive.TestHiveTableConcurrency Time > elapsed: 5.022 s <<< ERROR!org.apache.thrift.transport.TTransportException: > Socket is closed by peer. at > org.apache.thrift.transport.TIOStreamTransport.read(TIOStreamTransport.java:181) > at org.apache.thrift.transport.TTransport.readAll(TTransport.java:109) at > org.apache.thrift.protocol.TBinaryProtocol.readAll(TBinaryProtocol.java:463) > at > org.apache.thrift.protocol.TBinaryProtocol.readI32(TBinaryProtocol.java:361) > at > org.apache.thrift.protocol.TBinaryProtocol.readMessageBegin(TBinaryProtocol.java:244) > at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:77) at > org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.recv_create_database(ThriftHiveMetastore.java:1295) > at > org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.create_database(ThriftHiveMetastore.java:1282) > at > org.apache.hadoop.hive.metastore.HiveMetaStoreClient.createDatabase(HiveMetaStoreClient.java:1148) > at > org.apache.iceberg.hive.HiveMetastoreTest.startMetastore(HiveMetastoreTest.java:51) > The same problem does not occur when running it from IntelliJ or on CI. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (HIVE-25310) Fix local test run problems with Iceberg tests: Socket closed by peer
[ https://issues.apache.org/jira/browse/HIVE-25310?focusedWorklogId=620401=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-620401 ] ASF GitHub Bot logged work on HIVE-25310: - Author: ASF GitHub Bot Created on: 08/Jul/21 08:08 Start Date: 08/Jul/21 08:08 Worklog Time Spent: 10m Work Description: kuczoram merged pull request #2453: URL: https://github.com/apache/hive/pull/2453 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 620401) Time Spent: 20m (was: 10m) > Fix local test run problems with Iceberg tests: Socket closed by peer > - > > Key: HIVE-25310 > URL: https://issues.apache.org/jira/browse/HIVE-25310 > Project: Hive > Issue Type: Test >Reporter: Marta Kuczora >Assignee: Marta Kuczora >Priority: Major > Labels: pull-request-available > Time Spent: 20m > Remaining Estimate: 0h > > When running the tests in the iceberg-catalog and iceberg-handler module, > locally using mvn, we often get errors like this: > [ERROR] org.apache.iceberg.hive.TestHiveTableConcurrency Time elapsed: 5.022 > s <<< ERROR![ERROR] org.apache.iceberg.hive.TestHiveTableConcurrency Time > elapsed: 5.022 s <<< ERROR!org.apache.thrift.transport.TTransportException: > Socket is closed by peer. at > org.apache.thrift.transport.TIOStreamTransport.read(TIOStreamTransport.java:181) > at org.apache.thrift.transport.TTransport.readAll(TTransport.java:109) at > org.apache.thrift.protocol.TBinaryProtocol.readAll(TBinaryProtocol.java:463) > at > org.apache.thrift.protocol.TBinaryProtocol.readI32(TBinaryProtocol.java:361) > at > org.apache.thrift.protocol.TBinaryProtocol.readMessageBegin(TBinaryProtocol.java:244) > at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:77) at > org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.recv_create_database(ThriftHiveMetastore.java:1295) > at > org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.create_database(ThriftHiveMetastore.java:1282) > at > org.apache.hadoop.hive.metastore.HiveMetaStoreClient.createDatabase(HiveMetaStoreClient.java:1148) > at > org.apache.iceberg.hive.HiveMetastoreTest.startMetastore(HiveMetastoreTest.java:51) > The same problem does not occur when running it from IntelliJ or on CI. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (HIVE-25299) Casting timestamp to numeric data types is incorrect for non-UTC timezones
[
https://issues.apache.org/jira/browse/HIVE-25299?focusedWorklogId=620391=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-620391
]
ASF GitHub Bot logged work on HIVE-25299:
-
Author: ASF GitHub Bot
Created on: 08/Jul/21 07:27
Start Date: 08/Jul/21 07:27
Worklog Time Spent: 10m
Work Description: adesh-rao commented on a change in pull request #2447:
URL: https://github.com/apache/hive/pull/2447#discussion_r665937887
##
File path: ql/src/test/results/clientpositive/llap/timestamp_1.q.out
##
@@ -257,7 +329,7 @@ POSTHOOK: query: select cast(t as double) from timestamp_1
limit 1
POSTHOOK: type: QUERY
POSTHOOK: Input: default@timestamp_1
A masked pattern was here
-1.2938436611E9
+1.293843661E9
Review comment:
Yes the precision value is same.
`beeline -u
"jdbc:hive2://zk0-nikhil.ae4yqb3genuuvaozdfax4l12ud.dx.internal.cloudapp.net:2181,zk2-nikhil.ae4yqb3genuuvaozdfax4l12ud.dx.internal.cloudapp.net:2181,zk4-nikhil.ae4yqb3genuuvaozdfax4l12ud.dx.internal.cloudapp.net:2181/;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=hiveserver2"
Connecting to
jdbc:hive2://zk0-nikhil.ae4yqb3genuuvaozdfax4l12ud.dx.internal.cloudapp.net:2181,zk2-nikhil.ae4yqb3genuuvaozdfax4l12ud.dx.internal.cloudapp.net:2181,zk4-nikhil.ae4yqb3genuuvaozdfax4l12ud.dx.internal.cloudapp.net:2181/;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=hiveserver2
Connected to: Apache Hive (version 1.2.1000.2.6.5.3033-1)
Driver: Hive JDBC (version 1.2.1000.2.6.5.3033-1)
Transaction isolation: TRANSACTION_REPEATABLE_READ
Beeline version 1.2.1000.2.6.5.3033-1 by Apache Hive
0: jdbc:hive2://zk0-nikhil.ae4yqb3genuuvaozdf> select cast(cast('2011-01-01
01:01:01' as timestamp) as double) ;;
++--+
| _c0 |
++--+
| 1.293843661E9 |
++--+
1 row selected (0.452 seconds)
0: jdbc:hive2://zk0-nikhil.ae4yqb3genuuvaozdf> select cast(cast('2011-01-01
01:01:01' as timestamp) as float) ;;
+---+--+
| _c0 |
+---+--+
| 1.29384371E9 |
+---+--+
1 row selected (0.81 seconds)
`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 620391)
Time Spent: 2h 20m (was: 2h 10m)
> Casting timestamp to numeric data types is incorrect for non-UTC timezones
> --
>
> Key: HIVE-25299
> URL: https://issues.apache.org/jira/browse/HIVE-25299
> Project: Hive
> Issue Type: Bug
> Components: UDF
>Affects Versions: 3.1.2, 4.0.0
>Reporter: Adesh Kumar Rao
>Assignee: Adesh Kumar Rao
>Priority: Major
> Labels: pull-request-available
> Fix For: 4.0.0
>
> Time Spent: 2h 20m
> Remaining Estimate: 0h
>
> *Hive 1.2.1*
> {noformat}
> Connected to: Apache Hive (version 1.2.1000.2.6.5.3033-1)
> Driver: Hive JDBC (version 1.2.1000.2.6.5.3033-1)
> Transaction isolation: TRANSACTION_REPEATABLE_READ
> Beeline version 1.2.1000.2.6.5.3033-1 by Apache Hive
> 0: jdbc:hive2://zk0-nikhil.ae4yqb3genuuvaozdf> select cast ( cast
> ("2021-03-14 01:00:00" as timestamp) as int);
> +-+--+
> | _c0 |
> +-+--+
> | 1615658400 |
> +-+--+
> 1 row selected (0.387 seconds)
> 0: jdbc:hive2://zk0-nikhil.ae4yqb3genuuvaozdf> select cast ( cast
> ("2021-03-14 01:00:00" as timestamp) as bigint);
> +-+--+
> | _c0 |
> +-+--+
> | 1615658400 |
> +-+--+
> 1 row selected (0.369 seconds)
> 0: jdbc:hive2://zk0-nikhil.ae4yqb3genuuvaozdf> select cast ( cast
> ("2021-03-14 01:00:00" as timestamp) as double);
> +--+--+
> | _c0 |
> +--+--+
> | 1.6156584E9 |
> +--+--+
> {noformat}
> *Hive 3.1, 4.0*
> {noformat}
> Connected to: Apache Hive (version 3.1.0.3.1.6.1-6)
> Driver: Hive JDBC (version 3.1.4.4.1.4.8)
> Transaction isolation: TRANSACTION_REPEATABLE_READ
> Beeline version 3.1.4.4.1.4.8 by Apache Hive
> 0: jdbc:hive2://zk0-nikhil.usmltwlt0ncuxmbost> select cast ( cast
> ("2021-03-14 01:00:00" as timestamp) as int);
> +-+
> | _c0 |
> +-+
> | 1615683600 |
> +-+
> 1 row selected (0.666 seconds)
> 0: jdbc:hive2://zk0-nikhil.usmltwlt0ncuxmbost> select cast ( cast
> ("2021-03-14 01:00:00" as timestamp) as bigint);
> +-+
> | _c0 |
>
[jira] [Work logged] (HIVE-25299) Casting timestamp to numeric data types is incorrect for non-UTC timezones
[
https://issues.apache.org/jira/browse/HIVE-25299?focusedWorklogId=620390=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-620390
]
ASF GitHub Bot logged work on HIVE-25299:
-
Author: ASF GitHub Bot
Created on: 08/Jul/21 07:26
Start Date: 08/Jul/21 07:26
Worklog Time Spent: 10m
Work Description: adesh-rao commented on a change in pull request #2447:
URL: https://github.com/apache/hive/pull/2447#discussion_r665937887
##
File path: ql/src/test/results/clientpositive/llap/timestamp_1.q.out
##
@@ -257,7 +329,7 @@ POSTHOOK: query: select cast(t as double) from timestamp_1
limit 1
POSTHOOK: type: QUERY
POSTHOOK: Input: default@timestamp_1
A masked pattern was here
-1.2938436611E9
+1.293843661E9
Review comment:
Yes the precision value is same.
`beeline -u
"jdbc:hive2://zk0-nikhil.ae4yqb3genuuvaozdfax4l12ud.dx.internal.cloudapp.net:2181,zk2-nikhil.ae4yqb3genuuvaozdfax4l12ud.dx.internal.cloudapp.net:2181,zk4-nikhil.ae4yqb3genuuvaozdfax4l12ud.dx.internal.cloudapp.net:2181/;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=hiveserver2"
select cast(cast('2011-01-01 01:01:01' as timestamp) as double) ;;
Connecting to
jdbc:hive2://zk0-nikhil.ae4yqb3genuuvaozdfax4l12ud.dx.internal.cloudapp.net:2181,zk2-nikhil.ae4yqb3genuuvaozdfax4l12ud.dx.internal.cloudapp.net:2181,zk4-nikhil.ae4yqb3genuuvaozdfax4l12ud.dx.internal.cloudapp.net:2181/;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=hiveserver2
Connected to: Apache Hive (version 1.2.1000.2.6.5.3033-1)
Driver: Hive JDBC (version 1.2.1000.2.6.5.3033-1)
Transaction isolation: TRANSACTION_REPEATABLE_READ
Beeline version 1.2.1000.2.6.5.3033-1 by Apache Hive
0: jdbc:hive2://zk0-nikhil.ae4yqb3genuuvaozdf> select cast(cast('2011-01-01
01:01:01' as timestamp) as double) ;;
++--+
| _c0 |
++--+
| 1.293843661E9 |
++--+
1 row selected (0.452 seconds)
0: jdbc:hive2://zk0-nikhil.ae4yqb3genuuvaozdf> select cast(cast('2011-01-01
01:01:01' as timestamp) as float) ;;
+---+--+
| _c0 |
+---+--+
| 1.29384371E9 |
+---+--+
1 row selected (0.81 seconds)
0: jdbc:hive2://zk0-nikhil.ae4yqb3genuuvaozdf>`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 620390)
Time Spent: 2h 10m (was: 2h)
> Casting timestamp to numeric data types is incorrect for non-UTC timezones
> --
>
> Key: HIVE-25299
> URL: https://issues.apache.org/jira/browse/HIVE-25299
> Project: Hive
> Issue Type: Bug
> Components: UDF
>Affects Versions: 3.1.2, 4.0.0
>Reporter: Adesh Kumar Rao
>Assignee: Adesh Kumar Rao
>Priority: Major
> Labels: pull-request-available
> Fix For: 4.0.0
>
> Time Spent: 2h 10m
> Remaining Estimate: 0h
>
> *Hive 1.2.1*
> {noformat}
> Connected to: Apache Hive (version 1.2.1000.2.6.5.3033-1)
> Driver: Hive JDBC (version 1.2.1000.2.6.5.3033-1)
> Transaction isolation: TRANSACTION_REPEATABLE_READ
> Beeline version 1.2.1000.2.6.5.3033-1 by Apache Hive
> 0: jdbc:hive2://zk0-nikhil.ae4yqb3genuuvaozdf> select cast ( cast
> ("2021-03-14 01:00:00" as timestamp) as int);
> +-+--+
> | _c0 |
> +-+--+
> | 1615658400 |
> +-+--+
> 1 row selected (0.387 seconds)
> 0: jdbc:hive2://zk0-nikhil.ae4yqb3genuuvaozdf> select cast ( cast
> ("2021-03-14 01:00:00" as timestamp) as bigint);
> +-+--+
> | _c0 |
> +-+--+
> | 1615658400 |
> +-+--+
> 1 row selected (0.369 seconds)
> 0: jdbc:hive2://zk0-nikhil.ae4yqb3genuuvaozdf> select cast ( cast
> ("2021-03-14 01:00:00" as timestamp) as double);
> +--+--+
> | _c0 |
> +--+--+
> | 1.6156584E9 |
> +--+--+
> {noformat}
> *Hive 3.1, 4.0*
> {noformat}
> Connected to: Apache Hive (version 3.1.0.3.1.6.1-6)
> Driver: Hive JDBC (version 3.1.4.4.1.4.8)
> Transaction isolation: TRANSACTION_REPEATABLE_READ
> Beeline version 3.1.4.4.1.4.8 by Apache Hive
> 0: jdbc:hive2://zk0-nikhil.usmltwlt0ncuxmbost> select cast ( cast
> ("2021-03-14 01:00:00" as timestamp) as int);
> +-+
> | _c0 |
> +-+
> | 1615683600 |
> +-+
> 1 row selected (0.666 seconds)
> 0: jdbc:hive2://zk0-nikhil.usmltwlt0ncuxmbost> select cast ( cast
> ("2021-03-14 01:00:00"
[jira] [Work logged] (HIVE-25299) Casting timestamp to numeric data types is incorrect for non-UTC timezones
[
https://issues.apache.org/jira/browse/HIVE-25299?focusedWorklogId=620387=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-620387
]
ASF GitHub Bot logged work on HIVE-25299:
-
Author: ASF GitHub Bot
Created on: 08/Jul/21 07:21
Start Date: 08/Jul/21 07:21
Worklog Time Spent: 10m
Work Description: adesh-rao commented on a change in pull request #2447:
URL: https://github.com/apache/hive/pull/2447#discussion_r665934575
##
File path: ql/src/test/results/clientpositive/llap/timestamp_1.q.out
##
@@ -101,6 +101,78 @@ POSTHOOK: type: QUERY
POSTHOOK: Input: default@timestamp_1
A masked pattern was here
2011-01-01 01:01:01
+PREHOOK: query: select cast(t as boolean) from timestamp_1 limit 1
+PREHOOK: type: QUERY
+PREHOOK: Input: default@timestamp_1
+ A masked pattern was here
+POSTHOOK: query: select cast(t as boolean) from timestamp_1 limit 1
+POSTHOOK: type: QUERY
+POSTHOOK: Input: default@timestamp_1
+ A masked pattern was here
+true
+PREHOOK: query: select cast(t as tinyint) from timestamp_1 limit 1
+PREHOOK: type: QUERY
+PREHOOK: Input: default@timestamp_1
+ A masked pattern was here
+POSTHOOK: query: select cast(t as tinyint) from timestamp_1 limit 1
+POSTHOOK: type: QUERY
+POSTHOOK: Input: default@timestamp_1
+ A masked pattern was here
+NULL
+PREHOOK: query: select cast(t as smallint) from timestamp_1 limit 1
+PREHOOK: type: QUERY
+PREHOOK: Input: default@timestamp_1
+ A masked pattern was here
+POSTHOOK: query: select cast(t as smallint) from timestamp_1 limit 1
+POSTHOOK: type: QUERY
+POSTHOOK: Input: default@timestamp_1
+ A masked pattern was here
+NULL
+PREHOOK: query: select cast(t as int) from timestamp_1 limit 1
+PREHOOK: type: QUERY
+PREHOOK: Input: default@timestamp_1
+ A masked pattern was here
+POSTHOOK: query: select cast(t as int) from timestamp_1 limit 1
+POSTHOOK: type: QUERY
+POSTHOOK: Input: default@timestamp_1
+ A masked pattern was here
+1293843661
+PREHOOK: query: select cast(t as bigint) from timestamp_1 limit 1
+PREHOOK: type: QUERY
+PREHOOK: Input: default@timestamp_1
+ A masked pattern was here
+POSTHOOK: query: select cast(t as bigint) from timestamp_1 limit 1
+POSTHOOK: type: QUERY
+POSTHOOK: Input: default@timestamp_1
+ A masked pattern was here
+1293818461
+PREHOOK: query: select cast(t as float) from timestamp_1 limit 1
+PREHOOK: type: QUERY
+PREHOOK: Input: default@timestamp_1
+ A masked pattern was here
+POSTHOOK: query: select cast(t as float) from timestamp_1 limit 1
+POSTHOOK: type: QUERY
+POSTHOOK: Input: default@timestamp_1
+ A masked pattern was here
+1.2938185E9
Review comment:
This is coming because of timezone difference between UTC and
Asia/Bangkok (+7hrs).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 620387)
Time Spent: 2h (was: 1h 50m)
> Casting timestamp to numeric data types is incorrect for non-UTC timezones
> --
>
> Key: HIVE-25299
> URL: https://issues.apache.org/jira/browse/HIVE-25299
> Project: Hive
> Issue Type: Bug
> Components: UDF
>Affects Versions: 3.1.2, 4.0.0
>Reporter: Adesh Kumar Rao
>Assignee: Adesh Kumar Rao
>Priority: Major
> Labels: pull-request-available
> Fix For: 4.0.0
>
> Time Spent: 2h
> Remaining Estimate: 0h
>
> *Hive 1.2.1*
> {noformat}
> Connected to: Apache Hive (version 1.2.1000.2.6.5.3033-1)
> Driver: Hive JDBC (version 1.2.1000.2.6.5.3033-1)
> Transaction isolation: TRANSACTION_REPEATABLE_READ
> Beeline version 1.2.1000.2.6.5.3033-1 by Apache Hive
> 0: jdbc:hive2://zk0-nikhil.ae4yqb3genuuvaozdf> select cast ( cast
> ("2021-03-14 01:00:00" as timestamp) as int);
> +-+--+
> | _c0 |
> +-+--+
> | 1615658400 |
> +-+--+
> 1 row selected (0.387 seconds)
> 0: jdbc:hive2://zk0-nikhil.ae4yqb3genuuvaozdf> select cast ( cast
> ("2021-03-14 01:00:00" as timestamp) as bigint);
> +-+--+
> | _c0 |
> +-+--+
> | 1615658400 |
> +-+--+
> 1 row selected (0.369 seconds)
> 0: jdbc:hive2://zk0-nikhil.ae4yqb3genuuvaozdf> select cast ( cast
> ("2021-03-14 01:00:00" as timestamp) as double);
> +--+--+
> | _c0 |
> +--+--+
> | 1.6156584E9 |
> +--+--+
> {noformat}
> *Hive 3.1, 4.0*
> {noformat}
> Connected
[jira] [Work logged] (HIVE-25299) Casting timestamp to numeric data types is incorrect for non-UTC timezones
[
https://issues.apache.org/jira/browse/HIVE-25299?focusedWorklogId=620382=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-620382
]
ASF GitHub Bot logged work on HIVE-25299:
-
Author: ASF GitHub Bot
Created on: 08/Jul/21 07:06
Start Date: 08/Jul/21 07:06
Worklog Time Spent: 10m
Work Description: adesh-rao commented on a change in pull request #2447:
URL: https://github.com/apache/hive/pull/2447#discussion_r665925432
##
File path: ql/src/test/queries/clientpositive/timestamp_2.q
##
@@ -17,6 +17,19 @@ select cast(t as float) from timestamp_2 limit 1;
select cast(t as double) from timestamp_2 limit 1;
select cast(t as string) from timestamp_2 limit 1;
+set hive.local.time.zone=Asia/Bangkok;
+
+select cast(t as boolean) from timestamp_2 limit 1;
Review comment:
The difference is between serde of table and the insertion data format.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 620382)
Time Spent: 1h 40m (was: 1.5h)
> Casting timestamp to numeric data types is incorrect for non-UTC timezones
> --
>
> Key: HIVE-25299
> URL: https://issues.apache.org/jira/browse/HIVE-25299
> Project: Hive
> Issue Type: Bug
> Components: UDF
>Affects Versions: 3.1.2, 4.0.0
>Reporter: Adesh Kumar Rao
>Assignee: Adesh Kumar Rao
>Priority: Major
> Labels: pull-request-available
> Fix For: 4.0.0
>
> Time Spent: 1h 40m
> Remaining Estimate: 0h
>
> *Hive 1.2.1*
> {noformat}
> Connected to: Apache Hive (version 1.2.1000.2.6.5.3033-1)
> Driver: Hive JDBC (version 1.2.1000.2.6.5.3033-1)
> Transaction isolation: TRANSACTION_REPEATABLE_READ
> Beeline version 1.2.1000.2.6.5.3033-1 by Apache Hive
> 0: jdbc:hive2://zk0-nikhil.ae4yqb3genuuvaozdf> select cast ( cast
> ("2021-03-14 01:00:00" as timestamp) as int);
> +-+--+
> | _c0 |
> +-+--+
> | 1615658400 |
> +-+--+
> 1 row selected (0.387 seconds)
> 0: jdbc:hive2://zk0-nikhil.ae4yqb3genuuvaozdf> select cast ( cast
> ("2021-03-14 01:00:00" as timestamp) as bigint);
> +-+--+
> | _c0 |
> +-+--+
> | 1615658400 |
> +-+--+
> 1 row selected (0.369 seconds)
> 0: jdbc:hive2://zk0-nikhil.ae4yqb3genuuvaozdf> select cast ( cast
> ("2021-03-14 01:00:00" as timestamp) as double);
> +--+--+
> | _c0 |
> +--+--+
> | 1.6156584E9 |
> +--+--+
> {noformat}
> *Hive 3.1, 4.0*
> {noformat}
> Connected to: Apache Hive (version 3.1.0.3.1.6.1-6)
> Driver: Hive JDBC (version 3.1.4.4.1.4.8)
> Transaction isolation: TRANSACTION_REPEATABLE_READ
> Beeline version 3.1.4.4.1.4.8 by Apache Hive
> 0: jdbc:hive2://zk0-nikhil.usmltwlt0ncuxmbost> select cast ( cast
> ("2021-03-14 01:00:00" as timestamp) as int);
> +-+
> | _c0 |
> +-+
> | 1615683600 |
> +-+
> 1 row selected (0.666 seconds)
> 0: jdbc:hive2://zk0-nikhil.usmltwlt0ncuxmbost> select cast ( cast
> ("2021-03-14 01:00:00" as timestamp) as bigint);
> +-+
> | _c0 |
> +-+
> | 1615683600 |
> +-+
> 1 row selected (0.536 seconds)
> 0: jdbc:hive2://zk0-nikhil.usmltwlt0ncuxmbost> select cast ( cast
> ("2021-03-14 01:00:00" as timestamp) as double);
> +--+
> | _c0 |
> +--+
> | 1.6156836E9 |
> +--+
> 1 row selected (0.696 seconds)
> {noformat}
>
> The issue occurs for non-UTC timezone (VM timezone is set to 'Asia/Bangkok').
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (HIVE-25299) Casting timestamp to numeric data types is incorrect for non-UTC timezones
[
https://issues.apache.org/jira/browse/HIVE-25299?focusedWorklogId=620384=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-620384
]
ASF GitHub Bot logged work on HIVE-25299:
-
Author: ASF GitHub Bot
Created on: 08/Jul/21 07:06
Start Date: 08/Jul/21 07:06
Worklog Time Spent: 10m
Work Description: adesh-rao commented on a change in pull request #2447:
URL: https://github.com/apache/hive/pull/2447#discussion_r665925612
##
File path: ql/src/test/results/clientpositive/llap/timestamp_1.q.out
##
@@ -101,6 +101,78 @@ POSTHOOK: type: QUERY
POSTHOOK: Input: default@timestamp_1
A masked pattern was here
2011-01-01 01:01:01
+PREHOOK: query: select cast(t as boolean) from timestamp_1 limit 1
+PREHOOK: type: QUERY
+PREHOOK: Input: default@timestamp_1
+ A masked pattern was here
+POSTHOOK: query: select cast(t as boolean) from timestamp_1 limit 1
+POSTHOOK: type: QUERY
+POSTHOOK: Input: default@timestamp_1
+ A masked pattern was here
+true
+PREHOOK: query: select cast(t as tinyint) from timestamp_1 limit 1
+PREHOOK: type: QUERY
+PREHOOK: Input: default@timestamp_1
+ A masked pattern was here
+POSTHOOK: query: select cast(t as tinyint) from timestamp_1 limit 1
+POSTHOOK: type: QUERY
+POSTHOOK: Input: default@timestamp_1
+ A masked pattern was here
+NULL
+PREHOOK: query: select cast(t as smallint) from timestamp_1 limit 1
+PREHOOK: type: QUERY
+PREHOOK: Input: default@timestamp_1
+ A masked pattern was here
+POSTHOOK: query: select cast(t as smallint) from timestamp_1 limit 1
+POSTHOOK: type: QUERY
+POSTHOOK: Input: default@timestamp_1
+ A masked pattern was here
+NULL
+PREHOOK: query: select cast(t as int) from timestamp_1 limit 1
+PREHOOK: type: QUERY
+PREHOOK: Input: default@timestamp_1
+ A masked pattern was here
+POSTHOOK: query: select cast(t as int) from timestamp_1 limit 1
+POSTHOOK: type: QUERY
+POSTHOOK: Input: default@timestamp_1
+ A masked pattern was here
+1293843661
+PREHOOK: query: select cast(t as bigint) from timestamp_1 limit 1
+PREHOOK: type: QUERY
+PREHOOK: Input: default@timestamp_1
+ A masked pattern was here
+POSTHOOK: query: select cast(t as bigint) from timestamp_1 limit 1
+POSTHOOK: type: QUERY
+POSTHOOK: Input: default@timestamp_1
+ A masked pattern was here
+1293818461
Review comment:
Fixed the output.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 620384)
Time Spent: 1h 50m (was: 1h 40m)
> Casting timestamp to numeric data types is incorrect for non-UTC timezones
> --
>
> Key: HIVE-25299
> URL: https://issues.apache.org/jira/browse/HIVE-25299
> Project: Hive
> Issue Type: Bug
> Components: UDF
>Affects Versions: 3.1.2, 4.0.0
>Reporter: Adesh Kumar Rao
>Assignee: Adesh Kumar Rao
>Priority: Major
> Labels: pull-request-available
> Fix For: 4.0.0
>
> Time Spent: 1h 50m
> Remaining Estimate: 0h
>
> *Hive 1.2.1*
> {noformat}
> Connected to: Apache Hive (version 1.2.1000.2.6.5.3033-1)
> Driver: Hive JDBC (version 1.2.1000.2.6.5.3033-1)
> Transaction isolation: TRANSACTION_REPEATABLE_READ
> Beeline version 1.2.1000.2.6.5.3033-1 by Apache Hive
> 0: jdbc:hive2://zk0-nikhil.ae4yqb3genuuvaozdf> select cast ( cast
> ("2021-03-14 01:00:00" as timestamp) as int);
> +-+--+
> | _c0 |
> +-+--+
> | 1615658400 |
> +-+--+
> 1 row selected (0.387 seconds)
> 0: jdbc:hive2://zk0-nikhil.ae4yqb3genuuvaozdf> select cast ( cast
> ("2021-03-14 01:00:00" as timestamp) as bigint);
> +-+--+
> | _c0 |
> +-+--+
> | 1615658400 |
> +-+--+
> 1 row selected (0.369 seconds)
> 0: jdbc:hive2://zk0-nikhil.ae4yqb3genuuvaozdf> select cast ( cast
> ("2021-03-14 01:00:00" as timestamp) as double);
> +--+--+
> | _c0 |
> +--+--+
> | 1.6156584E9 |
> +--+--+
> {noformat}
> *Hive 3.1, 4.0*
> {noformat}
> Connected to: Apache Hive (version 3.1.0.3.1.6.1-6)
> Driver: Hive JDBC (version 3.1.4.4.1.4.8)
> Transaction isolation: TRANSACTION_REPEATABLE_READ
> Beeline version 3.1.4.4.1.4.8 by Apache Hive
> 0: jdbc:hive2://zk0-nikhil.usmltwlt0ncuxmbost> select cast ( cast
> ("2021-03-14 01:00:00" as timestamp) as int);
> +-+
> | _c0 |
> +-+
> | 1615683600 |
> +-+
> 1
[jira] [Work logged] (HIVE-25299) Casting timestamp to numeric data types is incorrect for non-UTC timezones
[
https://issues.apache.org/jira/browse/HIVE-25299?focusedWorklogId=620377=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-620377
]
ASF GitHub Bot logged work on HIVE-25299:
-
Author: ASF GitHub Bot
Created on: 08/Jul/21 07:01
Start Date: 08/Jul/21 07:01
Worklog Time Spent: 10m
Work Description: adesh-rao commented on a change in pull request #2447:
URL: https://github.com/apache/hive/pull/2447#discussion_r665922243
##
File path: ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToByte.java
##
@@ -211,7 +217,10 @@ public ByteWritable evaluate(TimestampWritableV2 i) {
if (i == null) {
return null;
} else {
- final long longValue = i.getSeconds();
+ ZoneId zone = SessionState.get() == null ?
+new HiveConf().getLocalTimeZone() :
SessionState.get().getConf().getLocalTimeZone();
+ TimestampTZ timestamp = TimestampTZUtil.convert(i.getTimestamp(), zone);
Review comment:
Can't be done in TimestampTZUtil (it will create a circular dependency
in common/exec modules). Instead create a new UDFUtils.java for the same.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 620377)
Time Spent: 1h 20m (was: 1h 10m)
> Casting timestamp to numeric data types is incorrect for non-UTC timezones
> --
>
> Key: HIVE-25299
> URL: https://issues.apache.org/jira/browse/HIVE-25299
> Project: Hive
> Issue Type: Bug
> Components: UDF
>Affects Versions: 3.1.2, 4.0.0
>Reporter: Adesh Kumar Rao
>Assignee: Adesh Kumar Rao
>Priority: Major
> Labels: pull-request-available
> Fix For: 4.0.0
>
> Time Spent: 1h 20m
> Remaining Estimate: 0h
>
> *Hive 1.2.1*
> {noformat}
> Connected to: Apache Hive (version 1.2.1000.2.6.5.3033-1)
> Driver: Hive JDBC (version 1.2.1000.2.6.5.3033-1)
> Transaction isolation: TRANSACTION_REPEATABLE_READ
> Beeline version 1.2.1000.2.6.5.3033-1 by Apache Hive
> 0: jdbc:hive2://zk0-nikhil.ae4yqb3genuuvaozdf> select cast ( cast
> ("2021-03-14 01:00:00" as timestamp) as int);
> +-+--+
> | _c0 |
> +-+--+
> | 1615658400 |
> +-+--+
> 1 row selected (0.387 seconds)
> 0: jdbc:hive2://zk0-nikhil.ae4yqb3genuuvaozdf> select cast ( cast
> ("2021-03-14 01:00:00" as timestamp) as bigint);
> +-+--+
> | _c0 |
> +-+--+
> | 1615658400 |
> +-+--+
> 1 row selected (0.369 seconds)
> 0: jdbc:hive2://zk0-nikhil.ae4yqb3genuuvaozdf> select cast ( cast
> ("2021-03-14 01:00:00" as timestamp) as double);
> +--+--+
> | _c0 |
> +--+--+
> | 1.6156584E9 |
> +--+--+
> {noformat}
> *Hive 3.1, 4.0*
> {noformat}
> Connected to: Apache Hive (version 3.1.0.3.1.6.1-6)
> Driver: Hive JDBC (version 3.1.4.4.1.4.8)
> Transaction isolation: TRANSACTION_REPEATABLE_READ
> Beeline version 3.1.4.4.1.4.8 by Apache Hive
> 0: jdbc:hive2://zk0-nikhil.usmltwlt0ncuxmbost> select cast ( cast
> ("2021-03-14 01:00:00" as timestamp) as int);
> +-+
> | _c0 |
> +-+
> | 1615683600 |
> +-+
> 1 row selected (0.666 seconds)
> 0: jdbc:hive2://zk0-nikhil.usmltwlt0ncuxmbost> select cast ( cast
> ("2021-03-14 01:00:00" as timestamp) as bigint);
> +-+
> | _c0 |
> +-+
> | 1615683600 |
> +-+
> 1 row selected (0.536 seconds)
> 0: jdbc:hive2://zk0-nikhil.usmltwlt0ncuxmbost> select cast ( cast
> ("2021-03-14 01:00:00" as timestamp) as double);
> +--+
> | _c0 |
> +--+
> | 1.6156836E9 |
> +--+
> 1 row selected (0.696 seconds)
> {noformat}
>
> The issue occurs for non-UTC timezone (VM timezone is set to 'Asia/Bangkok').
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (HIVE-25299) Casting timestamp to numeric data types is incorrect for non-UTC timezones
[
https://issues.apache.org/jira/browse/HIVE-25299?focusedWorklogId=620378=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-620378
]
ASF GitHub Bot logged work on HIVE-25299:
-
Author: ASF GitHub Bot
Created on: 08/Jul/21 07:01
Start Date: 08/Jul/21 07:01
Worklog Time Spent: 10m
Work Description: adesh-rao commented on a change in pull request #2447:
URL: https://github.com/apache/hive/pull/2447#discussion_r665922357
##
File path:
common/src/java/org/apache/hadoop/hive/common/type/TimestampTZUtil.java
##
@@ -186,4 +186,8 @@ public static Timestamp convertTimestampToZone(Timestamp
ts, ZoneId fromZone, Zo
return
Timestamp.ofEpochSecond(localDateTimeAtToZone.toEpochSecond(ZoneOffset.UTC),
localDateTimeAtToZone.getNano());
}
+
+ public static double convertTimestampTZToDouble(TimestampTZ timestampTZ) {
+return timestampTZ.getEpochSecond() + timestampTZ.getNanos() / 10;
Review comment:
Done.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 620378)
Time Spent: 1.5h (was: 1h 20m)
> Casting timestamp to numeric data types is incorrect for non-UTC timezones
> --
>
> Key: HIVE-25299
> URL: https://issues.apache.org/jira/browse/HIVE-25299
> Project: Hive
> Issue Type: Bug
> Components: UDF
>Affects Versions: 3.1.2, 4.0.0
>Reporter: Adesh Kumar Rao
>Assignee: Adesh Kumar Rao
>Priority: Major
> Labels: pull-request-available
> Fix For: 4.0.0
>
> Time Spent: 1.5h
> Remaining Estimate: 0h
>
> *Hive 1.2.1*
> {noformat}
> Connected to: Apache Hive (version 1.2.1000.2.6.5.3033-1)
> Driver: Hive JDBC (version 1.2.1000.2.6.5.3033-1)
> Transaction isolation: TRANSACTION_REPEATABLE_READ
> Beeline version 1.2.1000.2.6.5.3033-1 by Apache Hive
> 0: jdbc:hive2://zk0-nikhil.ae4yqb3genuuvaozdf> select cast ( cast
> ("2021-03-14 01:00:00" as timestamp) as int);
> +-+--+
> | _c0 |
> +-+--+
> | 1615658400 |
> +-+--+
> 1 row selected (0.387 seconds)
> 0: jdbc:hive2://zk0-nikhil.ae4yqb3genuuvaozdf> select cast ( cast
> ("2021-03-14 01:00:00" as timestamp) as bigint);
> +-+--+
> | _c0 |
> +-+--+
> | 1615658400 |
> +-+--+
> 1 row selected (0.369 seconds)
> 0: jdbc:hive2://zk0-nikhil.ae4yqb3genuuvaozdf> select cast ( cast
> ("2021-03-14 01:00:00" as timestamp) as double);
> +--+--+
> | _c0 |
> +--+--+
> | 1.6156584E9 |
> +--+--+
> {noformat}
> *Hive 3.1, 4.0*
> {noformat}
> Connected to: Apache Hive (version 3.1.0.3.1.6.1-6)
> Driver: Hive JDBC (version 3.1.4.4.1.4.8)
> Transaction isolation: TRANSACTION_REPEATABLE_READ
> Beeline version 3.1.4.4.1.4.8 by Apache Hive
> 0: jdbc:hive2://zk0-nikhil.usmltwlt0ncuxmbost> select cast ( cast
> ("2021-03-14 01:00:00" as timestamp) as int);
> +-+
> | _c0 |
> +-+
> | 1615683600 |
> +-+
> 1 row selected (0.666 seconds)
> 0: jdbc:hive2://zk0-nikhil.usmltwlt0ncuxmbost> select cast ( cast
> ("2021-03-14 01:00:00" as timestamp) as bigint);
> +-+
> | _c0 |
> +-+
> | 1615683600 |
> +-+
> 1 row selected (0.536 seconds)
> 0: jdbc:hive2://zk0-nikhil.usmltwlt0ncuxmbost> select cast ( cast
> ("2021-03-14 01:00:00" as timestamp) as double);
> +--+
> | _c0 |
> +--+
> | 1.6156836E9 |
> +--+
> 1 row selected (0.696 seconds)
> {noformat}
>
> The issue occurs for non-UTC timezone (VM timezone is set to 'Asia/Bangkok').
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (HIVE-25299) Casting timestamp to numeric data types is incorrect for non-UTC timezones
[
https://issues.apache.org/jira/browse/HIVE-25299?focusedWorklogId=620376=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-620376
]
ASF GitHub Bot logged work on HIVE-25299:
-
Author: ASF GitHub Bot
Created on: 08/Jul/21 07:00
Start Date: 08/Jul/21 07:00
Worklog Time Spent: 10m
Work Description: adesh-rao commented on a change in pull request #2447:
URL: https://github.com/apache/hive/pull/2447#discussion_r665921335
##
File path: ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToBoolean.java
##
@@ -213,7 +219,10 @@ public BooleanWritable evaluate(TimestampWritableV2 i) {
if (i == null) {
return null;
} else {
- booleanWritable.set(i.getSeconds() != 0 || i.getNanos() != 0);
+ ZoneId zone = SessionState.get() == null ?
Review comment:
Done.
##
File path: ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToBoolean.java
##
@@ -213,7 +219,10 @@ public BooleanWritable evaluate(TimestampWritableV2 i) {
if (i == null) {
return null;
} else {
- booleanWritable.set(i.getSeconds() != 0 || i.getNanos() != 0);
+ ZoneId zone = SessionState.get() == null ?
+new HiveConf().getLocalTimeZone() :
SessionState.get().getConf().getLocalTimeZone();
+ TimestampTZ timestamp = TimestampTZUtil.convert(i.getTimestamp(), zone);
+ booleanWritable.set(timestamp.getEpochSecond() != 0 ||
timestamp.getNanos() != 0);
Review comment:
Done.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 620376)
Time Spent: 1h 10m (was: 1h)
> Casting timestamp to numeric data types is incorrect for non-UTC timezones
> --
>
> Key: HIVE-25299
> URL: https://issues.apache.org/jira/browse/HIVE-25299
> Project: Hive
> Issue Type: Bug
> Components: UDF
>Affects Versions: 3.1.2, 4.0.0
>Reporter: Adesh Kumar Rao
>Assignee: Adesh Kumar Rao
>Priority: Major
> Labels: pull-request-available
> Fix For: 4.0.0
>
> Time Spent: 1h 10m
> Remaining Estimate: 0h
>
> *Hive 1.2.1*
> {noformat}
> Connected to: Apache Hive (version 1.2.1000.2.6.5.3033-1)
> Driver: Hive JDBC (version 1.2.1000.2.6.5.3033-1)
> Transaction isolation: TRANSACTION_REPEATABLE_READ
> Beeline version 1.2.1000.2.6.5.3033-1 by Apache Hive
> 0: jdbc:hive2://zk0-nikhil.ae4yqb3genuuvaozdf> select cast ( cast
> ("2021-03-14 01:00:00" as timestamp) as int);
> +-+--+
> | _c0 |
> +-+--+
> | 1615658400 |
> +-+--+
> 1 row selected (0.387 seconds)
> 0: jdbc:hive2://zk0-nikhil.ae4yqb3genuuvaozdf> select cast ( cast
> ("2021-03-14 01:00:00" as timestamp) as bigint);
> +-+--+
> | _c0 |
> +-+--+
> | 1615658400 |
> +-+--+
> 1 row selected (0.369 seconds)
> 0: jdbc:hive2://zk0-nikhil.ae4yqb3genuuvaozdf> select cast ( cast
> ("2021-03-14 01:00:00" as timestamp) as double);
> +--+--+
> | _c0 |
> +--+--+
> | 1.6156584E9 |
> +--+--+
> {noformat}
> *Hive 3.1, 4.0*
> {noformat}
> Connected to: Apache Hive (version 3.1.0.3.1.6.1-6)
> Driver: Hive JDBC (version 3.1.4.4.1.4.8)
> Transaction isolation: TRANSACTION_REPEATABLE_READ
> Beeline version 3.1.4.4.1.4.8 by Apache Hive
> 0: jdbc:hive2://zk0-nikhil.usmltwlt0ncuxmbost> select cast ( cast
> ("2021-03-14 01:00:00" as timestamp) as int);
> +-+
> | _c0 |
> +-+
> | 1615683600 |
> +-+
> 1 row selected (0.666 seconds)
> 0: jdbc:hive2://zk0-nikhil.usmltwlt0ncuxmbost> select cast ( cast
> ("2021-03-14 01:00:00" as timestamp) as bigint);
> +-+
> | _c0 |
> +-+
> | 1615683600 |
> +-+
> 1 row selected (0.536 seconds)
> 0: jdbc:hive2://zk0-nikhil.usmltwlt0ncuxmbost> select cast ( cast
> ("2021-03-14 01:00:00" as timestamp) as double);
> +--+
> | _c0 |
> +--+
> | 1.6156836E9 |
> +--+
> 1 row selected (0.696 seconds)
> {noformat}
>
> The issue occurs for non-UTC timezone (VM timezone is set to 'Asia/Bangkok').
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
