[jira] [Updated] (HIVE-24147) Table column names are not extracted correctly in Hive JDBC storage handler
[
https://issues.apache.org/jira/browse/HIVE-24147?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jesus Camacho Rodriguez updated HIVE-24147:
---
Description: It seems the `ResultSetMetaData` for the query used to
retrieve the table columns names contains fully qualified names, instead of
possibly supporting the {{getTableName}} method. This ends up throwing the
storage handler off and leading to exceptions, both in CBO path and non-CBO
path. (was: It seems the `ResultSetMetaData` extracted from the query to
retrieve the table columns names contains these columns as fully qualified
names instead of possibly using the {{getTableName}} method. This ends up
throwing the storage handler off and leading to exceptions, both in CBO path
and non-CBO path.)
> Table column names are not extracted correctly in Hive JDBC storage handler
> ---
>
> Key: HIVE-24147
> URL: https://issues.apache.org/jira/browse/HIVE-24147
> Project: Hive
> Issue Type: Bug
> Components: JDBC storage handler
>Affects Versions: 4.0.0
>Reporter: Jesus Camacho Rodriguez
>Priority: Major
> Labels: pull-request-available
> Time Spent: 10m
> Remaining Estimate: 0h
>
> It seems the `ResultSetMetaData` for the query used to retrieve the table
> columns names contains fully qualified names, instead of possibly supporting
> the {{getTableName}} method. This ends up throwing the storage handler off
> and leading to exceptions, both in CBO path and non-CBO path.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (HIVE-24145) Fix preemption issues in reducers and file sink operators
[ https://issues.apache.org/jira/browse/HIVE-24145?focusedWorklogId=481905=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-481905 ] ASF GitHub Bot logged work on HIVE-24145: - Author: ASF GitHub Bot Created on: 11/Sep/20 05:50 Start Date: 11/Sep/20 05:50 Worklog Time Spent: 10m Work Description: rbalamohan commented on a change in pull request #1485: URL: https://github.com/apache/hive/pull/1485#discussion_r486787799 ## File path: ql/src/java/org/apache/hadoop/hive/ql/optimizer/SortedDynPartitionOptimizer.java ## @@ -284,6 +285,11 @@ public Object process(Node nd, Stack stack, NodeProcessorCtx procCtx, // Create ReduceSink operator ReduceSinkOperator rsOp = getReduceSinkOp(partitionPositions, sortPositions, sortOrder, sortNullOrder, allRSCols, bucketColumns, numBuckets, fsParent, fsOp.getConf().getWriteType()); + // we have to make sure not to reorder the child operators as it might cause weird behavior in the tasks at + // the same level. when there is auto stats gather at the same level as another operation then it might + // cause unnecessary preemption. Maintaining the order here to avoid such preemption and possible errors Review comment: Plz add TEZ-3296 as ref if possible. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 481905) Time Spent: 0.5h (was: 20m) > Fix preemption issues in reducers and file sink operators > - > > Key: HIVE-24145 > URL: https://issues.apache.org/jira/browse/HIVE-24145 > Project: Hive > Issue Type: Bug >Reporter: Ramesh Kumar Thangarajan >Assignee: Ramesh Kumar Thangarajan >Priority: Major > Labels: pull-request-available > Time Spent: 0.5h > Remaining Estimate: 0h > > There are two issues because of preemption: > # Reducers are getting reordered as part of optimizations because of which > more preemption happen > # Preemption in the middle of writing can cause the file to not close and > lead to errors when we read the file later -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-24117) Fix for not setting managed table location in incremental load
[ https://issues.apache.org/jira/browse/HIVE-24117?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aasha Medhi updated HIVE-24117: --- Status: In Progress (was: Patch Available) > Fix for not setting managed table location in incremental load > -- > > Key: HIVE-24117 > URL: https://issues.apache.org/jira/browse/HIVE-24117 > Project: Hive > Issue Type: Task >Reporter: Aasha Medhi >Assignee: Aasha Medhi >Priority: Major > Labels: pull-request-available > Attachments: HIVE-24117.01.patch, HIVE-24117.02.patch > > Time Spent: 0.5h > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-24117) Fix for not setting managed table location in incremental load
[ https://issues.apache.org/jira/browse/HIVE-24117?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aasha Medhi updated HIVE-24117: --- Attachment: HIVE-24117.02.patch Status: Patch Available (was: In Progress) > Fix for not setting managed table location in incremental load > -- > > Key: HIVE-24117 > URL: https://issues.apache.org/jira/browse/HIVE-24117 > Project: Hive > Issue Type: Task >Reporter: Aasha Medhi >Assignee: Aasha Medhi >Priority: Major > Labels: pull-request-available > Attachments: HIVE-24117.01.patch, HIVE-24117.02.patch > > Time Spent: 0.5h > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (HIVE-24145) Fix preemption issues in reducers and file sink operators
[
https://issues.apache.org/jira/browse/HIVE-24145?focusedWorklogId=481902=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-481902
]
ASF GitHub Bot logged work on HIVE-24145:
-
Author: ASF GitHub Bot
Created on: 11/Sep/20 05:45
Start Date: 11/Sep/20 05:45
Worklog Time Spent: 10m
Work Description: rbalamohan commented on a change in pull request #1485:
URL: https://github.com/apache/hive/pull/1485#discussion_r486786544
##
File path: ql/src/java/org/apache/hadoop/hive/ql/exec/FileSinkOperator.java
##
@@ -216,29 +216,47 @@ public FSPaths(Path specPath, boolean isMmTable, boolean
isDirectInsert, boolean
}
public void closeWriters(boolean abort) throws HiveException {
+ Exception exception = null;
for (int idx = 0; idx < outWriters.length; idx++) {
if (outWriters[idx] != null) {
try {
outWriters[idx].close(abort);
updateProgress();
} catch (IOException e) {
-throw new HiveException(e);
+exception = e;
+LOG.error("Error closing " + outWriters[idx].toString(), e);
+// continue closing others
}
}
}
- try {
+ for (int i = 0; i < updaters.length; i++) {
+if (updaters[i] != null) {
+ SerDeStats stats = updaters[i].getStats();
+ // Ignore 0 row files except in case of insert overwrite
+ if (isDirectInsert && (stats.getRowCount() > 0 ||
isInsertOverwrite)) {
+outPathsCommitted[i] = updaters[i].getUpdatedFilePath();
+ }
+ try {
+updaters[i].close(abort);
+ } catch (IOException e) {
+exception = e;
+LOG.error("Error closing " + updaters[i].toString(), e);
+// continue closing others
+ }
+}
+ }
+ // Made an attempt to close all writers.
+ if (exception != null) {
for (int i = 0; i < updaters.length; i++) {
if (updaters[i] != null) {
-SerDeStats stats = updaters[i].getStats();
-// Ignore 0 row files except in case of insert overwrite
-if (isDirectInsert && (stats.getRowCount() > 0 ||
isInsertOverwrite)) {
- outPathsCommitted[i] = updaters[i].getUpdatedFilePath();
+try {
+ fs.delete(updaters[i].getUpdatedFilePath(), true);
+} catch (IOException e) {
+ e.printStackTrace();
Review comment:
LOG?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 481902)
Time Spent: 20m (was: 10m)
> Fix preemption issues in reducers and file sink operators
> -
>
> Key: HIVE-24145
> URL: https://issues.apache.org/jira/browse/HIVE-24145
> Project: Hive
> Issue Type: Bug
>Reporter: Ramesh Kumar Thangarajan
>Assignee: Ramesh Kumar Thangarajan
>Priority: Major
> Labels: pull-request-available
> Time Spent: 20m
> Remaining Estimate: 0h
>
> There are two issues because of preemption:
> # Reducers are getting reordered as part of optimizations because of which
> more preemption happen
> # Preemption in the middle of writing can cause the file to not close and
> lead to errors when we read the file later
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-24147) Table column names are not extracted correctly in Hive JDBC storage handler
[
https://issues.apache.org/jira/browse/HIVE-24147?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17194002#comment-17194002
]
Jesus Camacho Rodriguez commented on HIVE-24147:
The patch I put together in https://github.com/apache/hive/pull/1486 is
basically a bandage.
Instead, we should probably return proper column names in the JDBC result set
metadata call. It seems the appending is controlled by
{{hive.resultset.use.unique.column.names}}, which is set to {{true}} by
default. I am not sure about the implications of setting it to {{false}}, but
an option to uniquify those column names would be to use numeral after the
column name instead of prepending the {{table alias + '.'}}. In addition, we
could rely on the {{ResultSetMetaData.getTableName}} method to return the table
name if desired. However, if any application is relying somehow on the column
names having a certain format, this may result in a change of behavior for
those.
Thus, maybe the bandage works for the time being.
> Table column names are not extracted correctly in Hive JDBC storage handler
> ---
>
> Key: HIVE-24147
> URL: https://issues.apache.org/jira/browse/HIVE-24147
> Project: Hive
> Issue Type: Bug
> Components: JDBC storage handler
>Affects Versions: 4.0.0
>Reporter: Jesus Camacho Rodriguez
>Priority: Major
> Labels: pull-request-available
> Time Spent: 10m
> Remaining Estimate: 0h
>
> It seems the `ResultSetMetaData` extracted from the query to retrieve the
> table columns names contains these columns as fully qualified names instead
> of possibly using the {{getTableName}} method. This ends up throwing the
> storage handler off and leading to exceptions, both in CBO path and non-CBO
> path.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Assigned] (HIVE-24147) Table column names are not extracted correctly in Hive JDBC storage handler
[
https://issues.apache.org/jira/browse/HIVE-24147?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jesus Camacho Rodriguez reassigned HIVE-24147:
--
Assignee: (was: Jesus Camacho Rodriguez)
> Table column names are not extracted correctly in Hive JDBC storage handler
> ---
>
> Key: HIVE-24147
> URL: https://issues.apache.org/jira/browse/HIVE-24147
> Project: Hive
> Issue Type: Bug
> Components: JDBC storage handler
>Affects Versions: 4.0.0
>Reporter: Jesus Camacho Rodriguez
>Priority: Major
> Labels: pull-request-available
> Time Spent: 10m
> Remaining Estimate: 0h
>
> It seems the `ResultSetMetaData` extracted from the query to retrieve the
> table columns names contains these columns as fully qualified names instead
> of possibly using the {{getTableName}} method. This ends up throwing the
> storage handler off and leading to exceptions, both in CBO path and non-CBO
> path.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (HIVE-24147) Table column names are not extracted correctly in Hive JDBC storage handler
[
https://issues.apache.org/jira/browse/HIVE-24147?focusedWorklogId=481899=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-481899
]
ASF GitHub Bot logged work on HIVE-24147:
-
Author: ASF GitHub Bot
Created on: 11/Sep/20 05:35
Start Date: 11/Sep/20 05:35
Worklog Time Spent: 10m
Work Description: jcamachor opened a new pull request #1486:
URL: https://github.com/apache/hive/pull/1486
…BC storage handler
### What changes were proposed in this pull request?
### Why are the changes needed?
### Does this PR introduce _any_ user-facing change?
### How was this patch tested?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 481899)
Remaining Estimate: 0h
Time Spent: 10m
> Table column names are not extracted correctly in Hive JDBC storage handler
> ---
>
> Key: HIVE-24147
> URL: https://issues.apache.org/jira/browse/HIVE-24147
> Project: Hive
> Issue Type: Bug
> Components: JDBC storage handler
>Affects Versions: 4.0.0
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
>Priority: Major
> Time Spent: 10m
> Remaining Estimate: 0h
>
> It seems the `ResultSetMetaData` extracted from the query to retrieve the
> table columns names contains these columns as fully qualified names instead
> of possibly using the {{getTableName}} method. This ends up throwing the
> storage handler off and leading to exceptions, both in CBO path and non-CBO
> path.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Assigned] (HIVE-24147) Table column names are not extracted correctly in Hive JDBC storage handler
[
https://issues.apache.org/jira/browse/HIVE-24147?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jesus Camacho Rodriguez reassigned HIVE-24147:
--
Assignee: Jesus Camacho Rodriguez
> Table column names are not extracted correctly in Hive JDBC storage handler
> ---
>
> Key: HIVE-24147
> URL: https://issues.apache.org/jira/browse/HIVE-24147
> Project: Hive
> Issue Type: Bug
> Components: JDBC storage handler
>Affects Versions: 4.0.0
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
>Priority: Major
>
> It seems the `ResultSetMetaData` extracted from the query to retrieve the
> table columns names contains these columns as fully qualified names instead
> of possibly using the {{getTableName}} method. This ends up throwing the
> storage handler off and leading to exceptions, both in CBO path and non-CBO
> path.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-24147) Table column names are not extracted correctly in Hive JDBC storage handler
[
https://issues.apache.org/jira/browse/HIVE-24147?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated HIVE-24147:
--
Labels: pull-request-available (was: )
> Table column names are not extracted correctly in Hive JDBC storage handler
> ---
>
> Key: HIVE-24147
> URL: https://issues.apache.org/jira/browse/HIVE-24147
> Project: Hive
> Issue Type: Bug
> Components: JDBC storage handler
>Affects Versions: 4.0.0
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
>Priority: Major
> Labels: pull-request-available
> Time Spent: 10m
> Remaining Estimate: 0h
>
> It seems the `ResultSetMetaData` extracted from the query to retrieve the
> table columns names contains these columns as fully qualified names instead
> of possibly using the {{getTableName}} method. This ends up throwing the
> storage handler off and leading to exceptions, both in CBO path and non-CBO
> path.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Assigned] (HIVE-24147) Table column names are not extracted correctly in Hive JDBC storage handler
[
https://issues.apache.org/jira/browse/HIVE-24147?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jesus Camacho Rodriguez reassigned HIVE-24147:
--
Assignee: (was: Jesus Camacho Rodriguez)
> Table column names are not extracted correctly in Hive JDBC storage handler
> ---
>
> Key: HIVE-24147
> URL: https://issues.apache.org/jira/browse/HIVE-24147
> Project: Hive
> Issue Type: Bug
> Components: JDBC storage handler
>Affects Versions: 4.0.0
>Reporter: Jesus Camacho Rodriguez
>Priority: Major
>
> It seems the `ResultSetMetaData` extracted from the query to retrieve the
> table columns names contains these columns as fully qualified names instead
> of possibly using the {{getTableName}} method. This ends up throwing the
> storage handler off and leading to exceptions, both in CBO path and non-CBO
> path.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Assigned] (HIVE-24147) Table column names are not extracted correctly in Hive JDBC storage handler
[
https://issues.apache.org/jira/browse/HIVE-24147?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jesus Camacho Rodriguez reassigned HIVE-24147:
--
> Table column names are not extracted correctly in Hive JDBC storage handler
> ---
>
> Key: HIVE-24147
> URL: https://issues.apache.org/jira/browse/HIVE-24147
> Project: Hive
> Issue Type: Bug
> Components: JDBC storage handler
>Affects Versions: 4.0.0
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
>Priority: Major
>
> It seems the `ResultSetMetaData` extracted from the query to retrieve the
> table columns names contains these columns as fully qualified names instead
> of possibly using the {{getTableName}} method. This ends up throwing the
> storage handler off and leading to exceptions, both in CBO path and non-CBO
> path.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (HIVE-24145) Fix preemption issues in reducers and file sink operators
[ https://issues.apache.org/jira/browse/HIVE-24145?focusedWorklogId=481894=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-481894 ] ASF GitHub Bot logged work on HIVE-24145: - Author: ASF GitHub Bot Created on: 11/Sep/20 05:14 Start Date: 11/Sep/20 05:14 Worklog Time Spent: 10m Work Description: ramesh0201 opened a new pull request #1485: URL: https://github.com/apache/hive/pull/1485 ### What changes were proposed in this pull request? ### Why are the changes needed? ### Does this PR introduce _any_ user-facing change? ### How was this patch tested? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 481894) Remaining Estimate: 0h Time Spent: 10m > Fix preemption issues in reducers and file sink operators > - > > Key: HIVE-24145 > URL: https://issues.apache.org/jira/browse/HIVE-24145 > Project: Hive > Issue Type: Bug >Reporter: Ramesh Kumar Thangarajan >Assignee: Ramesh Kumar Thangarajan >Priority: Major > Time Spent: 10m > Remaining Estimate: 0h > > There are two issues because of preemption: > # Reducers are getting reordered as part of optimizations because of which > more preemption happen > # Preemption in the middle of writing can cause the file to not close and > lead to errors when we read the file later -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-24145) Fix preemption issues in reducers and file sink operators
[ https://issues.apache.org/jira/browse/HIVE-24145?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HIVE-24145: -- Labels: pull-request-available (was: ) > Fix preemption issues in reducers and file sink operators > - > > Key: HIVE-24145 > URL: https://issues.apache.org/jira/browse/HIVE-24145 > Project: Hive > Issue Type: Bug >Reporter: Ramesh Kumar Thangarajan >Assignee: Ramesh Kumar Thangarajan >Priority: Major > Labels: pull-request-available > Time Spent: 10m > Remaining Estimate: 0h > > There are two issues because of preemption: > # Reducers are getting reordered as part of optimizations because of which > more preemption happen > # Preemption in the middle of writing can cause the file to not close and > lead to errors when we read the file later -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22758) Create database with permission error when doas set to true
[
https://issues.apache.org/jira/browse/HIVE-22758?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chiran Ravani updated HIVE-22758:
-
Fix Version/s: 4.0.0
Resolution: Duplicate
Status: Resolved (was: Patch Available)
> Create database with permission error when doas set to true
> ---
>
> Key: HIVE-22758
> URL: https://issues.apache.org/jira/browse/HIVE-22758
> Project: Hive
> Issue Type: Bug
> Components: Standalone Metastore
>Affects Versions: 3.0.0, 3.1.0
>Reporter: Chiran Ravani
>Assignee: Chiran Ravani
>Priority: Critical
> Fix For: 4.0.0
>
> Attachments: HIVE-22758.1.patch, HIVE-22758.2.patch
>
>
> With doAs set to true, running create database on external location fails
> with permission denied for write access on the directory for hive user (User
> HMS is running as).
> Steps to reproduce the issue:
> 1. Turn on, Hive run as end-user to true.
> 2. Connect to hive as some user other than admin, eg:- chiran
> 3. Create a database with external location
> {code}
> create database externaldbexample location '/user/chiran/externaldbexample'
> {code}
> The above statement fails as write access is not available to hive service
> user on HDFS as below.
> {code}
> > create database externaldbexample location '/user/chiran/externaldbexample';
> INFO : Compiling
> command(queryId=hive_20200122043626_5c95e1fd-ce00-45fd-b58d-54f5e579f87d):
> create database externaldbexample location '/user/chiran/externaldbexample'
> INFO : Semantic Analysis Completed (retrial = false)
> INFO : Returning Hive schema: Schema(fieldSchemas:null, properties:null)
> INFO : Completed compiling
> command(queryId=hive_20200122043626_5c95e1fd-ce00-45fd-b58d-54f5e579f87d);
> Time taken: 1.377 seconds
> INFO : Executing
> command(queryId=hive_20200122043626_5c95e1fd-ce00-45fd-b58d-54f5e579f87d):
> create database externaldbexample location '/user/chiran/externaldbexample'
> INFO : Starting task [Stage-0:DDL] in serial mode
> ERROR : FAILED: Execution Error, return code 1 from
> org.apache.hadoop.hive.ql.exec.DDLTask.
> MetaException(message:java.lang.reflect.UndeclaredThrowableException)
> INFO : Completed executing
> command(queryId=hive_20200122043626_5c95e1fd-ce00-45fd-b58d-54f5e579f87d);
> Time taken: 0.238 seconds
> Error: Error while processing statement: FAILED: Execution Error, return code
> 1 from org.apache.hadoop.hive.ql.exec.DDLTask.
> MetaException(message:java.lang.reflect.UndeclaredThrowableException)
> (state=08S01,code=1)
> {code}
> From Hive Metastore service log, below is seen.
> {code}
> 2020-01-22T04:36:27,870 WARN [pool-6-thread-6]: metastore.ObjectStore
> (ObjectStore.java:getDatabase(1010)) - Failed to get database
> hive.externaldbexample, returning NoSuchObjectExcept
> ion
> 2020-01-22T04:36:27,898 INFO [pool-6-thread-6]: metastore.HiveMetaStore
> (HiveMetaStore.java:run(1339)) - Creating database path in managed directory
> hdfs://c470-node2.squadron.support.
> hortonworks.com:8020/user/chiran/externaldbexample
> 2020-01-22T04:36:27,903 INFO [pool-6-thread-6]: utils.FileUtils
> (FileUtils.java:mkdir(170)) - Creating directory if it doesn't exist:
> hdfs://namenodeaddress:8020/user/chiran/externaldbexample
> 2020-01-22T04:36:27,932 ERROR [pool-6-thread-6]: utils.MetaStoreUtils
> (MetaStoreUtils.java:logAndThrowMetaException(169)) - Got exception:
> org.apache.hadoop.security.AccessControlException Permission denied:
> user=hive, access=WRITE, inode="/user/chiran":chiran:chiran:drwxr-xr-x
> at
> org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.check(FSPermissionChecker.java:399)
> at
> org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.checkPermission(FSPermissionChecker.java:255)
> at
> org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.checkPermission(FSPermissionChecker.java:193)
> at
> org.apache.hadoop.hdfs.server.namenode.FSDirectory.checkPermission(FSDirectory.java:1859)
> at
> org.apache.hadoop.hdfs.server.namenode.FSDirectory.checkPermission(FSDirectory.java:1843)
> at
> org.apache.hadoop.hdfs.server.namenode.FSDirectory.checkAncestorAccess(FSDirectory.java:1802)
> at
> org.apache.hadoop.hdfs.server.namenode.FSDirMkdirOp.mkdirs(FSDirMkdirOp.java:59)
> at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.mkdirs(FSNamesystem.java:3150)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.mkdirs(NameNodeRpcServer.java:1126)
> at
> org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.mkdirs(ClientNamenodeProtocolServerSideTranslatorPB.java:707)
> at
>
[jira] [Commented] (HIVE-22758) Create database with permission error when doas set to true
[
https://issues.apache.org/jira/browse/HIVE-22758?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17193985#comment-17193985
]
Chiran Ravani commented on HIVE-22758:
--
Problem no longer exists after HIVE-23387. Closing Jira as duplicate of
HIVE-23387.
> Create database with permission error when doas set to true
> ---
>
> Key: HIVE-22758
> URL: https://issues.apache.org/jira/browse/HIVE-22758
> Project: Hive
> Issue Type: Bug
> Components: Standalone Metastore
>Affects Versions: 3.0.0, 3.1.0
>Reporter: Chiran Ravani
>Assignee: Chiran Ravani
>Priority: Critical
> Attachments: HIVE-22758.1.patch, HIVE-22758.2.patch
>
>
> With doAs set to true, running create database on external location fails
> with permission denied for write access on the directory for hive user (User
> HMS is running as).
> Steps to reproduce the issue:
> 1. Turn on, Hive run as end-user to true.
> 2. Connect to hive as some user other than admin, eg:- chiran
> 3. Create a database with external location
> {code}
> create database externaldbexample location '/user/chiran/externaldbexample'
> {code}
> The above statement fails as write access is not available to hive service
> user on HDFS as below.
> {code}
> > create database externaldbexample location '/user/chiran/externaldbexample';
> INFO : Compiling
> command(queryId=hive_20200122043626_5c95e1fd-ce00-45fd-b58d-54f5e579f87d):
> create database externaldbexample location '/user/chiran/externaldbexample'
> INFO : Semantic Analysis Completed (retrial = false)
> INFO : Returning Hive schema: Schema(fieldSchemas:null, properties:null)
> INFO : Completed compiling
> command(queryId=hive_20200122043626_5c95e1fd-ce00-45fd-b58d-54f5e579f87d);
> Time taken: 1.377 seconds
> INFO : Executing
> command(queryId=hive_20200122043626_5c95e1fd-ce00-45fd-b58d-54f5e579f87d):
> create database externaldbexample location '/user/chiran/externaldbexample'
> INFO : Starting task [Stage-0:DDL] in serial mode
> ERROR : FAILED: Execution Error, return code 1 from
> org.apache.hadoop.hive.ql.exec.DDLTask.
> MetaException(message:java.lang.reflect.UndeclaredThrowableException)
> INFO : Completed executing
> command(queryId=hive_20200122043626_5c95e1fd-ce00-45fd-b58d-54f5e579f87d);
> Time taken: 0.238 seconds
> Error: Error while processing statement: FAILED: Execution Error, return code
> 1 from org.apache.hadoop.hive.ql.exec.DDLTask.
> MetaException(message:java.lang.reflect.UndeclaredThrowableException)
> (state=08S01,code=1)
> {code}
> From Hive Metastore service log, below is seen.
> {code}
> 2020-01-22T04:36:27,870 WARN [pool-6-thread-6]: metastore.ObjectStore
> (ObjectStore.java:getDatabase(1010)) - Failed to get database
> hive.externaldbexample, returning NoSuchObjectExcept
> ion
> 2020-01-22T04:36:27,898 INFO [pool-6-thread-6]: metastore.HiveMetaStore
> (HiveMetaStore.java:run(1339)) - Creating database path in managed directory
> hdfs://c470-node2.squadron.support.
> hortonworks.com:8020/user/chiran/externaldbexample
> 2020-01-22T04:36:27,903 INFO [pool-6-thread-6]: utils.FileUtils
> (FileUtils.java:mkdir(170)) - Creating directory if it doesn't exist:
> hdfs://namenodeaddress:8020/user/chiran/externaldbexample
> 2020-01-22T04:36:27,932 ERROR [pool-6-thread-6]: utils.MetaStoreUtils
> (MetaStoreUtils.java:logAndThrowMetaException(169)) - Got exception:
> org.apache.hadoop.security.AccessControlException Permission denied:
> user=hive, access=WRITE, inode="/user/chiran":chiran:chiran:drwxr-xr-x
> at
> org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.check(FSPermissionChecker.java:399)
> at
> org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.checkPermission(FSPermissionChecker.java:255)
> at
> org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.checkPermission(FSPermissionChecker.java:193)
> at
> org.apache.hadoop.hdfs.server.namenode.FSDirectory.checkPermission(FSDirectory.java:1859)
> at
> org.apache.hadoop.hdfs.server.namenode.FSDirectory.checkPermission(FSDirectory.java:1843)
> at
> org.apache.hadoop.hdfs.server.namenode.FSDirectory.checkAncestorAccess(FSDirectory.java:1802)
> at
> org.apache.hadoop.hdfs.server.namenode.FSDirMkdirOp.mkdirs(FSDirMkdirOp.java:59)
> at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.mkdirs(FSNamesystem.java:3150)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.mkdirs(NameNodeRpcServer.java:1126)
> at
> org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.mkdirs(ClientNamenodeProtocolServerSideTranslatorPB.java:707)
> at
>
[jira] [Work logged] (HIVE-22290) ObjectStore.cleanWriteNotificationEvents and ObjectStore.cleanupEvents OutOfMemory on large number of pending events
[ https://issues.apache.org/jira/browse/HIVE-22290?focusedWorklogId=481878=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-481878 ] ASF GitHub Bot logged work on HIVE-22290: - Author: ASF GitHub Bot Created on: 11/Sep/20 03:58 Start Date: 11/Sep/20 03:58 Worklog Time Spent: 10m Work Description: nareshpr opened a new pull request #1484: URL: https://github.com/apache/hive/pull/1484 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 481878) Remaining Estimate: 0h Time Spent: 10m > ObjectStore.cleanWriteNotificationEvents and ObjectStore.cleanupEvents > OutOfMemory on large number of pending events > > > Key: HIVE-22290 > URL: https://issues.apache.org/jira/browse/HIVE-22290 > Project: Hive > Issue Type: Bug > Components: HCatalog, repl >Affects Versions: 4.0.0 >Reporter: Thomas Prelle >Assignee: Naresh P R >Priority: Major > Time Spent: 10m > Remaining Estimate: 0h > > As in [https://jira.apache.org/jira/browse/HIVE-19430] if there are large > number of events that haven't been cleaned up for some reason, then > ObjectStore.cleanWriteNotificationEvents() and ObjectStore.cleanupEvents can > run out of memory while it loads all the events to be deleted. > It should fetch events in batches. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22290) ObjectStore.cleanWriteNotificationEvents and ObjectStore.cleanupEvents OutOfMemory on large number of pending events
[ https://issues.apache.org/jira/browse/HIVE-22290?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HIVE-22290: -- Labels: pull-request-available (was: ) > ObjectStore.cleanWriteNotificationEvents and ObjectStore.cleanupEvents > OutOfMemory on large number of pending events > > > Key: HIVE-22290 > URL: https://issues.apache.org/jira/browse/HIVE-22290 > Project: Hive > Issue Type: Bug > Components: HCatalog, repl >Affects Versions: 4.0.0 >Reporter: Thomas Prelle >Assignee: Naresh P R >Priority: Major > Labels: pull-request-available > Time Spent: 10m > Remaining Estimate: 0h > > As in [https://jira.apache.org/jira/browse/HIVE-19430] if there are large > number of events that haven't been cleaned up for some reason, then > ObjectStore.cleanWriteNotificationEvents() and ObjectStore.cleanupEvents can > run out of memory while it loads all the events to be deleted. > It should fetch events in batches. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-24146) Cleanup TaskExecutionException in GenericUDTFExplode
[ https://issues.apache.org/jira/browse/HIVE-24146?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HIVE-24146: -- Labels: pull-request-available (was: ) > Cleanup TaskExecutionException in GenericUDTFExplode > > > Key: HIVE-24146 > URL: https://issues.apache.org/jira/browse/HIVE-24146 > Project: Hive > Issue Type: Improvement > Components: UDF >Reporter: Zhihua Deng >Assignee: Zhihua Deng >Priority: Minor > Labels: pull-request-available > Time Spent: 10m > Remaining Estimate: 0h > > - Remove TaskExecutionException, which may be not used anymore; > - Remove the default handling in GenericUDTFExplode#process, which has been > verified during the function initializing. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (HIVE-24146) Cleanup TaskExecutionException in GenericUDTFExplode
[ https://issues.apache.org/jira/browse/HIVE-24146?focusedWorklogId=481875=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-481875 ] ASF GitHub Bot logged work on HIVE-24146: - Author: ASF GitHub Bot Created on: 11/Sep/20 03:43 Start Date: 11/Sep/20 03:43 Worklog Time Spent: 10m Work Description: dengzhhu653 opened a new pull request #1483: URL: https://github.com/apache/hive/pull/1483 ### What changes were proposed in this pull request? ### Why are the changes needed? ### Does this PR introduce _any_ user-facing change? ### How was this patch tested? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 481875) Remaining Estimate: 0h Time Spent: 10m > Cleanup TaskExecutionException in GenericUDTFExplode > > > Key: HIVE-24146 > URL: https://issues.apache.org/jira/browse/HIVE-24146 > Project: Hive > Issue Type: Improvement > Components: UDF >Reporter: Zhihua Deng >Assignee: Zhihua Deng >Priority: Minor > Time Spent: 10m > Remaining Estimate: 0h > > - Remove TaskExecutionException, which may be not used anymore; > - Remove the default handling in GenericUDTFExplode#process, which has been > verified during the function initializing. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Assigned] (HIVE-24146) Cleanup TaskExecutionException in GenericUDTFExplode
[ https://issues.apache.org/jira/browse/HIVE-24146?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhihua Deng reassigned HIVE-24146: -- > Cleanup TaskExecutionException in GenericUDTFExplode > > > Key: HIVE-24146 > URL: https://issues.apache.org/jira/browse/HIVE-24146 > Project: Hive > Issue Type: Improvement > Components: UDF >Reporter: Zhihua Deng >Assignee: Zhihua Deng >Priority: Minor > > - Remove TaskExecutionException, which may be not used anymore; > - Remove the default handling in GenericUDTFExplode#process, which has been > verified during the function initializing. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Assigned] (HIVE-24145) Fix preemption issues in reducers and file sink operators
[ https://issues.apache.org/jira/browse/HIVE-24145?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ramesh Kumar Thangarajan reassigned HIVE-24145: --- > Fix preemption issues in reducers and file sink operators > - > > Key: HIVE-24145 > URL: https://issues.apache.org/jira/browse/HIVE-24145 > Project: Hive > Issue Type: Bug >Reporter: Ramesh Kumar Thangarajan >Assignee: Ramesh Kumar Thangarajan >Priority: Major > > There are two issues because of preemption: > # Reducers are getting reordered as part of optimizations because of which > more preemption happen > # Preemption in the middle of writing can cause the file to not close and > lead to errors when we read the file later -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Assigned] (HIVE-24107) Fix typo in ReloadFunctionsOperation
[ https://issues.apache.org/jira/browse/HIVE-24107?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhihua Deng reassigned HIVE-24107: -- Assignee: Zhihua Deng > Fix typo in ReloadFunctionsOperation > > > Key: HIVE-24107 > URL: https://issues.apache.org/jira/browse/HIVE-24107 > Project: Hive > Issue Type: Improvement > Components: HiveServer2 >Reporter: Zhihua Deng >Assignee: Zhihua Deng >Priority: Minor > Labels: pull-request-available > Time Spent: 40m > Remaining Estimate: 0h > > Hive.get() will register all functions as doRegisterAllFns is true, so > Hive.get().reloadFunctions() may load all functions from metastore twice, use > Hive.get(false) instead may be better. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-23800) Add hooks when HiveServer2 stops due to OutOfMemoryError
[ https://issues.apache.org/jira/browse/HIVE-23800?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhihua Deng updated HIVE-23800: --- Priority: Minor (was: Major) > Add hooks when HiveServer2 stops due to OutOfMemoryError > > > Key: HIVE-23800 > URL: https://issues.apache.org/jira/browse/HIVE-23800 > Project: Hive > Issue Type: Improvement > Components: HiveServer2 >Reporter: Zhihua Deng >Priority: Minor > Labels: pull-request-available > Time Spent: 3h 40m > Remaining Estimate: 0h > > Make oom hook an interface of HiveServer2, so user can implement the hook to > do something before HS2 stops, such as dumping the heap or altering the > devops. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-24044) Implement listPartitionNames on temporary tables
[
https://issues.apache.org/jira/browse/HIVE-24044?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zhihua Deng updated HIVE-24044:
---
Priority: Minor (was: Major)
> Implement listPartitionNames on temporary tables
> -
>
> Key: HIVE-24044
> URL: https://issues.apache.org/jira/browse/HIVE-24044
> Project: Hive
> Issue Type: Task
> Components: HiveServer2
>Affects Versions: 4.0.0
>Reporter: Zhihua Deng
>Priority: Minor
> Labels: pull-request-available
> Time Spent: 20m
> Remaining Estimate: 0h
>
> Temporary tables can have their own partitions, and IMetaStoreClient use
> {code:java}
> List listPartitionNames(PartitionsByExprRequest request){code}
> to filter or sort the results. This method can be implemented on temporary
> tables.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-24044) Implement listPartitionNames on temporary tables
[
https://issues.apache.org/jira/browse/HIVE-24044?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zhihua Deng updated HIVE-24044:
---
Summary: Implement listPartitionNames on temporary tables (was: Implement
listPartitionNames with filter or order on temporary tables )
> Implement listPartitionNames on temporary tables
> -
>
> Key: HIVE-24044
> URL: https://issues.apache.org/jira/browse/HIVE-24044
> Project: Hive
> Issue Type: Task
> Components: HiveServer2
>Affects Versions: 4.0.0
>Reporter: Zhihua Deng
>Priority: Major
> Labels: pull-request-available
> Time Spent: 20m
> Remaining Estimate: 0h
>
> Temporary tables can have their own partitions, and IMetaStoreClient use
> {code:java}
> List listPartitionNames(PartitionsByExprRequest request){code}
> to filter or sort the results. This method can be implemented on temporary
> tables.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-24044) Implement listPartitionNames with filter or order on temporary tables
[
https://issues.apache.org/jira/browse/HIVE-24044?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zhihua Deng updated HIVE-24044:
---
Component/s: (was: Metastore)
HiveServer2
> Implement listPartitionNames with filter or order on temporary tables
> --
>
> Key: HIVE-24044
> URL: https://issues.apache.org/jira/browse/HIVE-24044
> Project: Hive
> Issue Type: Task
> Components: HiveServer2
>Affects Versions: 4.0.0
>Reporter: Zhihua Deng
>Priority: Minor
> Labels: pull-request-available
> Time Spent: 20m
> Remaining Estimate: 0h
>
> Temporary tables can have their own partitions, and IMetaStoreClient use
> {code:java}
> List listPartitionNames(PartitionsByExprRequest request){code}
> to filter or sort the results. This method can be implemented on temporary
> tables.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-24044) Implement listPartitionNames with filter or order on temporary tables
[
https://issues.apache.org/jira/browse/HIVE-24044?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zhihua Deng updated HIVE-24044:
---
Priority: Major (was: Minor)
> Implement listPartitionNames with filter or order on temporary tables
> --
>
> Key: HIVE-24044
> URL: https://issues.apache.org/jira/browse/HIVE-24044
> Project: Hive
> Issue Type: Task
> Components: HiveServer2
>Affects Versions: 4.0.0
>Reporter: Zhihua Deng
>Priority: Major
> Labels: pull-request-available
> Time Spent: 20m
> Remaining Estimate: 0h
>
> Temporary tables can have their own partitions, and IMetaStoreClient use
> {code:java}
> List listPartitionNames(PartitionsByExprRequest request){code}
> to filter or sort the results. This method can be implemented on temporary
> tables.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (HIVE-23700) HiveConf static initialization fails when JAR URI is opaque
[ https://issues.apache.org/jira/browse/HIVE-23700?focusedWorklogId=481817=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-481817 ] ASF GitHub Bot logged work on HIVE-23700: - Author: ASF GitHub Bot Created on: 11/Sep/20 00:43 Start Date: 11/Sep/20 00:43 Worklog Time Spent: 10m Work Description: github-actions[bot] commented on pull request #1240: URL: https://github.com/apache/hive/pull/1240#issuecomment-690805738 This pull request has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Feel free to reach out on the d...@hive.apache.org list if the patch is in need of reviews. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 481817) Remaining Estimate: 118h 10m (was: 118h 20m) Time Spent: 1h 50m (was: 1h 40m) > HiveConf static initialization fails when JAR URI is opaque > --- > > Key: HIVE-23700 > URL: https://issues.apache.org/jira/browse/HIVE-23700 > Project: Hive > Issue Type: Bug > Components: Hive >Affects Versions: 2.3.7 >Reporter: Francisco Guerrero >Assignee: Francisco Guerrero >Priority: Minor > Labels: pull-request-available > Attachments: HIVE-23700.1.patch > > Original Estimate: 120h > Time Spent: 1h 50m > Remaining Estimate: 118h 10m > > HiveConf static initialization fails when the jar URI is opaque, for example > when it's embedded as a fat jar in a spring boot application. Then > initialization of the HiveConf static block fails and the HiveConf class does > not get classloaded. The opaque URI in my case looks like this > _jar:file:/usr/local/server/some-service-jar.jar!/BOOT-INF/lib/hive-common-2.3.7.jar!/_ > HiveConf#findConfigFile should be able to handle `IllegalArgumentException` > when the jar `URI` provided to `File` throws the exception. > To surface this issue three conditions need to be met. > 1. hive-site.xml should not be on the classpath > 2. hive-site.xml should not be on "HIVE_CONF_DIR" > 3. hive-site.xml should not be on "HIVE_HOME" -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Assigned] (HIVE-24144) getIdentifierQuoteString in HiveDatabaseMetaData returns incorrect value
[
https://issues.apache.org/jira/browse/HIVE-24144?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Kishen Das reassigned HIVE-24144:
-
Assignee: Kishen Das
> getIdentifierQuoteString in HiveDatabaseMetaData returns incorrect value
>
>
> Key: HIVE-24144
> URL: https://issues.apache.org/jira/browse/HIVE-24144
> Project: Hive
> Issue Type: Bug
> Components: JDBC
>Reporter: Jesus Camacho Rodriguez
>Assignee: Kishen Das
>Priority: Major
>
> {code}
> public String getIdentifierQuoteString() throws SQLException {
> return " ";
> }
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Assigned] (HIVE-24144) getIdentifierQuoteString in HiveDatabaseMetaData returns incorrect value
[
https://issues.apache.org/jira/browse/HIVE-24144?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jesus Camacho Rodriguez reassigned HIVE-24144:
--
> getIdentifierQuoteString in HiveDatabaseMetaData returns incorrect value
>
>
> Key: HIVE-24144
> URL: https://issues.apache.org/jira/browse/HIVE-24144
> Project: Hive
> Issue Type: Bug
> Components: JDBC
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
>Priority: Major
>
> {code}
> public String getIdentifierQuoteString() throws SQLException {
> return " ";
> }
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Assigned] (HIVE-24144) getIdentifierQuoteString in HiveDatabaseMetaData returns incorrect value
[
https://issues.apache.org/jira/browse/HIVE-24144?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jesus Camacho Rodriguez reassigned HIVE-24144:
--
Assignee: (was: Jesus Camacho Rodriguez)
> getIdentifierQuoteString in HiveDatabaseMetaData returns incorrect value
>
>
> Key: HIVE-24144
> URL: https://issues.apache.org/jira/browse/HIVE-24144
> Project: Hive
> Issue Type: Bug
> Components: JDBC
>Reporter: Jesus Camacho Rodriguez
>Priority: Major
>
> {code}
> public String getIdentifierQuoteString() throws SQLException {
> return " ";
> }
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-24131) Use original src location always when data copy runs on target
[ https://issues.apache.org/jira/browse/HIVE-24131?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pravin Sinha updated HIVE-24131: Attachment: HIVE-24131.02.patch > Use original src location always when data copy runs on target > --- > > Key: HIVE-24131 > URL: https://issues.apache.org/jira/browse/HIVE-24131 > Project: Hive > Issue Type: Bug >Reporter: Pravin Sinha >Assignee: Pravin Sinha >Priority: Major > Labels: pull-request-available > Attachments: HIVE-24131.01.patch, HIVE-24131.02.patch > > Time Spent: 20m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-24143) Include convention in JDBC converter operator in Calcite plan
[
https://issues.apache.org/jira/browse/HIVE-24143?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated HIVE-24143:
--
Labels: pull-request-available (was: )
> Include convention in JDBC converter operator in Calcite plan
> -
>
> Key: HIVE-24143
> URL: https://issues.apache.org/jira/browse/HIVE-24143
> Project: Hive
> Issue Type: Improvement
> Components: CBO
>Affects Versions: 4.0.0
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
>Priority: Minor
> Labels: pull-request-available
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Among others, it will be useful to debug the dialect being chosen for query
> generation. For instance:
> {code}
> HiveProject(jdbc_type_conversion_table1.ikey=[$0],
> jdbc_type_conversion_table1.bkey=[$1], jdbc_type_conversion_table1.fkey=[$2],
> jdbc_type_conversion_table1.dkey=[$3],
> jdbc_type_conversion_table1.chkey=[$4],
> jdbc_type_conversion_table1.dekey=[$5],
> jdbc_type_conversion_table1.dtkey=[$6], jdbc_type_conversion_table1.tkey=[$7])
> HiveProject(ikey=[$0], bkey=[$1], fkey=[$2], dkey=[$3], chkey=[$4],
> dekey=[$5], dtkey=[$6], tkey=[$7])
> ->HiveJdbcConverter(convention=[JDBC.DERBY])
> JdbcHiveTableScan(table=[[default, jdbc_type_conversion_table1]],
> table:alias=[jdbc_type_conversion_table1])
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (HIVE-24143) Include convention in JDBC converter operator in Calcite plan
[
https://issues.apache.org/jira/browse/HIVE-24143?focusedWorklogId=481696=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-481696
]
ASF GitHub Bot logged work on HIVE-24143:
-
Author: ASF GitHub Bot
Created on: 10/Sep/20 21:01
Start Date: 10/Sep/20 21:01
Worklog Time Spent: 10m
Work Description: jcamachor opened a new pull request #1482:
URL: https://github.com/apache/hive/pull/1482
…plan
### What changes were proposed in this pull request?
### Why are the changes needed?
### Does this PR introduce _any_ user-facing change?
### How was this patch tested?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 481696)
Remaining Estimate: 0h
Time Spent: 10m
> Include convention in JDBC converter operator in Calcite plan
> -
>
> Key: HIVE-24143
> URL: https://issues.apache.org/jira/browse/HIVE-24143
> Project: Hive
> Issue Type: Improvement
> Components: CBO
>Affects Versions: 4.0.0
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
>Priority: Minor
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Among others, it will be useful to debug the dialect being chosen for query
> generation. For instance:
> {code}
> HiveProject(jdbc_type_conversion_table1.ikey=[$0],
> jdbc_type_conversion_table1.bkey=[$1], jdbc_type_conversion_table1.fkey=[$2],
> jdbc_type_conversion_table1.dkey=[$3],
> jdbc_type_conversion_table1.chkey=[$4],
> jdbc_type_conversion_table1.dekey=[$5],
> jdbc_type_conversion_table1.dtkey=[$6], jdbc_type_conversion_table1.tkey=[$7])
> HiveProject(ikey=[$0], bkey=[$1], fkey=[$2], dkey=[$3], chkey=[$4],
> dekey=[$5], dtkey=[$6], tkey=[$7])
> ->HiveJdbcConverter(convention=[JDBC.DERBY])
> JdbcHiveTableScan(table=[[default, jdbc_type_conversion_table1]],
> table:alias=[jdbc_type_conversion_table1])
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-24143) Include convention in JDBC converter operator in Calcite plan
[
https://issues.apache.org/jira/browse/HIVE-24143?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jesus Camacho Rodriguez updated HIVE-24143:
---
Status: Patch Available (was: In Progress)
> Include convention in JDBC converter operator in Calcite plan
> -
>
> Key: HIVE-24143
> URL: https://issues.apache.org/jira/browse/HIVE-24143
> Project: Hive
> Issue Type: Improvement
> Components: CBO
>Affects Versions: 4.0.0
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
>Priority: Minor
>
> Among others, it will be useful to debug the dialect being chosen for query
> generation. For instance:
> {code}
> HiveProject(jdbc_type_conversion_table1.ikey=[$0],
> jdbc_type_conversion_table1.bkey=[$1], jdbc_type_conversion_table1.fkey=[$2],
> jdbc_type_conversion_table1.dkey=[$3],
> jdbc_type_conversion_table1.chkey=[$4],
> jdbc_type_conversion_table1.dekey=[$5],
> jdbc_type_conversion_table1.dtkey=[$6], jdbc_type_conversion_table1.tkey=[$7])
> HiveProject(ikey=[$0], bkey=[$1], fkey=[$2], dkey=[$3], chkey=[$4],
> dekey=[$5], dtkey=[$6], tkey=[$7])
> ->HiveJdbcConverter(convention=[JDBC.DERBY])
> JdbcHiveTableScan(table=[[default, jdbc_type_conversion_table1]],
> table:alias=[jdbc_type_conversion_table1])
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work started] (HIVE-24143) Include convention in JDBC converter operator in Calcite plan
[
https://issues.apache.org/jira/browse/HIVE-24143?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Work on HIVE-24143 started by Jesus Camacho Rodriguez.
--
> Include convention in JDBC converter operator in Calcite plan
> -
>
> Key: HIVE-24143
> URL: https://issues.apache.org/jira/browse/HIVE-24143
> Project: Hive
> Issue Type: Improvement
> Components: CBO
>Affects Versions: 4.0.0
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
>Priority: Minor
>
> Among others, it will be useful to debug the dialect being chosen for query
> generation. For instance:
> {code}
> HiveProject(jdbc_type_conversion_table1.ikey=[$0],
> jdbc_type_conversion_table1.bkey=[$1], jdbc_type_conversion_table1.fkey=[$2],
> jdbc_type_conversion_table1.dkey=[$3],
> jdbc_type_conversion_table1.chkey=[$4],
> jdbc_type_conversion_table1.dekey=[$5],
> jdbc_type_conversion_table1.dtkey=[$6], jdbc_type_conversion_table1.tkey=[$7])
> HiveProject(ikey=[$0], bkey=[$1], fkey=[$2], dkey=[$3], chkey=[$4],
> dekey=[$5], dtkey=[$6], tkey=[$7])
> ->HiveJdbcConverter(convention=[JDBC.DERBY])
> JdbcHiveTableScan(table=[[default, jdbc_type_conversion_table1]],
> table:alias=[jdbc_type_conversion_table1])
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Assigned] (HIVE-24143) Include convention in JDBC converter operator in Calcite plan
[
https://issues.apache.org/jira/browse/HIVE-24143?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jesus Camacho Rodriguez reassigned HIVE-24143:
--
> Include convention in JDBC converter operator in Calcite plan
> -
>
> Key: HIVE-24143
> URL: https://issues.apache.org/jira/browse/HIVE-24143
> Project: Hive
> Issue Type: Improvement
> Components: CBO
>Affects Versions: 4.0.0
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
>Priority: Minor
>
> Among others, it will be useful to debug the dialect being chosen for query
> generation. For instance:
> {code}
> HiveProject(jdbc_type_conversion_table1.ikey=[$0],
> jdbc_type_conversion_table1.bkey=[$1], jdbc_type_conversion_table1.fkey=[$2],
> jdbc_type_conversion_table1.dkey=[$3],
> jdbc_type_conversion_table1.chkey=[$4],
> jdbc_type_conversion_table1.dekey=[$5],
> jdbc_type_conversion_table1.dtkey=[$6], jdbc_type_conversion_table1.tkey=[$7])
> HiveProject(ikey=[$0], bkey=[$1], fkey=[$2], dkey=[$3], chkey=[$4],
> dekey=[$5], dtkey=[$6], tkey=[$7])
> ->HiveJdbcConverter(convention=[JDBC.DERBY])
> JdbcHiveTableScan(table=[[default, jdbc_type_conversion_table1]],
> table:alias=[jdbc_type_conversion_table1])
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-24141) Cannot retrieve results using a Select * on a given table
[ https://issues.apache.org/jira/browse/HIVE-24141?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17193766#comment-17193766 ] srinivas vemulapalli commented on HIVE-24141: - This is an issue with formatting, so closing this as it is irrelevant and not a bug. > Cannot retrieve results using a Select * on a given table > - > > Key: HIVE-24141 > URL: https://issues.apache.org/jira/browse/HIVE-24141 > Project: Hive > Issue Type: Bug > Components: Hive, Parquet >Reporter: srinivas vemulapalli >Priority: Blocker > > Receiving following error on querying select * on given table > > * Bad status for request TFetchResultsReq(fetchType=0, > operationHandle=TOperationHandle(hasResultSet=True, modifiedRowCount=None, > operationType=0, > operationId=THandleIdentifier(secret='\xaa\xd5d\x0e\xae\xd4E\x9e\xad\xc5w\x80Y\x0f\x07\xcd', > guid='1\x80\xc1\xa0\x9cKM\xab\x85\x11\xaa\x95\x19\xcd8\xb4')), > orientation=4, maxRows=100): TFetchResultsResp(status=TStatus(errorCode=0, > errorMessage='java.io.IOException: > org.apache.hadoop.hive.ql.metadata.HiveException: > java.lang.ClassCastException', sqlState=None, > infoMessages=['*org.apache.hive.service.cli.HiveSQLException:java.io.IOException: > org.apache.hadoop.hive.ql.metadata.HiveException: > java.lang.ClassCastException:25:24', > 'org.apache.hive.service.cli.operation.SQLOperation:getNextRowSet:SQLOperation.java:463', > > 'org.apache.hive.service.cli.operation.OperationManager:getOperationNextRowSet:OperationManager.java:294', > > 'org.apache.hive.service.cli.session.HiveSessionImpl:fetchResults:HiveSessionImpl.java:769', > 'sun.reflect.GeneratedMethodAccessor59:invoke::-1', > 'sun.reflect.DelegatingMethodAccessorImpl:invoke:DelegatingMethodAccessorImpl.java:43', > 'java.lang.reflect.Method:invoke:Method.java:498', > 'org.apache.hive.service.cli.session.HiveSessionProxy:invoke:HiveSessionProxy.java:78', > > 'org.apache.hive.service.cli.session.HiveSessionProxy:access$000:HiveSessionProxy.java:36', > > 'org.apache.hive.service.cli.session.HiveSessionProxy$1:run:HiveSessionProxy.java:63', > 'java.security.AccessController:doPrivileged:AccessController.java:-2', > 'javax.security.auth.Subject:doAs:Subject.java:422', > 'org.apache.hadoop.security.UserGroupInformation:doAs:UserGroupInformation.java:1924', > > 'org.apache.hive.service.cli.session.HiveSessionProxy:invoke:HiveSessionProxy.java:59', > 'com.sun.proxy.$Proxy23:fetchResults::-1', > 'org.apache.hive.service.cli.CLIService:fetchResults:CLIService.java:462', > 'org.apache.hive.service.cli.thrift.ThriftCLIService:FetchResults:ThriftCLIService.java:696', > > 'org.apache.hive.service.cli.thrift.TCLIService$Processor$FetchResults:getResult:TCLIService.java:1553', > > 'org.apache.hive.service.cli.thrift.TCLIService$Processor$FetchResults:getResult:TCLIService.java:1538', > 'org.apache.thrift.ProcessFunction:process:ProcessFunction.java:39', > 'org.apache.thrift.TBaseProcessor:process:TBaseProcessor.java:39', > 'org.apache.hadoop.hive.thrift.HadoopThriftAuthBridge$Server$TUGIAssumingProcessor:process:HadoopThriftAuthBridge.java:747', > > 'org.apache.thrift.server.TThreadPoolServer$WorkerProcess:run:TThreadPoolServer.java:286', > > 'java.util.concurrent.ThreadPoolExecutor:runWorker:ThreadPoolExecutor.java:1142', > > 'java.util.concurrent.ThreadPoolExecutor$Worker:run:ThreadPoolExecutor.java:617', > 'java.lang.Thread:run:Thread.java:745', > '*java.io.IOException:org.apache.hadoop.hive.ql.metadata.HiveException: > java.lang.ClassCastException:27:2', > 'org.apache.hadoop.hive.ql.exec.FetchTask:fetch:FetchTask.java:154', > 'org.apache.hadoop.hive.ql.Driver:getResults:Driver.java:2071', > 'org.apache.hive.service.cli.operation.SQLOperation:getNextRowSet:SQLOperation.java:458', > > '*org.apache.hadoop.hive.ql.metadata.HiveException:java.lang.ClassCastException:34:7', > > 'org.apache.hadoop.hive.ql.exec.ListSinkOperator:processOp:ListSinkOperator.java:90', > 'org.apache.hadoop.hive.ql.exec.Operator:forward:Operator.java:815', > 'org.apache.hadoop.hive.ql.exec.SelectOperator:processOp:SelectOperator.java:84', > 'org.apache.hadoop.hive.ql.exec.Operator:forward:Operator.java:815', > 'org.apache.hadoop.hive.ql.exec.TableScanOperator:processOp:TableScanOperator.java:98', > > 'org.apache.hadoop.hive.ql.exec.FetchOperator:pushRow:FetchOperator.java:425', > > 'org.apache.hadoop.hive.ql.exec.FetchOperator:pushRow:FetchOperator.java:417', > 'org.apache.hadoop.hive.ql.exec.FetchTask:fetch:FetchTask.java:140', > '*java.lang.ClassCastException:null:0:-1'], statusCode=3), results=None, > hasMoreRows=None) -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (HIVE-24141) Cannot retrieve results using a Select * on a given table
[ https://issues.apache.org/jira/browse/HIVE-24141?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] srinivas vemulapalli resolved HIVE-24141. - Resolution: Invalid > Cannot retrieve results using a Select * on a given table > - > > Key: HIVE-24141 > URL: https://issues.apache.org/jira/browse/HIVE-24141 > Project: Hive > Issue Type: Bug > Components: Hive, Parquet >Reporter: srinivas vemulapalli >Priority: Blocker > > Receiving following error on querying select * on given table > > * Bad status for request TFetchResultsReq(fetchType=0, > operationHandle=TOperationHandle(hasResultSet=True, modifiedRowCount=None, > operationType=0, > operationId=THandleIdentifier(secret='\xaa\xd5d\x0e\xae\xd4E\x9e\xad\xc5w\x80Y\x0f\x07\xcd', > guid='1\x80\xc1\xa0\x9cKM\xab\x85\x11\xaa\x95\x19\xcd8\xb4')), > orientation=4, maxRows=100): TFetchResultsResp(status=TStatus(errorCode=0, > errorMessage='java.io.IOException: > org.apache.hadoop.hive.ql.metadata.HiveException: > java.lang.ClassCastException', sqlState=None, > infoMessages=['*org.apache.hive.service.cli.HiveSQLException:java.io.IOException: > org.apache.hadoop.hive.ql.metadata.HiveException: > java.lang.ClassCastException:25:24', > 'org.apache.hive.service.cli.operation.SQLOperation:getNextRowSet:SQLOperation.java:463', > > 'org.apache.hive.service.cli.operation.OperationManager:getOperationNextRowSet:OperationManager.java:294', > > 'org.apache.hive.service.cli.session.HiveSessionImpl:fetchResults:HiveSessionImpl.java:769', > 'sun.reflect.GeneratedMethodAccessor59:invoke::-1', > 'sun.reflect.DelegatingMethodAccessorImpl:invoke:DelegatingMethodAccessorImpl.java:43', > 'java.lang.reflect.Method:invoke:Method.java:498', > 'org.apache.hive.service.cli.session.HiveSessionProxy:invoke:HiveSessionProxy.java:78', > > 'org.apache.hive.service.cli.session.HiveSessionProxy:access$000:HiveSessionProxy.java:36', > > 'org.apache.hive.service.cli.session.HiveSessionProxy$1:run:HiveSessionProxy.java:63', > 'java.security.AccessController:doPrivileged:AccessController.java:-2', > 'javax.security.auth.Subject:doAs:Subject.java:422', > 'org.apache.hadoop.security.UserGroupInformation:doAs:UserGroupInformation.java:1924', > > 'org.apache.hive.service.cli.session.HiveSessionProxy:invoke:HiveSessionProxy.java:59', > 'com.sun.proxy.$Proxy23:fetchResults::-1', > 'org.apache.hive.service.cli.CLIService:fetchResults:CLIService.java:462', > 'org.apache.hive.service.cli.thrift.ThriftCLIService:FetchResults:ThriftCLIService.java:696', > > 'org.apache.hive.service.cli.thrift.TCLIService$Processor$FetchResults:getResult:TCLIService.java:1553', > > 'org.apache.hive.service.cli.thrift.TCLIService$Processor$FetchResults:getResult:TCLIService.java:1538', > 'org.apache.thrift.ProcessFunction:process:ProcessFunction.java:39', > 'org.apache.thrift.TBaseProcessor:process:TBaseProcessor.java:39', > 'org.apache.hadoop.hive.thrift.HadoopThriftAuthBridge$Server$TUGIAssumingProcessor:process:HadoopThriftAuthBridge.java:747', > > 'org.apache.thrift.server.TThreadPoolServer$WorkerProcess:run:TThreadPoolServer.java:286', > > 'java.util.concurrent.ThreadPoolExecutor:runWorker:ThreadPoolExecutor.java:1142', > > 'java.util.concurrent.ThreadPoolExecutor$Worker:run:ThreadPoolExecutor.java:617', > 'java.lang.Thread:run:Thread.java:745', > '*java.io.IOException:org.apache.hadoop.hive.ql.metadata.HiveException: > java.lang.ClassCastException:27:2', > 'org.apache.hadoop.hive.ql.exec.FetchTask:fetch:FetchTask.java:154', > 'org.apache.hadoop.hive.ql.Driver:getResults:Driver.java:2071', > 'org.apache.hive.service.cli.operation.SQLOperation:getNextRowSet:SQLOperation.java:458', > > '*org.apache.hadoop.hive.ql.metadata.HiveException:java.lang.ClassCastException:34:7', > > 'org.apache.hadoop.hive.ql.exec.ListSinkOperator:processOp:ListSinkOperator.java:90', > 'org.apache.hadoop.hive.ql.exec.Operator:forward:Operator.java:815', > 'org.apache.hadoop.hive.ql.exec.SelectOperator:processOp:SelectOperator.java:84', > 'org.apache.hadoop.hive.ql.exec.Operator:forward:Operator.java:815', > 'org.apache.hadoop.hive.ql.exec.TableScanOperator:processOp:TableScanOperator.java:98', > > 'org.apache.hadoop.hive.ql.exec.FetchOperator:pushRow:FetchOperator.java:425', > > 'org.apache.hadoop.hive.ql.exec.FetchOperator:pushRow:FetchOperator.java:417', > 'org.apache.hadoop.hive.ql.exec.FetchTask:fetch:FetchTask.java:140', > '*java.lang.ClassCastException:null:0:-1'], statusCode=3), results=None, > hasMoreRows=None) -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (HIVE-24035) Add Jenkinsfile for branch-2.3
[ https://issues.apache.org/jira/browse/HIVE-24035?focusedWorklogId=481577=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-481577 ] ASF GitHub Bot logged work on HIVE-24035: - Author: ASF GitHub Bot Created on: 10/Sep/20 16:57 Start Date: 10/Sep/20 16:57 Worklog Time Spent: 10m Work Description: sunchao commented on pull request #1398: URL: https://github.com/apache/hive/pull/1398#issuecomment-690505966 > and seems like more than 1 pod have tried to execute the TestMiniLlapLocalCliDriver; @kgyrtkirk thanks, yes I noticed this too and I think this is causing the tests timeout. Let me check if the branch-2 is indeed using some outdated surefire plugin. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 481577) Time Spent: 2h 20m (was: 2h 10m) > Add Jenkinsfile for branch-2.3 > -- > > Key: HIVE-24035 > URL: https://issues.apache.org/jira/browse/HIVE-24035 > Project: Hive > Issue Type: Test >Reporter: Chao Sun >Assignee: Chao Sun >Priority: Major > Labels: pull-request-available > Time Spent: 2h 20m > Remaining Estimate: 0h > > To enable precommit tests for github PR, we need to have a Jenkinsfile in the > repo. This is already done for master and branch-2. This adds the same for > branch-2.3 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (HIVE-20001) With doas set to true, running select query as hrt_qa user on external table fails due to permission denied to read /warehouse/tablespace/managed directory.
[
https://issues.apache.org/jira/browse/HIVE-20001?focusedWorklogId=481578=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-481578
]
ASF GitHub Bot logged work on HIVE-20001:
-
Author: ASF GitHub Bot
Created on: 10/Sep/20 16:57
Start Date: 10/Sep/20 16:57
Worklog Time Spent: 10m
Work Description: cravani closed pull request #1451:
URL: https://github.com/apache/hive/pull/1451
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 481578)
Time Spent: 1h 10m (was: 1h)
> With doas set to true, running select query as hrt_qa user on external table
> fails due to permission denied to read /warehouse/tablespace/managed
> directory.
>
>
> Key: HIVE-20001
> URL: https://issues.apache.org/jira/browse/HIVE-20001
> Project: Hive
> Issue Type: Bug
>Affects Versions: 3.0.0
>Reporter: Jaume M
>Assignee: Jaume M
>Priority: Critical
> Labels: pull-request-available
> Attachments: HIVE-20001.1.patch, HIVE-20001.1.patch,
> HIVE-20001.2.patch, HIVE-20001.3.patch, HIVE-20001.4.patch, HIVE-20001.5.patch
>
> Time Spent: 1h 10m
> Remaining Estimate: 0h
>
> Hive: With doas set to true, running select query as hrt_qa user on external
> table fails due to permission denied to read /warehouse/tablespace/managed
> directory.
> Steps:
> 1. Create a external table.
> 2. Set doas to true.
> 3. run select count(*) using user hrt_qa.
> Table creation query.
> {code}
> beeline -n hrt_qa -p pwd -u
> "jdbc:hive2://ctr-e138-1518143905142-375925-01-06.hwx.site:2181,ctr-e138-1518143905142-375925-01-05.hwx.site:2181,ctr-e138-1518143905142-375925-01-07.hwx.site:2181/;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=hiveserver2;principal=hive/_h...@example.com;transportMode=http;httpPath=cliservice;ssl=true;sslTrustStore=/etc/security/serverKeys/hivetruststore.jks;trustStorePassword=changeit"
> --outputformat=tsv -e "drop table if exists test_table purge;
> create external table test_table(id int, age int) row format delimited fields
> terminated by '|' stored as textfile;
> load data inpath '/tmp/table1.dat' overwrite into table test_table;
> {code}
> select count(*) query execution fails
> {code}
> beeline -n hrt_qa -p pwd -u
> "jdbc:hive2://ctr-e138-1518143905142-375925-01-06.hwx.site:2181,ctr-e138-1518143905142-375925-01-05.hwx.site:2181,ctr-e138-1518143905142-375925-01-07.hwx.site:2181/;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=hiveserver2;principal=hive/_h...@example.com;transportMode=http;httpPath=cliservice;ssl=true;sslTrustStore=/etc/security/serverKeys/hivetruststore.jks;trustStorePassword=changeit"
> --outputformat=tsv -e "select count(*) from test_table where age>30 and
> id<10100;"
> 2018-06-22 10:22:29,328|INFO|Thread-126|machine.py:111 -
> tee_pipe()||b3a493ec-99be-483e-91fe-4b701ec27ebc|SLF4J: Class path contains
> multiple SLF4J bindings.
> 2018-06-22 10:22:29,330|INFO|Thread-126|machine.py:111 -
> tee_pipe()||b3a493ec-99be-483e-91fe-4b701ec27ebc|SLF4J: See
> http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
> 2018-06-22 10:22:29,335|INFO|Thread-126|machine.py:111 -
> tee_pipe()||b3a493ec-99be-483e-91fe-4b701ec27ebc|SLF4J: Actual binding is of
> type [org.apache.logging.slf4j.Log4jLoggerFactory]
> 2018-06-22 10:22:31,408|INFO|Thread-126|machine.py:111 -
> tee_pipe()||b3a493ec-99be-483e-91fe-4b701ec27ebc|Format tsv is deprecated,
> please use tsv2
> 2018-06-22 10:22:31,529|INFO|Thread-126|machine.py:111 -
> tee_pipe()||b3a493ec-99be-483e-91fe-4b701ec27ebc|Connecting to
> jdbc:hive2://ctr-e138-1518143905142-375925-01-06.hwx.site:2181,ctr-e138-1518143905142-375925-01-05.hwx.site:2181,ctr-e138-1518143905142-375925-01-07.hwx.site:2181/;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=hiveserver2;principal=hive/_h...@example.com;transportMode=http;httpPath=cliservice;ssl=true;sslTrustStore=/etc/security/serverKeys/hivetruststore.jks;trustStorePassword=changeit
> 2018-06-22 10:22:32,031|INFO|Thread-126|machine.py:111 -
> tee_pipe()||b3a493ec-99be-483e-91fe-4b701ec27ebc|18/06/22 10:22:32 [main]:
> INFO jdbc.HiveConnection: Connected to
> ctr-e138-1518143905142-375925-01-04.hwx.site:10001
> 2018-06-22 10:22:34,130|INFO|Thread-126|machine.py:111 -
> tee_pipe()||b3a493ec-99be-483e-91fe-4b701ec27ebc|18/06/22
[jira] [Work logged] (HIVE-24139) VectorGroupByOperator is not flushing hash table entries as needed
[
https://issues.apache.org/jira/browse/HIVE-24139?focusedWorklogId=481548=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-481548
]
ASF GitHub Bot logged work on HIVE-24139:
-
Author: ASF GitHub Bot
Created on: 10/Sep/20 16:07
Start Date: 10/Sep/20 16:07
Worklog Time Spent: 10m
Work Description: mustafaiman commented on a change in pull request #1481:
URL: https://github.com/apache/hive/pull/1481#discussion_r486463497
##
File path:
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/VectorGroupByOperator.java
##
@@ -427,8 +428,12 @@ public void initialize(Configuration hconf) throws
HiveException {
computeMemoryLimits();
LOG.debug("using hash aggregation processing mode");
+ reusableBufferSize = (int) (maxHtEntries * percentEntriesToFlush);
+
+ reusableAggregationBufferRows = new ArrayDeque<>(reusableBufferSize);
+
if (keyWrappersBatch.getVectorHashKeyWrappers()[0] instanceof
VectorHashKeyWrapperGeneral) {
Review comment:
@rbalamohan
The array that is returned from
`keyWrappersBatch.getVectorHashKeyWrappers()` consists of only one type of
wrappers, isn't it? So the first element is representative of all the elements
in that array.
We resort to this optimization only when the batch consists of
VectorHashKeyWrapperGeneral. The other wrapper types have only one two
primitive types. If needed, only thing we need to do is just override their
copyKey(KeyWrapper) method and generalize this `instanceOf` check to cover them
too. Right now, I dont think that is very important.
So when the first element is not VectorHashKeyWrapperGeneral, the rest of
the array are not either. In that case, we just skip reusing.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 481548)
Time Spent: 0.5h (was: 20m)
> VectorGroupByOperator is not flushing hash table entries as needed
> --
>
> Key: HIVE-24139
> URL: https://issues.apache.org/jira/browse/HIVE-24139
> Project: Hive
> Issue Type: Bug
>Reporter: Mustafa Iman
>Assignee: Mustafa Iman
>Priority: Major
> Labels: pull-request-available
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> After https://issues.apache.org/jira/browse/HIVE-23975 introduced a bug where
> copyKey mutates some key wrappers while copying. This Jira is to fix it.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-24141) Cannot retrieve results using a Select * on a given table
[ https://issues.apache.org/jira/browse/HIVE-24141?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17193684#comment-17193684 ] srinivas vemulapalli commented on HIVE-24141: - ERROR: * Bad status for request TFetchResultsReq(fetchType=0, operationHandle=TOperationHandle(hasResultSet=True, modifiedRowCount=None, operationType=0, operationId=THandleIdentifier(secret='\xaa\xd5d\x0e\xae\xd4E\x9e\xad\xc5w\x80Y\x0f\x07\xcd', guid='1\x80\xc1\xa0\x9cKM\xab\x85\x11\xaa\x95\x19\xcd8\xb4')), orientation=4, maxRows=100): TFetchResultsResp(status=TStatus(errorCode=0, errorMessage='java.io.IOException: org.apache.hadoop.hive.ql.metadata.HiveException: java.lang.ClassCastException', sqlState=None, infoMessages=['*org.apache.hive.service.cli.HiveSQLException:java.io.IOException: org.apache.hadoop.hive.ql.metadata.HiveException: java.lang.ClassCastException:25:24', 'org.apache.hive.service.cli.operation.SQLOperation:getNextRowSet:SQLOperation.java:463', 'org.apache.hive.service.cli.operation.OperationManager:getOperationNextRowSet:OperationManager.java:294', 'org.apache.hive.service.cli.session.HiveSessionImpl:fetchResults:HiveSessionImpl.java:769', 'sun.reflect.GeneratedMethodAccessor59:invoke::-1', 'sun.reflect.DelegatingMethodAccessorImpl:invoke:DelegatingMethodAccessorImpl.java:43', 'java.lang.reflect.Method:invoke:Method.java:498', 'org.apache.hive.service.cli.session.HiveSessionProxy:invoke:HiveSessionProxy.java:78', 'org.apache.hive.service.cli.session.HiveSessionProxy:access$000:HiveSessionProxy.java:36', 'org.apache.hive.service.cli.session.HiveSessionProxy$1:run:HiveSessionProxy.java:63', 'java.security.AccessController:doPrivileged:AccessController.java:-2', 'javax.security.auth.Subject:doAs:Subject.java:422', 'org.apache.hadoop.security.UserGroupInformation:doAs:UserGroupInformation.java:1924', 'org.apache.hive.service.cli.session.HiveSessionProxy:invoke:HiveSessionProxy.java:59', 'com.sun.proxy.$Proxy23:fetchResults::-1', 'org.apache.hive.service.cli.CLIService:fetchResults:CLIService.java:462', 'org.apache.hive.service.cli.thrift.ThriftCLIService:FetchResults:ThriftCLIService.java:696', 'org.apache.hive.service.cli.thrift.TCLIService$Processor$FetchResults:getResult:TCLIService.java:1553', 'org.apache.hive.service.cli.thrift.TCLIService$Processor$FetchResults:getResult:TCLIService.java:1538', 'org.apache.thrift.ProcessFunction:process:ProcessFunction.java:39', 'org.apache.thrift.TBaseProcessor:process:TBaseProcessor.java:39', 'org.apache.hadoop.hive.thrift.HadoopThriftAuthBridge$Server$TUGIAssumingProcessor:process:HadoopThriftAuthBridge.java:747', 'org.apache.thrift.server.TThreadPoolServer$WorkerProcess:run:TThreadPoolServer.java:286', 'java.util.concurrent.ThreadPoolExecutor:runWorker:ThreadPoolExecutor.java:1142', 'java.util.concurrent.ThreadPoolExecutor$Worker:run:ThreadPoolExecutor.java:617', 'java.lang.Thread:run:Thread.java:745', '*java.io.IOException:org.apache.hadoop.hive.ql.metadata.HiveException: java.lang.ClassCastException:27:2', 'org.apache.hadoop.hive.ql.exec.FetchTask:fetch:FetchTask.java:154', 'org.apache.hadoop.hive.ql.Driver:getResults:Driver.java:2071', 'org.apache.hive.service.cli.operation.SQLOperation:getNextRowSet:SQLOperation.java:458', '*org.apache.hadoop.hive.ql.metadata.HiveException:java.lang.ClassCastException:34:7', 'org.apache.hadoop.hive.ql.exec.ListSinkOperator:processOp:ListSinkOperator.java:90', 'org.apache.hadoop.hive.ql.exec.Operator:forward:Operator.java:815', 'org.apache.hadoop.hive.ql.exec.SelectOperator:processOp:SelectOperator.java:84', 'org.apache.hadoop.hive.ql.exec.Operator:forward:Operator.java:815', 'org.apache.hadoop.hive.ql.exec.TableScanOperator:processOp:TableScanOperator.java:98', 'org.apache.hadoop.hive.ql.exec.FetchOperator:pushRow:FetchOperator.java:425', 'org.apache.hadoop.hive.ql.exec.FetchOperator:pushRow:FetchOperator.java:417', 'org.apache.hadoop.hive.ql.exec.FetchTask:fetch:FetchTask.java:140', '*java.lang.ClassCastException:null:0:-1'], statusCode=3), results=None, hasMoreRows=None) > Cannot retrieve results using a Select * on a given table > - > > Key: HIVE-24141 > URL: https://issues.apache.org/jira/browse/HIVE-24141 > Project: Hive > Issue Type: Bug > Components: Hive, Parquet >Reporter: srinivas vemulapalli >Priority: Blocker > > Receiving following error on querying select * on given table > > * Bad status for request TFetchResultsReq(fetchType=0, > operationHandle=TOperationHandle(hasResultSet=True, modifiedRowCount=None, > operationType=0, > operationId=THandleIdentifier(secret='\xaa\xd5d\x0e\xae\xd4E\x9e\xad\xc5w\x80Y\x0f\x07\xcd', > guid='1\x80\xc1\xa0\x9cKM\xab\x85\x11\xaa\x95\x19\xcd8\xb4')), > orientation=4, maxRows=100):
[jira] [Resolved] (HIVE-24092) Implement additional JDBC methods required by JDBC storage handler
[
https://issues.apache.org/jira/browse/HIVE-24092?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jesus Camacho Rodriguez resolved HIVE-24092.
Fix Version/s: 4.0.0
Resolution: Fixed
Pushed to master, thanks [~kishendas]!
> Implement additional JDBC methods required by JDBC storage handler
> --
>
> Key: HIVE-24092
> URL: https://issues.apache.org/jira/browse/HIVE-24092
> Project: Hive
> Issue Type: Bug
> Components: JDBC storage handler
>Reporter: Jesus Camacho Rodriguez
>Assignee: Kishen Das
>Priority: Major
> Fix For: 4.0.0
>
>
> Calcite may rely on the following JDBC methods to generate SQL queries for
> Hive JDBC storage handler, which in the case of Hive itself, return a
> {{Method not supported}} exception. We should implement such methods:
> {code}
> nullsAreSortedAtEnd
> nullsAreSortedAtStart
> nullsAreSortedLow
> nullsAreSortedHigh
> storesLowerCaseIdentifiers
> storesLowerCaseQuotedIdentifiers
> storesMixedCaseIdentifiers
> storesMixedCaseQuotedIdentifiers
> storesUpperCaseIdentifiers
> storesUpperCaseQuotedIdentifiers
> supportsMixedCaseIdentifiers
> supportsMixedCaseQuotedIdentifiers
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-23025) Support add a yarn application name for tez on hiveserver2
[ https://issues.apache.org/jira/browse/HIVE-23025?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xi Chen updated HIVE-23025: --- Fix Version/s: (was: 2.3.5) > Support add a yarn application name for tez on hiveserver2 > -- > > Key: HIVE-23025 > URL: https://issues.apache.org/jira/browse/HIVE-23025 > Project: Hive > Issue Type: New Feature > Components: Tez >Affects Versions: 2.3.5 >Reporter: Jake Xie >Priority: Major > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-24044) Implement listPartitionNames with filter or order on temporary tables
[
https://issues.apache.org/jira/browse/HIVE-24044?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zhihua Deng updated HIVE-24044:
---
Issue Type: Task (was: Improvement)
Priority: Minor (was: Major)
> Implement listPartitionNames with filter or order on temporary tables
> --
>
> Key: HIVE-24044
> URL: https://issues.apache.org/jira/browse/HIVE-24044
> Project: Hive
> Issue Type: Task
> Components: Metastore
>Affects Versions: 4.0.0
>Reporter: Zhihua Deng
>Priority: Minor
> Labels: pull-request-available
> Time Spent: 20m
> Remaining Estimate: 0h
>
> Temporary tables can have their own partitions, and IMetaStoreClient use
> {code:java}
> List listPartitionNames(PartitionsByExprRequest request){code}
> to filter or sort the results. This method can be implemented on temporary
> tables.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (HIVE-24139) VectorGroupByOperator is not flushing hash table entries as needed
[
https://issues.apache.org/jira/browse/HIVE-24139?focusedWorklogId=481414=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-481414
]
ASF GitHub Bot logged work on HIVE-24139:
-
Author: ASF GitHub Bot
Created on: 10/Sep/20 12:10
Start Date: 10/Sep/20 12:10
Worklog Time Spent: 10m
Work Description: rbalamohan commented on a change in pull request #1481:
URL: https://github.com/apache/hive/pull/1481#discussion_r486285233
##
File path:
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/VectorGroupByOperator.java
##
@@ -427,8 +428,12 @@ public void initialize(Configuration hconf) throws
HiveException {
computeMemoryLimits();
LOG.debug("using hash aggregation processing mode");
+ reusableBufferSize = (int) (maxHtEntries * percentEntriesToFlush);
+
+ reusableAggregationBufferRows = new ArrayDeque<>(reusableBufferSize);
+
if (keyWrappersBatch.getVectorHashKeyWrappers()[0] instanceof
VectorHashKeyWrapperGeneral) {
Review comment:
This decision is made purely based on first item in the hashKeyWrapper.
Would there be issue if there are different wrappers (e.g
VectorHashKeyWrapperSingleLong) in other indices? Or would there be a miss if
the first index isn't VectorHashKeyWrapperGeneral?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 481414)
Time Spent: 20m (was: 10m)
> VectorGroupByOperator is not flushing hash table entries as needed
> --
>
> Key: HIVE-24139
> URL: https://issues.apache.org/jira/browse/HIVE-24139
> Project: Hive
> Issue Type: Bug
>Reporter: Mustafa Iman
>Assignee: Mustafa Iman
>Priority: Major
> Labels: pull-request-available
> Time Spent: 20m
> Remaining Estimate: 0h
>
> After https://issues.apache.org/jira/browse/HIVE-23975 introduced a bug where
> copyKey mutates some key wrappers while copying. This Jira is to fix it.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-24138) Llap external client flow is broken due to netty shading
[
https://issues.apache.org/jira/browse/HIVE-24138?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17193571#comment-17193571
]
Shubham Chaurasia commented on HIVE-24138:
--
[~abstractdog] [~thejas] [~ashutoshc]
Any suggestions on how to proceed on this ?
> Llap external client flow is broken due to netty shading
>
>
> Key: HIVE-24138
> URL: https://issues.apache.org/jira/browse/HIVE-24138
> Project: Hive
> Issue Type: Bug
> Components: llap
>Reporter: Shubham Chaurasia
>Assignee: Shubham Chaurasia
>Priority: Critical
>
> We shaded netty in hive-exec in -
> https://issues.apache.org/jira/browse/HIVE-23073
> This breaks LLAP external client flow on LLAP daemon side -
> LLAP daemon stacktrace -
> {code}
> 2020-09-09T18:22:13,413 INFO [TezTR-222977_4_0_0_0_0
> (497418324441977_0004_0_00_00_0)] llap.LlapOutputFormat: Returning
> writer for: attempt_497418324441977_0004_0_00_00_0
> 2020-09-09T18:22:13,419 ERROR [TezTR-222977_4_0_0_0_0
> (497418324441977_0004_0_00_00_0)] tez.MapRecordSource:
> java.lang.NoSuchMethodError:
> org.apache.arrow.memory.BufferAllocator.buffer(I)Lorg/apache/hive/io/netty/buffer/ArrowBuf;
> at
> org.apache.hadoop.hive.llap.WritableByteChannelAdapter.write(WritableByteChannelAdapter.java:96)
> at org.apache.arrow.vector.ipc.WriteChannel.write(WriteChannel.java:74)
> at org.apache.arrow.vector.ipc.WriteChannel.write(WriteChannel.java:57)
> at
> org.apache.arrow.vector.ipc.WriteChannel.writeIntLittleEndian(WriteChannel.java:89)
> at
> org.apache.arrow.vector.ipc.message.MessageSerializer.serialize(MessageSerializer.java:88)
> at
> org.apache.arrow.vector.ipc.ArrowWriter.ensureStarted(ArrowWriter.java:130)
> at
> org.apache.arrow.vector.ipc.ArrowWriter.writeBatch(ArrowWriter.java:102)
> at
> org.apache.hadoop.hive.llap.LlapArrowRecordWriter.write(LlapArrowRecordWriter.java:85)
> at
> org.apache.hadoop.hive.llap.LlapArrowRecordWriter.write(LlapArrowRecordWriter.java:46)
> at
> org.apache.hadoop.hive.ql.exec.vector.filesink.VectorFileSinkArrowOperator.process(VectorFileSinkArrowOperator.java:137)
> at
> org.apache.hadoop.hive.ql.exec.Operator.vectorForward(Operator.java:969)
> at
> org.apache.hadoop.hive.ql.exec.vector.VectorSelectOperator.process(VectorSelectOperator.java:158)
> at
> org.apache.hadoop.hive.ql.exec.Operator.vectorForward(Operator.java:969)
> at
> org.apache.hadoop.hive.ql.exec.TableScanOperator.process(TableScanOperator.java:172)
> at
> org.apache.hadoop.hive.ql.exec.vector.VectorMapOperator.deliverVectorizedRowBatch(VectorMapOperator.java:809)
> at
> org.apache.hadoop.hive.ql.exec.vector.VectorMapOperator.process(VectorMapOperator.java:842)
> at
> org.apache.hadoop.hive.ql.exec.tez.MapRecordSource.processRow(MapRecordSource.java:92)
> at
> org.apache.hadoop.hive.ql.exec.tez.MapRecordSource.pushRecord(MapRecordSource.java:76)
> at
> org.apache.hadoop.hive.ql.exec.tez.MapRecordProcessor.run(MapRecordProcessor.java:426)
> at
> org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:267)
> at
> org.apache.hadoop.hive.ql.exec.tez.TezProcessor.run(TezProcessor.java:250)
> at
> org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:374)
> at
> org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:75)
> at
> org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:62)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1876)
> at
> org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:62)
> at
> org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:38)
> at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
> at
> org.apache.hadoop.hive.llap.daemon.impl.StatsRecordingThreadPool$WrappedCallable.call(StatsRecordingThreadPool.java:118)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> {code}
> Arrow method signature mismatch mainly happens due to the fact that arrow
> contains some classes which are packaged under {{io.netty.buffer.*}} -
> {code}
> io.netty.buffer.ArrowBuf
> io.netty.buffer.ExpandableByteBuf
>
[jira] [Assigned] (HIVE-24138) Llap external client flow is broken due to netty shading
[
https://issues.apache.org/jira/browse/HIVE-24138?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Shubham Chaurasia reassigned HIVE-24138:
Assignee: Shubham Chaurasia
> Llap external client flow is broken due to netty shading
>
>
> Key: HIVE-24138
> URL: https://issues.apache.org/jira/browse/HIVE-24138
> Project: Hive
> Issue Type: Bug
> Components: llap
>Reporter: Shubham Chaurasia
>Assignee: Shubham Chaurasia
>Priority: Critical
>
> We shaded netty in hive-exec in -
> https://issues.apache.org/jira/browse/HIVE-23073
> This breaks LLAP external client flow on LLAP daemon side -
> LLAP daemon stacktrace -
> {code}
> 2020-09-09T18:22:13,413 INFO [TezTR-222977_4_0_0_0_0
> (497418324441977_0004_0_00_00_0)] llap.LlapOutputFormat: Returning
> writer for: attempt_497418324441977_0004_0_00_00_0
> 2020-09-09T18:22:13,419 ERROR [TezTR-222977_4_0_0_0_0
> (497418324441977_0004_0_00_00_0)] tez.MapRecordSource:
> java.lang.NoSuchMethodError:
> org.apache.arrow.memory.BufferAllocator.buffer(I)Lorg/apache/hive/io/netty/buffer/ArrowBuf;
> at
> org.apache.hadoop.hive.llap.WritableByteChannelAdapter.write(WritableByteChannelAdapter.java:96)
> at org.apache.arrow.vector.ipc.WriteChannel.write(WriteChannel.java:74)
> at org.apache.arrow.vector.ipc.WriteChannel.write(WriteChannel.java:57)
> at
> org.apache.arrow.vector.ipc.WriteChannel.writeIntLittleEndian(WriteChannel.java:89)
> at
> org.apache.arrow.vector.ipc.message.MessageSerializer.serialize(MessageSerializer.java:88)
> at
> org.apache.arrow.vector.ipc.ArrowWriter.ensureStarted(ArrowWriter.java:130)
> at
> org.apache.arrow.vector.ipc.ArrowWriter.writeBatch(ArrowWriter.java:102)
> at
> org.apache.hadoop.hive.llap.LlapArrowRecordWriter.write(LlapArrowRecordWriter.java:85)
> at
> org.apache.hadoop.hive.llap.LlapArrowRecordWriter.write(LlapArrowRecordWriter.java:46)
> at
> org.apache.hadoop.hive.ql.exec.vector.filesink.VectorFileSinkArrowOperator.process(VectorFileSinkArrowOperator.java:137)
> at
> org.apache.hadoop.hive.ql.exec.Operator.vectorForward(Operator.java:969)
> at
> org.apache.hadoop.hive.ql.exec.vector.VectorSelectOperator.process(VectorSelectOperator.java:158)
> at
> org.apache.hadoop.hive.ql.exec.Operator.vectorForward(Operator.java:969)
> at
> org.apache.hadoop.hive.ql.exec.TableScanOperator.process(TableScanOperator.java:172)
> at
> org.apache.hadoop.hive.ql.exec.vector.VectorMapOperator.deliverVectorizedRowBatch(VectorMapOperator.java:809)
> at
> org.apache.hadoop.hive.ql.exec.vector.VectorMapOperator.process(VectorMapOperator.java:842)
> at
> org.apache.hadoop.hive.ql.exec.tez.MapRecordSource.processRow(MapRecordSource.java:92)
> at
> org.apache.hadoop.hive.ql.exec.tez.MapRecordSource.pushRecord(MapRecordSource.java:76)
> at
> org.apache.hadoop.hive.ql.exec.tez.MapRecordProcessor.run(MapRecordProcessor.java:426)
> at
> org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:267)
> at
> org.apache.hadoop.hive.ql.exec.tez.TezProcessor.run(TezProcessor.java:250)
> at
> org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:374)
> at
> org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:75)
> at
> org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:62)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1876)
> at
> org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:62)
> at
> org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:38)
> at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
> at
> org.apache.hadoop.hive.llap.daemon.impl.StatsRecordingThreadPool$WrappedCallable.call(StatsRecordingThreadPool.java:118)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> {code}
> Arrow method signature mismatch mainly happens due to the fact that arrow
> contains some classes which are packaged under {{io.netty.buffer.*}} -
> {code}
> io.netty.buffer.ArrowBuf
> io.netty.buffer.ExpandableByteBuf
> io.netty.buffer.LargeBuffer
> io.netty.buffer.MutableWrappedByteBuf
>
[jira] [Commented] (HIVE-18284) NPE when inserting data with 'distribute by' clause with dynpart sort optimization
[
https://issues.apache.org/jira/browse/HIVE-18284?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17193518#comment-17193518
]
Syed Shameerur Rahman commented on HIVE-18284:
--
[~kgyrtkirk] [~jcamachorodriguez] Could you Please review the PR?
> NPE when inserting data with 'distribute by' clause with dynpart sort
> optimization
> --
>
> Key: HIVE-18284
> URL: https://issues.apache.org/jira/browse/HIVE-18284
> Project: Hive
> Issue Type: Bug
> Components: Query Processor
>Affects Versions: 2.3.1, 2.3.2
>Reporter: Aki Tanaka
>Assignee: Syed Shameerur Rahman
>Priority: Major
> Labels: pull-request-available
> Time Spent: 1h 20m
> Remaining Estimate: 0h
>
> A Null Pointer Exception occurs when inserting data with 'distribute by'
> clause. The following snippet query reproduces this issue:
> *(non-vectorized , non-llap mode)*
> {code:java}
> create table table1 (col1 string, datekey int);
> insert into table1 values ('ROW1', 1), ('ROW2', 2), ('ROW3', 1);
> create table table2 (col1 string) partitioned by (datekey int);
> set hive.vectorized.execution.enabled=false;
> set hive.optimize.sort.dynamic.partition=true;
> set hive.exec.dynamic.partition.mode=nonstrict;
> insert into table table2
> PARTITION(datekey)
> select col1,
> datekey
> from table1
> distribute by datekey ;
> {code}
> I could run the insert query without the error if I remove Distribute By or
> use Cluster By clause.
> It seems that the issue happens because Distribute By does not guarantee
> clustering or sorting properties on the distributed keys.
> FileSinkOperator removes the previous fsp. FileSinkOperator will remove the
> previous fsp which might be re-used when we use Distribute By.
> https://github.com/apache/hive/blob/branch-2.3/ql/src/java/org/apache/hadoop/hive/ql/exec/FileSinkOperator.java#L972
> The following stack trace is logged.
> {code:java}
> Vertex failed, vertexName=Reducer 2, vertexId=vertex_1513111717879_0056_1_01,
> diagnostics=[Task failed, taskId=task_1513111717879_0056_1_01_00,
> diagnostics=[TaskAttempt 0 failed, info=[Error: Error while running task (
> failure ) :
> attempt_1513111717879_0056_1_01_00_0:java.lang.RuntimeException:
> org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while
> processing row (tag=0) {"key":{},"value":{"_col0":"ROW3","_col1":1}}
> at
> org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:211)
> at
> org.apache.hadoop.hive.ql.exec.tez.TezProcessor.run(TezProcessor.java:168)
> at
> org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:370)
> at
> org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:73)
> at
> org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:61)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1698)
> at
> org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:61)
> at
> org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:37)
> at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime
> Error while processing row (tag=0)
> {"key":{},"value":{"_col0":"ROW3","_col1":1}}
> at
> org.apache.hadoop.hive.ql.exec.tez.ReduceRecordSource$GroupIterator.next(ReduceRecordSource.java:365)
> at
> org.apache.hadoop.hive.ql.exec.tez.ReduceRecordSource.pushRecord(ReduceRecordSource.java:250)
> at
> org.apache.hadoop.hive.ql.exec.tez.ReduceRecordProcessor.run(ReduceRecordProcessor.java:317)
> at
> org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:185)
> ... 14 more
> Caused by: java.lang.NullPointerException
> at
> org.apache.hadoop.hive.ql.exec.FileSinkOperator.process(FileSinkOperator.java:762)
> at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:897)
> at
> org.apache.hadoop.hive.ql.exec.SelectOperator.process(SelectOperator.java:95)
> at
> org.apache.hadoop.hive.ql.exec.tez.ReduceRecordSource$GroupIterator.next(ReduceRecordSource.java:356)
>
[jira] [Issue Comment Deleted] (HIVE-18284) NPE when inserting data with 'distribute by' clause with dynpart sort optimization
[
https://issues.apache.org/jira/browse/HIVE-18284?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Syed Shameerur Rahman updated HIVE-18284:
-
Comment: was deleted
(was: [~jcamachorodriguez] Could you please review the PR?)
> NPE when inserting data with 'distribute by' clause with dynpart sort
> optimization
> --
>
> Key: HIVE-18284
> URL: https://issues.apache.org/jira/browse/HIVE-18284
> Project: Hive
> Issue Type: Bug
> Components: Query Processor
>Affects Versions: 2.3.1, 2.3.2
>Reporter: Aki Tanaka
>Assignee: Syed Shameerur Rahman
>Priority: Major
> Labels: pull-request-available
> Time Spent: 1h 20m
> Remaining Estimate: 0h
>
> A Null Pointer Exception occurs when inserting data with 'distribute by'
> clause. The following snippet query reproduces this issue:
> *(non-vectorized , non-llap mode)*
> {code:java}
> create table table1 (col1 string, datekey int);
> insert into table1 values ('ROW1', 1), ('ROW2', 2), ('ROW3', 1);
> create table table2 (col1 string) partitioned by (datekey int);
> set hive.vectorized.execution.enabled=false;
> set hive.optimize.sort.dynamic.partition=true;
> set hive.exec.dynamic.partition.mode=nonstrict;
> insert into table table2
> PARTITION(datekey)
> select col1,
> datekey
> from table1
> distribute by datekey ;
> {code}
> I could run the insert query without the error if I remove Distribute By or
> use Cluster By clause.
> It seems that the issue happens because Distribute By does not guarantee
> clustering or sorting properties on the distributed keys.
> FileSinkOperator removes the previous fsp. FileSinkOperator will remove the
> previous fsp which might be re-used when we use Distribute By.
> https://github.com/apache/hive/blob/branch-2.3/ql/src/java/org/apache/hadoop/hive/ql/exec/FileSinkOperator.java#L972
> The following stack trace is logged.
> {code:java}
> Vertex failed, vertexName=Reducer 2, vertexId=vertex_1513111717879_0056_1_01,
> diagnostics=[Task failed, taskId=task_1513111717879_0056_1_01_00,
> diagnostics=[TaskAttempt 0 failed, info=[Error: Error while running task (
> failure ) :
> attempt_1513111717879_0056_1_01_00_0:java.lang.RuntimeException:
> org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while
> processing row (tag=0) {"key":{},"value":{"_col0":"ROW3","_col1":1}}
> at
> org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:211)
> at
> org.apache.hadoop.hive.ql.exec.tez.TezProcessor.run(TezProcessor.java:168)
> at
> org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:370)
> at
> org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:73)
> at
> org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:61)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1698)
> at
> org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:61)
> at
> org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:37)
> at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime
> Error while processing row (tag=0)
> {"key":{},"value":{"_col0":"ROW3","_col1":1}}
> at
> org.apache.hadoop.hive.ql.exec.tez.ReduceRecordSource$GroupIterator.next(ReduceRecordSource.java:365)
> at
> org.apache.hadoop.hive.ql.exec.tez.ReduceRecordSource.pushRecord(ReduceRecordSource.java:250)
> at
> org.apache.hadoop.hive.ql.exec.tez.ReduceRecordProcessor.run(ReduceRecordProcessor.java:317)
> at
> org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:185)
> ... 14 more
> Caused by: java.lang.NullPointerException
> at
> org.apache.hadoop.hive.ql.exec.FileSinkOperator.process(FileSinkOperator.java:762)
> at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:897)
> at
> org.apache.hadoop.hive.ql.exec.SelectOperator.process(SelectOperator.java:95)
> at
> org.apache.hadoop.hive.ql.exec.tez.ReduceRecordSource$GroupIterator.next(ReduceRecordSource.java:356)
> ... 17 more
> {code}
[jira] [Work logged] (HIVE-23851) MSCK REPAIR Command With Partition Filtering Fails While Dropping Partitions
[
https://issues.apache.org/jira/browse/HIVE-23851?focusedWorklogId=481350=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-481350
]
ASF GitHub Bot logged work on HIVE-23851:
-
Author: ASF GitHub Bot
Created on: 10/Sep/20 10:08
Start Date: 10/Sep/20 10:08
Worklog Time Spent: 10m
Work Description: shameersss1 commented on a change in pull request #1271:
URL: https://github.com/apache/hive/pull/1271#discussion_r486220996
##
File path:
standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/MsckPartitionExpressionProxy.java
##
@@ -1,114 +0,0 @@
-package org.apache.hadoop.hive.metastore;
-/*
- * Licensed to the Apache Software Foundation (ASF) under one
- * or more contributor license agreements. See the NOTICE file
- * distributed with this work for additional information
- * regarding copyright ownership. The ASF licenses this file
- * to you under the Apache License, Version 2.0 (the
- * "License"); you may not use this file except in compliance
- * with the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-
-import java.nio.charset.StandardCharsets;
-import java.util.ArrayList;
-import java.util.HashSet;
-import java.util.List;
-import java.util.Set;
-
-import org.apache.hadoop.hive.metastore.api.FieldSchema;
-import org.apache.hadoop.hive.metastore.api.FileMetadataExprType;
-import org.apache.hadoop.hive.metastore.api.MetaException;
-import org.apache.hadoop.hive.metastore.utils.FileUtils;
-import org.apache.hadoop.hive.ql.io.sarg.SearchArgument;
-import org.apache.hadoop.util.StringUtils;
-import org.slf4j.Logger;
-import org.slf4j.LoggerFactory;
-
-// This is added as part of moving MSCK code from ql to standalone-metastore.
There is a metastore API to drop
-// partitions by name but we cannot use it because msck typically will contain
partition value (year=2014). We almost
-// never drop partition by name (year). So we need to construct expression
filters, the current
-// PartitionExpressionProxy implementations (PartitionExpressionForMetastore
and HCatClientHMSImpl.ExpressionBuilder)
-// all depend on ql code to build ExprNodeDesc for the partition expressions.
It also depends on kryo for serializing
-// the expression objects to byte[]. For MSCK drop partition, we don't need
complex expression generator. For now,
-// all we do is split the partition spec (year=2014/month=24) into filter
expression year='2014' and month='24' and
-// rely on metastore database to deal with type conversions. Ideally,
PartitionExpressionProxy default implementation
-// should use SearchArgument (storage-api) to construct the filter expression
and not depend on ql, but the usecase
-// for msck is pretty simple and this specific implementation should suffice.
Review comment:
I have changed the approach, Please take a look at the new
implementation.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 481350)
Time Spent: 4h 10m (was: 4h)
> MSCK REPAIR Command With Partition Filtering Fails While Dropping Partitions
>
>
> Key: HIVE-23851
> URL: https://issues.apache.org/jira/browse/HIVE-23851
> Project: Hive
> Issue Type: Bug
>Affects Versions: 4.0.0
>Reporter: Syed Shameerur Rahman
>Assignee: Syed Shameerur Rahman
>Priority: Major
> Labels: pull-request-available
> Fix For: 4.0.0
>
> Time Spent: 4h 10m
> Remaining Estimate: 0h
>
> *Steps to reproduce:*
> # Create external table
> # Run msck command to sync all the partitions with metastore
> # Remove one of the partition path
> # Run msck repair with partition filtering
> *Stack Trace:*
> {code:java}
> 2020-07-15T02:10:29,045 ERROR [4dad298b-28b1-4e6b-94b6-aa785b60c576 main]
> ppr.PartitionExpressionForMetastore: Failed to deserialize the expression
> java.lang.IndexOutOfBoundsException: Index: 110, Size: 0
> at java.util.ArrayList.rangeCheck(ArrayList.java:657) ~[?:1.8.0_192]
> at java.util.ArrayList.get(ArrayList.java:433) ~[?:1.8.0_192]
> at
>
[jira] [Commented] (HIVE-24133) Hive query with Hbase storagehandler can give back incorrect results when predicate contains null check
[
https://issues.apache.org/jira/browse/HIVE-24133?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17193453#comment-17193453
]
zhishui commented on HIVE-24133:
This is caused by scan of hbase which only scan CF:COL of what your sql need.
In method of HiveHBaseInputFormatUtil.getScan, there only add necessary col id
to scan and it's may have some trouble about the scenario you provide, but you
could not be critical about this. However, we could not always scan all columns.
The basic difference is that how to understand between row-based storage (hive)
and column-based storage(hbase)
> Hive query with Hbase storagehandler can give back incorrect results when
> predicate contains null check
> ---
>
> Key: HIVE-24133
> URL: https://issues.apache.org/jira/browse/HIVE-24133
> Project: Hive
> Issue Type: Bug
>Reporter: Marton Bod
>Assignee: zhishui
>Priority: Major
>
> It has been observed that when using Hbase storage handler and the table
> contains null values, Hive can give back wrong query results, depending on
> what columns we select for and whether the where clause predicate contains
> any null checks.
> For example:
> create 'default:hive_test', 'cf'
> put 'default:hive_test', '1', 'cf:col1', 'val1'
> put 'default:hive_test', '1', 'cf:col2', 'val2'
> put 'default:hive_test', '2', 'cf:col1', 'val1_2'
> put 'default:hive_test', '2', 'cf:col2', 'val2_2'
> put 'default:hive_test', '3', 'cf:col1', 'val1_3'
> put 'default:hive_test', '3', 'cf:col2', 'val2_3'
> put 'default:hive_test', '3', 'cf:col3', 'val3_3'
> put 'default:hive_test', '3', 'cf:col4', "\x00\x00\x00\x00\x00\x02\xC2"
> put 'default:hive_test', '4', 'cf:col1', 'val1_4'
> put 'default:hive_test', '4', 'cf:col2', 'val2_4'
> scan 'default:hive_test'
> = HIVE
> CREATE EXTERNAL TABLE hbase_hive_test (
> rowkey string,
> col1 string,
> col2 string,
> col3 string
> )
> STORED BY 'org.apache.hadoop.hive.hbase.HBaseStorageHandler'
> WITH SERDEPROPERTIES (
> "hbase.columns.mapping" = ":key,cf:col1,cf:col2,cf:col3"
> )
> TBLPROPERTIES("hbase.table.name" = "default:hive_test");
> query: select * from hbase_hive_test where col3 is null;
> result:
> Total MapReduce CPU Time Spent: 10 seconds 980 msec
> OK
> 1 val1 val2 NULL
> 2 val1_2 val2_2 NULL
> 4 val1_4 val2_4 NULL
> query: select rowkey from hbase_hive_test where col3 is null;
> This does not produce any records.
> However, select rowkey, col2 from hbase_hive_test where col3 is null;
> This gives back the correct results again.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (HIVE-24035) Add Jenkinsfile for branch-2.3
[ https://issues.apache.org/jira/browse/HIVE-24035?focusedWorklogId=481291=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-481291 ] ASF GitHub Bot logged work on HIVE-24035: - Author: ASF GitHub Bot Created on: 10/Sep/20 07:25 Start Date: 10/Sep/20 07:25 Worklog Time Spent: 10m Work Description: kgyrtkirk commented on pull request #1398: URL: https://github.com/apache/hive/pull/1398#issuecomment-690045898 I've taken a quick look at the Jenkinsfile - I don't see reason for running the same stuff in multiple pods I'm not sure but it might be possible that branch-2 uses an ancient surefire plugin which doesn't even understand the inclusion/exclusion options? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 481291) Time Spent: 2h 10m (was: 2h) > Add Jenkinsfile for branch-2.3 > -- > > Key: HIVE-24035 > URL: https://issues.apache.org/jira/browse/HIVE-24035 > Project: Hive > Issue Type: Test >Reporter: Chao Sun >Assignee: Chao Sun >Priority: Major > Labels: pull-request-available > Time Spent: 2h 10m > Remaining Estimate: 0h > > To enable precommit tests for github PR, we need to have a Jenkinsfile in the > repo. This is already done for master and branch-2. This adds the same for > branch-2.3 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (HIVE-24035) Add Jenkinsfile for branch-2.3
[ https://issues.apache.org/jira/browse/HIVE-24035?focusedWorklogId=481290=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-481290 ] ASF GitHub Bot logged work on HIVE-24035: - Author: ASF GitHub Bot Created on: 10/Sep/20 07:24 Start Date: 10/Sep/20 07:24 Worklog Time Spent: 10m Work Description: kgyrtkirk commented on pull request #1398: URL: https://github.com/apache/hive/pull/1398#issuecomment-690045066 something is off here: ``` [2020-09-09T05:06:53.645Z] Running org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver [2020-09-09T17:17:29.852Z] Sending interrupt signal to process [2020-09-09T17:17:29.852Z] Killing processes [2020-09-09T17:17:30.065Z] kill finished with exit code 0 [2020-09-09T17:17:44.109Z] Terminated [2020-09-09T17:17:44.120Z] script returned exit code 143 ``` and seems like more than 1 pod have tried to execute the TestMiniLlapLocalCliDriver; This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 481290) Time Spent: 2h (was: 1h 50m) > Add Jenkinsfile for branch-2.3 > -- > > Key: HIVE-24035 > URL: https://issues.apache.org/jira/browse/HIVE-24035 > Project: Hive > Issue Type: Test >Reporter: Chao Sun >Assignee: Chao Sun >Priority: Major > Labels: pull-request-available > Time Spent: 2h > Remaining Estimate: 0h > > To enable precommit tests for github PR, we need to have a Jenkinsfile in the > repo. This is already done for master and branch-2. This adds the same for > branch-2.3 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-24129) Deleting the previous successful dump directory should be based on config
[
https://issues.apache.org/jira/browse/HIVE-24129?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Anishek Agarwal updated HIVE-24129:
---
Resolution: Fixed
Status: Resolved (was: Patch Available)
Committed to master, Thanks for the patch [~^sharma] and review [~aasha]
> Deleting the previous successful dump directory should be based on config
> -
>
> Key: HIVE-24129
> URL: https://issues.apache.org/jira/browse/HIVE-24129
> Project: Hive
> Issue Type: Task
>Reporter: Pravin Sinha
>Assignee: Arko Sharma
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-24129.01.patch, HIVE-24129.02.patch
>
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> {color:#22}Description: Provide a policy level config defaulted to
> true.{color}
> {color:#22}This can help debug any issue in the production.{color}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
