[hive] branch master updated: HIVE-22816 : QueryCache: Queries using views can have them cached after CTE expansion (Gopal V via Jesus Camacho Rodriguez)
This is an automated email from the ASF dual-hosted git repository.
hashutosh pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/hive.git
The following commit(s) were added to refs/heads/master by this push:
new f826283 HIVE-22816 : QueryCache: Queries using views can have them
cached after CTE expansion (Gopal V via Jesus Camacho Rodriguez)
f826283 is described below
commit f826283d36503eb7598b1e33b10bd5d7d1ddd074
Author: Gopal V
AuthorDate: Thu Feb 20 20:01:01 2020 -0800
HIVE-22816 : QueryCache: Queries using views can have them cached after CTE
expansion (Gopal V via Jesus Camacho Rodriguez)
Signed-off-by: Ashutosh Chauhan
---
ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java | 2 ++
1 file changed, 2 insertions(+)
diff --git a/ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java
b/ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java
index f46739e..2328eed 100644
--- a/ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java
+++ b/ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java
@@ -15279,8 +15279,10 @@ public class SemanticAnalyzer extends
BaseSemanticAnalyzer {
}
private List getNonTransactionalTables() {
+// views have been expanded by CBO already and can be ignored
return tablesFromReadEntities(inputs)
.stream()
+.filter(table -> !table.isView())
.filter(table -> !AcidUtils.isTransactionalTable(table))
.collect(Collectors.toList());
}
[hive] branch master updated: HIVE-22880 : ACID: All delete event readers should ignore ORC SARGs (Gopal V via Rajesh Balamohan)
This is an automated email from the ASF dual-hosted git repository.
hashutosh pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/hive.git
The following commit(s) were added to refs/heads/master by this push:
new 657b510 HIVE-22880 : ACID: All delete event readers should ignore ORC
SARGs (Gopal V via Rajesh Balamohan)
657b510 is described below
commit 657b5107a34a205aae76fde42020453f04f40ead
Author: Gopal V
AuthorDate: Thu Feb 20 19:26:51 2020 -0800
HIVE-22880 : ACID: All delete event readers should ignore ORC SARGs (Gopal
V via Rajesh Balamohan)
Signed-off-by: Ashutosh Chauhan
---
.../apache/hadoop/hive/ql/io/orc/VectorizedOrcAcidRowBatchReader.java| 1 +
1 file changed, 1 insertion(+)
diff --git
a/ql/src/java/org/apache/hadoop/hive/ql/io/orc/VectorizedOrcAcidRowBatchReader.java
b/ql/src/java/org/apache/hadoop/hive/ql/io/orc/VectorizedOrcAcidRowBatchReader.java
index 2543dc6..0361872 100644
---
a/ql/src/java/org/apache/hadoop/hive/ql/io/orc/VectorizedOrcAcidRowBatchReader.java
+++
b/ql/src/java/org/apache/hadoop/hive/ql/io/orc/VectorizedOrcAcidRowBatchReader.java
@@ -236,6 +236,7 @@ public class VectorizedOrcAcidRowBatchReader
// Set the range on the deleteEventReaderOptions to 0 to INTEGER_MAX
because
// we always want to read all the delete delta files.
deleteEventReaderOptions.range(0, Long.MAX_VALUE);
+deleteEventReaderOptions.searchArgument(null, null);
keyInterval = findMinMaxKeys(orcSplit, conf, deleteEventReaderOptions);
DeleteEventRegistry der;
try {
[hive] branch master updated: HIVE-22908: AM caching connections to LLAP based on hostname and port does not work in kubernetes (Prasanth Jayachandran reviewed by Gopal Vijayaraghavan)
This is an automated email from the ASF dual-hosted git repository.
prasanthj pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/hive.git
The following commit(s) were added to refs/heads/master by this push:
new 703cf29 HIVE-22908: AM caching connections to LLAP based on hostname
and port does not work in kubernetes (Prasanth Jayachandran reviewed by Gopal
Vijayaraghavan)
703cf29 is described below
commit 703cf29fa14fffe7a7e0ee30cd11d16a5245bf1c
Author: Prasanth Jayachandran
AuthorDate: Thu Feb 20 14:38:33 2020 -0800
HIVE-22908: AM caching connections to LLAP based on hostname and port does
not work in kubernetes (Prasanth Jayachandran reviewed by Gopal Vijayaraghavan)
---
.../org/apache/hadoop/hive/llap/AsyncPbRpcProxy.java | 16 +++-
1 file changed, 15 insertions(+), 1 deletion(-)
diff --git
a/llap-common/src/java/org/apache/hadoop/hive/llap/AsyncPbRpcProxy.java
b/llap-common/src/java/org/apache/hadoop/hive/llap/AsyncPbRpcProxy.java
index ad39963..25e10f4 100644
--- a/llap-common/src/java/org/apache/hadoop/hive/llap/AsyncPbRpcProxy.java
+++ b/llap-common/src/java/org/apache/hadoop/hive/llap/AsyncPbRpcProxy.java
@@ -15,6 +15,8 @@
package org.apache.hadoop.hive.llap;
import java.io.IOException;
+import java.net.InetAddress;
+import java.net.UnknownHostException;
import java.security.PrivilegedAction;
import java.util.HashSet;
import java.util.Iterator;
@@ -430,6 +432,9 @@ public abstract class AsyncPbRpcProxy nodeToken) {
String hostId = getHostIdentifier(nodeId.getHostname(), nodeId.getPort());
+if (LOG.isDebugEnabled()) {
+ LOG.debug("Getting host proxies for {}", hostId);
+}
try {
return hostProxies.get(hostId, new Callable() {
@Override
@@ -481,7 +486,16 @@ public abstract class AsyncPbRpcProxy
[hive] branch master updated: HIVE-22831: Add option in HiveStrictManagedMigration to also move tables converted to external living in old WH (Adam Szita, reviewed by Peter Vary)
This is an automated email from the ASF dual-hosted git repository.
szita pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/hive.git
The following commit(s) were added to refs/heads/master by this push:
new faaf2c3 HIVE-22831: Add option in HiveStrictManagedMigration to also
move tables converted to external living in old WH (Adam Szita, reviewed by
Peter Vary)
faaf2c3 is described below
commit faaf2c3fda527cbebf0ed22ab95691ddb9ad8588
Author: Adam Szita
AuthorDate: Thu Feb 6 17:10:39 2020 +0100
HIVE-22831: Add option in HiveStrictManagedMigration to also move tables
converted to external living in old WH (Adam Szita, reviewed by Peter Vary)
---
.../hive/ql/util/HiveStrictManagedMigration.java | 251 ++---
.../ql/util/TestHiveStrictManagedMigration.java| 202 -
2 files changed, 368 insertions(+), 85 deletions(-)
diff --git
a/ql/src/java/org/apache/hadoop/hive/ql/util/HiveStrictManagedMigration.java
b/ql/src/java/org/apache/hadoop/hive/ql/util/HiveStrictManagedMigration.java
index 19fb750..e2e5983 100644
--- a/ql/src/java/org/apache/hadoop/hive/ql/util/HiveStrictManagedMigration.java
+++ b/ql/src/java/org/apache/hadoop/hive/ql/util/HiveStrictManagedMigration.java
@@ -73,6 +73,10 @@ import org.slf4j.LoggerFactory;
import com.google.common.annotations.VisibleForTesting;
import com.google.common.collect.Lists;
+import static java.util.stream.Collectors.toList;
+import static org.apache.hadoop.hive.metastore.TableType.EXTERNAL_TABLE;
+import static org.apache.hadoop.hive.metastore.TableType.MANAGED_TABLE;
+
public class HiveStrictManagedMigration {
private static final Logger LOG =
LoggerFactory.getLogger(HiveStrictManagedMigration.class);
@@ -91,9 +95,10 @@ public class HiveStrictManagedMigration {
final String oldWarehouseRoot;
final TableMigrationOption migrationOption;
final Properties confProps;
-final boolean shouldModifyManagedTableLocation;
+boolean shouldModifyManagedTableLocation;
final boolean shouldModifyManagedTableOwner;
final boolean shouldModifyManagedTablePermissions;
+boolean shouldMoveExternal;
final boolean dryRun;
final TableType tableType;
final int tablePoolSize;
@@ -106,6 +111,7 @@ public class HiveStrictManagedMigration {
boolean shouldModifyManagedTableLocation,
boolean shouldModifyManagedTableOwner,
boolean shouldModifyManagedTablePermissions,
+ boolean shouldMoveExternal,
boolean dryRun,
TableType tableType,
int tablePoolSize) {
@@ -118,24 +124,18 @@ public class HiveStrictManagedMigration {
this.shouldModifyManagedTableLocation = shouldModifyManagedTableLocation;
this.shouldModifyManagedTableOwner = shouldModifyManagedTableOwner;
this.shouldModifyManagedTablePermissions =
shouldModifyManagedTablePermissions;
+ this.shouldMoveExternal = shouldMoveExternal;
this.dryRun = dryRun;
this.tableType = tableType;
this.tablePoolSize = tablePoolSize;
}
-public RunOptions setShouldModifyManagedTableLocation(boolean
shouldModifyManagedTableLocation) {
- return new RunOptions(
- this.dbRegex,
- this.tableRegex,
- this.oldWarehouseRoot,
- this.migrationOption,
- this.confProps,
- shouldModifyManagedTableLocation,
- this.shouldModifyManagedTableOwner,
- this.shouldModifyManagedTablePermissions,
- this.dryRun,
- this.tableType,
- this.tablePoolSize);
+public void setShouldModifyManagedTableLocation(boolean
shouldModifyManagedTableLocation) {
+ this.shouldModifyManagedTableLocation = shouldModifyManagedTableLocation;
+}
+
+public void setShouldMoveExternal(boolean shouldMoveExternal) {
+ this.shouldMoveExternal = shouldMoveExternal;
}
@Override
@@ -149,6 +149,7 @@ public class HiveStrictManagedMigration {
", shouldModifyManagedTableLocation=" +
shouldModifyManagedTableLocation +
", shouldModifyManagedTableOwner=" +
shouldModifyManagedTableOwner +
", shouldModifyManagedTablePermissions=" +
shouldModifyManagedTablePermissions +
+ ", shouldMoveExternal=" + shouldMoveExternal +
", dryRun=" + dryRun +
", tableType=" + tableType +
", tablePoolSize=" + tablePoolSize +
@@ -172,16 +173,22 @@ public class HiveStrictManagedMigration {
private static class WarehouseRootCheckResult {
final boolean shouldModifyManagedTableLocation;
-final Path curWhRootPath;
+final boolean shouldMoveExternal;
+final Path targetPath;
final HadoopShims.HdfsEncryptionShim encryptionShim;
+final HadoopShims.HdfsErasureCodingShim ecShim;
WarehouseRootCheckResult(
boolean
[hive] branch master updated (fd08239 -> 2705e93)
This is an automated email from the ASF dual-hosted git repository. mgergely pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/hive.git. from fd08239 HIVE-22881: Revise non-recommended Calcite api calls (Zoltan Haindrich reviewed by Jesus Camacho Rodriguez) add 2705e93 HIVE-22897 Remove enforcing of package-info.java files from the rest of the checkstyle files (Miklos Gergely, reviewed by Peter Vary) No new revisions were added by this update. Summary of changes: standalone-metastore/checkstyle/checkstyle.xml | 3 --- storage-api/checkstyle/checkstyle.xml | 3 --- 2 files changed, 6 deletions(-)
