dongjoon-hyun commented on a change in pull request #23694: [SPARK-24360][SQL]
Support Hive 3.1 metastore
URL: https://github.com/apache/spark/pull/23694#discussion_r252532112
##########
File path:
sql/hive/src/main/scala/org/apache/spark/sql/hive/client/HiveShim.scala
##########
@@ -1179,3 +1180,128 @@ private[client] class Shim_v2_1 extends Shim_v2_0 {
private[client] class Shim_v2_2 extends Shim_v2_1
private[client] class Shim_v2_3 extends Shim_v2_1
+
+private[client] class Shim_v3_1 extends Shim_v2_3 {
+ // Spark supports only non-ACID operations
+ protected lazy val isAcidIUDoperation = JBoolean.FALSE
Review comment:
Historically, `isAcidIUDoperation` is an evolved one from `isAcid`.
In Hive code, `isAcid` was a general ACID operation and `isAcidIUDoperation`
is now used for ACID Insert/Update/Delete operations. Also, they checks
`isFullAcidTable` and use it together like this.
```java
else if(!isAcidIUDoperation && isFullAcidTable) {
destPath = fixFullAcidPathForLoadData(loadFileType, destPath, txnId,
stmtId, tbl);
}
```
And yes for your last question. We don't know the future of Hive. So, for
the different parameter name, we had better handle differently. That was my
logic.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
