[GitHub] [spark] Eric5553 commented on a change in pull request #27952: [SPARK-31184][SQL] Support getTablesByType API of Hive Client

2020-03-19 Thread GitBox 
Eric5553 commented on a change in pull request #27952: [SPARK-31184][SQL] 
Support getTablesByType API of Hive Client
URL: https://github.com/apache/spark/pull/27952#discussion_r395056474
 
 

 ##
 File path: 
sql/hive/src/main/scala/org/apache/spark/sql/hive/client/HiveShim.scala
 ##
 @@ -363,6 +370,15 @@ private[client] class Shim_v0_12 extends Shim with 
Logging {
 conf.getIntVar(HiveConf.ConfVars.METASTORE_CLIENT_CONNECT_RETRY_DELAY) * 
1000L
   }
 
+  override def getTablesByType(
+  hive: Hive,
+  dbName: String,
+  pattern: String,
+  tableType: TableType): Seq[String] = {
+throw new UnsupportedOperationException("Hive 2.2 and lower versions don't 
support " +
+  "getTablesByType. Please use Hive 2.3 or higher version.")
 
 Review comment:
   The fallback logic needs to call `HiveClient.getTablesByName`, which got a 
lot of logic in `HiveClient`. Thus I think it needs to handle the fallback 
logic in `HiveClient` instead of `HiveShim`. 
0a550b17ff24e43144d841ac449b510ef1c297cf WDYT @maropu @cloud-fan @HyukjinKwon , 
thanks!
   
   Also I renamed the `HiveClient.listViews` API to 
`HiveClient.listTablesByType` to provide a more common interface.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] [spark] Eric5553 commented on a change in pull request #27952: [SPARK-31184][SQL] Support getTablesByType API of Hive Client

2020-03-19 Thread GitBox 
Eric5553 commented on a change in pull request #27952: [SPARK-31184][SQL] 
Support getTablesByType API of Hive Client
URL: https://github.com/apache/spark/pull/27952#discussion_r395056474
 
 

 ##
 File path: 
sql/hive/src/main/scala/org/apache/spark/sql/hive/client/HiveShim.scala
 ##
 @@ -363,6 +370,15 @@ private[client] class Shim_v0_12 extends Shim with 
Logging {
 conf.getIntVar(HiveConf.ConfVars.METASTORE_CLIENT_CONNECT_RETRY_DELAY) * 
1000L
   }
 
+  override def getTablesByType(
+  hive: Hive,
+  dbName: String,
+  pattern: String,
+  tableType: TableType): Seq[String] = {
+throw new UnsupportedOperationException("Hive 2.2 and lower versions don't 
support " +
+  "getTablesByType. Please use Hive 2.3 or higher version.")
 
 Review comment:
   The fallback logic needs to call `HiveClient.getTablesByName`, which got a 
lot of logic in `HiveClient`. Thus I think it needs to handle the fallback 
logic in `HiveClient` instead of `HiveShim`. WDYT @maropu @cloud-fan 
@HyukjinKwon , thanks!
   
   Also I renamed the `HiveClient.listViews` API to 
`HiveClient.listTablesByType` to provide a more common interface.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] [spark] Eric5553 commented on a change in pull request #27952: [SPARK-31184][SQL] Support getTablesByType API of Hive Client

2020-03-19 Thread GitBox 
Eric5553 commented on a change in pull request #27952: [SPARK-31184][SQL] 
Support getTablesByType API of Hive Client
URL: https://github.com/apache/spark/pull/27952#discussion_r394815124
 
 

 ##
 File path: 
sql/hive/src/main/scala/org/apache/spark/sql/hive/client/HiveShim.scala
 ##
 @@ -363,6 +370,15 @@ private[client] class Shim_v0_12 extends Shim with 
Logging {
 conf.getIntVar(HiveConf.ConfVars.METASTORE_CLIENT_CONNECT_RETRY_DELAY) * 
1000L
   }
 
+  override def getTablesByType(
+  hive: Hive,
+  dbName: String,
+  pattern: String,
+  tableType: TableType): Seq[String] = {
+throw new UnsupportedOperationException("Hive 2.2 and lower versions don't 
support " +
+  "getTablesByType. Please use Hive 2.3 or higher version.")
 
 Review comment:
   Previously I thought `HiveShim` is a thin layer close to Hive API. Yea, it 
should be better if we handle it in low-level API. I'll make the change here.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] [spark] Eric5553 commented on a change in pull request #27952: [SPARK-31184][SQL] Support getTablesByType API of Hive Client

2020-03-19 Thread GitBox 
Eric5553 commented on a change in pull request #27952: [SPARK-31184][SQL] 
Support getTablesByType API of Hive Client
URL: https://github.com/apache/spark/pull/27952#discussion_r394815124
 
 

 ##
 File path: 
sql/hive/src/main/scala/org/apache/spark/sql/hive/client/HiveShim.scala
 ##
 @@ -363,6 +370,15 @@ private[client] class Shim_v0_12 extends Shim with 
Logging {
 conf.getIntVar(HiveConf.ConfVars.METASTORE_CLIENT_CONNECT_RETRY_DELAY) * 
1000L
   }
 
+  override def getTablesByType(
+  hive: Hive,
+  dbName: String,
+  pattern: String,
+  tableType: TableType): Seq[String] = {
+throw new UnsupportedOperationException("Hive 2.2 and lower versions don't 
support " +
+  "getTablesByType. Please use Hive 2.3 or higher version.")
 
 Review comment:
   I thought `HiveShim` is a thin layer close to Hive API. Yea, it should be 
better if we handle it in low-level API. I'll make the change here.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] [spark] Eric5553 commented on a change in pull request #27952: [SPARK-31184][SQL] Support getTablesByType API of Hive Client

2020-03-18 Thread GitBox 
Eric5553 commented on a change in pull request #27952: [SPARK-31184][SQL] 
Support getTablesByType API of Hive Client
URL: https://github.com/apache/spark/pull/27952#discussion_r394767919
 
 

 ##
 File path: 
sql/hive/src/main/scala/org/apache/spark/sql/hive/client/HiveShim.scala
 ##
 @@ -363,6 +370,15 @@ private[client] class Shim_v0_12 extends Shim with 
Logging {
 conf.getIntVar(HiveConf.ConfVars.METASTORE_CLIENT_CONNECT_RETRY_DELAY) * 
1000L
   }
 
+  override def getTablesByType(
+  hive: Hive,
+  dbName: String,
+  pattern: String,
+  tableType: TableType): Seq[String] = {
+throw new UnsupportedOperationException("Hive 2.2 and lower versions don't 
support " +
+  "getTablesByType. Please use Hive 2.3 or higher version.")
 
 Review comment:
   Sure, that's a mismatch. Updated the description, thanks :-)


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] [spark] Eric5553 commented on a change in pull request #27952: [SPARK-31184][SQL] Support getTablesByType API of Hive Client

2020-03-18 Thread GitBox 
Eric5553 commented on a change in pull request #27952: [SPARK-31184][SQL] 
Support getTablesByType API of Hive Client
URL: https://github.com/apache/spark/pull/27952#discussion_r394766333
 
 

 ##
 File path: 
sql/hive/src/main/scala/org/apache/spark/sql/hive/client/HiveShim.scala
 ##
 @@ -363,6 +370,15 @@ private[client] class Shim_v0_12 extends Shim with 
Logging {
 conf.getIntVar(HiveConf.ConfVars.METASTORE_CLIENT_CONNECT_RETRY_DELAY) * 
1000L
   }
 
+  override def getTablesByType(
+  hive: Hive,
+  dbName: String,
+  pattern: String,
+  tableType: TableType): Seq[String] = {
+throw new UnsupportedOperationException("Hive 2.2 and lower versions don't 
support " +
+  "getTablesByType. Please use Hive 2.3 or higher version.")
 
 Review comment:
   As @cloud-fan suggested in 
https://github.com/apache/spark/pull/27897#discussion_r394391663, for versions 
before Hive 2.3, I'll implement the fallback solution using `listTables` and 
filter the result in PR https://github.com/apache/spark/pull/27897.
   Thanks for the review! @HyukjinKwon 


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org