Github user gatorsmile commented on a diff in the pull request:
https://github.com/apache/spark/pull/12146#discussion_r59094324
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/command/ddl.scala ---
@@ -176,12 +176,70 @@ case class DescribeDatabase(
}
/**
+ * Drops a table/view from the metastore and removes it if it is cached.
+ *
+ * The syntax of this command is:
+ * {{{
+ * DROP TABLE [IF EXISTS] table_name [PURGE] [FOR REPLICATION(event_id)];
+ * DROP VIEW [IF EXISTS] [db_name.]view_name;
+ * }}}
+ *
+ * Note, PURGE and replication clauses are not supported
+ */
+case class DropTable(
+ tableName: TableIdentifier,
+ ifExists: Boolean,
+ isView: Boolean) extends RunnableCommand {
+
+ override def run(sqlContext: SQLContext): Seq[Row] = {
+ val catalog = sqlContext.sessionState.catalog
+ if (isView && !catalog.isViewSupported) {
+ throw new AnalysisException(s"Not supported object: views")
+ }
+ // If the command DROP VIEW is to drop a table or DROP TABLE is to
drop a view
+ // issue an exception.
+ catalog.getTableOption(tableName).map(_.tableType match {
+ case CatalogTableType.VIRTUAL_VIEW if !isView =>
+ throw new AnalysisException(s"Cannot drop a view with DROP TABLE")
+ case o if o != CatalogTableType.VIRTUAL_VIEW && isView =>
+ throw new AnalysisException(s"Cannot drop a table with DROP VIEW")
+ case _ =>
+ })
+
+ try {
+
sqlContext.cacheManager.tryUncacheQuery(sqlContext.table(tableName.quotedString))
+ } catch {
+ // This table's metadata is not in Hive metastore (e.g. the table
does not exist).
+ case e if e.getClass.getName ==
"org.apache.hadoop.hive.ql.metadata.InvalidTableException" =>
+ case _: org.apache.spark.sql.catalyst.analysis.NoSuchTableException
=>
+ // Other Throwables can be caused by users providing wrong
parameters in OPTIONS
+ // (e.g. invalid paths). We catch it and log a warning message.
+ // Users should be able to drop such kinds of tables regardless if
there is an error.
+ case e: Throwable => log.warn(s"${e.getMessage}", e)
+ }
+ catalog.invalidateTable(tableName)
+
+ try {
+ catalog.dropTable(tableName, ifExists)
+ } catch {
+ // Quiesce the exception issued from Hive. Log it as an error message
+ // This is just to make it consistent with Hive Native Command;
Otherwise, many test cases
+ // in HiveCompatibilitySuite fail. In Hive, the orders of test cases
matter. That is, the
+ // test cases are not independent.
+ case e: org.apache.spark.sql.AnalysisException
+ if e.getMessage.contains("NoSuchObjectException") =>
logError(s"${e.getMessage}", e)
--- End diff --
Yeah. Agree. Now, we hit 30 test failures in `HiveCompatibilitySuite` if we
issue an exception. The reason why we did hot hit them is we will not get
exceptions when we use Hive Native SQL command. Basically, this is the existing
behavior.
You might also want to know why Hive does not have this issue. The test
cases are dependent on each other. They did not clean the environment at the
end of test cases . They expect the next/subsequent test cases drop the tables.
They did not put `IF EXISTS` clause in the `DROP TABLE` statement.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
