William1104 opened a new pull request #24747: [SPARK-27772][SQL]SQLTestUtils Refactoring URL: https://github.com/apache/spark/pull/24747 ## What changes were proposed in this pull request? The current `SQLTestUtils` created many `withXXX` utility functions to clean up tables/views/caches created for testing purpose. Some of those `withXXX` functions ignore certain exceptions, like `NoSuchTableException` in the clean up block (ie, the finally block). ``` /** * Drops temporary view `viewNames` after calling `f`. */ protected def withTempView(viewNames: String*)(f: => Unit): Unit = { try f finally { // If the test failed part way, we don't want to mask the failure by failing to remove // temp views that never got created. try viewNames.foreach(spark.catalog.dropTempView) catch { case _: NoSuchTableException => } } } ``` I believe it is not the best approach. Because it is hard to anticipate what exception should or should not be ignored. Java's `try-with-resources` statement does not mask exception throwing in the try block with any exception caught in the 'close()' statement. Exception caught in the 'close()' statement would add as a suppressed exception instead. It sounds a better approach. Therefore, I proposed to standardise those 'withXXX' function with following `tryWithFinally` function, which does something similar to Java's try-with-resources statement. ``` /** * Drops temporary view `viewNames` after calling `f`. */ protected def withTempView(viewNames: String*)(f: => Unit): Unit = { tryWithFinally(f)(viewNames.foreach(spark.catalog.dropTempView)) } /** * Executes the given tryBlock and then the given finallyBlock no matter whether tryBlock throws * an exception. If both tryBlock and finallyBlock throw exceptions, the exception thrown * from the finallyBlock with be added to the exception thrown from tryBlock as a * suppress exception. It helps to avoid masking the exception from tryBlock with exception * from finallyBlock */ private def tryWithFinally(tryBlock: => Unit)(finallyBlock: => Unit): Unit = { var fromTryBlock: Throwable = null try tryBlock catch { case cause: Throwable => fromTryBlock = cause throw cause } finally { if (fromTryBlock != null) { try finallyBlock catch { case fromFinallyBlock: Throwable => fromTryBlock.addSuppressed(fromFinallyBlock) throw fromTryBlock } } else { finallyBlock } } } ``` If a feature is well written, we show not hit any exception in those closing method in testcase. The purpose of this proposal is to help developers to identify what actually break their tests. ## How was this patch tested? A unit test is added to make sure those withXXX (eg withCache) methods would not mask the original exception.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
