[
https://issues.apache.org/jira/browse/SPARK-62097?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18147758#comment-18147758
]
Pedrum Jalali commented on SPARK-62097:
---------------------------------------
CC [~LuciferYang] [~dongjoon]
> Overridable Test Assertions
> ---------------------------
>
> Key: SPARK-62097
> URL: https://issues.apache.org/jira/browse/SPARK-62097
> Project: Spark
> Issue Type: Improvement
> Components: Tests
> Affects Versions: 4.0.6
> Reporter: Pedrum Jalali
> Priority: Trivial
> Fix For: 4.0.6
>
>
> Proposing to refactor Spark test assertion statements to be protected
> functions instead of inline statements.
> ---
> h3. Before
> {code:java}
> private def someTest(): Unit = {
> ...
> checkError(
> exception = intercept[SparkRuntimeException] {
> df5.select(map_from_arrays($"k", $"v")).collect()
> },
> condition = "NULL_MAP_KEY",
> parameters = Map.empty
> )
> }
> {code}
> ----
> h3. After
> We first define a helper protected function like this:
> {code:java}
> protected def assertNullMapKeyFailure(func: => Any): Unit = {
> checkError(
> exception = intercept[SparkRuntimeException](func),
> condition = "NULL_MAP_KEY",
> parameters = Map.empty
> )
> }
> {code}
> And call the function in the tests like this
> {code:java}
> private def someTest(): Unit = {
> ...
> assertNullMapKeyFailure {
> df5.select(map_from_arrays($"k", $"v")).collect()
> }
> }
> {code}
> ----
> h3. Why
> * Better code re-use in tests
> * Downstream repositories (i.e Gluten) import Spark tests and run them as is.
> Changes to exceptions have caused these downstream repositories to have to
> copy entire tests just to change the assertion clause. This would enable much
> better code re-use in downstream repositories as well (please see
> https://github.com/apache/gluten/issues/13647)
> ----
> h3. Example
> https://github.com/apache/spark/pull/58878
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]