Pedrum Jalali created SPARK-62097:
-------------------------------------

             Summary: Overridable Test Assertions
                 Key: SPARK-62097
                 URL: https://issues.apache.org/jira/browse/SPARK-62097
             Project: Spark
          Issue Type: Improvement
          Components: Tests
    Affects Versions: 4.0.6
            Reporter: Pedrum Jalali
             Fix For: 4.0.6


Proposing to refactor Spark test assertion statements to be protected functions 
instead of inline statements. 
---
h3. Before
{code:java}
private def someTest(): Unit = {
    ...
    checkError(
      exception = intercept[SparkRuntimeException] {
        df5.select(map_from_arrays($"k", $"v")).collect()
      },
      condition = "NULL_MAP_KEY",
      parameters = Map.empty
    )
}
{code}
----
h3. After
We first define a helper protected function like this:

{code:java}
  protected def assertNullMapKeyFailure(func: => Any): Unit = {
    checkError(
      exception = intercept[SparkRuntimeException](func),
      condition = "NULL_MAP_KEY",
      parameters = Map.empty
    )
  }
{code}

And call the function in the tests like this
{code:java}
private def someTest(): Unit = {
    ...
      assertNullMapKeyFailure {
      df5.select(map_from_arrays($"k", $"v")).collect()
    }
}
{code}

----
h3. Why
* Better code re-use in tests
* Downstream repositories (i.e Gluten) import Spark tests and run them as is. 
Changes to exceptions have caused these downstream repositories to have to copy 
entire tests just to change the assertion clause. This would enable much better 
code re-use in downstream repositories as well (please see 
https://github.com/apache/gluten/issues/13647)
----
h3. Example

https://github.com/apache/spark/pull/58878





--
This message was sent by Atlassian Jira
(v10.3.26#10030026)

---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to