Pedrum Jalali created SPARK-62097:
-------------------------------------
Summary: Overridable Test Assertions
Key: SPARK-62097
URL: https://issues.apache.org/jira/browse/SPARK-62097
Project: Spark
Issue Type: Improvement
Components: Tests
Affects Versions: 4.0.6
Reporter: Pedrum Jalali
Fix For: 4.0.6
Proposing to refactor Spark test assertion statements to be protected functions
instead of inline statements.
---
h3. Before
{code:java}
private def someTest(): Unit = {
...
checkError(
exception = intercept[SparkRuntimeException] {
df5.select(map_from_arrays($"k", $"v")).collect()
},
condition = "NULL_MAP_KEY",
parameters = Map.empty
)
}
{code}
----
h3. After
We first define a helper protected function like this:
{code:java}
protected def assertNullMapKeyFailure(func: => Any): Unit = {
checkError(
exception = intercept[SparkRuntimeException](func),
condition = "NULL_MAP_KEY",
parameters = Map.empty
)
}
{code}
And call the function in the tests like this
{code:java}
private def someTest(): Unit = {
...
assertNullMapKeyFailure {
df5.select(map_from_arrays($"k", $"v")).collect()
}
}
{code}
----
h3. Why
* Better code re-use in tests
* Downstream repositories (i.e Gluten) import Spark tests and run them as is.
Changes to exceptions have caused these downstream repositories to have to copy
entire tests just to change the assertion clause. This would enable much better
code re-use in downstream repositories as well (please see
https://github.com/apache/gluten/issues/13647)
----
h3. Example
https://github.com/apache/spark/pull/58878
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]