GitHub user icexelloss opened a pull request:
https://github.com/apache/spark/pull/21531
[SPARK-24521][SQL] Fix ineffective test in CachedTableSuite
## What changes were proposed in this pull request?
test("withColumn doesn't invalidate cached dataframe") in CachedTableSuite
doesn't not work because:
The UDF is executed and test count incremented when "df.cache()" is called
and the subsequent "df.collect()" has no effect on the test result.
This PR fixed this test and add another test for caching UDF.
## How was this patch tested?
Add new tests.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/icexelloss/spark fix-cache-test
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/21531.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #21531
----
----
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
