Github user mgaido91 commented on a diff in the pull request:
https://github.com/apache/spark/pull/22263#discussion_r213568423
--- Diff:
sql/core/src/test/scala/org/apache/spark/sql/CachedTableSuite.scala ---
@@ -64,6 +66,13 @@ class CachedTableSuite extends QueryTest with
SQLTestUtils with SharedSQLContext
maybeBlock.nonEmpty
}
+ def isExpectStorageLevel(rddId: Int, level:
DataReadMethod.DataReadMethod): Boolean = {
+ val maybeBlock = sparkContext.env.blockManager.get(RDDBlockId(rddId,
0))
--- End diff --
shall we also check that the block is present? If nothing is cached this
would (wrongly IMHO) return true. I see that since you are checking
`isMaterialized` this doesn't happen in your test cases, but this function may
be reused in the future so I think it'd be better to have it behaving
consistently to its name. Do you agree?
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
