MaxGekk commented on pull request #30995: URL: https://github.com/apache/spark/pull/30995#issuecomment-754784200
@advancedxy There should be 2 conditions to fail: 1. Canonicalized plan includes table stats 2. Different table stats in already cached plan in the cache and in the plan to check. Both 3.0 and master includes table stats to canonicalized plan of `HiveTableRelation` but cached plan in master does not contains table stats due to this PR https://github.com/apache/spark/pull/30598. In the master, we have another code for `CACHE TABLE` comparing to 3.0. master: ``` sparkSession.sharedState.cacheManager.cacheQuery( sparkSession, planToCache, Some(relationName)) ``` 3.0: ``` sparkSession.catalog.cacheTable(tableIdent.quotedString) ``` 3.0 puts plans with stats to the cache but master doesn't. And the test performs the check `assert(spark.catalog.isCached(t))` which internally looks for `sparkSession.table(tableName)` in the cache. And the plan for `sparkSession.table(tableName)` includes tables stats. As a consequence, the assert passes. ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
