doki23 commented on PR #45181: URL: https://github.com/apache/spark/pull/45181#issuecomment-1979974166
> I don't think it fixes the issue completely and there are some problems with the solution. I believe a proper solution is in the following comment: [#45181 (comment)](https://github.com/apache/spark/pull/45181#issuecomment-1969241145) > > Also many more tests are needed. Maybe it is. But we need check logicalPlan's children like this: ```scala def queryExecution: QueryExecution = { val cacheManager = queryExecutionUnPersisted.sparkSession.sharedState.cacheManager val plan = queryExecutionUnPersisted.logical val findCachedFragment = plan.find({ case command: IgnoreCachedData => false case currentFragment => cacheManager.lookupCachedData(currentFragment).isEmpty }).isDefined if (findCachedFragment) queryExecutionUnPersisted else queryExecutionPersisted } ``` And I don't know if it breaks the binary compatibility. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
