Huang-yi-3456 edited a comment on issue #10577: [SPARK-12616] [SQL] Making Logical Operator `Union` Support Arbitrary Number of Children URL: https://github.com/apache/spark/pull/10577#issuecomment-610782532 @cloud-fan thanks for your quick response. I have a simple test, in which a is cached and b is not and here is the output of explain method: > == Parsed Logical Plan == Union :- AnalysisBarrier : +- LogicalRDD [number#2, word#3], false +- AnalysisBarrier +- LogicalRDD [number#8, word#9], false > == Analyzed Logical Plan == number: int, word: string Union :- LogicalRDD [number#2, word#3], false +- LogicalRDD [number#8, word#9], false > == Optimized Logical Plan == Union :- InMemoryRelation [number#2, word#3], true, 10000, StorageLevel(disk, memory, deserialized, 1 replicas) : +- Scan ExistingRDD[number#2,word#3] +- LogicalRDD [number#8, word#9], false > == Physical Plan == Union :- InMemoryTableScan [number#2, word#3] : +- InMemoryRelation [number#2, word#3], true, 10000, StorageLevel(disk, memory, deserialized, 1 replicas) : +- Scan ExistingRDD[number#2,word#3] +- Scan ExistingRDD[number#8,word#9] It seems the cached a is used. Please bear my ignorance and correct me what's wrong here. Thanks.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
