[ https://issues.apache.org/jira/browse/HIVE-10058?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14376552#comment-14376552 ]
Xuefu Zhang edited comment on HIVE-10058 at 3/23/15 8:16 PM: ------------------------------------------------------------- [~chinnalalam], Sorry that I wasn't clear. I was thinking more on HIVE-8858, where I'd like to have visual representation of SparkPlan. As you can see in the class definition, A SparkPlan consists of a graph of SparkTrans. SparkTran has a few subclasses. Some subclass, such as MapInput has property such as "toCache". What is desirable is that we log a SparkPlan in a graphical way similar to what's show for work graph in explain plan, such as: {code} MapInput (cache off) -> Shuffle (cache on) -> Reduce \-> Reduce {code} This is will give us some idea about SparkPlan that we are executing. Let me know if you have any questions. was (Author: xuefuz): [~chinnalalam], Sorry that I wasn't clear. I was thinking more on HIVE-8858, where I'd like to have visual representation of SparkPlan. As you can see in the class definition, A SparkPlan consists of a graph of SparkTrans. SparkTran has a few subclasses. Some subclass, such as MapInput has property such as "toCache". What is desirable is that we log a SparkPlan in a graphical way similar to what's show for work graph in explain plan, such as: {code} MapInput (cache off) -> Shuffle (cache on) -> Reduce \-> Reduce {code} This is will give us some idea about SparkPlan that we are executing. Let me know if you have any questions. > Log the information of cached RDD [Spark Branch] > ------------------------------------------------ > > Key: HIVE-10058 > URL: https://issues.apache.org/jira/browse/HIVE-10058 > Project: Hive > Issue Type: Sub-task > Components: Spark > Reporter: Chinna Rao Lalam > Assignee: Chinna Rao Lalam > Fix For: spark-branch > > Attachments: HIVE-10058.1-spark.patch, HIVE-10058.2-spark.patch > > > Log the cached RDD Id's at info level. -- This message was sent by Atlassian JIRA (v6.3.4#6332)