[jira] [Commented] (SPARK-13298) DAG visualization does not render correctly for jobs
[ https://issues.apache.org/jira/browse/SPARK-13298?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15192324#comment-15192324 ] Todd Leo commented on SPARK-13298: -- Pls also see SPARK-13645: DAG Diagram not shown properly in Chrome > DAG visualization does not render correctly for jobs > > > Key: SPARK-13298 > URL: https://issues.apache.org/jira/browse/SPARK-13298 > Project: Spark > Issue Type: Bug > Components: Web UI >Affects Versions: 1.6.0 >Reporter: Lucas Woltmann >Assignee: Shixiong Zhu > Fix For: 1.6.1, 2.0.0 > > Attachments: dag_full.png, dag_viz.png, no-dag.png > > > Whenever I try to open the DAG for a job, I get something like this: > !dag_viz.png! > Obviously the svg doesn't get resized, but if I resize it manually, only the > first of four stages in the DAG is shown. > The js console says (variable v is null in peg$c34): > {code:javascript} > Uncaught TypeError: Cannot read property '3' of null > peg$c34 @ graphlib-dot.min.js:1 > peg$parseidDef @ graphlib-dot.min.js:1 > peg$parseaList @ graphlib-dot.min.js:1 > peg$parseattrListBlock @ graphlib-dot.min.js:1 > peg$parseattrList @ graphlib-dot.min.js:1 > peg$parsenodeStmt @ graphlib-dot.min.js:1 > peg$parsestmt @ graphlib-dot.min.js:1 > peg$parsestmtList @ graphlib-dot.min.js:1 > peg$parsesubgraphStmt @ graphlib-dot.min.js:1 > peg$parsenodeIdOrSubgraph @ graphlib-dot.min.js:1 > peg$parseedgeStmt @ graphlib-dot.min.js:1 > peg$parsestmt @ graphlib-dot.min.js:1 > peg$parsestmtList @ graphlib-dot.min.js:1 > peg$parsesubgraphStmt @ graphlib-dot.min.js:1 > peg$parsenodeIdOrSubgraph @ graphlib-dot.min.js:1 > peg$parseedgeStmt @ graphlib-dot.min.js:1 > peg$parsestmt @ graphlib-dot.min.js:1 > peg$parsestmtList @ graphlib-dot.min.js:1 > peg$parsegraphStmt @ graphlib-dot.min.js:1 > parse @ graphlib-dot.min.js:2 > readOne @ graphlib-dot.min.js:2 > renderDot @ spark-dag-viz.js:281 > (anonymous function) @ spark-dag-viz.js:248 > (anonymous function) @ d3.min.js: > 3Y @ d3.min.js:1 > _a.each @ d3.min.js:3 > renderDagVizForJob @ spark-dag-viz.js:207 > renderDagViz @ spark-dag-viz.js:163 > toggleDagViz @ spark-dag-viz.js:100 > onclick @ ?id=2:153 > {code} > (tested in FIrefox 44.0.1 and Chromium 48.0.2564.103) -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-13298) DAG visualization does not render correctly for jobs
[ https://issues.apache.org/jira/browse/SPARK-13298?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15157633#comment-15157633 ] Shixiong Zhu commented on SPARK-13298: -- Thanks for the reproducer. Just submit a PR to fix it. > DAG visualization does not render correctly for jobs > > > Key: SPARK-13298 > URL: https://issues.apache.org/jira/browse/SPARK-13298 > Project: Spark > Issue Type: Bug > Components: Web UI >Affects Versions: 1.6.0 >Reporter: Lucas Woltmann > Attachments: dag_full.png, dag_viz.png, no-dag.png > > > Whenever I try to open the DAG for a job, I get something like this: > !dag_viz.png! > Obviously the svg doesn't get resized, but if I resize it manually, only the > first of four stages in the DAG is shown. > The js console says (variable v is null in peg$c34): > {code:javascript} > Uncaught TypeError: Cannot read property '3' of null > peg$c34 @ graphlib-dot.min.js:1 > peg$parseidDef @ graphlib-dot.min.js:1 > peg$parseaList @ graphlib-dot.min.js:1 > peg$parseattrListBlock @ graphlib-dot.min.js:1 > peg$parseattrList @ graphlib-dot.min.js:1 > peg$parsenodeStmt @ graphlib-dot.min.js:1 > peg$parsestmt @ graphlib-dot.min.js:1 > peg$parsestmtList @ graphlib-dot.min.js:1 > peg$parsesubgraphStmt @ graphlib-dot.min.js:1 > peg$parsenodeIdOrSubgraph @ graphlib-dot.min.js:1 > peg$parseedgeStmt @ graphlib-dot.min.js:1 > peg$parsestmt @ graphlib-dot.min.js:1 > peg$parsestmtList @ graphlib-dot.min.js:1 > peg$parsesubgraphStmt @ graphlib-dot.min.js:1 > peg$parsenodeIdOrSubgraph @ graphlib-dot.min.js:1 > peg$parseedgeStmt @ graphlib-dot.min.js:1 > peg$parsestmt @ graphlib-dot.min.js:1 > peg$parsestmtList @ graphlib-dot.min.js:1 > peg$parsegraphStmt @ graphlib-dot.min.js:1 > parse @ graphlib-dot.min.js:2 > readOne @ graphlib-dot.min.js:2 > renderDot @ spark-dag-viz.js:281 > (anonymous function) @ spark-dag-viz.js:248 > (anonymous function) @ d3.min.js: > 3Y @ d3.min.js:1 > _a.each @ d3.min.js:3 > renderDagVizForJob @ spark-dag-viz.js:207 > renderDagViz @ spark-dag-viz.js:163 > toggleDagViz @ spark-dag-viz.js:100 > onclick @ ?id=2:153 > {code} > (tested in FIrefox 44.0.1 and Chromium 48.0.2564.103) -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-13298) DAG visualization does not render correctly for jobs
[ https://issues.apache.org/jira/browse/SPARK-13298?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15157637#comment-15157637 ] Apache Spark commented on SPARK-13298: -- User 'zsxwing' has created a pull request for this issue: https://github.com/apache/spark/pull/11309 > DAG visualization does not render correctly for jobs > > > Key: SPARK-13298 > URL: https://issues.apache.org/jira/browse/SPARK-13298 > Project: Spark > Issue Type: Bug > Components: Web UI >Affects Versions: 1.6.0 >Reporter: Lucas Woltmann >Assignee: Shixiong Zhu > Attachments: dag_full.png, dag_viz.png, no-dag.png > > > Whenever I try to open the DAG for a job, I get something like this: > !dag_viz.png! > Obviously the svg doesn't get resized, but if I resize it manually, only the > first of four stages in the DAG is shown. > The js console says (variable v is null in peg$c34): > {code:javascript} > Uncaught TypeError: Cannot read property '3' of null > peg$c34 @ graphlib-dot.min.js:1 > peg$parseidDef @ graphlib-dot.min.js:1 > peg$parseaList @ graphlib-dot.min.js:1 > peg$parseattrListBlock @ graphlib-dot.min.js:1 > peg$parseattrList @ graphlib-dot.min.js:1 > peg$parsenodeStmt @ graphlib-dot.min.js:1 > peg$parsestmt @ graphlib-dot.min.js:1 > peg$parsestmtList @ graphlib-dot.min.js:1 > peg$parsesubgraphStmt @ graphlib-dot.min.js:1 > peg$parsenodeIdOrSubgraph @ graphlib-dot.min.js:1 > peg$parseedgeStmt @ graphlib-dot.min.js:1 > peg$parsestmt @ graphlib-dot.min.js:1 > peg$parsestmtList @ graphlib-dot.min.js:1 > peg$parsesubgraphStmt @ graphlib-dot.min.js:1 > peg$parsenodeIdOrSubgraph @ graphlib-dot.min.js:1 > peg$parseedgeStmt @ graphlib-dot.min.js:1 > peg$parsestmt @ graphlib-dot.min.js:1 > peg$parsestmtList @ graphlib-dot.min.js:1 > peg$parsegraphStmt @ graphlib-dot.min.js:1 > parse @ graphlib-dot.min.js:2 > readOne @ graphlib-dot.min.js:2 > renderDot @ spark-dag-viz.js:281 > (anonymous function) @ spark-dag-viz.js:248 > (anonymous function) @ d3.min.js: > 3Y @ d3.min.js:1 > _a.each @ d3.min.js:3 > renderDagVizForJob @ spark-dag-viz.js:207 > renderDagViz @ spark-dag-viz.js:163 > toggleDagViz @ spark-dag-viz.js:100 > onclick @ ?id=2:153 > {code} > (tested in FIrefox 44.0.1 and Chromium 48.0.2564.103) -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-13298) DAG visualization does not render correctly for jobs
[ https://issues.apache.org/jira/browse/SPARK-13298?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15152057#comment-15152057 ] Lucas Woltmann commented on SPARK-13298: Yes, of course: {code} from pyspark import SparkContext from pyspark.sql import SQLContext sc = SparkContext(appName="HierarchyGroup") sqlContext = SQLContext(sc) df = sqlContext.read.format('com.databricks.spark.csv').options(header='true', inferschema='true').load('cars.csv') stats = df \ .groupby("year", "make", "model") \ .pivot("model") \ .count() \ .cache() stats_elem1 = stats.groupby("make").sum() stats_elem1 = stats_elem1.unionAll(stats.groupby("year").sum()) stats_elem1.show() {code} I use {{--packages com.databricks:spark-csv_2.10:1.3.0}} with {{spark-submit}}. The example csv can be found here (it's the standard example from the csv package documentation): {code} wget https://github.com/databricks/spark-csv/raw/master/src/test/resources/cars.csv {code} > DAG visualization does not render correctly for jobs > > > Key: SPARK-13298 > URL: https://issues.apache.org/jira/browse/SPARK-13298 > Project: Spark > Issue Type: Bug > Components: Web UI >Affects Versions: 1.6.0 >Reporter: Lucas Woltmann > Attachments: dag_full.png, dag_viz.png > > > Whenever I try to open the DAG for a job, I get something like this: > !dag_viz.png! > Obviously the svg doesn't get resized, but if I resize it manually, only the > first of four stages in the DAG is shown. > The js console says (variable v is null in peg$c34): > {code:javascript} > Uncaught TypeError: Cannot read property '3' of null > peg$c34 @ graphlib-dot.min.js:1 > peg$parseidDef @ graphlib-dot.min.js:1 > peg$parseaList @ graphlib-dot.min.js:1 > peg$parseattrListBlock @ graphlib-dot.min.js:1 > peg$parseattrList @ graphlib-dot.min.js:1 > peg$parsenodeStmt @ graphlib-dot.min.js:1 > peg$parsestmt @ graphlib-dot.min.js:1 > peg$parsestmtList @ graphlib-dot.min.js:1 > peg$parsesubgraphStmt @ graphlib-dot.min.js:1 > peg$parsenodeIdOrSubgraph @ graphlib-dot.min.js:1 > peg$parseedgeStmt @ graphlib-dot.min.js:1 > peg$parsestmt @ graphlib-dot.min.js:1 > peg$parsestmtList @ graphlib-dot.min.js:1 > peg$parsesubgraphStmt @ graphlib-dot.min.js:1 > peg$parsenodeIdOrSubgraph @ graphlib-dot.min.js:1 > peg$parseedgeStmt @ graphlib-dot.min.js:1 > peg$parsestmt @ graphlib-dot.min.js:1 > peg$parsestmtList @ graphlib-dot.min.js:1 > peg$parsegraphStmt @ graphlib-dot.min.js:1 > parse @ graphlib-dot.min.js:2 > readOne @ graphlib-dot.min.js:2 > renderDot @ spark-dag-viz.js:281 > (anonymous function) @ spark-dag-viz.js:248 > (anonymous function) @ d3.min.js: > 3Y @ d3.min.js:1 > _a.each @ d3.min.js:3 > renderDagVizForJob @ spark-dag-viz.js:207 > renderDagViz @ spark-dag-viz.js:163 > toggleDagViz @ spark-dag-viz.js:100 > onclick @ ?id=2:153 > {code} > (tested in FIrefox 44.0.1 and Chromium 48.0.2564.103) -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-13298) DAG visualization does not render correctly for jobs
[ https://issues.apache.org/jira/browse/SPARK-13298?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15149457#comment-15149457 ] Shixiong Zhu commented on SPARK-13298: -- Do you have a reproducer? > DAG visualization does not render correctly for jobs > > > Key: SPARK-13298 > URL: https://issues.apache.org/jira/browse/SPARK-13298 > Project: Spark > Issue Type: Bug > Components: Web UI >Affects Versions: 1.6.0 >Reporter: Lucas Woltmann > Attachments: dag_full.png, dag_viz.png > > > Whenever I try to open the DAG for a job, I get something like this: > !dag_viz.png! > Obviously the svg doesn't get resized, but if I resize it manually, only the > first of four stages in the DAG is shown. > The js console says (variable v is null in peg$c34): > {code:javascript} > Uncaught TypeError: Cannot read property '3' of null > peg$c34 @ graphlib-dot.min.js:1 > peg$parseidDef @ graphlib-dot.min.js:1 > peg$parseaList @ graphlib-dot.min.js:1 > peg$parseattrListBlock @ graphlib-dot.min.js:1 > peg$parseattrList @ graphlib-dot.min.js:1 > peg$parsenodeStmt @ graphlib-dot.min.js:1 > peg$parsestmt @ graphlib-dot.min.js:1 > peg$parsestmtList @ graphlib-dot.min.js:1 > peg$parsesubgraphStmt @ graphlib-dot.min.js:1 > peg$parsenodeIdOrSubgraph @ graphlib-dot.min.js:1 > peg$parseedgeStmt @ graphlib-dot.min.js:1 > peg$parsestmt @ graphlib-dot.min.js:1 > peg$parsestmtList @ graphlib-dot.min.js:1 > peg$parsesubgraphStmt @ graphlib-dot.min.js:1 > peg$parsenodeIdOrSubgraph @ graphlib-dot.min.js:1 > peg$parseedgeStmt @ graphlib-dot.min.js:1 > peg$parsestmt @ graphlib-dot.min.js:1 > peg$parsestmtList @ graphlib-dot.min.js:1 > peg$parsegraphStmt @ graphlib-dot.min.js:1 > parse @ graphlib-dot.min.js:2 > readOne @ graphlib-dot.min.js:2 > renderDot @ spark-dag-viz.js:281 > (anonymous function) @ spark-dag-viz.js:248 > (anonymous function) @ d3.min.js: > 3Y @ d3.min.js:1 > _a.each @ d3.min.js:3 > renderDagVizForJob @ spark-dag-viz.js:207 > renderDagViz @ spark-dag-viz.js:163 > toggleDagViz @ spark-dag-viz.js:100 > onclick @ ?id=2:153 > {code} > (tested in FIrefox 44.0.1 and Chromium 48.0.2564.103) -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-13298) DAG visualization does not render correctly for jobs
[ https://issues.apache.org/jira/browse/SPARK-13298?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15148232#comment-15148232 ] Lucas Woltmann commented on SPARK-13298: Looks like .cache() breaks it. DAG without cache(): !dag_full-png! > DAG visualization does not render correctly for jobs > > > Key: SPARK-13298 > URL: https://issues.apache.org/jira/browse/SPARK-13298 > Project: Spark > Issue Type: Bug > Components: Web UI >Affects Versions: 1.6.0 >Reporter: Lucas Woltmann > Attachments: dag_viz.png > > > Whenever I try to open the DAG for a job, I get something like this: > !dag_viz.png! > Obviously the svg doesn't get resized, but if I resize it manually, only the > first of four stages in the DAG is shown. > The js console says (variable v is null in peg$c34): > {code:javascript} > Uncaught TypeError: Cannot read property '3' of null > peg$c34 @ graphlib-dot.min.js:1 > peg$parseidDef @ graphlib-dot.min.js:1 > peg$parseaList @ graphlib-dot.min.js:1 > peg$parseattrListBlock @ graphlib-dot.min.js:1 > peg$parseattrList @ graphlib-dot.min.js:1 > peg$parsenodeStmt @ graphlib-dot.min.js:1 > peg$parsestmt @ graphlib-dot.min.js:1 > peg$parsestmtList @ graphlib-dot.min.js:1 > peg$parsesubgraphStmt @ graphlib-dot.min.js:1 > peg$parsenodeIdOrSubgraph @ graphlib-dot.min.js:1 > peg$parseedgeStmt @ graphlib-dot.min.js:1 > peg$parsestmt @ graphlib-dot.min.js:1 > peg$parsestmtList @ graphlib-dot.min.js:1 > peg$parsesubgraphStmt @ graphlib-dot.min.js:1 > peg$parsenodeIdOrSubgraph @ graphlib-dot.min.js:1 > peg$parseedgeStmt @ graphlib-dot.min.js:1 > peg$parsestmt @ graphlib-dot.min.js:1 > peg$parsestmtList @ graphlib-dot.min.js:1 > peg$parsegraphStmt @ graphlib-dot.min.js:1 > parse @ graphlib-dot.min.js:2 > readOne @ graphlib-dot.min.js:2 > renderDot @ spark-dag-viz.js:281 > (anonymous function) @ spark-dag-viz.js:248 > (anonymous function) @ d3.min.js: > 3Y @ d3.min.js:1 > _a.each @ d3.min.js:3 > renderDagVizForJob @ spark-dag-viz.js:207 > renderDagViz @ spark-dag-viz.js:163 > toggleDagViz @ spark-dag-viz.js:100 > onclick @ ?id=2:153 > {code} > (tested in FIrefox 44.0.1 and Chromium 48.0.2564.103) -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org