Github user zsxwing commented on the pull request:
https://github.com/apache/spark/pull/5473#issuecomment-92621825
Why would we get wrong properties in this case? We are explicitly setting
it in the thread that is launching the Spark job, so there is no question of
inheriting local
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/5473#issuecomment-92647115
[Test build #30223 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/30223/consoleFull)
for PR 5473 at commit
Github user zsxwing commented on the pull request:
https://github.com/apache/spark/pull/5473#issuecomment-92646872
Please ignore that `Processing time` and `Total delay` are `-`. It's an
issue about `JobScheduler`. I will open a JIRA for it.
---
If your project is set up for it, you
Github user zsxwing commented on the pull request:
https://github.com/apache/spark/pull/5473#issuecomment-92660739
Please ignore that Processing time and Total delay are -. It's an issue
about JobScheduler. I will open a JIRA for it.
After thinking it carefully, this may not
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/5473#issuecomment-92662703
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/5473#issuecomment-92662661
[Test build #30220 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/30220/consoleFull)
for PR 5473 at commit
Github user zsxwing commented on the pull request:
https://github.com/apache/spark/pull/5473#issuecomment-92646413
I set the max-width of the error column to `300px`. Here is the new
screenshot:
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/5473#issuecomment-92682373
[Test build #30223 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/30223/consoleFull)
for PR 5473 at commit
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/5473#issuecomment-92682417
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user zsxwing commented on a diff in the pull request:
https://github.com/apache/spark/pull/5473#discussion_r28301652
--- Diff:
streaming/src/main/scala/org/apache/spark/streaming/ui/BatchPage.scala ---
@@ -0,0 +1,224 @@
+/*
+ * Licensed to the Apache Software
Github user zsxwing commented on a diff in the pull request:
https://github.com/apache/spark/pull/5473#discussion_r28301670
--- Diff:
streaming/src/main/scala/org/apache/spark/streaming/scheduler/JobScheduler.scala
---
@@ -179,7 +181,14 @@ class JobScheduler(val ssc:
Github user zsxwing commented on a diff in the pull request:
https://github.com/apache/spark/pull/5473#discussion_r28301657
--- Diff:
streaming/src/main/scala/org/apache/spark/streaming/scheduler/JobScheduler.scala
---
@@ -170,8 +170,10 @@ class JobScheduler(val ssc:
Github user tdas commented on the pull request:
https://github.com/apache/spark/pull/5473#issuecomment-92618458
Why would we get wrong properties in this case? We are explicitly setting
it in the thread that is launching the Spark job, so there is no question of
inheriting local
Github user tdas commented on the pull request:
https://github.com/apache/spark/pull/5473#issuecomment-92606473
@zsxwing Any thoughts on this?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have
Github user zsxwing commented on the pull request:
https://github.com/apache/spark/pull/5473#issuecomment-92613622
There should be tests in the StreamingListenerSuite that tests the local
property setting.
As I explained in
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/5473#issuecomment-92611601
[Test build #30220 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/30220/consoleFull)
for PR 5473 at commit
Github user tdas commented on the pull request:
https://github.com/apache/spark/pull/5473#issuecomment-92288077
Its looking pretty good! Here are some preliminary comments. I will do a
more detailed pass tomorrow during the day.
1. There should be tests in the
Github user tdas commented on a diff in the pull request:
https://github.com/apache/spark/pull/5473#discussion_r28223951
--- Diff:
streaming/src/main/scala/org/apache/spark/streaming/scheduler/JobScheduler.scala
---
@@ -179,7 +181,14 @@ class JobScheduler(val ssc:
Github user tdas commented on a diff in the pull request:
https://github.com/apache/spark/pull/5473#discussion_r28224797
--- Diff:
streaming/src/main/scala/org/apache/spark/streaming/ui/BatchPage.scala ---
@@ -0,0 +1,224 @@
+/*
+ * Licensed to the Apache Software
Github user tdas commented on a diff in the pull request:
https://github.com/apache/spark/pull/5473#discussion_r28223843
--- Diff:
streaming/src/main/scala/org/apache/spark/streaming/scheduler/BatchInfo.scala
---
@@ -58,4 +58,11 @@ case class BatchInfo(
*/
def
Github user tdas commented on a diff in the pull request:
https://github.com/apache/spark/pull/5473#discussion_r28223913
--- Diff:
streaming/src/main/scala/org/apache/spark/streaming/scheduler/JobScheduler.scala
---
@@ -170,8 +170,10 @@ class JobScheduler(val ssc:
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/5473#issuecomment-91907788
[Test build #30084 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/30084/consoleFull)
for PR 5473 at commit
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/5473#issuecomment-91907795
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
GitHub user zsxwing opened a pull request:
https://github.com/apache/spark/pull/5473
[SPARK-6862][Streaming][WebUI][WIP] Add BatchPage to display details of a
batch
This is an initial commit for SPARK-6862. Once SPARK-6796 is merged, I will
add the links to StreamingPage so that
Github user zsxwing commented on the pull request:
https://github.com/apache/spark/pull/5473#issuecomment-91890461
cc @tdas
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
Github user zsxwing commented on the pull request:
https://github.com/apache/spark/pull/5473#issuecomment-91890386
My test codes:
```Scala
import org.apache.spark._
import org.apache.spark.streaming._
object StreamingApp {
def main(args: Array[String]):
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/5473#issuecomment-91890772
[Test build #30084 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/30084/consoleFull)
for PR 5473 at commit
27 matches
Mail list logo
