Github user JoshRosen commented on the pull request:
https://github.com/apache/spark/pull/1723#issuecomment-57575023
Hi @mubarak,
This issue has been fixed by #2464, so do you mind closing this? Thanks!
(Due to the way that this GitHub mirror is set up, we don't have
Github user mubarak commented on the pull request:
https://github.com/apache/spark/pull/1723#issuecomment-57581072
Fixed using https://github.com/apache/spark/pull/2464
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If
Github user mubarak closed the pull request at:
https://github.com/apache/spark/pull/1723
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is
Github user tdas commented on the pull request:
https://github.com/apache/spark/pull/2464#issuecomment-56628837
@mubarak Thank you very much for this fix! Its finally merged!
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as
Github user tdas commented on a diff in the pull request:
https://github.com/apache/spark/pull/2464#discussion_r17932762
--- Diff: core/src/main/scala/org/apache/spark/util/Utils.scala ---
@@ -859,18 +864,27 @@ private[spark] object Utils extends Logging {
}
}
Github user tdas commented on a diff in the pull request:
https://github.com/apache/spark/pull/2464#discussion_r17932801
--- Diff: core/src/main/scala/org/apache/spark/util/Utils.scala ---
@@ -859,18 +864,27 @@ private[spark] object Utils extends Logging {
}
}
Github user andrewor14 commented on a diff in the pull request:
https://github.com/apache/spark/pull/2464#discussion_r17933233
--- Diff: core/src/main/scala/org/apache/spark/util/Utils.scala ---
@@ -859,18 +864,27 @@ private[spark] object Utils extends Logging {
}
}
Github user andrewor14 commented on a diff in the pull request:
https://github.com/apache/spark/pull/2464#discussion_r17933382
--- Diff:
streaming/src/test/scala/org/apache/spark/streaming/StreamingContextSuite.scala
---
@@ -293,3 +300,37 @@ class TestReceiver extends
Github user andrewor14 commented on a diff in the pull request:
https://github.com/apache/spark/pull/2464#discussion_r17933492
--- Diff:
streaming/src/main/scala/org/apache/spark/streaming/dstream/DStream.scala ---
@@ -106,6 +107,9 @@ abstract class DStream[T: ClassTag] (
Github user tdas commented on the pull request:
https://github.com/apache/spark/pull/1723#issuecomment-56577498
@mubarak mind closing this?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have
Github user andrewor14 commented on a diff in the pull request:
https://github.com/apache/spark/pull/2464#discussion_r17933782
--- Diff: core/src/main/scala/org/apache/spark/util/Utils.scala ---
@@ -859,18 +864,27 @@ private[spark] object Utils extends Logging {
}
}
Github user tdas commented on a diff in the pull request:
https://github.com/apache/spark/pull/2464#discussion_r17934308
--- Diff:
streaming/src/test/scala/org/apache/spark/streaming/StreamingContextSuite.scala
---
@@ -293,3 +300,37 @@ class TestReceiver extends
Github user andrewor14 commented on a diff in the pull request:
https://github.com/apache/spark/pull/2464#discussion_r17934310
--- Diff:
streaming/src/main/scala/org/apache/spark/streaming/dstream/DStream.scala ---
@@ -272,43 +276,41 @@ abstract class DStream[T: ClassTag] (
Github user andrewor14 commented on a diff in the pull request:
https://github.com/apache/spark/pull/2464#discussion_r17934399
--- Diff:
streaming/src/main/scala/org/apache/spark/streaming/dstream/DStream.scala ---
@@ -799,3 +801,29 @@ abstract class DStream[T: ClassTag] (
Github user andrewor14 commented on a diff in the pull request:
https://github.com/apache/spark/pull/2464#discussion_r17934707
--- Diff:
streaming/src/main/scala/org/apache/spark/streaming/dstream/DStream.scala ---
@@ -272,43 +276,41 @@ abstract class DStream[T: ClassTag] (
Github user andrewor14 commented on a diff in the pull request:
https://github.com/apache/spark/pull/2464#discussion_r17934743
--- Diff:
streaming/src/main/scala/org/apache/spark/streaming/dstream/DStream.scala ---
@@ -272,43 +286,41 @@ abstract class DStream[T: ClassTag] (
Github user andrewor14 commented on a diff in the pull request:
https://github.com/apache/spark/pull/2464#discussion_r17934863
--- Diff:
streaming/src/test/scala/org/apache/spark/streaming/StreamingContextSuite.scala
---
@@ -293,3 +300,37 @@ class TestReceiver extends
Github user tdas commented on a diff in the pull request:
https://github.com/apache/spark/pull/2464#discussion_r17934946
--- Diff: core/src/main/scala/org/apache/spark/util/Utils.scala ---
@@ -859,18 +864,27 @@ private[spark] object Utils extends Logging {
}
}
Github user tdas commented on a diff in the pull request:
https://github.com/apache/spark/pull/2464#discussion_r17935081
--- Diff:
streaming/src/main/scala/org/apache/spark/streaming/dstream/DStream.scala ---
@@ -106,6 +107,9 @@ abstract class DStream[T: ClassTag] (
/**
Github user tdas commented on a diff in the pull request:
https://github.com/apache/spark/pull/2464#discussion_r17935069
--- Diff: core/src/main/scala/org/apache/spark/util/Utils.scala ---
@@ -859,18 +864,27 @@ private[spark] object Utils extends Logging {
}
}
Github user andrewor14 commented on a diff in the pull request:
https://github.com/apache/spark/pull/2464#discussion_r17935435
--- Diff: core/src/main/scala/org/apache/spark/util/Utils.scala ---
@@ -859,18 +864,27 @@ private[spark] object Utils extends Logging {
}
}
Github user andrewor14 commented on a diff in the pull request:
https://github.com/apache/spark/pull/2464#discussion_r17935509
--- Diff: core/src/main/scala/org/apache/spark/util/Utils.scala ---
@@ -859,18 +864,27 @@ private[spark] object Utils extends Logging {
}
}
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/2464#issuecomment-56585324
[QA tests have
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/20727/consoleFull)
for PR 2464 at commit
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/2464#issuecomment-56595105
[QA tests have
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/20727/consoleFull)
for PR 2464 at commit
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/2464#issuecomment-56595110
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/20727/
Github user tdas commented on a diff in the pull request:
https://github.com/apache/spark/pull/2464#discussion_r17942668
--- Diff:
streaming/src/main/scala/org/apache/spark/streaming/dstream/DStream.scala ---
@@ -272,43 +276,41 @@ abstract class DStream[T: ClassTag] (
}
Github user andrewor14 commented on the pull request:
https://github.com/apache/spark/pull/2464#issuecomment-56597921
Thanks, LGTM. I'm merging this into master.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your
Github user asfgit closed the pull request at:
https://github.com/apache/spark/pull/2464
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is
Github user pwendell commented on a diff in the pull request:
https://github.com/apache/spark/pull/2464#discussion_r17826891
--- Diff: core/src/main/scala/org/apache/spark/util/Utils.scala ---
@@ -859,18 +864,27 @@ private[spark] object Utils extends Logging {
}
}
Github user pwendell commented on a diff in the pull request:
https://github.com/apache/spark/pull/2464#discussion_r17826897
--- Diff: core/src/main/scala/org/apache/spark/util/Utils.scala ---
@@ -859,18 +864,27 @@ private[spark] object Utils extends Logging {
}
}
Github user tdas commented on the pull request:
https://github.com/apache/spark/pull/1723#issuecomment-56239335
@mubarak I have opened a new PR with my changes. Could you please take a
look at https://github.com/apache/spark/pull/2464
And can you close this PR?
---
If your
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/2464#issuecomment-56239789
[QA tests have
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/20591/consoleFull)
for PR 2464 at commit
Github user tdas commented on a diff in the pull request:
https://github.com/apache/spark/pull/2464#discussion_r17811250
--- Diff:
streaming/src/main/scala/org/apache/spark/streaming/dstream/DStream.scala ---
@@ -272,43 +286,41 @@ abstract class DStream[T: ClassTag] (
}
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/2464#issuecomment-56240718
[QA tests have
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/20593/consoleFull)
for PR 2464 at commit
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/2464#issuecomment-56245119
[QA tests have
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/20593/consoleFull)
for PR 2464 at commit
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/2464#issuecomment-56245278
[QA tests have
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/20591/consoleFull)
for PR 2464 at commit
Github user andrewor14 commented on the pull request:
https://github.com/apache/spark/pull/2464#issuecomment-56252261
Thanks for working on this @tdas while on vacation. I will look at this in
the next day or two.
---
If your project is set up for it, you can reply to this email and
Github user mubarak commented on the pull request:
https://github.com/apache/spark/pull/1723#issuecomment-54606563
@tdas
Can you please review? Thanks
![screen shot 2014-09-05 at 1 42 28
Github user mubarak commented on the pull request:
https://github.com/apache/spark/pull/1723#issuecomment-54606650
Jenkins, this is ok to test.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not
Github user tdas commented on the pull request:
https://github.com/apache/spark/pull/1723#issuecomment-54676763
Jenkins, test this please.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have
Github user tdas commented on a diff in the pull request:
https://github.com/apache/spark/pull/1723#discussion_r17196753
--- Diff: core/src/main/scala/org/apache/spark/util/Utils.scala ---
@@ -856,13 +859,27 @@ private[spark] object Utils extends Logging {
* finding the
Github user tdas commented on the pull request:
https://github.com/apache/spark/pull/1723#issuecomment-54682054
Jenkins is currently having issues so cannot test it. But this looks pretty
good except comment on the streaming regex - it should be in
streaming/util/Utils.scala not in
Github user andrewor14 commented on a diff in the pull request:
https://github.com/apache/spark/pull/1723#discussion_r17201437
--- Diff: core/src/main/scala/org/apache/spark/util/Utils.scala ---
@@ -856,13 +859,27 @@ private[spark] object Utils extends Logging {
* finding
Github user andrewor14 commented on a diff in the pull request:
https://github.com/apache/spark/pull/1723#discussion_r17201452
--- Diff: core/src/main/scala/org/apache/spark/util/Utils.scala ---
@@ -883,8 +900,8 @@ private[spark] object Utils extends Logging {
for
Github user andrewor14 commented on the pull request:
https://github.com/apache/spark/pull/1723#issuecomment-54688918
This sets a flag in `SparkContext` to reflect whether we want streaming
call sites vs normal call sites. Doesn't this mean if we use this same
`SparkContext` for
Github user tdas commented on the pull request:
https://github.com/apache/spark/pull/1723#issuecomment-54689188
No no, we want to preserve the behavior for Spark, and only change it for
streaming.
---
If your project is set up for it, you can reply to this email and have your
reply
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/1723#issuecomment-54694525
Can one of the admins verify this patch?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project
Github user andrewor14 commented on the pull request:
https://github.com/apache/spark/pull/1723#issuecomment-54697540
ok to test
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
Github user tdas commented on the pull request:
https://github.com/apache/spark/pull/1723#issuecomment-54698297
Jenkins, this is ok to test.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/1723#issuecomment-54698936
[QA tests have
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/19880/consoleFull)
for PR 1723 at commit
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/1723#issuecomment-54700379
[QA tests have
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/19880/consoleFull)
for PR 1723 at commit
Github user tdas commented on the pull request:
https://github.com/apache/spark/pull/1723#issuecomment-54700500
Hey @mubarak, @andrewor14 and I discussed this PR, and it would be cool use
this opportunity refactor a the callSite stuff a bit, and document this
callsite stuff (which is
Github user mubarak commented on the pull request:
https://github.com/apache/spark/pull/1723#issuecomment-54701472
@tdas
+1. Can you please review? Thanks
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your
Github user mubarak commented on the pull request:
https://github.com/apache/spark/pull/1723#issuecomment-54701499
Jenkins, this is ok to test.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not
Github user tdas commented on the pull request:
https://github.com/apache/spark/pull/1723#issuecomment-54701514
@mubarak I commented earlier about me making some small refactoring. Do you
mind? See earlier comment.
---
If your project is set up for it, you can reply to this email
Github user mubarak commented on the pull request:
https://github.com/apache/spark/pull/1723#issuecomment-54701520
@tdas I don't mind. Please.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have
Github user tdas commented on the pull request:
https://github.com/apache/spark/pull/1723#issuecomment-54701548
Thanks @mubarak I will create a new PR and ask you to take a look.
---
If your project is set up for it, you can reply to this email and have your
reply appear on
Github user mubarak commented on the pull request:
https://github.com/apache/spark/pull/1723#issuecomment-54547524
Will take a look and update you tonight. Thanks.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your
Github user tdas commented on the pull request:
https://github.com/apache/spark/pull/1723#issuecomment-54376381
Any updates?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
Github user tdas commented on the pull request:
https://github.com/apache/spark/pull/1723#issuecomment-53509499
Yeah, better to not modify Spark's default behavior. One solution for this
is to modify Utils.getCallSite to take a parameter - a filter function. There
will be the default
Github user mubarak commented on the pull request:
https://github.com/apache/spark/pull/1723#issuecomment-53229308
@tdas
I don't think the new (proposed) REGEX in `Utils.getCallSite` works for
test suite. For instance,
Github user mubarak commented on a diff in the pull request:
https://github.com/apache/spark/pull/1723#discussion_r16463252
--- Diff:
streaming/src/main/scala/org/apache/spark/streaming/dstream/DStream.scala ---
@@ -106,6 +106,27 @@ abstract class DStream[T: ClassTag] (
/**
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/1723#issuecomment-52746058
[QA tests have
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/18943/consoleFull)
for PR 1723 at commit
Github user mubarak commented on the pull request:
https://github.com/apache/spark/pull/1723#issuecomment-52746109
Jenkins, this is ok to test.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/1723#issuecomment-52750288
[QA tests have
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/18943/consoleFull)
for PR 1723 at commit
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/1723#issuecomment-52598483
[QA tests have
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/18821/consoleFull)
for PR 1723 at commit
Github user mubarak commented on the pull request:
https://github.com/apache/spark/pull/1723#issuecomment-52598991
*Streaming example:*
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/1723#issuecomment-52602246
[QA tests have
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/18821/consoleFull)
for PR 1723 at commit
Github user mubarak commented on the pull request:
https://github.com/apache/spark/pull/1723#issuecomment-52605797
```
[info] - call sites report correct locations *** FAILED ***
[info] [apply] did not equal [makeRDD]
(SparkContextInfoSuite.scala:85)
```
CallSite
Github user tdas commented on the pull request:
https://github.com/apache/spark/pull/1723#issuecomment-52615460
Oh yes, that failure makes sense. Since we are capturing all Spark classes,
it is not able to identify the callsite inside SparkContextInfoSuite. This is
because the
Github user tdas commented on a diff in the pull request:
https://github.com/apache/spark/pull/1723#discussion_r16407131
--- Diff:
streaming/src/main/scala/org/apache/spark/streaming/dstream/DStream.scala ---
@@ -106,6 +106,27 @@ abstract class DStream[T: ClassTag] (
/**
Github user tdas commented on a diff in the pull request:
https://github.com/apache/spark/pull/1723#discussion_r16407308
--- Diff:
streaming/src/main/scala/org/apache/spark/streaming/dstream/DStream.scala ---
@@ -106,6 +106,27 @@ abstract class DStream[T: ClassTag] (
/**
Github user mubarak commented on a diff in the pull request:
https://github.com/apache/spark/pull/1723#discussion_r16339260
--- Diff: core/src/main/scala/org/apache/spark/util/Utils.scala ---
@@ -813,7 +816,10 @@ private[spark] object Utils extends Logging {
* A regular
Github user tdas commented on the pull request:
https://github.com/apache/spark/pull/1723#issuecomment-52443825
Jenkins, this is ok to test.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have
Github user tdas commented on a diff in the pull request:
https://github.com/apache/spark/pull/1723#discussion_r16334743
--- Diff: core/src/main/scala/org/apache/spark/util/Utils.scala ---
@@ -813,7 +816,10 @@ private[spark] object Utils extends Logging {
* A regular
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/1723#issuecomment-52443994
[QA tests have
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/18724/consoleFull)
for PR 1723 at commit
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/1723#issuecomment-52444020
[QA tests have
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/18724/consoleFull)
for PR 1723 at commit
Github user tdas commented on a diff in the pull request:
https://github.com/apache/spark/pull/1723#discussion_r16334830
--- Diff:
streaming/src/main/scala/org/apache/spark/streaming/StreamingContext.scala ---
@@ -38,7 +38,7 @@ import org.apache.spark.streaming.dstream._
Github user tdas commented on a diff in the pull request:
https://github.com/apache/spark/pull/1723#discussion_r16334866
--- Diff:
streaming/src/main/scala/org/apache/spark/streaming/dstream/DStream.scala ---
@@ -106,6 +106,27 @@ abstract class DStream[T: ClassTag] (
/**
Github user tdas commented on a diff in the pull request:
https://github.com/apache/spark/pull/1723#discussion_r16334878
--- Diff:
streaming/src/main/scala/org/apache/spark/streaming/dstream/ReducedWindowedDStream.scala
---
@@ -52,6 +52,8 @@ class ReducedWindowedDStream[K:
Github user tdas commented on the pull request:
https://github.com/apache/spark/pull/1723#issuecomment-52444695
This looks pretty promising!! However why do every DStream need to change?
The DStream parent class can do the following. Before calling compute(), it can
store the
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/1723#issuecomment-52448650
[QA tests have
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/18731/consoleFull)
for PR 1723 at commit
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/1723#issuecomment-52448704
[QA tests have
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/18731/consoleFull)
for PR 1723 at commit
Github user tdas commented on the pull request:
https://github.com/apache/spark/pull/1723#issuecomment-52449035
@mubarak This is what I had in mind. I have a few more comments in the
code. Can you please comment/address those? Also, can you post a new screenshot?
---
If your project
Github user tdas commented on a diff in the pull request:
https://github.com/apache/spark/pull/1723#discussion_r16336535
--- Diff:
streaming/src/main/scala/org/apache/spark/streaming/dstream/DStream.scala ---
@@ -302,8 +325,10 @@ abstract class DStream[T: ClassTag] (
Github user mubarak commented on the pull request:
https://github.com/apache/spark/pull/1723#issuecomment-52450485
New screenshot
---
If
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/1723#issuecomment-52450636
[QA tests have
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/18733/consoleFull)
for PR 1723 at commit
Github user mubarak commented on a diff in the pull request:
https://github.com/apache/spark/pull/1723#discussion_r16336958
--- Diff: core/src/main/scala/org/apache/spark/util/Utils.scala ---
@@ -813,7 +816,10 @@ private[spark] object Utils extends Logging {
* A regular
Github user tdas commented on the pull request:
https://github.com/apache/spark/pull/1723#issuecomment-52452666
Screenshot is great! The tests dont pass because of style check errors.
Please address them.
error
Github user tdas commented on a diff in the pull request:
https://github.com/apache/spark/pull/1723#discussion_r16337534
--- Diff:
streaming/src/main/scala/org/apache/spark/streaming/dstream/DStream.scala ---
@@ -106,6 +106,27 @@ abstract class DStream[T: ClassTag] (
/**
Github user tdas commented on a diff in the pull request:
https://github.com/apache/spark/pull/1723#discussion_r16337606
--- Diff: core/src/main/scala/org/apache/spark/util/Utils.scala ---
@@ -813,7 +816,10 @@ private[spark] object Utils extends Logging {
* A regular
Github user tdas commented on a diff in the pull request:
https://github.com/apache/spark/pull/1723#discussion_r16337761
--- Diff: core/src/main/scala/org/apache/spark/util/Utils.scala ---
@@ -813,7 +816,10 @@ private[spark] object Utils extends Logging {
* A regular
Github user mubarak commented on the pull request:
https://github.com/apache/spark/pull/1723#issuecomment-51712229
@tdas
I have removed 'name' from DStream and addressed your review comments. Can
you please review? Thanks.
---
If your project is set up for it, you can reply to
Github user tdas commented on a diff in the pull request:
https://github.com/apache/spark/pull/1723#discussion_r15727686
--- Diff:
streaming/src/main/scala/org/apache/spark/streaming/StreamingContext.scala ---
@@ -112,6 +112,7 @@ class StreamingContext private[streaming] (
GitHub user mubarak opened a pull request:
https://github.com/apache/spark/pull/1723
[SPARK-1853] Show Streaming application code context (file, line number) in
Spark Stages UI
Please review: [SPARK-1853] Show Streaming application code context (file,
line number) in Spark Stages
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/1723#issuecomment-50943156
Can one of the admins verify this patch?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your
Github user tdas commented on a diff in the pull request:
https://github.com/apache/spark/pull/1723#discussion_r15726022
--- Diff:
streaming/src/main/scala/org/apache/spark/streaming/StreamingContext.scala ---
@@ -112,6 +112,7 @@ class StreamingContext private[streaming] (
Github user tdas commented on a diff in the pull request:
https://github.com/apache/spark/pull/1723#discussion_r15726777
--- Diff:
streaming/src/main/scala/org/apache/spark/streaming/dstream/DStream.scala ---
@@ -106,6 +106,24 @@ abstract class DStream[T: ClassTag] (
/**
Github user tdas commented on a diff in the pull request:
https://github.com/apache/spark/pull/1723#discussion_r15726784
--- Diff:
streaming/src/main/scala/org/apache/spark/streaming/dstream/FilteredDStream.scala
---
@@ -27,12 +27,17 @@ class FilteredDStream[T: ClassTag](
Github user tdas commented on the pull request:
https://github.com/apache/spark/pull/1723#issuecomment-50952562
This is a great start @mubarak thanks for navigating all the control flow
graph and putting this together. Though I feel there is a better way of doing
this that does not
1 - 100 of 101 matches
Mail list logo
