[GitHub] spark pull request: [SPARK-5154] [PySpark] [Streaming] Kafka strea...

2015-02-02 Thread mateiz
Github user mateiz commented on a diff in the pull request: https://github.com/apache/spark/pull/3715#discussion_r23966188 --- Diff: core/src/main/scala/org/apache/spark/api/python/PythonRDD.scala --- @@ -313,6 +313,7 @@ private object SpecialLengths { val

[GitHub] spark pull request: [SPARK-5154] [PySpark] [Streaming] Kafka strea...

2015-02-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/3715#issuecomment-72586628 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-5154] [PySpark] [Streaming] Kafka strea...

2015-02-02 Thread pwendell
Github user pwendell commented on a diff in the pull request: https://github.com/apache/spark/pull/3715#discussion_r23977407 --- Diff: make-distribution.sh --- @@ -188,6 +188,7 @@ echo Build flags: $@ $DISTDIR/RELEASE # Copy jars cp

[GitHub] spark pull request: [SPARK-5154] [PySpark] [Streaming] Kafka strea...

2015-02-02 Thread pwendell
Github user pwendell commented on a diff in the pull request: https://github.com/apache/spark/pull/3715#discussion_r23977374 --- Diff: external/kafka-assembly/pom.xml --- @@ -0,0 +1,106 @@ +?xml version=1.0 encoding=UTF-8? +!-- + ~ Licensed to the Apache Software

[GitHub] spark pull request: [SPARK-5154] [PySpark] [Streaming] Kafka strea...

2015-02-02 Thread davies
Github user davies commented on the pull request: https://github.com/apache/spark/pull/3715#issuecomment-72579626 @tdas @pwendell done --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request: [SPARK-5154] [PySpark] [Streaming] Kafka strea...

2015-02-02 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/3715#discussion_r23978974 --- Diff: python/pyspark/streaming/kafka.py --- @@ -0,0 +1,83 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one or more +#

[GitHub] spark pull request: [SPARK-5154] [PySpark] [Streaming] Kafka strea...

2015-02-02 Thread tdas
Github user tdas commented on the pull request: https://github.com/apache/spark/pull/3715#issuecomment-72585386 I am merging despite a small comment from me. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-5154] [PySpark] [Streaming] Kafka strea...

2015-02-02 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/3715#discussion_r23977669 --- Diff: make-distribution.sh --- @@ -188,6 +188,7 @@ echo Build flags: $@ $DISTDIR/RELEASE # Copy jars cp

[GitHub] spark pull request: [SPARK-5154] [PySpark] [Streaming] Kafka strea...

2015-02-02 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3715#issuecomment-72579598 [Test build #26578 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/26578/consoleFull) for PR 3715 at commit

[GitHub] spark pull request: [SPARK-5154] [PySpark] [Streaming] Kafka strea...

2015-02-02 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/3715 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request: [SPARK-5154] [PySpark] [Streaming] Kafka strea...

2015-02-02 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3715#issuecomment-72586619 [Test build #26578 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/26578/consoleFull) for PR 3715 at commit

[GitHub] spark pull request: [SPARK-5154] [PySpark] [Streaming] Kafka strea...

2015-02-02 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/3715#discussion_r23950665 --- Diff: core/src/main/scala/org/apache/spark/api/python/PythonRDD.scala --- @@ -313,6 +313,7 @@ private object SpecialLengths { val

[GitHub] spark pull request: [SPARK-5154] [PySpark] [Streaming] Kafka strea...

2015-02-01 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/3715#discussion_r23908186 --- Diff: core/src/test/scala/org/apache/spark/api/python/PythonRDDSuite.scala --- @@ -23,11 +23,21 @@ import org.scalatest.FunSuite class

[GitHub] spark pull request: [SPARK-5154] [PySpark] [Streaming] Kafka strea...

2015-02-01 Thread davies
Github user davies commented on a diff in the pull request: https://github.com/apache/spark/pull/3715#discussion_r23909226 --- Diff: core/src/test/scala/org/apache/spark/api/python/PythonRDDSuite.scala --- @@ -23,11 +23,21 @@ import org.scalatest.FunSuite class

[GitHub] spark pull request: [SPARK-5154] [PySpark] [Streaming] Kafka strea...

2015-01-29 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/3715#discussion_r23791005 --- Diff: python/pyspark/streaming/kafka.py --- @@ -0,0 +1,82 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one or more +#

[GitHub] spark pull request: [SPARK-5154] [PySpark] [Streaming] Kafka strea...

2015-01-29 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/3715#discussion_r23788839 --- Diff: core/src/main/scala/org/apache/spark/api/python/PythonRDD.scala --- @@ -313,6 +313,7 @@ private object SpecialLengths { val

[GitHub] spark pull request: [SPARK-5154] [PySpark] [Streaming] Kafka strea...

2015-01-29 Thread davies
Github user davies commented on a diff in the pull request: https://github.com/apache/spark/pull/3715#discussion_r23789864 --- Diff: core/src/main/scala/org/apache/spark/api/python/PythonRDD.scala --- @@ -313,6 +313,7 @@ private object SpecialLengths { val

[GitHub] spark pull request: [SPARK-5154] [PySpark] [Streaming] Kafka strea...

2015-01-29 Thread tdas
Github user tdas commented on the pull request: https://github.com/apache/spark/pull/3715#issuecomment-72074737 Jenkins, test this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-5154] [PySpark] [Streaming] Kafka strea...

2015-01-29 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3715#issuecomment-72075443 [Test build #26324 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/26324/consoleFull) for PR 3715 at commit

[GitHub] spark pull request: [SPARK-5154] [PySpark] [Streaming] Kafka strea...

2015-01-29 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/3715#discussion_r23789381 --- Diff: core/src/main/scala/org/apache/spark/api/python/PythonRDD.scala --- @@ -313,6 +313,7 @@ private object SpecialLengths { val

[GitHub] spark pull request: [SPARK-5154] [PySpark] [Streaming] Kafka strea...

2015-01-29 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/3715#discussion_r23790293 --- Diff: python/pyspark/streaming/kafka.py --- @@ -0,0 +1,82 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one or more +#

[GitHub] spark pull request: [SPARK-5154] [PySpark] [Streaming] Kafka strea...

2015-01-29 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/3715#discussion_r23791037 --- Diff: python/pyspark/streaming/kafka.py --- @@ -0,0 +1,82 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one or more +#

[GitHub] spark pull request: [SPARK-5154] [PySpark] [Streaming] Kafka strea...

2015-01-29 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/3715#discussion_r23792016 --- Diff: core/src/main/scala/org/apache/spark/api/python/PythonRDD.scala --- @@ -313,6 +313,7 @@ private object SpecialLengths { val

[GitHub] spark pull request: [SPARK-5154] [PySpark] [Streaming] Kafka strea...

2015-01-29 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/3715#discussion_r23792793 --- Diff: python/pyspark/streaming/kafka.py --- @@ -0,0 +1,82 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one or more +#

[GitHub] spark pull request: [SPARK-5154] [PySpark] [Streaming] Kafka strea...

2015-01-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/3715#issuecomment-72087160 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-5154] [PySpark] [Streaming] Kafka strea...

2015-01-29 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3715#issuecomment-72087147 [Test build #26324 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/26324/consoleFull) for PR 3715 at commit

[GitHub] spark pull request: [SPARK-5154] [PySpark] [Streaming] Kafka strea...

2015-01-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/3715#issuecomment-72110650 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-5154] [PySpark] [Streaming] Kafka strea...

2015-01-29 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3715#issuecomment-72110638 [Test build #26331 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/26331/consoleFull) for PR 3715 at commit

[GitHub] spark pull request: [SPARK-5154] [PySpark] [Streaming] Kafka strea...

2015-01-29 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3715#issuecomment-72119539 [Test build #26339 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/26339/consoleFull) for PR 3715 at commit

[GitHub] spark pull request: [SPARK-5154] [PySpark] [Streaming] Kafka strea...

2015-01-29 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/3715#discussion_r23817720 --- Diff: core/src/main/scala/org/apache/spark/api/python/PythonRDD.scala --- @@ -371,54 +372,28 @@ private[spark] object PythonRDD extends Logging { }

[GitHub] spark pull request: [SPARK-5154] [PySpark] [Streaming] Kafka strea...

2015-01-29 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/3715#discussion_r23817104 --- Diff: core/src/main/scala/org/apache/spark/api/python/PythonRDD.scala --- @@ -371,54 +372,28 @@ private[spark] object PythonRDD extends Logging { }

[GitHub] spark pull request: [SPARK-5154] [PySpark] [Streaming] Kafka strea...

2015-01-29 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3715#issuecomment-72128069 [Test build #26339 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/26339/consoleFull) for PR 3715 at commit

[GitHub] spark pull request: [SPARK-5154] [PySpark] [Streaming] Kafka strea...

2015-01-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/3715#issuecomment-72128081 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-5154] [PySpark] [Streaming] Kafka strea...

2015-01-29 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3715#issuecomment-72138481 [Test build #26361 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/26361/consoleFull) for PR 3715 at commit

[GitHub] spark pull request: [SPARK-5154] [PySpark] [Streaming] Kafka strea...

2015-01-29 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/3715#discussion_r23817285 --- Diff: core/src/test/scala/org/apache/spark/api/python/PythonRDDSuite.scala --- @@ -23,11 +23,21 @@ import org.scalatest.FunSuite class

[GitHub] spark pull request: [SPARK-5154] [PySpark] [Streaming] Kafka strea...

2015-01-29 Thread davies
Github user davies commented on a diff in the pull request: https://github.com/apache/spark/pull/3715#discussion_r23817573 --- Diff: core/src/main/scala/org/apache/spark/api/python/PythonRDD.scala --- @@ -371,54 +372,28 @@ private[spark] object PythonRDD extends Logging { }

[GitHub] spark pull request: [SPARK-5154] [PySpark] [Streaming] Kafka strea...

2015-01-29 Thread mateiz
Github user mateiz commented on a diff in the pull request: https://github.com/apache/spark/pull/3715#discussion_r23814616 --- Diff: core/src/main/scala/org/apache/spark/api/python/PythonRDD.scala --- @@ -313,6 +313,7 @@ private object SpecialLengths { val

[GitHub] spark pull request: [SPARK-5154] [PySpark] [Streaming] Kafka strea...

2015-01-29 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/3715#discussion_r23816755 --- Diff: core/src/main/scala/org/apache/spark/api/python/PythonRDD.scala --- @@ -371,54 +372,28 @@ private[spark] object PythonRDD extends Logging { }

[GitHub] spark pull request: [SPARK-5154] [PySpark] [Streaming] Kafka strea...

2015-01-29 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/3715#discussion_r23816764 --- Diff: core/src/main/scala/org/apache/spark/api/python/PythonRDD.scala --- @@ -371,54 +372,28 @@ private[spark] object PythonRDD extends Logging { }

[GitHub] spark pull request: [SPARK-5154] [PySpark] [Streaming] Kafka strea...

2015-01-29 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/3715#discussion_r23816747 --- Diff: core/src/main/scala/org/apache/spark/api/python/PythonRDD.scala --- @@ -371,54 +372,28 @@ private[spark] object PythonRDD extends Logging { }

[GitHub] spark pull request: [SPARK-5154] [PySpark] [Streaming] Kafka strea...

2015-01-29 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/3715#discussion_r23817241 --- Diff: core/src/test/scala/org/apache/spark/api/python/PythonRDDSuite.scala --- @@ -23,11 +23,21 @@ import org.scalatest.FunSuite class

[GitHub] spark pull request: [SPARK-5154] [PySpark] [Streaming] Kafka strea...

2015-01-29 Thread davies
Github user davies commented on a diff in the pull request: https://github.com/apache/spark/pull/3715#discussion_r23815013 --- Diff: core/src/main/scala/org/apache/spark/api/python/PythonRDD.scala --- @@ -313,6 +313,7 @@ private object SpecialLengths { val

[GitHub] spark pull request: [SPARK-5154] [PySpark] [Streaming] Kafka strea...

2015-01-29 Thread davies
Github user davies commented on a diff in the pull request: https://github.com/apache/spark/pull/3715#discussion_r23817527 --- Diff: core/src/main/scala/org/apache/spark/api/python/PythonRDD.scala --- @@ -371,54 +372,28 @@ private[spark] object PythonRDD extends Logging { }

[GitHub] spark pull request: [SPARK-5154] [PySpark] [Streaming] Kafka strea...

2015-01-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/3715#issuecomment-72145159 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-5154] [PySpark] [Streaming] Kafka strea...

2015-01-29 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3715#issuecomment-72145154 [Test build #26361 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/26361/consoleFull) for PR 3715 at commit

[GitHub] spark pull request: [SPARK-5154] [PySpark] [Streaming] Kafka strea...

2015-01-29 Thread davies
Github user davies commented on a diff in the pull request: https://github.com/apache/spark/pull/3715#discussion_r23817918 --- Diff: core/src/main/scala/org/apache/spark/api/python/PythonRDD.scala --- @@ -371,54 +372,28 @@ private[spark] object PythonRDD extends Logging { }

[GitHub] spark pull request: [SPARK-5154] [PySpark] [Streaming] Kafka strea...

2015-01-28 Thread pwendell
Github user pwendell commented on a diff in the pull request: https://github.com/apache/spark/pull/3715#discussion_r23672024 --- Diff: make-distribution.sh --- @@ -188,6 +188,7 @@ echo Build flags: $@ $DISTDIR/RELEASE # Copy jars cp

[GitHub] spark pull request: [SPARK-5154] [PySpark] [Streaming] Kafka strea...

2015-01-28 Thread JoshRosen
Github user JoshRosen commented on the pull request: https://github.com/apache/spark/pull/3715#issuecomment-71922614 Jenkins, retest this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-5154] [PySpark] [Streaming] Kafka strea...

2015-01-28 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3715#issuecomment-71941597 [Test build #569 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/569/consoleFull) for PR 3715 at commit

[GitHub] spark pull request: [SPARK-5154] [PySpark] [Streaming] Kafka strea...

2015-01-28 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3715#issuecomment-71947480 [Test build #569 has finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/569/consoleFull) for PR 3715 at commit

[GitHub] spark pull request: [SPARK-5154] [PySpark] [Streaming] Kafka strea...

2015-01-27 Thread prabeesh
Github user prabeesh commented on a diff in the pull request: https://github.com/apache/spark/pull/3715#discussion_r23636136 --- Diff: examples/src/main/python/streaming/kafka_wordcount.py --- @@ -0,0 +1,57 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one

[GitHub] spark pull request: [SPARK-5154] [PySpark] [Streaming] Kafka strea...

2015-01-27 Thread prabeesh
Github user prabeesh commented on a diff in the pull request: https://github.com/apache/spark/pull/3715#discussion_r23637548 --- Diff: examples/src/main/python/streaming/kafka_wordcount.py --- @@ -0,0 +1,57 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one

[GitHub] spark pull request: [SPARK-5154] [PySpark] [Streaming] Kafka strea...

2015-01-27 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3715#issuecomment-71715606 [Test build #26180 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/26180/consoleFull) for PR 3715 at commit

[GitHub] spark pull request: [SPARK-5154] [PySpark] [Streaming] Kafka strea...

2015-01-27 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3715#issuecomment-71710113 [Test build #26177 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/26177/consoleFull) for PR 3715 at commit

[GitHub] spark pull request: [SPARK-5154] [PySpark] [Streaming] Kafka strea...

2015-01-27 Thread davies
Github user davies commented on a diff in the pull request: https://github.com/apache/spark/pull/3715#discussion_r23636560 --- Diff: examples/src/main/python/streaming/kafka_wordcount.py --- @@ -0,0 +1,57 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one

[GitHub] spark pull request: [SPARK-5154] [PySpark] [Streaming] Kafka strea...

2015-01-27 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3715#issuecomment-71715803 [Test build #26180 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/26180/consoleFull) for PR 3715 at commit

[GitHub] spark pull request: [SPARK-5154] [PySpark] [Streaming] Kafka strea...

2015-01-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/3715#issuecomment-71715805 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-5154] [PySpark] [Streaming] Kafka strea...

2015-01-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/3715#issuecomment-71722057 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-5154] [PySpark] [Streaming] Kafka strea...

2015-01-27 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3715#issuecomment-71722042 [Test build #26177 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/26177/consoleFull) for PR 3715 at commit

[GitHub] spark pull request: [SPARK-5154] [PySpark] [Streaming] Kafka strea...

2015-01-27 Thread JoshRosen
Github user JoshRosen commented on a diff in the pull request: https://github.com/apache/spark/pull/3715#discussion_r23660457 --- Diff: core/src/main/scala/org/apache/spark/api/python/PythonRDD.scala --- @@ -374,49 +375,63 @@ private[spark] object PythonRDD extends Logging {

[GitHub] spark pull request: [SPARK-5154] [PySpark] [Streaming] Kafka strea...

2015-01-27 Thread JoshRosen
Github user JoshRosen commented on a diff in the pull request: https://github.com/apache/spark/pull/3715#discussion_r23661073 --- Diff: core/src/main/scala/org/apache/spark/api/python/PythonRDD.scala --- @@ -374,49 +375,63 @@ private[spark] object PythonRDD extends Logging {

[GitHub] spark pull request: [SPARK-5154] [PySpark] [Streaming] Kafka strea...

2015-01-27 Thread JoshRosen
Github user JoshRosen commented on a diff in the pull request: https://github.com/apache/spark/pull/3715#discussion_r23661069 --- Diff: core/src/main/scala/org/apache/spark/api/python/PythonRDD.scala --- @@ -374,49 +375,63 @@ private[spark] object PythonRDD extends Logging {

[GitHub] spark pull request: [SPARK-5154] [PySpark] [Streaming] Kafka strea...

2015-01-27 Thread JoshRosen
Github user JoshRosen commented on the pull request: https://github.com/apache/spark/pull/3715#issuecomment-71769379 Should we have tests for this? Do we have tests for the other Python streaming sources? --- If your project is set up for it, you can reply to this email and have

[GitHub] spark pull request: [SPARK-5154] [PySpark] [Streaming] Kafka strea...

2015-01-27 Thread JoshRosen
Github user JoshRosen commented on a diff in the pull request: https://github.com/apache/spark/pull/3715#discussion_r23660342 --- Diff: core/src/main/scala/org/apache/spark/api/python/PythonRDD.scala --- @@ -374,49 +375,63 @@ private[spark] object PythonRDD extends Logging {

[GitHub] spark pull request: [SPARK-5154] [PySpark] [Streaming] Kafka strea...

2015-01-27 Thread JoshRosen
Github user JoshRosen commented on a diff in the pull request: https://github.com/apache/spark/pull/3715#discussion_r23661140 --- Diff: python/pyspark/serializers.py --- @@ -484,6 +487,8 @@ def loads(self, stream): length = read_int(stream) if length ==

[GitHub] spark pull request: [SPARK-5154] [PySpark] [Streaming] Kafka strea...

2015-01-27 Thread tdas
Github user tdas commented on the pull request: https://github.com/apache/spark/pull/3715#issuecomment-71766450 @JoshRosen Can you take brief look at the python stuff? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If

[GitHub] spark pull request: [SPARK-5154] [PySpark] [Streaming] Kafka strea...

2015-01-27 Thread pwendell
Github user pwendell commented on a diff in the pull request: https://github.com/apache/spark/pull/3715#discussion_r23661317 --- Diff: make-distribution.sh --- @@ -188,6 +188,7 @@ echo Build flags: $@ $DISTDIR/RELEASE # Copy jars cp

[GitHub] spark pull request: [SPARK-5154] [PySpark] [Streaming] Kafka strea...

2015-01-27 Thread pwendell
Github user pwendell commented on a diff in the pull request: https://github.com/apache/spark/pull/3715#discussion_r23661298 --- Diff: make-distribution.sh --- @@ -188,6 +188,7 @@ echo Build flags: $@ $DISTDIR/RELEASE # Copy jars cp

[GitHub] spark pull request: [SPARK-5154] [PySpark] [Streaming] Kafka strea...

2015-01-27 Thread davies
Github user davies commented on a diff in the pull request: https://github.com/apache/spark/pull/3715#discussion_r23668764 --- Diff: make-distribution.sh --- @@ -188,6 +188,7 @@ echo Build flags: $@ $DISTDIR/RELEASE # Copy jars cp

[GitHub] spark pull request: [SPARK-5154] [PySpark] [Streaming] Kafka strea...

2015-01-27 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3715#issuecomment-71789728 [Test build #26209 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/26209/consoleFull) for PR 3715 at commit

[GitHub] spark pull request: [SPARK-5154] [PySpark] [Streaming] Kafka strea...

2015-01-27 Thread davies
Github user davies commented on a diff in the pull request: https://github.com/apache/spark/pull/3715#discussion_r23668817 --- Diff: core/src/main/scala/org/apache/spark/api/python/PythonRDD.scala --- @@ -374,49 +375,63 @@ private[spark] object PythonRDD extends Logging {

[GitHub] spark pull request: [SPARK-5154] [PySpark] [Streaming] Kafka strea...

2015-01-27 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3715#issuecomment-71793802 [Test build #26209 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/26209/consoleFull) for PR 3715 at commit

[GitHub] spark pull request: [SPARK-5154] [PySpark] [Streaming] Kafka strea...

2015-01-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/3715#issuecomment-71793808 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-5154] [PySpark] [Streaming] Kafka strea...

2015-01-27 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/3715#discussion_r23662499 --- Diff: core/src/main/scala/org/apache/spark/api/python/PythonRDD.scala --- @@ -313,6 +313,7 @@ private object SpecialLengths { val

[GitHub] spark pull request: [SPARK-5154] [PySpark] [Streaming] Kafka strea...

2015-01-27 Thread JoshRosen
Github user JoshRosen commented on a diff in the pull request: https://github.com/apache/spark/pull/3715#discussion_r23663046 --- Diff: core/src/main/scala/org/apache/spark/api/python/PythonRDD.scala --- @@ -313,6 +313,7 @@ private object SpecialLengths { val

[GitHub] spark pull request: [SPARK-5154] [PySpark] [Streaming] Kafka strea...

2015-01-27 Thread JoshRosen
Github user JoshRosen commented on a diff in the pull request: https://github.com/apache/spark/pull/3715#discussion_r23661437 --- Diff: python/pyspark/streaming/kafka.py --- @@ -0,0 +1,82 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one or more +#

[GitHub] spark pull request: [SPARK-5154] [PySpark] [Streaming] Kafka strea...

2015-01-27 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/3715#discussion_r23662457 --- Diff: core/src/main/scala/org/apache/spark/api/python/PythonRDD.scala --- @@ -374,49 +375,63 @@ private[spark] object PythonRDD extends Logging {

[GitHub] spark pull request: [SPARK-5154] [PySpark] [Streaming] Kafka strea...

2015-01-27 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/3715#discussion_r23662537 --- Diff: make-distribution.sh --- @@ -188,6 +188,7 @@ echo Build flags: $@ $DISTDIR/RELEASE # Copy jars cp

[GitHub] spark pull request: [SPARK-5154] [PySpark] [Streaming] Kafka strea...

2015-01-26 Thread kennethmyers
Github user kennethmyers commented on the pull request: https://github.com/apache/spark/pull/3715#issuecomment-71515443 Hello, I made a fork of the main Spark repo and pulled this pull request into it. I was able to build it successfully but I get the followng error when

[GitHub] spark pull request: [SPARK-5154] [PySpark] [Streaming] Kafka strea...

2015-01-26 Thread davies
Github user davies commented on the pull request: https://github.com/apache/spark/pull/3715#issuecomment-71531465 @kennethmyers fixed, thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-5154] [PySpark] [Streaming] Kafka strea...

2015-01-26 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3715#issuecomment-71531745 [Test build #26106 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/26106/consoleFull) for PR 3715 at commit

[GitHub] spark pull request: [SPARK-5154] [PySpark] [Streaming] Kafka strea...

2015-01-26 Thread kennethmyers
Github user kennethmyers commented on the pull request: https://github.com/apache/spark/pull/3715#issuecomment-71524679 Also, I believe the warning message on line 77 of ```python/pyspark/streaming/kafka.py``` should read ```$ bin/spark-submit``` rather than

[GitHub] spark pull request: [SPARK-5154] [PySpark] [Streaming] Kafka strea...

2015-01-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/3715#issuecomment-71542788 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-5154] [PySpark] [Streaming] Kafka strea...

2015-01-26 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3715#issuecomment-71542772 [Test build #26106 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/26106/consoleFull) for PR 3715 at commit

[GitHub] spark pull request: [SPARK-5154] [PySpark] [Streaming] Kafka strea...

2015-01-26 Thread kennethmyers
Github user kennethmyers commented on the pull request: https://github.com/apache/spark/pull/3715#issuecomment-71533859 @davies Thanks! That worked! I'm new to Spark and misunderstood the function of the spark-submit command. --- If your project is set up for it, you can reply to

[GitHub] spark pull request: [SPARK-5154] [PySpark] [Streaming] Kafka strea...

2015-01-26 Thread davies
Github user davies commented on the pull request: https://github.com/apache/spark/pull/3715#issuecomment-71531793 @kennethmyers I think you should put the path of your scripts after spark-submit options, such as: ``` bin/spark-submit --driver-class-path x

[GitHub] spark pull request: [SPARK-5154] [PySpark] [Streaming] Kafka strea...

2015-01-22 Thread davies
Github user davies commented on the pull request: https://github.com/apache/spark/pull/3715#issuecomment-70984720 @tdas I think this PR is almost ready, please follow the example to double check it, thanks! --- If your project is set up for it, you can reply to this email and have

[GitHub] spark pull request: [SPARK-5154] [PySpark] [Streaming] Kafka strea...

2015-01-22 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3715#issuecomment-70985022 [Test build #25958 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/25958/consoleFull) for PR 3715 at commit

[GitHub] spark pull request: [SPARK-5154] [PySpark] [Streaming] Kafka strea...

2015-01-22 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3715#issuecomment-70984126 [Test build #25956 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/25956/consoleFull) for PR 3715 at commit

[GitHub] spark pull request: [SPARK-5154] [PySpark] [Streaming] Kafka strea...

2015-01-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/3715#issuecomment-70988894 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-5154] [PySpark] [Streaming] Kafka strea...

2015-01-22 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3715#issuecomment-7099 [Test build #25951 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/25951/consoleFull) for PR 3715 at commit

[GitHub] spark pull request: [SPARK-5154] [PySpark] [Streaming] Kafka strea...

2015-01-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/3715#issuecomment-70992015 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-5154] [PySpark] [Streaming] Kafka strea...

2015-01-22 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3715#issuecomment-70992003 [Test build #25958 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/25958/consoleFull) for PR 3715 at commit

[GitHub] spark pull request: [SPARK-5154] [PySpark] [Streaming] Kafka strea...

2015-01-22 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3715#issuecomment-70990806 [Test build #25956 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/25956/consoleFull) for PR 3715 at commit

[GitHub] spark pull request: [SPARK-5154] [PySpark] [Streaming] Kafka strea...

2015-01-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/3715#issuecomment-70990814 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-5154] [PySpark] [Streaming] Kafka strea...

2015-01-21 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3715#issuecomment-70980275 [Test build #25951 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/25951/consoleFull) for PR 3715 at commit

[GitHub] spark pull request: [SPARK-5154] [PySpark] [Streaming] Kafka strea...

2015-01-08 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/3715#discussion_r22693622 --- Diff: core/src/main/scala/org/apache/spark/api/python/PythonRDD.scala --- @@ -374,49 +375,61 @@ private[spark] object PythonRDD extends Logging {

[GitHub] spark pull request: [SPARK-5154] [PySpark] [Streaming] Kafka strea...

2015-01-08 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3715#issuecomment-69261048 [Test build #25267 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/25267/consoleFull) for PR 3715 at commit

[GitHub] spark pull request: [SPARK-5154] [PySpark] [Streaming] Kafka strea...

2015-01-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/3715#issuecomment-69261055 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-5154] [PySpark] [Streaming] Kafka strea...

2015-01-08 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3715#issuecomment-69253047 [Test build #25267 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/25267/consoleFull) for PR 3715 at commit

  1   2   >