[GitHub] spark pull request: [STREAMING] SPARK-4986 Wait for receivers to d...

2015-02-03 Thread tdas
Github user tdas commented on the pull request: https://github.com/apache/spark/pull/3868#issuecomment-72719061 Thanks! @cleaton Could you close this PR, and then create a new PR with the new stuff. Also create a new JIRA saying something like Make the gracefulStopTimeout more

[GitHub] spark pull request: [SPARK-5554] [SQL] [PySpark] add more tests fo...

2015-02-03 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4331#issuecomment-72719892 [Test build #26661 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/26661/consoleFull) for PR 4331 at commit

[GitHub] spark pull request: [SPARK-5554] [SQL] [PySpark] add more tests fo...

2015-02-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/4331#issuecomment-72719914 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-4964] [Streaming] Exactly-once semantic...

2015-02-03 Thread koeninger
Github user koeninger commented on a diff in the pull request: https://github.com/apache/spark/pull/3798#discussion_r24033509 --- Diff: external/kafka/src/main/scala/org/apache/spark/streaming/kafka/KafkaUtils.scala --- @@ -144,4 +150,174 @@ object KafkaUtils {

[GitHub] spark pull request: [SPARK-5426][SQL] Add SparkSQL Java API helper...

2015-02-03 Thread marmbrus
Github user marmbrus commented on the pull request: https://github.com/apache/spark/pull/4243#issuecomment-72723284 ok to test --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark pull request: [SPARK-4987] [SQL] parquet timestamp type supp...

2015-02-03 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/3820 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request: [SPARK-4987] [SQL] parquet timestamp type supp...

2015-02-03 Thread marmbrus
Github user marmbrus commented on the pull request: https://github.com/apache/spark/pull/3820#issuecomment-72724023 Thanks! Merged to master. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-4964] [Streaming] Exactly-once semantic...

2015-02-03 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/3798#discussion_r24028643 --- Diff: external/kafka/src/main/scala/org/apache/spark/streaming/kafka/KafkaUtils.scala --- @@ -144,4 +150,174 @@ object KafkaUtils {

[GitHub] spark pull request: [SPARK-4964] [Streaming] Exactly-once semantic...

2015-02-03 Thread pwendell
Github user pwendell commented on a diff in the pull request: https://github.com/apache/spark/pull/3798#discussion_r24029463 --- Diff: external/kafka/src/main/scala/org/apache/spark/streaming/kafka/KafkaUtils.scala --- @@ -144,4 +150,249 @@ object KafkaUtils {

[GitHub] spark pull request: [SPARK-4964] [Streaming] Exactly-once semantic...

2015-02-03 Thread pwendell
Github user pwendell commented on a diff in the pull request: https://github.com/apache/spark/pull/3798#discussion_r24029652 --- Diff: external/kafka/src/main/scala/org/apache/spark/streaming/kafka/OffsetRange.scala --- @@ -0,0 +1,69 @@ +/* + * Licensed to the Apache

[GitHub] spark pull request: [STREAMING] SPARK-4986 Wait for receivers to d...

2015-02-03 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/4338#discussion_r24029813 --- Diff: streaming/src/test/scala/org/apache/spark/streaming/StreamingContextSuite.scala --- @@ -205,6 +205,33 @@ class StreamingContextSuite extends FunSuite

[GitHub] spark pull request: [STREAMING] SPARK-4986 Wait for receivers to d...

2015-02-03 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/4338#discussion_r24029997 --- Diff: streaming/src/test/scala/org/apache/spark/streaming/StreamingContextSuite.scala --- @@ -205,6 +205,33 @@ class StreamingContextSuite extends FunSuite

[GitHub] spark pull request: [SPARK-4964] [Streaming] Exactly-once semantic...

2015-02-03 Thread koeninger
Github user koeninger commented on a diff in the pull request: https://github.com/apache/spark/pull/3798#discussion_r24031208 --- Diff: external/kafka/src/main/scala/org/apache/spark/streaming/kafka/KafkaUtils.scala --- @@ -144,4 +150,174 @@ object KafkaUtils {

[GitHub] spark pull request: [SPARK-5155] [PySpark] [Streaming] Mqtt stream...

2015-02-03 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4229#issuecomment-72717970 [Test build #2 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/2/consoleFull) for PR 4229 at commit

[GitHub] spark pull request: [STREAMING] SPARK-3505: Augmenting SparkStream...

2015-02-03 Thread tdas
Github user tdas commented on the pull request: https://github.com/apache/spark/pull/2267#issuecomment-72719145 This is not closed yet :( --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request: [SPARK-5155] [PySpark] [Streaming] Mqtt stream...

2015-02-03 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4229#issuecomment-72721443 [Test build #26668 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/26668/consoleFull) for PR 4229 at commit

[GitHub] spark pull request: [SQL] Minor changes for dataframe implementati...

2015-02-03 Thread marmbrus
Github user marmbrus commented on a diff in the pull request: https://github.com/apache/spark/pull/4336#discussion_r24034304 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/DataFrameImpl.scala --- @@ -260,11 +260,11 @@ private[sql] class DataFrameImpl protected[sql](

[GitHub] spark pull request: [SQL][Hiveconsole] Bring hive console code up ...

2015-02-03 Thread marmbrus
Github user marmbrus commented on the pull request: https://github.com/apache/spark/pull/4330#issuecomment-72722689 ok to test --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark pull request: [SPARK-5426][SQL] Add SparkSQL Java API helper...

2015-02-03 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4243#issuecomment-72724110 [Test build #26670 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/26670/consoleFull) for PR 4243 at commit

[GitHub] spark pull request: [SPARK-5574] use given name prefix in dir

2015-02-03 Thread squito
Github user squito commented on the pull request: https://github.com/apache/spark/pull/4344#issuecomment-72752833 whoops, sorry I forgot about the title, just updated. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If

[GitHub] spark pull request: [STREAMING] SPARK-4986 Wait for receivers to d...

2015-02-03 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/4338#discussion_r24048309 --- Diff: streaming/src/test/scala/org/apache/spark/streaming/StreamingContextSuite.scala --- @@ -205,6 +205,33 @@ class StreamingContextSuite extends FunSuite

[GitHub] spark pull request: [SQL][DataFrame] defineUDF.

2015-02-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/4345#issuecomment-72754081 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-4939] move to next locality when no pen...

2015-02-03 Thread markhamstra
Github user markhamstra commented on a diff in the pull request: https://github.com/apache/spark/pull/3779#discussion_r24048516 --- Diff: core/src/main/scala/org/apache/spark/scheduler/TaskSetManager.scala --- @@ -506,13 +506,59 @@ private[spark] class TaskSetManager( *

[GitHub] spark pull request: [SPARK-5576] saveAsTable into Hive fails due t...

2015-02-03 Thread sryza
Github user sryza commented on the pull request: https://github.com/apache/spark/pull/4346#issuecomment-72754158 Mind adding [SQL] to the title so it gets sorted properly? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well.

[GitHub] spark pull request: [SPARK-4879] [WIP] Use driver to coordinate Ha...

2015-02-03 Thread JoshRosen
Github user JoshRosen commented on a diff in the pull request: https://github.com/apache/spark/pull/4066#discussion_r24048906 --- Diff: core/src/test/scala/org/apache/spark/scheduler/DAGSchedulerSuite.scala --- @@ -766,5 +753,10 @@ class DAGSchedulerSuite extends FunSuiteLike

[GitHub] spark pull request: [SQL][DataFrame] defineUDF.

2015-02-03 Thread mengxr
Github user mengxr commented on a diff in the pull request: https://github.com/apache/spark/pull/4345#discussion_r24049119 --- Diff: mllib/src/main/scala/org/apache/spark/ml/classification/LogisticRegression.scala --- @@ -132,14 +132,14 @@ class LogisticRegressionModel

[GitHub] spark pull request: [SPARK-5341] Use maven coordinates as dependen...

2015-02-03 Thread pwendell
Github user pwendell commented on a diff in the pull request: https://github.com/apache/spark/pull/4215#discussion_r24049531 --- Diff: core/src/main/scala/org/apache/spark/deploy/SparkSubmit.scala --- @@ -475,6 +500,199 @@ object SparkSubmit { } } +/**

[GitHub] spark pull request: [SPARK-5341] Use maven coordinates as dependen...

2015-02-03 Thread pwendell
Github user pwendell commented on a diff in the pull request: https://github.com/apache/spark/pull/4215#discussion_r24049602 --- Diff: core/src/main/scala/org/apache/spark/deploy/SparkSubmit.scala --- @@ -475,6 +500,199 @@ object SparkSubmit { } } +/**

[GitHub] spark pull request: [SQL][DataFrame] defineUDF.

2015-02-03 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4345#issuecomment-72756711 [Test build #26687 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/26687/consoleFull) for PR 4345 at commit

[GitHub] spark pull request: [SPARK-4874] [CORE] Collect record count metri...

2015-02-03 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4067#issuecomment-72756645 [Test build #26678 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/26678/consoleFull) for PR 4067 at commit

[GitHub] spark pull request: [SPARK-4874] [CORE] Collect record count metri...

2015-02-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/4067#issuecomment-72756658 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SQL][DataFrame] defineUDF.

2015-02-03 Thread davies
Github user davies commented on a diff in the pull request: https://github.com/apache/spark/pull/4345#discussion_r24050248 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/Dsl.scala --- @@ -213,188 +211,189 @@ object Dsl { } } */ + /**

[GitHub] spark pull request: [SPARK-4964] [Streaming] Exactly-once semantic...

2015-02-03 Thread koeninger
Github user koeninger commented on the pull request: https://github.com/apache/spark/pull/3798#issuecomment-72758821 Besides introducing 2 classes where 1 would do, it implies that there are (or could be) multiple implementations of the abstract class. You're not using it

[GitHub] spark pull request: [SPARK-4964] [Streaming] Exactly-once semantic...

2015-02-03 Thread tdas
Github user tdas commented on the pull request: https://github.com/apache/spark/pull/3798#issuecomment-72759859 But of course there can be multiple implementations! For example, there is both KafkaReceiver and ReliableKafkaReceiver. The second was introduced so that the code path for

[GitHub] spark pull request: [SPARK-4969][STREAMING][PYTHON] Add binaryReco...

2015-02-03 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3803#issuecomment-72761349 [Test build #26680 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/26680/consoleFull) for PR 3803 at commit

[GitHub] spark pull request: [SPARK-4969][STREAMING][PYTHON] Add binaryReco...

2015-02-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/3803#issuecomment-72761356 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-5341] Use maven coordinates as dependen...

2015-02-03 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4215#issuecomment-72761758 [Test build #26685 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/26685/consoleFull) for PR 4215 at commit

[GitHub] spark pull request: [SPARK-5341] Use maven coordinates as dependen...

2015-02-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/4215#issuecomment-72761769 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-4969][STREAMING][PYTHON] Add binaryReco...

2015-02-03 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3803#issuecomment-72750244 [Test build #26680 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/26680/consoleFull) for PR 3803 at commit

[GitHub] spark pull request: use given name prefix in dir

2015-02-03 Thread squito
GitHub user squito opened a pull request: https://github.com/apache/spark/pull/4344 use given name prefix in dir https://issues.apache.org/jira/browse/SPARK-5574 very minor, doesn't effect external behavior at all. Note that after this change, some of these dirs no

[GitHub] spark pull request: [SPARK-4879] [WIP] Use driver to coordinate Ha...

2015-02-03 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4066#issuecomment-72751842 [Test build #26682 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/26682/consoleFull) for PR 4066 at commit

[GitHub] spark pull request: [SPARK-5341] Use maven coordinates as dependen...

2015-02-03 Thread brkyvz
Github user brkyvz commented on a diff in the pull request: https://github.com/apache/spark/pull/4215#discussion_r24047567 --- Diff: core/src/main/scala/org/apache/spark/deploy/SparkSubmit.scala --- @@ -25,6 +25,18 @@ import scala.collection.mutable.{ArrayBuffer, HashMap, Map}

[GitHub] spark pull request: use given name prefix in dir

2015-02-03 Thread vanzin
Github user vanzin commented on the pull request: https://github.com/apache/spark/pull/4344#issuecomment-72752324 hey @squito , could you add `[SPARK-5574]` to the PR title to follow the usual convention? Thx! --- If your project is set up for it, you can reply to this email and

[GitHub] spark pull request: use given name prefix in dir

2015-02-03 Thread vanzin
Github user vanzin commented on the pull request: https://github.com/apache/spark/pull/4344#issuecomment-72752418 other than that, LGTM. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request: [STREAMING] SPARK-4986 Wait for receivers to d...

2015-02-03 Thread tdas
Github user tdas commented on the pull request: https://github.com/apache/spark/pull/4338#issuecomment-72753609 Alright, I like this. I am merging this. Please submit the other PR as well :) --- If your project is set up for it, you can reply to this email and have your reply

[GitHub] spark pull request: [SPARK-5576] saveAsTable into Hive fails due t...

2015-02-03 Thread danosipov
GitHub user danosipov opened a pull request: https://github.com/apache/spark/pull/4346 [SPARK-5576] saveAsTable into Hive fails due to duplicate columns Fix issue with Hive incompatibility with case sensitive columns You can merge this pull request into a Git repository by running:

[GitHub] spark pull request: [SQL][DataFrame] defineUDF.

2015-02-03 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4345#issuecomment-72754355 [Test build #26684 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/26684/consoleFull) for PR 4345 at commit

[GitHub] spark pull request: [SPARK-5576] saveAsTable into Hive fails due t...

2015-02-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/4346#issuecomment-72754270 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-5341] Use maven coordinates as dependen...

2015-02-03 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4215#issuecomment-72754387 [Test build #26685 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/26685/consoleFull) for PR 4215 at commit

[GitHub] spark pull request: [MLLIB] [spark-2352] Implementation of an Arti...

2015-02-03 Thread avulanov
Github user avulanov commented on the pull request: https://github.com/apache/spark/pull/1290#issuecomment-72754968 There are few PR related to different types of artificial neural networks. It makes sense to come up with common interfaces, reuse the code and make it easily

[GitHub] spark pull request: [SQL][DataFrame] defineUDF.

2015-02-03 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4345#issuecomment-72754917 [Test build #26684 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/26684/consoleFull) for PR 4345 at commit

[GitHub] spark pull request: [SQL][DataFrame] defineUDF.

2015-02-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/4345#issuecomment-72754920 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SQL][DataFrame] defineUDF.

2015-02-03 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/4345#discussion_r24049499 --- Diff: mllib/src/main/scala/org/apache/spark/ml/classification/LogisticRegression.scala --- @@ -132,14 +132,14 @@ class LogisticRegressionModel private[ml]

[GitHub] spark pull request: [SPARK-4964] [Streaming] Exactly-once semantic...

2015-02-03 Thread tdas
Github user tdas commented on the pull request: https://github.com/apache/spark/pull/3798#issuecomment-72758029 I dont get it, what's the complication with abstract classes? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as

[GitHub] spark pull request: [SPARK-4964] [Streaming] Exactly-once semantic...

2015-02-03 Thread koeninger
Github user koeninger commented on the pull request: https://github.com/apache/spark/pull/3798#issuecomment-72760900 To put it another way, the type you return has to be public. If you return a public abstract class, what are you going to do when someone else subclasses

[GitHub] spark pull request: [SPARK-5341] Use maven coordinates as dependen...

2015-02-03 Thread brkyvz
Github user brkyvz commented on a diff in the pull request: https://github.com/apache/spark/pull/4215#discussion_r24047115 --- Diff: core/src/main/scala/org/apache/spark/deploy/SparkSubmit.scala --- @@ -25,6 +25,18 @@ import scala.collection.mutable.{ArrayBuffer, HashMap, Map}

[GitHub] spark pull request: use given name prefix in dir

2015-02-03 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4344#issuecomment-72751913 [Test build #26681 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/26681/consoleFull) for PR 4344 at commit

[GitHub] spark pull request: [SQL][DataFrame] defineUDF.

2015-02-03 Thread rxin
GitHub user rxin opened a pull request: https://github.com/apache/spark/pull/4345 [SQL][DataFrame] defineUDF. A more convenient way to define user-defined functions. You can merge this pull request into a Git repository by running: $ git pull https://github.com/rxin/spark

[GitHub] spark pull request: [SQL][DataFrame] defineUDF.

2015-02-03 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4345#issuecomment-72754075 [Test build #26683 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/26683/consoleFull) for PR 4345 at commit

[GitHub] spark pull request: [STREAMING] SPARK-4986 Wait for receivers to d...

2015-02-03 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/4338 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request: [SPARK-4964] [Streaming] Exactly-once semantic...

2015-02-03 Thread koeninger
Github user koeninger commented on the pull request: https://github.com/apache/spark/pull/3798#issuecomment-72754442 Like patrick said, I really don't see any reason not to just expose KafkaRDD. You can still hide its constructor without making a superflous abstract class, and

[GitHub] spark pull request: [SPARK-4879] [WIP] Use driver to coordinate Ha...

2015-02-03 Thread JoshRosen
Github user JoshRosen commented on a diff in the pull request: https://github.com/apache/spark/pull/4066#discussion_r24048694 --- Diff: core/src/test/scala/org/apache/spark/scheduler/DAGSchedulerSuite.scala --- @@ -766,5 +753,10 @@ class DAGSchedulerSuite extends FunSuiteLike

[GitHub] spark pull request: [SPARK-4939] move to next locality when no pen...

2015-02-03 Thread markhamstra
Github user markhamstra commented on a diff in the pull request: https://github.com/apache/spark/pull/3779#discussion_r24049293 --- Diff: core/src/main/scala/org/apache/spark/scheduler/TaskSetManager.scala --- @@ -506,13 +506,59 @@ private[spark] class TaskSetManager( *

[GitHub] spark pull request: [SPARK-4879] Use the Spark driver to authorize...

2015-02-03 Thread JoshRosen
Github user JoshRosen commented on the pull request: https://github.com/apache/spark/pull/4155#issuecomment-72755930 I'm continuing work on this output commit coordination patch over at #4066, in case anyone would like to help with further review. --- If your project is set up for

[GitHub] spark pull request: [SPARK-4795][Core] Redesign the primitive typ...

2015-02-03 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/3642#discussion_r24049421 --- Diff: graphx/src/test/scala/org/apache/spark/graphx/lib/ShortestPathsSuite.scala --- @@ -40,7 +40,7 @@ class ShortestPathsSuite extends FunSuite with

[GitHub] spark pull request: [SPARK-4879] [WIP] Use driver to coordinate Ha...

2015-02-03 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4066#issuecomment-72755993 [Test build #26686 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/26686/consoleFull) for PR 4066 at commit

[GitHub] spark pull request: [SPARK-4964] [Streaming] Exactly-once semantic...

2015-02-03 Thread tdas
Github user tdas commented on the pull request: https://github.com/apache/spark/pull/3798#issuecomment-72756440 I spent some time talking to Patrick offline about this. If we expose the KafkaRDD as is (while keeping its constructor private), then the simplified createRDD would be

[GitHub] spark pull request: [SPARK-4964] [Streaming] Exactly-once semantic...

2015-02-03 Thread koeninger
Github user koeninger commented on the pull request: https://github.com/apache/spark/pull/3798#issuecomment-72757731 Just make the simplified createRDD return a static type of RDD[(K, V)], that's what I'm saying. You're already going to have to deal with those other type

[GitHub] spark pull request: [SPARK-5576][SQL] saveAsTable into Hive fails ...

2015-02-03 Thread danosipov
Github user danosipov commented on the pull request: https://github.com/apache/spark/pull/4346#issuecomment-72759153 @sryza no problem, done --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-4795][Core] Redesign the primitive typ...

2015-02-03 Thread ankurdave
Github user ankurdave commented on a diff in the pull request: https://github.com/apache/spark/pull/3642#discussion_r24050985 --- Diff: graphx/src/test/scala/org/apache/spark/graphx/lib/ShortestPathsSuite.scala --- @@ -40,7 +40,7 @@ class ShortestPathsSuite extends FunSuite with

[GitHub] spark pull request: [SQL][DataFrame] defineUDF.

2015-02-03 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4345#issuecomment-72760362 [Test build #26688 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/26688/consoleFull) for PR 4345 at commit

[GitHub] spark pull request: [SPARK-4964] [Streaming] Exactly-once semantic...

2015-02-03 Thread tdas
Github user tdas commented on the pull request: https://github.com/apache/spark/pull/3798#issuecomment-72761134 In terms of number of classes, the abstract KafkaRDD is essentially replacing HasOffsetRanges. There is no need for this HasOffsetRanges trait that gets used only (assuming

[GitHub] spark pull request: [SPARK-5520][MLlib] Make FP-Growth implementat...

2015-02-03 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4340#issuecomment-72761075 [Test build #26689 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/26689/consoleFull) for PR 4340 at commit

[GitHub] spark pull request: [SPARK-5550] [SQL] Support the case insensitiv...

2015-02-03 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/4326 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request: [SPARK-5550] [SQL] Support the case insensitiv...

2015-02-03 Thread marmbrus
Github user marmbrus commented on the pull request: https://github.com/apache/spark/pull/4326#issuecomment-72725158 Thanks! Merged to master. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-5383][SQL] Support alias for udtfs

2015-02-03 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/4186 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request: SPARK-4705:Creating different log directories ...

2015-02-03 Thread vanzin
Github user vanzin commented on the pull request: https://github.com/apache/spark/pull/4311#issuecomment-72729132 Hi @twinkle-sachdeva, I left some comments about your design in the bug. Also, in general, changes first go into the master branch, and then are backported to

[GitHub] spark pull request: SPARK-4687. [WIP] Add an addDirectory API

2015-02-03 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/3670#discussion_r24038361 --- Diff: core/src/main/scala/org/apache/spark/util/Utils.scala --- @@ -513,13 +516,44 @@ private[spark] object Utils extends Logging {

[GitHub] spark pull request: [STREAMING] SPARK-4986 Wait for receivers to d...

2015-02-03 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4338#issuecomment-72733188 [Test build #26674 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/26674/consoleFull) for PR 4338 at commit

[GitHub] spark pull request: [SPARK-4879] [WIP] Use driver to coordinate Ha...

2015-02-03 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/4066#discussion_r24039388 --- Diff: core/src/main/scala/org/apache/spark/SparkHadoopWriter.scala --- @@ -105,24 +106,61 @@ class SparkHadoopWriter(@transient jobConf: JobConf)

[GitHub] spark pull request: [SPARK-4879] [WIP] Use driver to coordinate Ha...

2015-02-03 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/4066#discussion_r24039906 --- Diff: core/src/main/scala/org/apache/spark/scheduler/OutputCommitCoordinator.scala --- @@ -0,0 +1,199 @@ +/* + * Licensed to the Apache Software

[GitHub] spark pull request: SPARK-5548: Fixed a race condition in AkkaUtil...

2015-02-03 Thread jacek-lewandowski
GitHub user jacek-lewandowski opened a pull request: https://github.com/apache/spark/pull/4343 SPARK-5548: Fixed a race condition in AkkaUtilsSuite You can merge this pull request into a Git repository by running: $ git pull https://github.com/jacek-lewandowski/spark

[GitHub] spark pull request: [SPARK-5554] [SQL] [PySpark] add more tests fo...

2015-02-03 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4331#issuecomment-72726721 [Test build #26671 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/26671/consoleFull) for PR 4331 at commit

[GitHub] spark pull request: [SPARK-5153][Streaming][Test] Increased timeou...

2015-02-03 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4342#issuecomment-72728498 [Test build #26664 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/26664/consoleFull) for PR 4342 at commit

[GitHub] spark pull request: [SPARK-5155] [PySpark] [Streaming] Mqtt stream...

2015-02-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/4229#issuecomment-72730109 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-5155] [PySpark] [Streaming] Mqtt stream...

2015-02-03 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4229#issuecomment-72730095 [Test build #2 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/2/consoleFull) for PR 4229 at commit

[GitHub] spark pull request: [STREAMING] SPARK-4986 Wait for receivers to d...

2015-02-03 Thread cleaton
Github user cleaton commented on a diff in the pull request: https://github.com/apache/spark/pull/4338#discussion_r24038573 --- Diff: streaming/src/test/scala/org/apache/spark/streaming/StreamingContextSuite.scala --- @@ -205,6 +205,33 @@ class StreamingContextSuite extends

[GitHub] spark pull request: [SPARK-4879] [WIP] Use driver to coordinate Ha...

2015-02-03 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/4066#discussion_r24039251 --- Diff: core/src/main/scala/org/apache/spark/SparkEnv.scala --- @@ -20,6 +20,8 @@ package org.apache.spark import java.io.File import

[GitHub] spark pull request: [SPARK-4879] [WIP] Use driver to coordinate Ha...

2015-02-03 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/4066#discussion_r24039221 --- Diff: core/src/main/scala/org/apache/spark/CommitDeniedException.scala --- @@ -0,0 +1,32 @@ +/* + * Licensed to the Apache Software Foundation

[GitHub] spark pull request: [SPARK-5155] [PySpark] [Streaming] Mqtt stream...

2015-02-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/4229#issuecomment-72733961 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-5182] [SPARK-5528] [SPARK-5509] [SPARK-...

2015-02-03 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4308#issuecomment-72734610 [Test build #26667 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/26667/consoleFull) for PR 4308 at commit

[GitHub] spark pull request: SPARK-5548: Fixed a race condition in AkkaUtil...

2015-02-03 Thread jacek-lewandowski
Github user jacek-lewandowski commented on the pull request: https://github.com/apache/spark/pull/4343#issuecomment-72735186 Once approved, I'll create another PR for master. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as

[GitHub] spark pull request: [SPARK-5383][SQL] Support alias for udtfs

2015-02-03 Thread marmbrus
Github user marmbrus commented on the pull request: https://github.com/apache/spark/pull/4186#issuecomment-72725706 Thanks! Merged to master. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-4508] [SQL] build native date type to c...

2015-02-03 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/4325 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request: [SPARK-5554] [SQL] [PySpark] add more tests fo...

2015-02-03 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4331#issuecomment-72729526 [Test build #26672 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/26672/consoleFull) for PR 4331 at commit

[GitHub] spark pull request: [SPARK-5520][MLlib] Make FP-Growth implementat...

2015-02-03 Thread mengxr
Github user mengxr commented on the pull request: https://github.com/apache/spark/pull/4340#issuecomment-7272 @jackylk I sent you a PR update for Java at: https://github.com/jackylk/spark/pull/2 --- If your project is set up for it, you can reply to this email and have your

[GitHub] spark pull request: [SQL] Correct the default size of TimestampTyp...

2015-02-03 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4314#issuecomment-72730429 [Test build #26673 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/26673/consoleFull) for PR 4314 at commit

[GitHub] spark pull request: [SPARK-4877] Allow user first classes to exten...

2015-02-03 Thread stephenh
Github user stephenh commented on the pull request: https://github.com/apache/spark/pull/3725#issuecomment-72731355 @pwendell any chance this could make the 1.3 branch? @holdenk has (I believe) signed off, but doesn't have commit access. We're currently running a patched

[GitHub] spark pull request: [SPARK-4877] Allow user first classes to exten...

2015-02-03 Thread stephenh
Github user stephenh commented on the pull request: https://github.com/apache/spark/pull/3725#issuecomment-72731411 Oh, I see it has merge conflicts now...I'll work on rebasing it... --- If your project is set up for it, you can reply to this email and have your reply appear on

[GitHub] spark pull request: [SPARK-4979][MLLIB] Streaming logisitic regres...

2015-02-03 Thread freeman-lab
Github user freeman-lab commented on a diff in the pull request: https://github.com/apache/spark/pull/4306#discussion_r24038692 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/regression/StreamingLinearAlgorithm.scala --- @@ -58,14 +58,14 @@ abstract class

<    1   2   3   4   5   6   7   >