[GitHub] spark pull request: [SPARK-4588] ML Attributes

2015-03-06 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/4925#discussion_r25985633 --- Diff: mllib/src/main/scala/org/apache/spark/ml/attribute/AttributeGroup.scala --- @@ -0,0 +1,234 @@ +/* + * Licensed to the Apache Software

[GitHub] spark pull request: [SPARK-6139] [Streaming] Allow pre-populate sl...

2015-03-06 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4875#issuecomment-77655289 [Test build #28356 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/28356/consoleFull) for PR 4875 at commit

[GitHub] spark pull request: [SPARK-6139] [Streaming] Allow pre-populate sl...

2015-03-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/4875#issuecomment-77655296 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-6209] Clean up connections in ExecutorC...

2015-03-06 Thread JoshRosen
Github user JoshRosen commented on the pull request: https://github.com/apache/spark/pull/4935#issuecomment-77657753 This actually turns out to be a much more subtle issue than I originally anticipated, since we might need to do other work to handle HttpUrlConnection's KeepAlive

[GitHub] spark pull request: [SPARK-5651][SQL] Add input64 in blacklist and...

2015-03-06 Thread watermen
Github user watermen commented on the pull request: https://github.com/apache/spark/pull/4427#issuecomment-77667059 @marmbrus Any more comment on this? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark pull request: [SPARK-6209] Clean up connections in ExecutorC...

2015-03-06 Thread JoshRosen
Github user JoshRosen commented on the pull request: https://github.com/apache/spark/pull/4935#issuecomment-77652552 /cc @pwendell --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request: Fix python typo

2015-03-06 Thread srowen
Github user srowen commented on the pull request: https://github.com/apache/spark/pull/4840#issuecomment-77658585 Oh I realized the variable name is `accum` in this example and needs to be fixed from `acc` on usage. I can fix on merge. --- If your project is set up for it, you can

[GitHub] spark pull request: [SPARK-5342][YARN] Allow long running Spark ap...

2015-03-06 Thread harishreedharan
Github user harishreedharan commented on a diff in the pull request: https://github.com/apache/spark/pull/4688#discussion_r25988487 --- Diff: yarn/src/main/scala/org/apache/spark/deploy/yarn/YarnSparkHadoopUtil.scala --- @@ -82,6 +100,179 @@ class YarnSparkHadoopUtil extends

[GitHub] spark pull request: [SPARK-5929] Pyspark: Register a pip requireme...

2015-03-06 Thread davies
Github user davies commented on the pull request: https://github.com/apache/spark/pull/4897#issuecomment-77659741 Jenkins, Ok to test. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request: [SPARK-4588] ML Attributes

2015-03-06 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4925#issuecomment-77659647 [Test build #28357 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/28357/consoleFull) for PR 4925 at commit

[GitHub] spark pull request: [SPARK-5929] Pyspark: Register a pip requireme...

2015-03-06 Thread davies
Github user davies commented on the pull request: https://github.com/apache/spark/pull/4897#issuecomment-77659717 Good idea, this is a useful feature to have. Could you also a argument for spark-submit, (for example, --pip or --py-requirements). Also, could you add a test for it?

[GitHub] spark pull request: [SPARK-6030][CORE] Don't alignSize the compute...

2015-03-06 Thread advancedxy
Github user advancedxy commented on a diff in the pull request: https://github.com/apache/spark/pull/4783#discussion_r25992824 --- Diff: core/src/main/scala/org/apache/spark/util/SizeEstimator.scala --- @@ -49,6 +49,11 @@ private[spark] object SizeEstimator extends Logging {

[GitHub] spark pull request: [SPARK-6185][SQL] Deltele repeated TOKEN. TOK...

2015-03-06 Thread DoingDone9
Github user DoingDone9 commented on the pull request: https://github.com/apache/spark/pull/4907#issuecomment-77671541 Anyone could test it? @chenghao-intel @marmbrus --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If

[GitHub] spark pull request: [SPARK-6209] Clean up connections in ExecutorC...

2015-03-06 Thread JoshRosen
Github user JoshRosen commented on a diff in the pull request: https://github.com/apache/spark/pull/4935#discussion_r25986036 --- Diff: repl/src/main/scala/org/apache/spark/repl/ExecutorClassLoader.scala --- @@ -71,27 +72,64 @@ class ExecutorClassLoader(conf: SparkConf, classUri:

[GitHub] spark pull request: [SPARK-6051][Streaming] Add ZooKeeper offest p...

2015-03-06 Thread tdas
Github user tdas commented on the pull request: https://github.com/apache/spark/pull/4805#issuecomment-77656954 @koeninger Would be good to have you in the conversation. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well.

[GitHub] spark pull request: [SPARK-6209] Clean up connections in ExecutorC...

2015-03-06 Thread JoshRosen
Github user JoshRosen commented on a diff in the pull request: https://github.com/apache/spark/pull/4935#discussion_r25987903 --- Diff: repl/src/main/scala/org/apache/spark/repl/ExecutorClassLoader.scala --- @@ -71,27 +72,64 @@ class ExecutorClassLoader(conf: SparkConf, classUri:

[GitHub] spark pull request: [SPARK-5929] Pyspark: Register a pip requireme...

2015-03-06 Thread davies
Github user davies commented on a diff in the pull request: https://github.com/apache/spark/pull/4897#discussion_r25988691 --- Diff: python/pyspark/context.py --- @@ -65,8 +65,9 @@ class SparkContext(object): _python_includes = None # zip and egg files that need to be

[GitHub] spark pull request: Fix python typo

2015-03-06 Thread davies
Github user davies commented on the pull request: https://github.com/apache/spark/pull/4840#issuecomment-77658901 @srowen Good catch, I saw that, then forget it immediately, really need to take a rest.. --- If your project is set up for it, you can reply to this email and have your

[GitHub] spark pull request: [SPARK-6051][Streaming] Add ZooKeeper offest p...

2015-03-06 Thread koeninger
Github user koeninger commented on a diff in the pull request: https://github.com/apache/spark/pull/4805#discussion_r25991083 --- Diff: external/kafka/src/main/scala/org/apache/spark/streaming/kafka/DirectKafkaInputDStream.scala --- @@ -158,4 +166,37 @@ class

[GitHub] spark pull request: [SPARK-4588] ML Attributes

2015-03-06 Thread mengxr
Github user mengxr commented on a diff in the pull request: https://github.com/apache/spark/pull/4925#discussion_r25985352 --- Diff: mllib/src/main/scala/org/apache/spark/ml/attribute/attributes.scala --- @@ -0,0 +1,512 @@ +/* + * Licensed to the Apache Software Foundation

[GitHub] spark pull request: [SPARK-4588] ML Attributes

2015-03-06 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4925#issuecomment-77651210 [Test build #28357 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/28357/consoleFull) for PR 4925 at commit

[GitHub] spark pull request: [SPARK-5946][Streaming] Add Python API for dir...

2015-03-06 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/4723#discussion_r25986002 --- Diff: python/pyspark/streaming/kafka.py --- @@ -70,7 +71,103 @@ def createStream(ssc, zkQuorum, groupId, topics, kafkaParams={}, except

[GitHub] spark pull request: [SPARK-6199] [SQL] Support CTE in HiveContext ...

2015-03-06 Thread marmbrus
Github user marmbrus commented on the pull request: https://github.com/apache/spark/pull/4929#issuecomment-77654143 ok to test --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark pull request: [SPARK-6209] Clean up connections in ExecutorC...

2015-03-06 Thread JoshRosen
Github user JoshRosen commented on a diff in the pull request: https://github.com/apache/spark/pull/4935#discussion_r25987502 --- Diff: repl/src/main/scala/org/apache/spark/repl/ExecutorClassLoader.scala --- @@ -71,27 +72,64 @@ class ExecutorClassLoader(conf: SparkConf, classUri:

[GitHub] spark pull request: [SPARK-5946][Streaming] Add Python API for dir...

2015-03-06 Thread tdas
Github user tdas commented on the pull request: https://github.com/apache/spark/pull/4723#issuecomment-77655852 This is looking good, but unfortunately its hard to say due to the lack of unit tests. We have to test the kafka python API through python unit tests. I can open a separate

[GitHub] spark pull request: [SPARK-4588] ML Attributes

2015-03-06 Thread mengxr
Github user mengxr commented on a diff in the pull request: https://github.com/apache/spark/pull/4925#discussion_r25988254 --- Diff: mllib/src/main/scala/org/apache/spark/ml/attribute/AttributeGroup.scala --- @@ -0,0 +1,234 @@ +/* + * Licensed to the Apache Software

[GitHub] spark pull request: [SPARK-4588] ML Attributes

2015-03-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/4925#issuecomment-77659654 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-4588] ML Attributes

2015-03-06 Thread mengxr
Github user mengxr commented on a diff in the pull request: https://github.com/apache/spark/pull/4925#discussion_r25988900 --- Diff: mllib/src/main/scala/org/apache/spark/ml/attribute/AttributeGroup.scala --- @@ -0,0 +1,234 @@ +/* + * Licensed to the Apache Software

[GitHub] spark pull request: [SPARK-6051][Streaming] Add ZooKeeper offest p...

2015-03-06 Thread koeninger
Github user koeninger commented on a diff in the pull request: https://github.com/apache/spark/pull/4805#discussion_r25990683 --- Diff: external/kafka/src/main/scala/org/apache/spark/streaming/kafka/DirectKafkaInputDStream.scala --- @@ -158,4 +166,37 @@ class

[GitHub] spark pull request: [SPARK-6051][Streaming] Add ZooKeeper offest p...

2015-03-06 Thread koeninger
Github user koeninger commented on a diff in the pull request: https://github.com/apache/spark/pull/4805#discussion_r25990950 --- Diff: external/kafka/src/main/scala/org/apache/spark/streaming/kafka/DirectKafkaInputDStream.scala --- @@ -158,4 +166,37 @@ class

[GitHub] spark pull request: [SPARK-1503][MLLIB] Initial AcceleratedGradien...

2015-03-06 Thread mengxr
Github user mengxr commented on a diff in the pull request: https://github.com/apache/spark/pull/4934#discussion_r25985578 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/optimization/AcceleratedGradientDescent.scala --- @@ -0,0 +1,237 @@ +/* + * Licensed to the

[GitHub] spark pull request: [SPARK-6198][SQL] Support select current_data...

2015-03-06 Thread marmbrus
Github user marmbrus commented on the pull request: https://github.com/apache/spark/pull/4926#issuecomment-77654187 please add a test --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request: Fix python typo

2015-03-06 Thread davies
Github user davies commented on the pull request: https://github.com/apache/spark/pull/4840#issuecomment-77657916 Nice catch, LGTM, thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-4588] ML Attributes

2015-03-06 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/4925#discussion_r25988627 --- Diff: mllib/src/main/scala/org/apache/spark/ml/attribute/AttributeGroup.scala --- @@ -0,0 +1,234 @@ +/* + * Licensed to the Apache Software

[GitHub] spark pull request: [SPARK-6199] [SQL] Support CTE in HiveContext ...

2015-03-06 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4929#issuecomment-77662119 [Test build #28359 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/28359/consoleFull) for PR 4929 at commit

[GitHub] spark pull request: [SPARK-6199] [SQL] Support CTE in HiveContext ...

2015-03-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/4929#issuecomment-77662126 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-6051][Streaming] Add ZooKeeper offest p...

2015-03-06 Thread koeninger
Github user koeninger commented on a diff in the pull request: https://github.com/apache/spark/pull/4805#discussion_r25990568 --- Diff: external/kafka/src/main/scala/org/apache/spark/streaming/kafka/DirectKafkaInputDStream.scala --- @@ -82,8 +83,12 @@ class

[GitHub] spark pull request: [SPARK-6051][Streaming] Add ZooKeeper offest p...

2015-03-06 Thread koeninger
Github user koeninger commented on a diff in the pull request: https://github.com/apache/spark/pull/4805#discussion_r25990850 --- Diff: external/kafka/src/main/scala/org/apache/spark/streaming/kafka/SparkKafkaUtils.scala --- @@ -0,0 +1,56 @@ +/* + * Licensed to the Apache

[GitHub] spark pull request: [SPARK-4894][mllib] Added Bernoulli option to ...

2015-03-06 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4087#issuecomment-77667987 [Test build #28361 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/28361/consoleFull) for PR 4087 at commit

[GitHub] spark pull request: [SPARK-5342][YARN] Allow long running Spark ap...

2015-03-06 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4688#issuecomment-77669755 [Test build #28362 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/28362/consoleFull) for PR 4688 at commit

[GitHub] spark pull request: [SPARK-4588] ML Attributes

2015-03-06 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/4925#discussion_r25985506 --- Diff: mllib/src/main/scala/org/apache/spark/ml/attribute/attributes.scala --- @@ -0,0 +1,512 @@ +/* + * Licensed to the Apache Software Foundation

[GitHub] spark pull request: Fix python typo

2015-03-06 Thread srowen
Github user srowen commented on the pull request: https://github.com/apache/spark/pull/4840#issuecomment-77661389 Heh, actually the Scala and Java examples are wrong too. They should be respectively: ``` val accum = sc.accumulator(0) data.map { x = accum += x; f(x) }

[GitHub] spark pull request: [SPARK-5342][YARN] Allow long running Spark ap...

2015-03-06 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4688#issuecomment-77665761 [Test build #28362 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/28362/consoleFull) for PR 4688 at commit

[GitHub] spark pull request: [SPARK-6030][CORE] Don't alignSize the compute...

2015-03-06 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4783#issuecomment-77670754 [Test build #28363 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/28363/consoleFull) for PR 4783 at commit

[GitHub] spark pull request: [SPARK-6030][CORE] Using simulated field layou...

2015-03-06 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4783#issuecomment-77672792 [Test build #28363 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/28363/consoleFull) for PR 4783 at commit

[GitHub] spark pull request: [SPARK-6030][CORE] Using simulated field layou...

2015-03-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/4783#issuecomment-77672798 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-6209] Clean up connections in ExecutorC...

2015-03-06 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4935#issuecomment-77661325 [Test build #28358 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/28358/consoleFull) for PR 4935 at commit

[GitHub] spark pull request: [SPARK-6209] Clean up connections in ExecutorC...

2015-03-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/4935#issuecomment-77661330 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-4894][mllib] Added Bernoulli option to ...

2015-03-06 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4087#issuecomment-77665465 [Test build #28361 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/28361/consoleFull) for PR 4087 at commit

[GitHub] spark pull request: [SPARK-6186] [EC2] Make Tachyon version config...

2015-03-06 Thread uronce-cc
Github user uronce-cc commented on the pull request: https://github.com/apache/spark/pull/4901#issuecomment-77665462 @srowen pep8 resolved. @nchammas re-word help message, DEFAULT_TACHYON_VERSION removed. --- If your project is set up for it, you can reply to this email and have

[GitHub] spark pull request: [SPARK-6209] Clean up connections in ExecutorC...

2015-03-06 Thread JoshRosen
GitHub user JoshRosen opened a pull request: https://github.com/apache/spark/pull/4935 [SPARK-6209] Clean up connections in ExecutorClassLoader after failing to load classes ExecutorClassLoader does not ensure proper cleanup of network connections that it opens. If it fails to

[GitHub] spark pull request: [SPARK-6209] Clean up connections in ExecutorC...

2015-03-06 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4935#issuecomment-77652458 [Test build #28358 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/28358/consoleFull) for PR 4935 at commit

[GitHub] spark pull request: [SPARK-6209] Clean up connections in ExecutorC...

2015-03-06 Thread JoshRosen
Github user JoshRosen commented on a diff in the pull request: https://github.com/apache/spark/pull/4935#discussion_r25986151 --- Diff: repl/src/main/scala/org/apache/spark/repl/ExecutorClassLoader.scala --- @@ -71,27 +72,64 @@ class ExecutorClassLoader(conf: SparkConf, classUri:

[GitHub] spark pull request: [SPARK-6209] Clean up connections in ExecutorC...

2015-03-06 Thread JoshRosen
Github user JoshRosen commented on a diff in the pull request: https://github.com/apache/spark/pull/4935#discussion_r25987271 --- Diff: repl/src/main/scala/org/apache/spark/repl/ExecutorClassLoader.scala --- @@ -71,27 +72,64 @@ class ExecutorClassLoader(conf: SparkConf, classUri:

[GitHub] spark pull request: [SPARK-6051][Streaming] Add ZooKeeper offest p...

2015-03-06 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/4805#discussion_r25987964 --- Diff: project/MimaExcludes.scala --- @@ -153,6 +153,11 @@ object MimaExcludes {

[GitHub] spark pull request: Fix python typo

2015-03-06 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/4840 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request: [SPARK-5342][YARN] Allow long running Spark ap...

2015-03-06 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4688#issuecomment-77662292 [Test build #28360 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/28360/consoleFull) for PR 4688 at commit

[GitHub] spark pull request: [SPARK-6051][Streaming] Add ZooKeeper offest p...

2015-03-06 Thread koeninger
Github user koeninger commented on a diff in the pull request: https://github.com/apache/spark/pull/4805#discussion_r25990774 --- Diff: external/kafka/src/main/scala/org/apache/spark/streaming/kafka/ReliableKafkaReceiver.scala --- @@ -239,21 +239,7 @@ class ReliableKafkaReceiver[

[GitHub] spark pull request: [SPARK-6051][Streaming] Add ZooKeeper offest p...

2015-03-06 Thread koeninger
Github user koeninger commented on a diff in the pull request: https://github.com/apache/spark/pull/4805#discussion_r25990749 --- Diff: external/kafka/src/main/scala/org/apache/spark/streaming/kafka/DirectKafkaInputDStream.scala --- @@ -158,4 +166,37 @@ class

[GitHub] spark pull request: [SPARK-5342][YARN] Allow long running Spark ap...

2015-03-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/4688#issuecomment-77669758 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-5946][Streaming] Add Python API for dir...

2015-03-06 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/4723#discussion_r25986066 --- Diff: python/pyspark/streaming/kafka.py --- @@ -70,7 +71,103 @@ def createStream(ssc, zkQuorum, groupId, topics, kafkaParams={}, except

[GitHub] spark pull request: [SPARK-1503][MLLIB] Initial AcceleratedGradien...

2015-03-06 Thread staple
Github user staple commented on a diff in the pull request: https://github.com/apache/spark/pull/4934#discussion_r25986928 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/optimization/AcceleratedGradientDescent.scala --- @@ -0,0 +1,237 @@ +/* + * Licensed to the

[GitHub] spark pull request: [SPARK-5681][Streaming] Add tracker status and...

2015-03-06 Thread tdas
Github user tdas commented on the pull request: https://github.com/apache/spark/pull/4467#issuecomment-77659282 Hey ... lets continue the discussion. I took a quick look at the logic, sounds good. Let me think a bit more and look at the code. --- If your project is set up for it,

[GitHub] spark pull request: [SPARK-6209] Clean up connections in ExecutorC...

2015-03-06 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/4935#discussion_r25988751 --- Diff: repl/src/main/scala/org/apache/spark/repl/ExecutorClassLoader.scala --- @@ -71,27 +72,64 @@ class ExecutorClassLoader(conf: SparkConf, classUri:

[GitHub] spark pull request: [SPARK-5929] Pyspark: Register a pip requireme...

2015-03-06 Thread davies
Github user davies commented on a diff in the pull request: https://github.com/apache/spark/pull/4897#discussion_r25988754 --- Diff: python/pyspark/context.py --- @@ -710,6 +717,33 @@ def addPyFile(self, path): # for tests in local mode

[GitHub] spark pull request: [SPARK-5342][YARN] Allow long running Spark ap...

2015-03-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/4688#issuecomment-77662800 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-5342][YARN] Allow long running Spark ap...

2015-03-06 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4688#issuecomment-77662798 [Test build #28360 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/28360/consoleFull) for PR 4688 at commit

[GitHub] spark pull request: [SPARK-6183][Deploy] Skip bad workers when re-...

2015-03-06 Thread zhpengg
Github user zhpengg commented on a diff in the pull request: https://github.com/apache/spark/pull/4909#discussion_r25992491 --- Diff: core/src/main/scala/org/apache/spark/deploy/master/Master.scala --- @@ -467,7 +467,9 @@ private[spark] class Master( * two executors on the

[GitHub] spark pull request: [SPARK-6030][CORE] Don't alignSize the compute...

2015-03-06 Thread advancedxy
Github user advancedxy commented on the pull request: https://github.com/apache/spark/pull/4783#issuecomment-77671093 @shivaram I update the comment, please review again. And still unrelated test failure. --- If your project is set up for it, you can reply to this email and have

[GitHub] spark pull request: [SPARK-6199] [SQL] Support CTE in HiveContext ...

2015-03-06 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4929#issuecomment-77654757 [Test build #28359 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/28359/consoleFull) for PR 4929 at commit

[GitHub] spark pull request: [SPARK-6051][Streaming] Add ZooKeeper offest p...

2015-03-06 Thread koeninger
Github user koeninger commented on a diff in the pull request: https://github.com/apache/spark/pull/4805#discussion_r25990695 --- Diff: external/kafka/src/main/scala/org/apache/spark/streaming/kafka/DirectKafkaInputDStream.scala --- @@ -158,4 +166,37 @@ class

[GitHub] spark pull request: [SPARK-4894][mllib] Added Bernoulli option to ...

2015-03-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/4087#issuecomment-77667991 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-6177][MLlib] LDA should check partition...

2015-03-06 Thread hhbyyh
Github user hhbyyh closed the pull request at: https://github.com/apache/spark/pull/4899 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request: [SPARK-5342][YARN] Allow long running Spark ap...

2015-03-06 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4688#issuecomment-77532962 [Test build #28336 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/28336/consoleFull) for PR 4688 at commit

[GitHub] spark pull request: [SPARK-6145][SQL] fix ORDER BY on nested field...

2015-03-06 Thread cloud-fan
Github user cloud-fan commented on the pull request: https://github.com/apache/spark/pull/4918#issuecomment-77565205 Hi @marmbrus , I studied how we handle ORDER BY and had a more complete fix. For simple example SELECT a, b FROM t ORDER BY b, it will be parsed into

[GitHub] spark pull request: [SPARK-6191] [EC2] Generalize ability to downl...

2015-03-06 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4919#issuecomment-77566523 [Test build #28343 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/28343/consoleFull) for PR 4919 at commit

[GitHub] spark pull request: [EC2] [SPARK-6188] Instance types can be misla...

2015-03-06 Thread nchammas
Github user nchammas commented on the pull request: https://github.com/apache/spark/pull/4916#issuecomment-77567240 I also tried to change the instance type when starting a stopped cluster. That didn't work, and the instances were brought up with their original instance types.

[GitHub] spark pull request: [SPARK-6145][SQL] fix ORDER BY on nested field...

2015-03-06 Thread cloud-fan
Github user cloud-fan commented on the pull request: https://github.com/apache/spark/pull/4904#issuecomment-77567607 Seems Jenkins doesn't listen to me :( --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark pull request: [SPARK-6178][Shuffle] Removed unused imports

2015-03-06 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/4900 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request: [EC2] [SPARK-6188] Instance types can be misla...

2015-03-06 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4916#issuecomment-77570226 [Test build #28342 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/28342/consoleFull) for PR 4916 at commit

[GitHub] spark pull request: [EC2] [SPARK-6188] Instance types can be misla...

2015-03-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/4916#issuecomment-77570242 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-6095] [MLLIB] Support model save/load i...

2015-03-06 Thread yanboliang
Github user yanboliang commented on the pull request: https://github.com/apache/spark/pull/4911#issuecomment-77572142 @mengxr Yes, it make sense. After look through the code, I found we have two alternatives: 1, Implement a new PythonMLLibAPI looks like this def

[GitHub] spark pull request: [SPARK-6197][CORE] handle json exception when ...

2015-03-06 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4927#issuecomment-77576075 [Test build #28344 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/28344/consoleFull) for PR 4927 at commit

[GitHub] spark pull request: [SPARK-6197][CORE] handle json exception when ...

2015-03-06 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/4927#discussion_r25951712 --- Diff: core/src/main/scala/org/apache/spark/deploy/master/Master.scala --- @@ -764,8 +764,10 @@ private[spark] class Master( val replayBus = new

[GitHub] spark pull request: [SPARK-6178][Shuffle] Removed unused imports

2015-03-06 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4900#issuecomment-77568094 [Test build #28340 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/28340/consoleFull) for PR 4900 at commit

[GitHub] spark pull request: [SPARK-5969][PySpark] Fix descending pyspark.r...

2015-03-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/4761#issuecomment-77534574 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-4588] ML Attributes

2015-03-06 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/4925#discussion_r25939339 --- Diff: mllib/src/main/scala/org/apache/spark/ml/attribute/AttributeType.scala --- @@ -0,0 +1,61 @@ +/* + * Licensed to the Apache Software

[GitHub] spark pull request: [SPARK-6197][CORE] handle json exception when ...

2015-03-06 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/4927#discussion_r25939663 --- Diff: core/src/main/scala/org/apache/spark/scheduler/ReplayListenerBus.scala --- @@ -40,15 +41,31 @@ private[spark] class ReplayListenerBus extends

[GitHub] spark pull request: Fix python typo

2015-03-06 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/4840#discussion_r25940372 --- Diff: docs/programming-guide.md --- @@ -1353,7 +1353,7 @@ data.map(x - accum.add(x); f(x);); div data-lang=python markdown=1 {% highlight python

[GitHub] spark pull request: [SPARK-6183][Deploy] Skip bad workers when re-...

2015-03-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/4909#issuecomment-77562688 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-6183][Deploy] Skip bad workers when re-...

2015-03-06 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4909#issuecomment-77562676 **[Test build #28338 timed out](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/28338/consoleFull)** for PR 4909 at commit

[GitHub] spark pull request: [EC2] [SPARK-6188] Instance types can be misla...

2015-03-06 Thread nchammas
Github user nchammas commented on the pull request: https://github.com/apache/spark/pull/4916#issuecomment-77564954 @thvasilo can you clarify what flow you are seeing this behavior for? I can't reproduce the issue you're reporting. Here's what I tried: 1. Launch a

[GitHub] spark pull request: [SPARK-6145][SQL] fix ORDER BY on nested field...

2015-03-06 Thread cloud-fan
Github user cloud-fan commented on the pull request: https://github.com/apache/spark/pull/4904#issuecomment-77566870 retest it please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request: [SPARK-6178][Shuffle] Removed unused imports

2015-03-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/4900#issuecomment-77568103 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: SPARK-4705:[core] Write event logs of differen...

2015-03-06 Thread twinkle-sachdeva
Github user twinkle-sachdeva commented on a diff in the pull request: https://github.com/apache/spark/pull/4845#discussion_r25951518 --- Diff: core/src/main/scala/org/apache/spark/deploy/history/HistoryPage.scala --- @@ -34,18 +37,31 @@ private[spark] class HistoryPage(parent:

[GitHub] spark pull request: [SPARK-6197][CORE] handle json exception when ...

2015-03-06 Thread liyezhang556520
Github user liyezhang556520 commented on the pull request: https://github.com/apache/spark/pull/4927#issuecomment-77576837 @srowen , thanks for your comments, and I have updated the code. --- If your project is set up for it, you can reply to this email and have your reply appear on

[GitHub] spark pull request: [SPARK-6145][SQL] fix ORDER BY on nested field...

2015-03-06 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/4904#discussion_r25947236 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/basicOperators.scala --- @@ -152,6 +153,18 @@ case class Sort(

[GitHub] spark pull request: [SPARK-6202] [SQL] enable variable substitutio...

2015-03-06 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4930#issuecomment-77533722 [Test build #28333 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/28333/consoleFull) for PR 4930 at commit

[GitHub] spark pull request: [SPARK-6202] [SQL] enable variable substitutio...

2015-03-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/4930#issuecomment-77533732 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-6204][SQL] GenerateProjection's equals ...

2015-03-06 Thread viirya
GitHub user viirya opened a pull request: https://github.com/apache/spark/pull/4931 [SPARK-6204][SQL] GenerateProjection's equals should check length equality `GenerateProjection`'s `equals` now only checks column equality. It should also check length equality. You can merge this

  1   2   3   4   >