[GitHub] spark pull request: [SPARK-2179][SQL] A minor refactoring Java dat...

2014-08-01 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1712#issuecomment-50851677 QA results for PR 1712:br- This patch PASSES unit tests.br- This patch merges cleanlybr- This patch adds no public classesbrbrFor more information see test

[GitHub] spark pull request: [SPARK-2608] fix executor backend launch commo...

2014-08-01 Thread pwendell
Github user pwendell commented on the pull request: https://github.com/apache/spark/pull/1513#issuecomment-50851919 Jenkins, test this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-1470][SPARK-1842] Use the scala-logging...

2014-08-01 Thread witgo
Github user witgo commented on the pull request: https://github.com/apache/spark/pull/1369#issuecomment-50851936 Jenkins, retest this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-2608] fix executor backend launch commo...

2014-08-01 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1513#issuecomment-50852097 QA tests have started for PR 1513. This patch merges cleanly. brView progress: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/17650/consoleFull ---

[GitHub] spark pull request: [SPARK-1470][SPARK-1842] Use the scala-logging...

2014-08-01 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1369#issuecomment-50852101 QA tests have started for PR 1369. This patch merges cleanly. brView progress: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/17651/consoleFull ---

[GitHub] spark pull request: SPARK-2632, SPARK-2576. Fixed by only importin...

2014-08-01 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/1635 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request: [SPARK-2702][Core] Upgrade Tachyon dependency ...

2014-08-01 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/1651 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request: [SPARK-1812] upgrade dependency to scala-loggi...

2014-08-01 Thread ScrapCodes
Github user ScrapCodes commented on the pull request: https://github.com/apache/spark/pull/1701#issuecomment-50852258 Jenkins, test this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-2608] fix executor backend launch commo...

2014-08-01 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1513#issuecomment-50852319 QA results for PR 1513:br- This patch FAILED unit tests.br- This patch merges cleanlybr- This patch adds no public classesbrbrFor more information see test

[GitHub] spark pull request: [SPARK-1812] upgrade dependency to scala-loggi...

2014-08-01 Thread ScrapCodes
Github user ScrapCodes commented on a diff in the pull request: https://github.com/apache/spark/pull/1701#discussion_r15683473 --- Diff: sql/core/pom.xml --- @@ -83,6 +83,16 @@ artifactIdscalacheck_${scala.binary.version}/artifactId scopetest/scope

[GitHub] spark pull request: [SPARK-2103][Streaming] Change to ClassTag for...

2014-08-01 Thread tdas
Github user tdas commented on the pull request: https://github.com/apache/spark/pull/1508#issuecomment-50852462 OK, this is very confusing, sequence of two results is very confusing. Let me run the tests again. --- If your project is set up for it, you can reply to this email

[GitHub] spark pull request: [SPARK-2103][Streaming] Change to ClassTag for...

2014-08-01 Thread tdas
Github user tdas commented on the pull request: https://github.com/apache/spark/pull/1508#issuecomment-50852470 Jenkins, test this again. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request: [SPARK-1812] upgrade dependency to scala-loggi...

2014-08-01 Thread avati
Github user avati commented on a diff in the pull request: https://github.com/apache/spark/pull/1701#discussion_r15683518 --- Diff: sql/core/pom.xml --- @@ -83,6 +83,16 @@ artifactIdscalacheck_${scala.binary.version}/artifactId scopetest/scope

[GitHub] spark pull request: [SPARK-2786][mllib] Python correlations

2014-08-01 Thread dorx
GitHub user dorx opened a pull request: https://github.com/apache/spark/pull/1713 [SPARK-2786][mllib] Python correlations You can merge this pull request into a Git repository by running: $ git pull https://github.com/dorx/spark pythonCorrelation Alternatively you can review

[GitHub] spark pull request: [Spark 2557] fix LOCAL_N_REGEX in createTaskSc...

2014-08-01 Thread aarondav
Github user aarondav commented on the pull request: https://github.com/apache/spark/pull/1464#issuecomment-50853278 This seems totally reasonable, apologies for not seeing this. LGTM once Jenkins passes. --- If your project is set up for it, you can reply to this email and have your

[GitHub] spark pull request: [SPARK-2786][mllib] Python correlations

2014-08-01 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1713#issuecomment-50853328 QA tests have started for PR 1713. This patch merges cleanly. brView progress: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/17653/consoleFull ---

[GitHub] spark pull request: [SPARK-2033] Automatically cleanup checkpoint

2014-08-01 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/855#issuecomment-50853335 QA tests have started for PR 855. This patch merges cleanly. brView progress: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/17654/consoleFull ---

[GitHub] spark pull request: [SPARK-2786][mllib] Python correlations

2014-08-01 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1713#issuecomment-50853374 QA results for PR 1713:br- This patch FAILED unit tests.br- This patch merges cleanlybr- This patch adds the following public classes (experimental):brclass

[GitHub] spark pull request: [SPARK-2033] Automatically cleanup checkpoint

2014-08-01 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/855#issuecomment-50853502 QA results for PR 855:br- This patch FAILED unit tests.br- This patch merges cleanlybr- This patch adds no public classesbrbrFor more information see test

[GitHub] spark pull request: [Spark 2557] fix LOCAL_N_REGEX in createTaskSc...

2014-08-01 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1464#issuecomment-50853616 QA tests have started for PR 1464. This patch merges cleanly. brView progress: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/17655/consoleFull ---

[GitHub] spark pull request: [MLLIB] [spark-2352] Implementation of an 1-hi...

2014-08-01 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1290#issuecomment-50853746 QA results for PR 1290:br- This patch PASSES unit tests.br- This patch merges cleanlybr- This patch adds the following public classes (experimental):brabstract class

[GitHub] spark pull request: SPARK-2099. Report progress while task is runn...

2014-08-01 Thread sryza
Github user sryza commented on the pull request: https://github.com/apache/spark/pull/1056#issuecomment-50853874 Thanks @pwendell and @andrewor14 for your continued reviews. 10 seconds sounds fine to me. Not that it's a shining beacon of performance, but MapReduce actually

[GitHub] spark pull request: SPARK-2532: Minimal shuffle consolidation fixe...

2014-08-01 Thread aarondav
Github user aarondav commented on a diff in the pull request: https://github.com/apache/spark/pull/1678#discussion_r15683958 --- Diff: core/src/main/scala/org/apache/spark/storage/BlockObjectWriter.scala --- @@ -147,28 +147,36 @@ private[spark] class DiskBlockObjectWriter(

[GitHub] spark pull request: [SPARK-1997] update breeze to version 0.8.1

2014-08-01 Thread mengxr
Github user mengxr commented on the pull request: https://github.com/apache/spark/pull/940#issuecomment-50854138 @witgo Could you update the pom to exclude `commons-math3` from dependencies? I tried at local and LBFGS works well. It should be safe to remove `commons-math3`. For

[GitHub] spark pull request: SPARK-2532: Minimal shuffle consolidation fixe...

2014-08-01 Thread aarondav
Github user aarondav commented on a diff in the pull request: https://github.com/apache/spark/pull/1678#discussion_r15684100 --- Diff: core/src/main/scala/org/apache/spark/storage/BlockObjectWriter.scala --- @@ -147,28 +147,36 @@ private[spark] class DiskBlockObjectWriter(

[GitHub] spark pull request: [SPARK-2010] [PySpark] [SQL] support nested st...

2014-08-01 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1598#issuecomment-50854552 QA results for PR 1598:br- This patch PASSES unit tests.br- This patch merges cleanlybr- This patch adds the following public classes (experimental):brclass

[GitHub] spark pull request: [SPARK-2786][mllib] Python correlations

2014-08-01 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1713#issuecomment-50854587 QA tests have started for PR 1713. This patch merges cleanly. brView progress: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/17657/consoleFull ---

[GitHub] spark pull request: [SPARK-2635] Fix race condition at SchedulerBa...

2014-08-01 Thread kayousterhout
Github user kayousterhout commented on the pull request: https://github.com/apache/spark/pull/1525#issuecomment-50855070 I will take a look at this tomorrow. On Thu, Jul 31, 2014 at 10:37 PM, Zhihui Li notificati...@github.com wrote: @tgravescs

[GitHub] spark pull request: [SPARK-2670] FetchFailedException should be th...

2014-08-01 Thread mateiz
Github user mateiz commented on the pull request: https://github.com/apache/spark/pull/1578#issuecomment-50855127 Thanks for the changes! I've merged this in. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-1997] update breeze to version 0.8.1

2014-08-01 Thread witgo
Github user witgo commented on the pull request: https://github.com/apache/spark/pull/940#issuecomment-50855187 @mengxr done --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark pull request: [SPARK-1812] upgrade dependency to scala-loggi...

2014-08-01 Thread ScrapCodes
Github user ScrapCodes commented on the pull request: https://github.com/apache/spark/pull/1701#issuecomment-50855246 @pwendell Jenkins does not listen to my commands, Can you command it ? --- If your project is set up for it, you can reply to this email and have your reply appear on

[GitHub] spark pull request: [WIP] SPARK-2157 Ability to write tight firewa...

2014-08-01 Thread ash211
Github user ash211 commented on the pull request: https://github.com/apache/spark/pull/1107#issuecomment-50855345 I haven't tested this on an actual locked down cluster yet -- it's just been looking at netstat output so far On Jul 30, 2014 10:22 PM, Patrick Wendell

[GitHub] spark pull request: [SPARK-1997] update breeze to version 0.8.1

2014-08-01 Thread mengxr
Github user mengxr commented on a diff in the pull request: https://github.com/apache/spark/pull/940#discussion_r15684374 --- Diff: mllib/pom.xml --- @@ -60,6 +60,14 @@ groupIdjunit/groupId artifactIdjunit/artifactId /exclusion +

[GitHub] spark pull request: [SPARK-2786][mllib] Python correlations

2014-08-01 Thread mengxr
Github user mengxr commented on a diff in the pull request: https://github.com/apache/spark/pull/1713#discussion_r15684424 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/stat/correlation/Correlation.scala --- @@ -49,43 +49,48 @@ private[stat] trait Correlation { }

[GitHub] spark pull request: [SPARK-1997] update breeze to version 0.8.1

2014-08-01 Thread witgo
Github user witgo commented on a diff in the pull request: https://github.com/apache/spark/pull/940#discussion_r15684437 --- Diff: mllib/pom.xml --- @@ -60,6 +60,14 @@ groupIdjunit/groupId artifactIdjunit/artifactId /exclusion +

[GitHub] spark pull request: Streaming mllib [SPARK-2438][MLLIB]

2014-08-01 Thread freeman-lab
Github user freeman-lab commented on a diff in the pull request: https://github.com/apache/spark/pull/1361#discussion_r15684457 --- Diff: examples/src/main/scala/org/apache/spark/examples/mllib/StreamingLinearRegression.scala --- @@ -0,0 +1,56 @@ +/* + * Licensed to the

[GitHub] spark pull request: Streaming mllib [SPARK-2438][MLLIB]

2014-08-01 Thread freeman-lab
Github user freeman-lab commented on a diff in the pull request: https://github.com/apache/spark/pull/1361#discussion_r15684474 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/regression/StreamingLinearRegression.scala --- @@ -0,0 +1,104 @@ +/* + * Licensed to the

[GitHub] spark pull request: [SPARK-2678][Core] Prevents `spark-submit` fro...

2014-08-01 Thread liancheng
Github user liancheng commented on the pull request: https://github.com/apache/spark/pull/1699#issuecomment-50855873 Thanks for the great feedback, I agree, will update soon. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as

[GitHub] spark pull request: [WIP][SPARK-2316] Avoid O(blocks) operations i...

2014-08-01 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1679#issuecomment-50855850 QA tests have started for PR 1679. This patch merges cleanly. brView progress: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/17659/consoleFull ---

[GitHub] spark pull request: [SPARK-2490] Change recursive visiting on RDD ...

2014-08-01 Thread mateiz
Github user mateiz commented on the pull request: https://github.com/apache/spark/pull/1418#issuecomment-50855896 Jenkins, test this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-2670] FetchFailedException should be th...

2014-08-01 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/1578 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request: SPARK-2738. Remove redundant imports in BlockM...

2014-08-01 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/1642 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request: [SPARK-2678][Core] Prevents `spark-submit` fro...

2014-08-01 Thread liancheng
Github user liancheng commented on the pull request: https://github.com/apache/spark/pull/1699#issuecomment-50855977 Also, thank you @vanzin! You're right, it can be done in a downward compatible yet simple way. --- If your project is set up for it, you can reply to this email and

[GitHub] spark pull request: [SPARK-2490] Change recursive visiting on RDD ...

2014-08-01 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1418#issuecomment-50856200 QA tests have started for PR 1418. This patch merges cleanly. brView progress: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/17661/consoleFull ---

[GitHub] spark pull request: [SPARK-2786][mllib] Python correlations

2014-08-01 Thread mengxr
Github user mengxr commented on a diff in the pull request: https://github.com/apache/spark/pull/1713#discussion_r15684617 --- Diff: python/pyspark/mllib/stat.py --- @@ -0,0 +1,103 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one or more +#

[GitHub] spark pull request: [SPARK-2786][mllib] Python correlations

2014-08-01 Thread mengxr
Github user mengxr commented on a diff in the pull request: https://github.com/apache/spark/pull/1713#discussion_r15684640 --- Diff: python/pyspark/mllib/stat.py --- @@ -0,0 +1,103 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one or more +#

[GitHub] spark pull request: [SPARK-1470][SPARK-1842] Use the scala-logging...

2014-08-01 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1369#issuecomment-50856657 QA results for PR 1369:br- This patch FAILED unit tests.br- This patch merges cleanlybr- This patch adds no public classesbrbrFor more information see test

[GitHub] spark pull request: SPARK-2686 Add Length and Strlen support to Sp...

2014-08-01 Thread javadba
Github user javadba commented on the pull request: https://github.com/apache/spark/pull/1586#issuecomment-50856746 @ueshin I mostly agree except: let us keep the length which can be used for non-strings e.g. length(12345678) = 8 Then since length does handle

[GitHub] spark pull request: [SPARK-695] In DAGScheduler's getPreferredLocs...

2014-08-01 Thread mateiz
Github user mateiz commented on the pull request: https://github.com/apache/spark/pull/1362#issuecomment-50856749 Jenkins, test this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-695] In DAGScheduler's getPreferredLocs...

2014-08-01 Thread mateiz
Github user mateiz commented on a diff in the pull request: https://github.com/apache/spark/pull/1362#discussion_r15684786 --- Diff: core/src/main/scala/org/apache/spark/scheduler/DAGScheduler.scala --- @@ -1131,6 +1131,23 @@ class DAGScheduler( */ private[spark]

[GitHub] spark pull request: [SPARK-695] In DAGScheduler's getPreferredLocs...

2014-08-01 Thread mateiz
Github user mateiz commented on the pull request: https://github.com/apache/spark/pull/1362#issuecomment-50856865 Sorry for taking a bit of time to get to this, but it looks good. I'll merge it if the tests pass. --- If your project is set up for it, you can reply to this email and

[GitHub] spark pull request: [SPARK-695] In DAGScheduler's getPreferredLocs...

2014-08-01 Thread mateiz
Github user mateiz commented on a diff in the pull request: https://github.com/apache/spark/pull/1362#discussion_r15684804 --- Diff: core/src/test/scala/org/apache/spark/scheduler/DAGSchedulerSuite.scala --- @@ -291,6 +293,18 @@ class DAGSchedulerSuite extends

[GitHub] spark pull request: SPARK-2099. Report progress while task is runn...

2014-08-01 Thread sryza
Github user sryza commented on a diff in the pull request: https://github.com/apache/spark/pull/1056#discussion_r15684813 --- Diff: core/src/main/scala/org/apache/spark/SparkContext.scala --- @@ -991,6 +994,9 @@ class SparkContext(config: SparkConf) extends Logging {

[GitHub] spark pull request: [Spark 2557] fix LOCAL_N_REGEX in createTaskSc...

2014-08-01 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1464#issuecomment-50857070 QA results for PR 1464:br- This patch PASSES unit tests.br- This patch merges cleanlybr- This patch adds no public classesbrbrFor more information see test

[GitHub] spark pull request: SPARK-2134: Report metrics before application ...

2014-08-01 Thread mateiz
Github user mateiz commented on the pull request: https://github.com/apache/spark/pull/1076#issuecomment-50857088 Alright, I'm going to merge this as is then. Thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If

[GitHub] spark pull request: [Spark 2557] fix LOCAL_N_REGEX in createTaskSc...

2014-08-01 Thread aarondav
Github user aarondav commented on the pull request: https://github.com/apache/spark/pull/1464#issuecomment-50857152 Merging into master. Thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [WIP][SPARK-2316] Avoid O(blocks) operations i...

2014-08-01 Thread andrewor14
Github user andrewor14 commented on the pull request: https://github.com/apache/spark/pull/1679#issuecomment-50857283 I did some benchmarking by running the following job 100 times one immediately after another. Each job launches a many short-lived tasks, each of which persists a

[GitHub] spark pull request: [SQL][SPARK-2212]Hash Outer Join

2014-08-01 Thread chenghao-intel
Github user chenghao-intel commented on the pull request: https://github.com/apache/spark/pull/1147#issuecomment-50857406 Thank you @marmbrus I've updated the code as suggested. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as

[GitHub] spark pull request: SPARK-2294: fix locality inversion bug in Task...

2014-08-01 Thread mateiz
Github user mateiz commented on a diff in the pull request: https://github.com/apache/spark/pull/1313#discussion_r15684978 --- Diff: core/src/main/scala/org/apache/spark/scheduler/TaskSetManager.scala --- @@ -341,20 +354,27 @@ private[spark] class TaskSetManager( *

[GitHub] spark pull request: SPARK-2294: fix locality inversion bug in Task...

2014-08-01 Thread mateiz
Github user mateiz commented on the pull request: https://github.com/apache/spark/pull/1313#issuecomment-50857528 Awesome, looks like this passes now! What was it, was there a bug or tests being flaky? --- If your project is set up for it, you can reply to this email and have your

[GitHub] spark pull request: [SPARK-2786][mllib] Python correlations

2014-08-01 Thread mengxr
Github user mengxr commented on a diff in the pull request: https://github.com/apache/spark/pull/1713#discussion_r15685005 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/api/python/PythonMLLibAPI.scala --- @@ -456,6 +458,37 @@ class PythonMLLibAPI extends Serializable {

[GitHub] spark pull request: [SQL][SPARK-2212]Hash Outer Join

2014-08-01 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1147#issuecomment-50857544 QA tests have started for PR 1147. This patch merges cleanly. brView progress: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/17663/consoleFull ---

[GitHub] spark pull request: [SPARK-2786][mllib] Python correlations

2014-08-01 Thread mengxr
Github user mengxr commented on a diff in the pull request: https://github.com/apache/spark/pull/1713#discussion_r15684998 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/api/python/PythonMLLibAPI.scala --- @@ -456,6 +458,37 @@ class PythonMLLibAPI extends Serializable {

[GitHub] spark pull request: [SPARK-2786][mllib] Python correlations

2014-08-01 Thread mengxr
Github user mengxr commented on a diff in the pull request: https://github.com/apache/spark/pull/1713#discussion_r15685050 --- Diff: mllib/src/test/scala/org/apache/spark/mllib/api/python/PythonMLLibAPISuite.scala --- @@ -59,10 +59,25 @@ class PythonMLLibAPISuite extends FunSuite

[GitHub] spark pull request: [SPARK-2786][mllib] Python correlations

2014-08-01 Thread mengxr
Github user mengxr commented on a diff in the pull request: https://github.com/apache/spark/pull/1713#discussion_r15685067 --- Diff: mllib/src/test/scala/org/apache/spark/mllib/api/python/PythonMLLibAPISuite.scala --- @@ -59,10 +59,25 @@ class PythonMLLibAPISuite extends FunSuite

[GitHub] spark pull request: [SPARK-2033] Automatically cleanup checkpoint

2014-08-01 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/855#issuecomment-5085 QA results for PR 855:br- This patch PASSES unit tests.br- This patch merges cleanlybr- This patch adds no public classesbrbrFor more information see test

[GitHub] spark pull request: [SPARK-1812] upgrade dependency to scala-loggi...

2014-08-01 Thread ScrapCodes
Github user ScrapCodes commented on the pull request: https://github.com/apache/spark/pull/1701#issuecomment-50857755 LGTM. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark pull request: [SPARK-2786][mllib] Python correlations

2014-08-01 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1713#issuecomment-50858129 QA results for PR 1713:br- This patch PASSES unit tests.br- This patch merges cleanlybr- This patch adds the following public classes (experimental):brclass

[GitHub] spark pull request: SPARK-2686 Add Length and Strlen support to Sp...

2014-08-01 Thread marmbrus
Github user marmbrus commented on the pull request: https://github.com/apache/spark/pull/1586#issuecomment-50858199 let us keep the length which can be used for non-strings e.g. length(12345678) = 8 Non-string length as described above should probably be handled by

[GitHub] spark pull request: SPARK-2686 Add Length and Strlen support to Sp...

2014-08-01 Thread marmbrus
Github user marmbrus commented on the pull request: https://github.com/apache/spark/pull/1586#issuecomment-50858363 Also, `CharLength` for the `Expression` seems less ambiguous. Though we can certainly alias to `length` in the parser for compatibility. --- If your project is set up

[GitHub] spark pull request: SPARK-2740: allow user to specify ascending an...

2014-08-01 Thread lirui-intel
Github user lirui-intel commented on the pull request: https://github.com/apache/spark/pull/1645#issuecomment-50858418 Thanks @JoshRosen :) --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: SPARK-2686 Add Length and Strlen support to Sp...

2014-08-01 Thread javadba
Github user javadba commented on the pull request: https://github.com/apache/spark/pull/1586#issuecomment-50858453 @marmbrus OK fine with that. Then given the inputs from ueshin, we are presently at: len(gth)/char_length : take a single string argument and use

[GitHub] spark pull request: SPARK-2099. Report progress while task is runn...

2014-08-01 Thread pwendell
Github user pwendell commented on a diff in the pull request: https://github.com/apache/spark/pull/1056#discussion_r15685394 --- Diff: core/src/main/scala/org/apache/spark/scheduler/TaskSchedulerImpl.scala --- @@ -320,6 +323,26 @@ private[spark] class TaskSchedulerImpl( }

[GitHub] spark pull request: SPARK-2686 Add Length and Strlen support to Sp...

2014-08-01 Thread javadba
Github user javadba commented on the pull request: https://github.com/apache/spark/pull/1586#issuecomment-50858624 @marmbrusRE: Charlength for the expression - also fine, will do. (btw how did you highlight in the comment?) --- If your project is set up for it, you can reply

[GitHub] spark pull request: [SPARK-1812] core - upgrade to json4s-jackson ...

2014-08-01 Thread ScrapCodes
Github user ScrapCodes commented on the pull request: https://github.com/apache/spark/pull/1702#issuecomment-50858704 test this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request: [SPARK-1812] core - upgrade to json4s-jackson ...

2014-08-01 Thread ScrapCodes
Github user ScrapCodes commented on the pull request: https://github.com/apache/spark/pull/1702#issuecomment-50858698 LGTM, if jenkins build succeeds. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does

[GitHub] spark pull request: SPARK-2686 Add Length and Strlen support to Sp...

2014-08-01 Thread ueshin
Github user ueshin commented on the pull request: https://github.com/apache/spark/pull/1586#issuecomment-50858955 Oops, I had forgotten that Hive's `Length` can handle binary type. It would be better to use `Length` instead of `CharLength` and make it handle binary type. --- If

[GitHub] spark pull request: [SPARK-2010] [PySpark] [SQL] support nested st...

2014-08-01 Thread davies
Github user davies commented on the pull request: https://github.com/apache/spark/pull/1598#issuecomment-50859069 @marmbrus @mateiz @JoshRosen Could you take another look as this? I had removed schema as string, and added Row as the suggested type to infer schema. --- If your

[GitHub] spark pull request: [SPARK-2316] Avoid O(blocks) operations in lis...

2014-08-01 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1679#issuecomment-50859282 QA results for PR 1679:br- This patch FAILED unit tests.br- This patch merges cleanlybr- This patch adds the following public classes (experimental):brclass

[GitHub] spark pull request: [SPARK-1997] update breeze to version 0.8.1

2014-08-01 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/940#issuecomment-50859290 QA results for PR 940:br- This patch FAILED unit tests.br- This patch merges cleanlybr- This patch adds no public classesbrbrFor more information see test

[GitHub] spark pull request: [SPARK-2490] Change recursive visiting on RDD ...

2014-08-01 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1418#issuecomment-50859287 QA results for PR 1418:br- This patch FAILED unit tests.br- This patch merges cleanlybr- This patch adds no public classesbrbrFor more information see test

[GitHub] spark pull request: SPARK-2099. Report progress while task is runn...

2014-08-01 Thread pwendell
Github user pwendell commented on the pull request: https://github.com/apache/spark/pull/1056#issuecomment-50859536 Yeah I think it's fine to put that too another patch and just make it something a bit more on the conservative side (10 seconds) for now. --- If your project is set up

[GitHub] spark pull request: [SPARK-2700] [SQL] Hidden files (such as .impa...

2014-08-01 Thread chutium
Github user chutium commented on the pull request: https://github.com/apache/spark/pull/1691#issuecomment-50859555 done, thanks --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark pull request: [MLLIB] [spark-2352] Implementation of an 1-hi...

2014-08-01 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1290#issuecomment-50859737 QA tests have started for PR 1290. This patch merges cleanly. brView progress: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/17665/consoleFull ---

[GitHub] spark pull request: [SPARK-983] Support external sorting

2014-08-01 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/1090 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request: Fix JIRA-983 and support exteranl sort for sor...

2014-08-01 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/931 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request: SPARK-983. Support external sorting in sortByK...

2014-08-01 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/1677 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request: [Spark 2557] fix LOCAL_N_REGEX in createTaskSc...

2014-08-01 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/1464 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request: SPARK-2134: Report metrics before application ...

2014-08-01 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/1076 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request: [SPARK-1981] Add AWS Kinesis streaming support

2014-08-01 Thread cfregly
Github user cfregly commented on a diff in the pull request: https://github.com/apache/spark/pull/1434#discussion_r15685837 --- Diff: extras/kinesis-asl/src/main/scala/org/apache/spark/streaming/kinesis/KinesisRecordProcessorUtils.scala --- @@ -0,0 +1,79 @@ +/* + *

[GitHub] spark pull request: [SPARK-1981] Add AWS Kinesis streaming support

2014-08-01 Thread cfregly
Github user cfregly commented on a diff in the pull request: https://github.com/apache/spark/pull/1434#discussion_r15685828 --- Diff: extras/kinesis-asl/src/main/scala/org/apache/spark/streaming/kinesis/KinesisStringRecordSerializer.scala --- @@ -0,0 +1,44 @@ +/* + *

[GitHub] spark pull request: [SPARK-1981] Add AWS Kinesis streaming support

2014-08-01 Thread cfregly
Github user cfregly commented on a diff in the pull request: https://github.com/apache/spark/pull/1434#discussion_r15685855 --- Diff: examples/src/main/scala/org/apache/spark/examples/streaming/KinesisWordCount.scala --- @@ -0,0 +1,369 @@ +/* + * Licensed to the Apache

[GitHub] spark pull request: [SPARK-1981] Add AWS Kinesis streaming support

2014-08-01 Thread cfregly
Github user cfregly commented on a diff in the pull request: https://github.com/apache/spark/pull/1434#discussion_r15685887 --- Diff: examples/src/main/scala/org/apache/spark/examples/streaming/KinesisWordCount.scala --- @@ -0,0 +1,369 @@ +/* + * Licensed to the Apache

[GitHub] spark pull request: [SPARK-1981] Add AWS Kinesis streaming support

2014-08-01 Thread cfregly
Github user cfregly commented on a diff in the pull request: https://github.com/apache/spark/pull/1434#discussion_r15685865 --- Diff: examples/src/main/scala/org/apache/spark/examples/streaming/KinesisWordCount.scala --- @@ -0,0 +1,369 @@ +/* + * Licensed to the Apache

[GitHub] spark pull request: [SPARK-1997] update breeze to version 0.8.1

2014-08-01 Thread witgo
Github user witgo commented on the pull request: https://github.com/apache/spark/pull/940#issuecomment-50860276 Jenkins, retest this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-1997] update breeze to version 0.8.1

2014-08-01 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/940#issuecomment-50860395 QA tests have started for PR 940. This patch merges cleanly. brView progress: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/17666/consoleFull ---

[GitHub] spark pull request: [SPARK-2729] [SQL] Forgot to match Timestamp t...

2014-08-01 Thread liancheng
Github user liancheng commented on the pull request: https://github.com/apache/spark/pull/1636#issuecomment-50861117 @marmbrus I've discussed with @chutium offline. Since 1.1 code freeze deadline is near, we can merge this first. I tested this PR locally and it looks good. Will

[GitHub] spark pull request: SPARK-2099. Report progress while task is runn...

2014-08-01 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1056#issuecomment-50861124 QA tests have started for PR 1056. This patch merges cleanly. brView progress: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/17667/consoleFull ---

[GitHub] spark pull request: SPARK-2099. Report progress while task is runn...

2014-08-01 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1056#issuecomment-50861360 QA results for PR 1056:br- This patch FAILED unit tests.br- This patch merges cleanlybr- This patch adds the following public classes (experimental):brcase class

[GitHub] spark pull request: SPARK-2099. Report progress while task is runn...

2014-08-01 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1056#issuecomment-50861872 QA tests have started for PR 1056. This patch merges cleanly. brView progress: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/17668/consoleFull ---

  1   2   3   4   5   6   7   8   >