[GitHub] spark pull request: SPARK-4203 Partition directories in random ord...

2014-11-03 Thread marmbrus
Github user marmbrus commented on the pull request: https://github.com/apache/spark/pull/3076#issuecomment-61564976 /cc @chenglian --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request: fixed MLlib Naive-Bayes java example bug

2014-11-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/3081#issuecomment-61565165 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-3573][MLLIB] Make MLlib's Vector compat...

2014-11-03 Thread jkbradley
Github user jkbradley commented on a diff in the pull request: https://github.com/apache/spark/pull/3070#discussion_r19773840 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/linalg/Vectors.scala --- @@ -75,6 +80,73 @@ sealed trait Vector extends Serializable { }

[GitHub] spark pull request: [SPARK-3573][MLLIB] Make MLlib's Vector compat...

2014-11-03 Thread jkbradley
Github user jkbradley commented on a diff in the pull request: https://github.com/apache/spark/pull/3070#discussion_r19773847 --- Diff: python/pyspark/sql.py --- @@ -460,6 +527,13 @@ def _parse_datatype_json_string(json_string): ...

[GitHub] spark pull request: [SPARK-3573][MLLIB] Make MLlib's Vector compat...

2014-11-03 Thread jkbradley
Github user jkbradley commented on a diff in the pull request: https://github.com/apache/spark/pull/3070#discussion_r19773846 --- Diff: python/pyspark/sql.py --- @@ -408,6 +408,73 @@ def fromJson(cls, json): return StructType([StructField.fromJson(f) for f in

[GitHub] spark pull request: [SPARK-1021] Defer the data-driven computation...

2014-11-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/3079#issuecomment-61565401 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-1021] Defer the data-driven computation...

2014-11-03 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3079#issuecomment-61565392 [Test build #22828 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/22828/consoleFull) for PR 3079 at commit

[GitHub] spark pull request: [SPARK-3778] newAPIHadoopRDD doesn't properly ...

2014-11-03 Thread pwendell
Github user pwendell commented on a diff in the pull request: https://github.com/apache/spark/pull/2676#discussion_r19773894 --- Diff: core/src/main/scala/org/apache/spark/SparkContext.scala --- @@ -641,6 +641,7 @@ class SparkContext(config: SparkConf) extends Logging {

[GitHub] spark pull request: [SPARK-4145] Web UI job pages

2014-11-03 Thread kayousterhout
Github user kayousterhout commented on a diff in the pull request: https://github.com/apache/spark/pull/3009#discussion_r19773955 --- Diff: core/src/main/scala/org/apache/spark/ui/jobs/AllJobsPage.scala --- @@ -0,0 +1,139 @@ +/* + * Licensed to the Apache Software

[GitHub] spark pull request: [SPARK-3778] newAPIHadoopRDD doesn't properly ...

2014-11-03 Thread pwendell
Github user pwendell commented on a diff in the pull request: https://github.com/apache/spark/pull/2676#discussion_r19773950 --- Diff: core/src/main/scala/org/apache/spark/SparkContext.scala --- @@ -661,7 +662,10 @@ class SparkContext(config: SparkConf) extends Logging {

[GitHub] spark pull request: [SQL] Convert arguments to Scala UDFs

2014-11-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/3077#issuecomment-61565726 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SQL] Convert arguments to Scala UDFs

2014-11-03 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3077#issuecomment-61565719 [Test build #22825 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/22825/consoleFull) for PR 3077 at commit

[GitHub] spark pull request: [SPARK-4145] Web UI job pages

2014-11-03 Thread kayousterhout
Github user kayousterhout commented on a diff in the pull request: https://github.com/apache/spark/pull/3009#discussion_r19774251 --- Diff: core/src/main/scala/org/apache/spark/ui/jobs/JobPage.scala --- @@ -0,0 +1,101 @@ +/* + * Licensed to the Apache Software Foundation

[GitHub] spark pull request: [SPARK-3270] Spark API for Application Extensi...

2014-11-03 Thread mmalohlava
Github user mmalohlava commented on the pull request: https://github.com/apache/spark/pull/2691#issuecomment-61567145 Sorry for delayed answer. I was trying to provide better solution without modifying Spark. However, regarding Sean's question: * In our case we need to

[GitHub] spark pull request: [SPARK-4192][SQL] Internal API for Python UDT

2014-11-03 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3068#issuecomment-61567252 [Test build #22832 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/22832/consoleFull) for PR 3068 at commit

[GitHub] spark pull request: [SPARK-4145] Web UI job pages

2014-11-03 Thread kayousterhout
Github user kayousterhout commented on the pull request: https://github.com/apache/spark/pull/3009#issuecomment-61567337 @JoshRosen this looks great!! One thing that seems missing here is some kind of status bar on the jobs page. I know there was a bunch of debate on what an

[GitHub] spark pull request: [SQL] Convert arguments to Scala UDFs

2014-11-03 Thread mengxr
Github user mengxr commented on the pull request: https://github.com/apache/spark/pull/3077#issuecomment-61567570 This is great! LGTM. Thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: SPARK-4178. Hadoop input metrics ignore bytes ...

2014-11-03 Thread pwendell
Github user pwendell commented on the pull request: https://github.com/apache/spark/pull/3045#issuecomment-61567648 LGTM - thanks @ksakellis and @sryza. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark pull request: [SPARK-3886] [PySpark] simplify serializer, us...

2014-11-03 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2920#issuecomment-61567627 [Test build #22826 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/22826/consoleFull) for PR 2920 at commit

[GitHub] spark pull request: [SPARK-3886] [PySpark] simplify serializer, us...

2014-11-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/2920#issuecomment-61567632 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: SPARK-4178. Hadoop input metrics ignore bytes ...

2014-11-03 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/3045 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request: [SPARK-3957]: show broadcast variable resource...

2014-11-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/2851#issuecomment-61569172 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-3957]: show broadcast variable resource...

2014-11-03 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2851#issuecomment-61569165 [Test build #22829 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/22829/consoleFull) for PR 2851 at commit

[GitHub] spark pull request: [SPARK-3886] [PySpark] simplify serializer, us...

2014-11-03 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2920#issuecomment-61569440 [Test build #22827 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/22827/consoleFull) for PR 2920 at commit

[GitHub] spark pull request: [SPARK-3886] [PySpark] simplify serializer, us...

2014-11-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/2920#issuecomment-61569447 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-4186] add binaryFiles and binaryRecords...

2014-11-03 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3078#issuecomment-61569973 [Test build #509 has finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/509/consoleFull) for PR 3078 at commit

[GitHub] spark pull request: [SPARK-732][SPARK-3628][CORE][RESUBMIT] elimin...

2014-11-03 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2524#issuecomment-61570767 [Test build #22830 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/22830/consoleFull) for PR 2524 at commit

[GitHub] spark pull request: [SPARK-732][SPARK-3628][CORE][RESUBMIT] elimin...

2014-11-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/2524#issuecomment-61570773 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: fixed MLlib Naive-Bayes java example bug

2014-11-03 Thread mengxr
Github user mengxr commented on a diff in the pull request: https://github.com/apache/spark/pull/3081#discussion_r19777251 --- Diff: docs/mllib-naive-bayes.md --- @@ -90,7 +90,7 @@ JavaPairRDDDouble, Double predictionAndLabel = }); double accuracy = 1.0 *

[GitHub] spark pull request: [SPARK-611] Display executor thread dumps in w...

2014-11-03 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2944#issuecomment-61573067 [Test build #22831 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/22831/consoleFull) for PR 2944 at commit

[GitHub] spark pull request: [SPARK-611] Display executor thread dumps in w...

2014-11-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/2944#issuecomment-61573077 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-611] Display executor thread dumps in w...

2014-11-03 Thread andrewor14
Github user andrewor14 commented on the pull request: https://github.com/apache/spark/pull/2944#issuecomment-61573903 LGTM --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark pull request: Support for mesos DockerInfo in fine-grained m...

2014-11-03 Thread tnachen
Github user tnachen commented on the pull request: https://github.com/apache/spark/pull/3074#issuecomment-61575462 Very cool! I don't know what the Spark code style are, so can't really give any code wise comments. Have you tried this out already? And also there should be

[GitHub] spark pull request: [SPARK-4192][SQL] Internal API for Python UDT

2014-11-03 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3068#issuecomment-61575655 [Test build #22833 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/22833/consoleFull) for PR 3068 at commit

[GitHub] spark pull request: [WIP][SPARK-3797] Run external shuffle service...

2014-11-03 Thread andrewor14
GitHub user andrewor14 opened a pull request: https://github.com/apache/spark/pull/3082 [WIP][SPARK-3797] Run external shuffle service in Yarn NM This creates a new module `network/yarn` that depends on `network/shuffle` recently created in #3001. This PR introduces a custom Yarn

[GitHub] spark pull request: [SPARK-2261] Make event logger use a single fi...

2014-11-03 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1222#issuecomment-61575694 [Test build #22834 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/22834/consoleFull) for PR 1222 at commit

[GitHub] spark pull request: [WIP][SPARK-3797] Run external shuffle service...

2014-11-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/3082#issuecomment-61575888 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [WIP][SPARK-3797] Run external shuffle service...

2014-11-03 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3082#issuecomment-61575877 [Test build #22835 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/22835/consoleFull) for PR 3082 at commit

[GitHub] spark pull request: [WIP][SPARK-3797] Run external shuffle service...

2014-11-03 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3082#issuecomment-61575887 [Test build #22835 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/22835/consoleFull) for PR 3082 at commit

[GitHub] spark pull request: [SPARK-4048] Enhance and extend hadoop-provide...

2014-11-03 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2982#issuecomment-61576093 [Test build #22836 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/22836/consoleFull) for PR 2982 at commit

[GitHub] spark pull request: [SPARK-4048] Enhance and extend hadoop-provide...

2014-11-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/2982#issuecomment-61576179 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-4048] Enhance and extend hadoop-provide...

2014-11-03 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2982#issuecomment-61576178 [Test build #22836 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/22836/consoleFull) for PR 2982 at commit

[GitHub] spark pull request: [WIP][SPARK-3797] Run external shuffle service...

2014-11-03 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3082#issuecomment-61576335 [Test build #22837 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/22837/consoleFull) for PR 3082 at commit

[GitHub] spark pull request: [SPARK-4213] ParquetFilters - No support for L...

2014-11-03 Thread sarutak
GitHub user sarutak opened a pull request: https://github.com/apache/spark/pull/3083 [SPARK-4213] ParquetFilters - No support for LT, LTE, GT, GTE operators Following description is quoted from JIRA: When I issue a hql query against a HiveContext where my predicate uses a

[GitHub] spark pull request: [FIX][MLLIB] fix seed in BaggedPointSuite

2014-11-03 Thread mengxr
GitHub user mengxr opened a pull request: https://github.com/apache/spark/pull/3084 [FIX][MLLIB] fix seed in BaggedPointSuite Saw Jenkins test failures due to random seeds. @jkbradley You can merge this pull request into a Git repository by running: $ git pull

[GitHub] spark pull request: [SPARK-4213] ParquetFilters - No support for L...

2014-11-03 Thread sarutak
Github user sarutak commented on the pull request: https://github.com/apache/spark/pull/3083#issuecomment-61576851 CC @marmbrus --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark pull request: [WIP][SPARK-3797] Run external shuffle service...

2014-11-03 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/3082#discussion_r19779280 --- Diff: core/src/main/scala/org/apache/spark/ExecutorAllocationManager.scala --- @@ -124,6 +126,22 @@ private[spark] class ExecutorAllocationManager(sc:

[GitHub] spark pull request: [WIP][SPARK-3797] Run external shuffle service...

2014-11-03 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/3082#discussion_r19779296 --- Diff: core/src/main/scala/org/apache/spark/ExecutorAllocationManager.scala --- @@ -124,6 +126,22 @@ private[spark] class ExecutorAllocationManager(sc:

[GitHub] spark pull request: [SPARK-4048] Enhance and extend hadoop-provide...

2014-11-03 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2982#issuecomment-61577033 [Test build #22838 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/22838/consoleFull) for PR 2982 at commit

[GitHub] spark pull request: [FIX][MLLIB] fix seed in BaggedPointSuite

2014-11-03 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3084#issuecomment-61577191 [Test build #22839 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/22839/consoleFull) for PR 3084 at commit

[GitHub] spark pull request: [WIP][SPARK-3797] Run external shuffle service...

2014-11-03 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/3082#discussion_r19779445 --- Diff: core/src/main/scala/org/apache/spark/ExecutorAllocationManager.scala --- @@ -124,6 +126,22 @@ private[spark] class ExecutorAllocationManager(sc:

[GitHub] spark pull request: [FIX][MLLIB] fix seed in BaggedPointSuite

2014-11-03 Thread manishamde
Github user manishamde commented on the pull request: https://github.com/apache/spark/pull/3084#issuecomment-61577226 @mengxr Sorry, it's my fault. It looks good to me. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If

[GitHub] spark pull request: [WIP][SPARK-3797] Run external shuffle service...

2014-11-03 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/3082#discussion_r19779489 --- Diff: core/src/main/scala/org/apache/spark/ExecutorAllocationManager.scala --- @@ -124,6 +126,22 @@ private[spark] class ExecutorAllocationManager(sc:

[GitHub] spark pull request: [SPARK-4213][SQL] ParquetFilters - No support ...

2014-11-03 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3083#issuecomment-61577408 [Test build #22840 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/22840/consoleFull) for PR 3083 at commit

[GitHub] spark pull request: [WIP][SPARK-3797] Run external shuffle service...

2014-11-03 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/3082#discussion_r19779621 --- Diff: core/src/main/scala/org/apache/spark/storage/BlockManager.scala --- @@ -92,7 +93,19 @@ private[spark] class BlockManager(

[GitHub] spark pull request: [WIP][SPARK-3797] Run external shuffle service...

2014-11-03 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/3082#discussion_r19779863 --- Diff: network/yarn/src/main/java/org/apache/spark/network/yarn/YarnShuffleService.java --- @@ -0,0 +1,103 @@ +/* + * Licensed to the Apache

[GitHub] spark pull request: [WIP][SPARK-3797] Run external shuffle service...

2014-11-03 Thread vanzin
Github user vanzin commented on the pull request: https://github.com/apache/spark/pull/3082#issuecomment-61578165 LGTM aside some nits; I still have to catch up with the rest of the network code changes to make more informed comments, though. --- If your project is set up for it,

[GitHub] spark pull request: [WIP][SPARK-3797] Run external shuffle service...

2014-11-03 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/3082#discussion_r19779981 --- Diff: pom.xml --- @@ -93,6 +93,7 @@ moduletools/module modulenetwork/common/module modulenetwork/shuffle/module +

[GitHub] spark pull request: [SPARK-4192][SQL] Internal API for Python UDT

2014-11-03 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3068#issuecomment-61578547 [Test build #22832 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/22832/consoleFull) for PR 3068 at commit

[GitHub] spark pull request: [SPARK-4192][SQL] Internal API for Python UDT

2014-11-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/3068#issuecomment-61578558 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [WIP][SPARK-3797] Run external shuffle service...

2014-11-03 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/3082#discussion_r19780190 --- Diff: core/src/main/scala/org/apache/spark/storage/BlockManager.scala --- @@ -92,7 +93,19 @@ private[spark] class BlockManager(

[GitHub] spark pull request: [WIP][SPARK-3797] Run external shuffle service...

2014-11-03 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/3082#discussion_r19780219 --- Diff: network/yarn/src/main/java/org/apache/spark/network/yarn/YarnShuffleService.java --- @@ -0,0 +1,103 @@ +/* + * Licensed to the Apache

[GitHub] spark pull request: [SPARK-4186] add binaryFiles and binaryRecords...

2014-11-03 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3078#issuecomment-61579504 [Test build #22842 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/22842/consoleFull) for PR 3078 at commit

[GitHub] spark pull request: [SPARK-3573][MLLIB] Make MLlib's Vector compat...

2014-11-03 Thread jkbradley
Github user jkbradley commented on a diff in the pull request: https://github.com/apache/spark/pull/3070#discussion_r19780608 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/test/ExamplePointUDT.scala --- @@ -0,0 +1,64 @@ +/* + * Licensed to the Apache Software

[GitHub] spark pull request: [SPARK-3573][MLLIB] Make MLlib's Vector compat...

2014-11-03 Thread jkbradley
Github user jkbradley commented on the pull request: https://github.com/apache/spark/pull/3070#issuecomment-61579917 LGTM though I'll depend on @davies for feedback on the Python API on the other PR [https://github.com/apache/spark/pull/3068] --- If your project is set up for it,

[GitHub] spark pull request: [SPARK-4186] add binaryFiles and binaryRecords...

2014-11-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/3078#issuecomment-61580109 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [WIP][SPARK-3797] Run external shuffle service...

2014-11-03 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/3082#discussion_r19780952 --- Diff: core/src/main/scala/org/apache/spark/ExecutorAllocationManager.scala --- @@ -124,6 +126,22 @@ private[spark] class ExecutorAllocationManager(sc:

[GitHub] spark pull request: [SPARK-4192][SQL] Internal API for Python UDT

2014-11-03 Thread davies
Github user davies commented on the pull request: https://github.com/apache/spark/pull/3068#issuecomment-61580666 LGTM, let's ship it. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request: [SPARK-4186] add binaryFiles and binaryRecords...

2014-11-03 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3078#issuecomment-61580680 [Test build #510 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/510/consoleFull) for PR 3078 at commit

[GitHub] spark pull request: [SPARK-3954][Streaming] promote the speed of c...

2014-11-03 Thread tianyi
Github user tianyi commented on the pull request: https://github.com/apache/spark/pull/2811#issuecomment-61580910 I think this PR is more concentrate on logical optimization, not speed. Spark used three iterations to get the fileRDD list which is not necessary. --- If your project

[GitHub] spark pull request: [SPARK-4168][WebUI] web statges number should ...

2014-11-03 Thread liyezhang556520
Github user liyezhang556520 commented on the pull request: https://github.com/apache/spark/pull/3035#issuecomment-61580908 @andrewor14 , can you help to check whether this patch is fine now? --- If your project is set up for it, you can reply to this email and have your reply appear

[GitHub] spark pull request: [SPARK-4166][Core] Add a backward compatibilit...

2014-11-03 Thread zsxwing
Github user zsxwing commented on the pull request: https://github.com/apache/spark/pull/3085#issuecomment-61581027 /cc @andrewor14 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request: [SPARK-4166][Core] Add a backward compatibilit...

2014-11-03 Thread zsxwing
GitHub user zsxwing opened a pull request: https://github.com/apache/spark/pull/3085 [SPARK-4166][Core] Add a backward compatibility test for ExecutorLostFailure You can merge this pull request into a Git repository by running: $ git pull https://github.com/zsxwing/spark

[GitHub] spark pull request: [SPARK-4166][Core][WebUI] Display the executor...

2014-11-03 Thread zsxwing
Github user zsxwing commented on the pull request: https://github.com/apache/spark/pull/3033#issuecomment-61581056 Already added it in #3085 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: Support for mesos DockerInfo in fine-grained m...

2014-11-03 Thread hellertime
Github user hellertime commented on the pull request: https://github.com/apache/spark/pull/3074#issuecomment-61581145 This code has been in use for a while now. I'm currently building a workflow reliant on the ability for spark to spin up its tasks inside docker. Not sure

[GitHub] spark pull request: [SPARK-4151][SQL] Add string operation functio...

2014-11-03 Thread tianyi
Github user tianyi commented on a diff in the pull request: https://github.com/apache/spark/pull/2998#discussion_r19781336 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/stringOperations.scala --- @@ -49,7 +49,10 @@ trait StringRegexExpression {

[GitHub] spark pull request: [SPARK-4151][SQL] Add string operation functio...

2014-11-03 Thread tianyi
Github user tianyi commented on a diff in the pull request: https://github.com/apache/spark/pull/2998#discussion_r19781402 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/SQLQuerySuite.scala --- @@ -500,6 +500,23 @@ class SQLQuerySuite extends QueryTest with

[GitHub] spark pull request: [WIP][SPARK-3797] Run external shuffle service...

2014-11-03 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3082#issuecomment-61581363 [Test build #22843 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/22843/consoleFull) for PR 3082 at commit

[GitHub] spark pull request: [SPARK-4017] show progress bar in console and ...

2014-11-03 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3029#issuecomment-61581372 [Test build #22845 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/22845/consoleFull) for PR 3029 at commit

[GitHub] spark pull request: [WIP][SPARK-3797] Run external shuffle service...

2014-11-03 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/3082#discussion_r19781450 --- Diff: pom.xml --- @@ -93,6 +93,7 @@ moduletools/module modulenetwork/common/module modulenetwork/shuffle/module +

[GitHub] spark pull request: [SPARK-4166][Core] Add a backward compatibilit...

2014-11-03 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3085#issuecomment-61581580 [Test build #22844 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/22844/consoleFull) for PR 3085 at commit

[GitHub] spark pull request: [FIX][MLLIB] fix seed in BaggedPointSuite

2014-11-03 Thread jkbradley
Github user jkbradley commented on the pull request: https://github.com/apache/spark/pull/3084#issuecomment-61581589 LGTM --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] spark pull request: Support for mesos DockerInfo in fine-grained m...

2014-11-03 Thread tnachen
Github user tnachen commented on the pull request: https://github.com/apache/spark/pull/3074#issuecomment-61581607 @hellertime yes I was referring to the coarse mode, I think it should be straight forward enough to try to get coarse mode included in this patch (as I learned getting a

[GitHub] spark pull request: [WIP][SPARK-3797] Run external shuffle service...

2014-11-03 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3082#issuecomment-61581606 [Test build #22837 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/22837/consoleFull) for PR 3082 at commit

[GitHub] spark pull request: [WIP][SPARK-3797] Run external shuffle service...

2014-11-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/3082#issuecomment-61581614 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-4017] show progress bar in console and ...

2014-11-03 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3029#issuecomment-61581788 [Test build #22846 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/22846/consoleFull) for PR 3029 at commit

[GitHub] spark pull request: Shading the Jetty dependency.

2014-11-03 Thread mccheah
Github user mccheah commented on the pull request: https://github.com/apache/spark/pull/2984#issuecomment-61582083 Jenkins, test this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: Support for mesos DockerInfo in fine-grained m...

2014-11-03 Thread timothysc
Github user timothysc commented on the pull request: https://github.com/apache/spark/pull/3074#issuecomment-61582163 +1 try to be as comprehensive as possible. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-4213][SQL] ParquetFilters - No support ...

2014-11-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/3083#issuecomment-61582213 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-4213][SQL] ParquetFilters - No support ...

2014-11-03 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3083#issuecomment-61582205 [Test build #22840 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/22840/consoleFull) for PR 3083 at commit

[GitHub] spark pull request: [SPARK-4151][SQL] Add string operation functio...

2014-11-03 Thread tianyi
Github user tianyi commented on a diff in the pull request: https://github.com/apache/spark/pull/2998#discussion_r19781741 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/stringOperations.scala --- @@ -49,7 +49,10 @@ trait StringRegexExpression {

[GitHub] spark pull request: Support for mesos DockerInfo in fine-grained m...

2014-11-03 Thread hellertime
Github user hellertime commented on the pull request: https://github.com/apache/spark/pull/3074#issuecomment-61582366 Ok. I'll look into expanding this to coarse mode. It shouldn't be too bad. Just need to work backwards on the protobuf objects. --- If your project is set up for it,

[GitHub] spark pull request: Shading the Jetty dependency.

2014-11-03 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2984#issuecomment-61582421 [Test build #22847 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/22847/consoleFull) for PR 2984 at commit

[GitHub] spark pull request: [SPARK-4213][SQL] ParquetFilters - No support ...

2014-11-03 Thread marmbrus
Github user marmbrus commented on a diff in the pull request: https://github.com/apache/spark/pull/3083#discussion_r19781777 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/parquet/ParquetFilters.scala --- @@ -111,6 +111,11 @@ private[sql] object ParquetFilters {

[GitHub] spark pull request: [SPARK-4151][SQL] Add string operation functio...

2014-11-03 Thread tianyi
Github user tianyi commented on the pull request: https://github.com/apache/spark/pull/2998#issuecomment-61582413 You have added a empty file case sensitivity in golden files, is it related to this PR? --- If your project is set up for it, you can reply to this email and have your

[GitHub] spark pull request: [WIP][SPARK-3797] Run external shuffle service...

2014-11-03 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3082#issuecomment-61582852 [Test build #22848 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/22848/consoleFull) for PR 3082 at commit

[GitHub] spark pull request: [SPARK-2261] Make event logger use a single fi...

2014-11-03 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1222#issuecomment-61582899 [Test build #22834 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/22834/consoleFull) for PR 1222 at commit

[GitHub] spark pull request: [SPARK-2261] Make event logger use a single fi...

2014-11-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/1222#issuecomment-61582903 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SQL] Convert arguments to Scala UDFs

2014-11-03 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/3077 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request: [SPARK-4163][Core] Add a backward compatibilit...

2014-11-03 Thread zsxwing
GitHub user zsxwing opened a pull request: https://github.com/apache/spark/pull/3086 [SPARK-4163][Core] Add a backward compatibility test for FetchFailed /cc @aarondav You can merge this pull request into a Git repository by running: $ git pull

<    1   2   3   4   5   >