[GitHub] spark pull request: Unify the logic for column pruning, projection...

2014-03-24 Thread marmbrus
GitHub user marmbrus opened a pull request: https://github.com/apache/spark/pull/213 Unify the logic for column pruning, projection, and filtering of table scans. This removes a bunch of duplicated logic, dead code and casting when planning parquet table scans and hive table

[GitHub] spark pull request: [SPARK-1133] add small files input in MLlib

2014-03-24 Thread mengxr
Github user mengxr commented on a diff in the pull request: https://github.com/apache/spark/pull/164#discussion_r10873319 --- Diff: mllib/src/main/java/org/apache/spark/mllib/input/WholeTextFileInputFormat.java --- @@ -0,0 +1,53 @@ +/* + * Licensed to the Apache Software

[GitHub] spark pull request: SPARK-1094 Support MiMa for reporting binary c...

2014-03-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/207#issuecomment-38414578 Build finished. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request: SPARK-1094 Support MiMa for reporting binary c...

2014-03-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/207#issuecomment-38414579 One or more automated tests failed Refer to this link for build results: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/13381/ --- If your

[GitHub] spark pull request: Unify the logic for column pruning, projection...

2014-03-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/213#issuecomment-38414612 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: java.nio.charset.MalformedInputException

2014-03-24 Thread scwf
Github user scwf closed the pull request at: https://github.com/apache/spark/pull/212 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request: [SPARK-1141] [WIP] Parallelize Task Serializat...

2014-03-24 Thread qqsun8819
GitHub user qqsun8819 opened a pull request: https://github.com/apache/spark/pull/214 [SPARK-1141] [WIP] Parallelize Task Serialization https://spark-project.atlassian.net/browse/SPARK-1141 @kayousterhout copied from JIRA(design doc in JIRA is old, I'll update it later)

[GitHub] spark pull request: Fixed coding style issues in Spark SQL

2014-03-24 Thread rxin
Github user rxin commented on the pull request: https://github.com/apache/spark/pull/208#issuecomment-38415798 ```scala package org.apache.spark.sql package catalyst ``` vs ```scala package org.apache.spark.sql.catalyst ``` There are three reasons I

[GitHub] spark pull request: Fixed coding style issues in Spark SQL

2014-03-24 Thread liancheng
Github user liancheng commented on the pull request: https://github.com/apache/spark/pull/208#issuecomment-38416577 @pwendell When I started on this PR, I was also puzzled which option to use at first, until I saw the same usage in Scala standard library. But I should confess that I

[GitHub] spark pull request: [SPARK-1141] [WIP] Parallelize Task Serializat...

2014-03-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/214#issuecomment-38416907 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: SPARK-1057: Upgrade fastutil to 6.5.11

2014-03-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/215#issuecomment-38416906 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: SPARK-1144 Added license and RAT to check lice...

2014-03-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/125#issuecomment-38416924 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: SPARK-1294 Fix resolution of uppercase field n...

2014-03-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/202#issuecomment-38416911 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: SPARK-1144 Added license and RAT to check lice...

2014-03-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/125#issuecomment-38416925 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: SPARK-1294 Fix resolution of uppercase field n...

2014-03-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/202#issuecomment-38416912 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: SPARK-1144 Added license and RAT to check lice...

2014-03-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/125#issuecomment-38417050 One or more automated tests failed Refer to this link for build results: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/13386/ --- If your

[GitHub] spark pull request: SPARK-1057: Upgrade fastutil to 6.5.11

2014-03-24 Thread pwendell
Github user pwendell commented on the pull request: https://github.com/apache/spark/pull/215#issuecomment-38417092 LGTM - not sure about merging into 0.9.1 though. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: Fixed coding style issues in Spark SQL

2014-03-24 Thread liancheng
Github user liancheng commented on the pull request: https://github.com/apache/spark/pull/208#issuecomment-38417357 I think a point here is whether we should assume all contributors use IDEs like IntelliJ and their automated features. At least, I find in most scenarios, default

[GitHub] spark pull request: Fix SPARK-1280: Stage.name return apply at Op...

2014-03-24 Thread witgo
Github user witgo commented on the pull request: https://github.com/apache/spark/pull/180#issuecomment-38417874 Who can merge the improvement for web UI? @aarondav --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If

[GitHub] spark pull request: Fix SPARK-1280: Stage.name return apply at Op...

2014-03-24 Thread aarondav
Github user aarondav commented on the pull request: https://github.com/apache/spark/pull/180#issuecomment-38418878 Thanks for this patch. Would you mind providing an example stack trace where this helped? I want to get a better sense of the issue to see if this is specific to Option

[GitHub] spark pull request: [SPARK-1186] : Enrich the Spark Shell to suppo...

2014-03-24 Thread berngp
Github user berngp commented on the pull request: https://github.com/apache/spark/pull/116#issuecomment-38418948 @aarondav provided the squash commit with the agreed changes. Bellow the help message that we could have if we want to support the args used by #86. Please share

[GitHub] spark pull request: SPARK-1057: Upgrade fastutil to 6.5.11

2014-03-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/215#issuecomment-38419532 All automated tests passed. Refer to this link for build results: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/13384/ --- If your project

[GitHub] spark pull request: SPARK-1057: Upgrade fastutil to 6.5.11

2014-03-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/215#issuecomment-38419531 Merged build finished. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: SPARK-1294 Fix resolution of uppercase field n...

2014-03-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/202#issuecomment-38419522 All automated tests passed. Refer to this link for build results: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/13385/ --- If your project

[GitHub] spark pull request: SPARK-1094 Support MiMa for reporting binary c...

2014-03-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/207#issuecomment-38419549 Build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request: SPARK-1094 Support MiMa for reporting binary c...

2014-03-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/207#issuecomment-38419548 Build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request: SPARK-1096, a space after comment start style ...

2014-03-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/124#issuecomment-38419558 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: SPARK-1144 Added license and RAT to check lice...

2014-03-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/125#issuecomment-38419554 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-1186] : Enrich the Spark Shell to suppo...

2014-03-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/116#issuecomment-38419564 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-1186] : Enrich the Spark Shell to suppo...

2014-03-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/116#issuecomment-38419565 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: SPARK-1094 Support MiMa for reporting binary c...

2014-03-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/207#issuecomment-38420087 One or more automated tests failed Refer to this link for build results: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/13387/ --- If your

[GitHub] spark pull request: SPARK-1094 Support MiMa for reporting binary c...

2014-03-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/207#issuecomment-38420085 Build finished. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request: Fix SPARK-1280: Stage.name return apply at Op...

2014-03-24 Thread witgo
Github user witgo commented on the pull request: https://github.com/apache/spark/pull/180#issuecomment-38422713 val pairs = sc.parallelize(Array((1, 1), (1, 2), (1, 3), (2, 1))) pairs.take(1) http://host:4040/stages/ Completed Stages table Description Column

[GitHub] spark pull request: [SPARK-1186] : Enrich the Spark Shell to suppo...

2014-03-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/116#issuecomment-38423316 All automated tests passed. Refer to this link for build results: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/13390/ --- If your project

[GitHub] spark pull request: SPARK-1144 Added license and RAT to check lice...

2014-03-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/125#issuecomment-38423311 Merged build finished. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: SPARK-1096, a space after comment start style ...

2014-03-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/124#issuecomment-38423314 All automated tests passed. Refer to this link for build results: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/13389/ --- If your project

[GitHub] spark pull request: SPARK-1144 Added license and RAT to check lice...

2014-03-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/125#issuecomment-38423315 All automated tests passed. Refer to this link for build results: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/13388/ --- If your project

[GitHub] spark pull request: Adding an option to persist Spark RDD blocks ...

2014-03-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/158#issuecomment-38423375 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: Adding an option to persist Spark RDD blocks ...

2014-03-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/158#issuecomment-38423374 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: Unify the logic for column pruning, projection...

2014-03-24 Thread liancheng
Github user liancheng commented on the pull request: https://github.com/apache/spark/pull/213#issuecomment-38423970 Two unused imports are left in `HiveStrategies.scala`. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well.

[GitHub] spark pull request: [SPARK-1133] add small files input in MLlib

2014-03-24 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/164#discussion_r10877199 --- Diff: mllib/src/main/java/org/apache/spark/mllib/input/WholeTextFileRecordReader.java --- @@ -0,0 +1,104 @@ +/* + * Licensed to the Apache

[GitHub] spark pull request: Unify the logic for column pruning, projection...

2014-03-24 Thread liancheng
Github user liancheng commented on the pull request: https://github.com/apache/spark/pull/213#issuecomment-38428081 LGTM, much cleaner :) --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request: [SPARK-1210] Prevent ContextClassLoader of Act...

2014-03-24 Thread ueshin
Github user ueshin commented on the pull request: https://github.com/apache/spark/pull/15#issuecomment-38432140 Added a test case. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request: [SPARK-1210] Prevent ContextClassLoader of Act...

2014-03-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/15#issuecomment-38432384 Build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request: [SPARK-1210] Prevent ContextClassLoader of Act...

2014-03-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/15#issuecomment-38432385 Build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request: [SPARK-1133] add small files input in MLlib

2014-03-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/164#issuecomment-38436606 Build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request: [SPARK-1210] Prevent ContextClassLoader of Act...

2014-03-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/15#issuecomment-38436529 All automated tests passed. Refer to this link for build results: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/13392/ --- If your project

[GitHub] spark pull request: SPARK-1235: kill the application when DAGSched...

2014-03-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/186#issuecomment-38436603 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-1133] add small files input in MLlib

2014-03-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/164#issuecomment-38436605 Build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request: [SPARK-1210] Prevent ContextClassLoader of Act...

2014-03-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/15#issuecomment-38436528 Build finished. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request: Fixed coding style issues in Spark SQL

2014-03-24 Thread alig
Github user alig commented on the pull request: https://github.com/apache/spark/pull/208#issuecomment-38439045 Also +1 for ```package org.apache.spark.sql.catalyst```, just because it's simpler to understand for the majority of the programmers in the world ;) --- If your project is

[GitHub] spark pull request: [SPARK-1198] Allow pipes tasks to run in diffe...

2014-03-24 Thread tgravescs
Github user tgravescs commented on the pull request: https://github.com/apache/spark/pull/128#issuecomment-38440744 yes I can add something, although I don't really have a way to test it. Note that my original question was how we want to go about adding support for

[GitHub] spark pull request: [SPARK-1133] add small files input in MLlib

2014-03-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/164#issuecomment-38441593 Build finished. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request: SPARK-1235: kill the application when DAGSched...

2014-03-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/186#issuecomment-38441595 One or more automated tests failed Refer to this link for build results: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/13393/ --- If your

[GitHub] spark pull request: SPARK-1235: kill the application when DAGSched...

2014-03-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/186#issuecomment-38441592 Merged build finished. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: SPARK-1235: kill the application when DAGSched...

2014-03-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/186#issuecomment-38441655 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: SPARK-1235: kill the application when DAGSched...

2014-03-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/186#issuecomment-38441656 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: SPARK-1235: kill the application when DAGSched...

2014-03-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/186#issuecomment-38441791 Merged build finished. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: SPARK-1235: kill the application when DAGSched...

2014-03-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/186#issuecomment-38441793 One or more automated tests failed Refer to this link for build results: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/13395/ --- If your

[GitHub] spark pull request: SPARK-1235: kill the application when DAGSched...

2014-03-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/186#issuecomment-38441957 One or more automated tests failed Refer to this link for build results: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/13396/ --- If your

[GitHub] spark pull request: SPARK-1235: kill the application when DAGSched...

2014-03-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/186#issuecomment-38441955 Merged build finished. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: SPARK-1235: kill the application when DAGSched...

2014-03-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/186#issuecomment-38442090 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: SPARK-1235: kill the application when DAGSched...

2014-03-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/186#issuecomment-38442093 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-1141] [WIP] Parallelize Task Serializat...

2014-03-24 Thread CodingCat
Github user CodingCat commented on a diff in the pull request: https://github.com/apache/spark/pull/214#discussion_r10884042 --- Diff: core/src/main/scala/org/apache/spark/scheduler/TaskSchedulerImpl.scala --- @@ -93,6 +96,10 @@ private[spark] class TaskSchedulerImpl( val

[GitHub] spark pull request: [SPARK-1141] [WIP] Parallelize Task Serializat...

2014-03-24 Thread CodingCat
Github user CodingCat commented on a diff in the pull request: https://github.com/apache/spark/pull/214#discussion_r10884203 --- Diff: core/src/main/scala/org/apache/spark/scheduler/TaskSchedulerImpl.scala --- @@ -243,12 +275,18 @@ private[spark] class TaskSchedulerImpl(

[GitHub] spark pull request: [SPARK-1141] [WIP] Parallelize Task Serializat...

2014-03-24 Thread CodingCat
Github user CodingCat commented on a diff in the pull request: https://github.com/apache/spark/pull/214#discussion_r10884189 --- Diff: core/src/main/scala/org/apache/spark/scheduler/TaskSchedulerImpl.scala --- @@ -243,12 +275,18 @@ private[spark] class TaskSchedulerImpl(

[GitHub] spark pull request: [SPARK-1141] [WIP] Parallelize Task Serializat...

2014-03-24 Thread CodingCat
Github user CodingCat commented on a diff in the pull request: https://github.com/apache/spark/pull/214#discussion_r10884300 --- Diff: core/src/main/scala/org/apache/spark/scheduler/TaskSchedulerImpl.scala --- @@ -219,18 +226,43 @@ private[spark] class TaskSchedulerImpl(

[GitHub] spark pull request: [SPARK-1141] [WIP] Parallelize Task Serializat...

2014-03-24 Thread CodingCat
Github user CodingCat commented on a diff in the pull request: https://github.com/apache/spark/pull/214#discussion_r10884378 --- Diff: core/src/main/scala/org/apache/spark/scheduler/TaskSchedulerImpl.scala --- @@ -219,18 +226,43 @@ private[spark] class TaskSchedulerImpl(

[GitHub] spark pull request: [SPARK-1141] [WIP] Parallelize Task Serializat...

2014-03-24 Thread CodingCat
Github user CodingCat commented on a diff in the pull request: https://github.com/apache/spark/pull/214#discussion_r10884645 --- Diff: core/src/main/scala/org/apache/spark/scheduler/TaskSchedulerImpl.scala --- @@ -30,6 +30,9 @@ import scala.util.Random import

[GitHub] spark pull request: [SPARK-1141] [WIP] Parallelize Task Serializat...

2014-03-24 Thread CodingCat
Github user CodingCat commented on a diff in the pull request: https://github.com/apache/spark/pull/214#discussion_r10884928 --- Diff: core/src/main/scala/org/apache/spark/scheduler/TaskSchedulerImpl.scala --- @@ -219,18 +226,43 @@ private[spark] class TaskSchedulerImpl(

[GitHub] spark pull request: SPARK-1235: kill the application when DAGSched...

2014-03-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/186#issuecomment-38448296 Merged build finished. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: SPARK-1235: kill the application when DAGSched...

2014-03-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/186#issuecomment-38448297 All automated tests passed. Refer to this link for build results: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/13397/ --- If your project

[GitHub] spark pull request: [SPARK-1141] [WIP] Parallelize Task Serializat...

2014-03-24 Thread qqsun8819
Github user qqsun8819 commented on a diff in the pull request: https://github.com/apache/spark/pull/214#discussion_r10886619 --- Diff: core/src/main/scala/org/apache/spark/scheduler/TaskSchedulerImpl.scala --- @@ -219,18 +226,43 @@ private[spark] class TaskSchedulerImpl(

[GitHub] spark pull request: SPARK-1094 Support MiMa for reporting binary c...

2014-03-24 Thread pwendell
Github user pwendell commented on the pull request: https://github.com/apache/spark/pull/207#issuecomment-38462860 Jenkins, test this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-1141] [WIP] Parallelize Task Serializat...

2014-03-24 Thread qqsun8819
Github user qqsun8819 commented on the pull request: https://github.com/apache/spark/pull/214#issuecomment-38463531 Fix DriverSuite case fail put threadpool inside resourceoffer and shutdown it before it return some other fix according to @CodingCat 's review --- If your

[GitHub] spark pull request: Fixed coding style issues in Spark SQL

2014-03-24 Thread liancheng
Github user liancheng commented on the pull request: https://github.com/apache/spark/pull/208#issuecomment-38466998 Thanks for all your votes! I'll fix this in a separate PR ASAP. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub

[GitHub] spark pull request: Spark parquet improvements

2014-03-24 Thread AndreSchumacher
Github user AndreSchumacher commented on a diff in the pull request: https://github.com/apache/spark/pull/195#discussion_r10892494 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/parquet/ParquetRelation.scala --- @@ -72,16 +73,43 @@ case class ParquetRelation(val tableName:

[GitHub] spark pull request: [SPARK-1303] [MLLIB] Added discretization capa...

2014-03-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/216#issuecomment-38471741 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: SPARK-1094 Support MiMa for reporting binary c...

2014-03-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/207#issuecomment-38471725 All automated tests passed. Refer to this link for build results: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/13398/ --- If your project

[GitHub] spark pull request: SPARK-1094 Support MiMa for reporting binary c...

2014-03-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/207#issuecomment-38471724 Build finished. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request: [SPARK-1303] [MLLIB] Added discretization capa...

2014-03-24 Thread mengxr
Github user mengxr commented on the pull request: https://github.com/apache/spark/pull/216#issuecomment-38472118 Jenkins, test this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-1303] [MLLIB] Added discretization capa...

2014-03-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/216#issuecomment-38472286 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-1303] [MLLIB] Added discretization capa...

2014-03-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/216#issuecomment-38472412 Merged build finished. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-1303] [MLLIB] Added discretization capa...

2014-03-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/216#issuecomment-38472415 One or more automated tests failed Refer to this link for build results: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/13400/ --- If your

[GitHub] spark pull request: Adding an option to persist Spark RDD blocks ...

2014-03-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/158#issuecomment-38473429 Merged build finished. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: SPARK-1235: kill the application when DAGSched...

2014-03-24 Thread CodingCat
Github user CodingCat commented on the pull request: https://github.com/apache/spark/pull/186#issuecomment-38476549 I adjusted the code to capture the exception inside processEvent function, so that we can easily test the function in DAGSchedulerSuite --- If your project is

[GitHub] spark pull request: Adding an option to persist Spark RDD blocks ...

2014-03-24 Thread pwendell
Github user pwendell commented on the pull request: https://github.com/apache/spark/pull/158#issuecomment-38476606 @RongGu I just created a JIRA for this: https://spark-project.atlassian.net/browse/SPARK-1305 Do you mind updating the title here to start with

[GitHub] spark pull request: SPARK-1094 Support MiMa for reporting binary c...

2014-03-24 Thread aarondav
Github user aarondav commented on a diff in the pull request: https://github.com/apache/spark/pull/207#discussion_r10896563 --- Diff: tools/src/main/scala/org/apache/spark/tools/GenerateMIMAIgnore.scala --- @@ -0,0 +1,131 @@ +/* + * Licensed to the Apache Software

[GitHub] spark pull request: SPARK-1305: Support persisting RDD's directly ...

2014-03-24 Thread RongGu
Github user RongGu commented on the pull request: https://github.com/apache/spark/pull/158#issuecomment-38478522 @pwendell , I've updated the title of this PR with prefix 'SPARK-1305:'. Also, I've created my account on the Spark JIRA. It's named RongGu. --- If your project is set up

[GitHub] spark pull request: SPARK-1128: set hadoop task properties when co...

2014-03-24 Thread aarondav
Github user aarondav commented on the pull request: https://github.com/apache/spark/pull/101#issuecomment-38478514 Looks good to me. Merged into master - thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: SPARK-1128: set hadoop task properties when co...

2014-03-24 Thread aarondav
Github user aarondav commented on the pull request: https://github.com/apache/spark/pull/101#issuecomment-38478659 Oops, spoke to soon, some changes have apparently made this PR not cleanly mergeable. Mind doing a rebase? --- If your project is set up for it, you can reply to this

[GitHub] spark pull request: [SPARK-1303] [MLLIB] Added discretization capa...

2014-03-24 Thread mengxr
Github user mengxr commented on a diff in the pull request: https://github.com/apache/spark/pull/216#discussion_r10897487 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/discretization/EMDDiscretizer.scala --- @@ -0,0 +1,402 @@ +/* +* Licensed to the Apache Software

[GitHub] spark pull request: SPARK-1305: Support persisting RDD's directly ...

2014-03-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/158#issuecomment-38479671 One or more automated tests failed Refer to this link for build results: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/13399/ --- If your

[GitHub] spark pull request: SPARK-1094 Support MiMa for reporting binary c...

2014-03-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/207#issuecomment-38479779 Build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request: SPARK-1305: Support persisting RDD's directly ...

2014-03-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/158#issuecomment-38479800 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: SPARK-1094 Support MiMa for reporting binary c...

2014-03-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/207#issuecomment-38479780 Build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request: SPARK-1305: Support persisting RDD's directly ...

2014-03-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/158#issuecomment-38479965 Merged build finished. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: SPARK-1094 Support MiMa for reporting binary c...

2014-03-24 Thread aarondav
Github user aarondav commented on the pull request: https://github.com/apache/spark/pull/207#issuecomment-38480096 Looks good to me! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request: [SPARK-1303] [MLLIB] Added discretization capa...

2014-03-24 Thread mengxr
Github user mengxr commented on a diff in the pull request: https://github.com/apache/spark/pull/216#discussion_r10897825 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/discretization/MapAccumulator.scala --- @@ -0,0 +1,53 @@ +/* +* Licensed to the Apache Software

[GitHub] spark pull request: SPARK-1305: Support persisting RDD's directly ...

2014-03-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/158#issuecomment-38480623 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

  1   2   3   >