[GitHub] spark pull request: [SPARK-2737] Add retag() method for changing R...

2014-07-30 Thread mateiz
Github user mateiz commented on the pull request: https://github.com/apache/spark/pull/1639#issuecomment-50576705 Sure, sounds good. Did you see my comments on preserving partitions too though? --- If your project is set up for it, you can reply to this email and have your reply

[GitHub] spark pull request: [SPARK-2737] Add retag() method for changing R...

2014-07-30 Thread JoshRosen
Github user JoshRosen commented on a diff in the pull request: https://github.com/apache/spark/pull/1639#discussion_r15567618 --- Diff: core/src/main/scala/org/apache/spark/rdd/RDD.scala --- @@ -1239,6 +1239,28 @@ abstract class RDD[T: ClassTag]( /** The

[GitHub] spark pull request: [SPARK-1981] Add AWS Kinesis streaming support

2014-07-30 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/1434#discussion_r15567656 --- Diff: extras/spark-kinesis-asl/src/main/java/org/apache/spark/examples/streaming/JavaKinesisWordCount.java --- @@ -0,0 +1,310 @@ +/* + * Licensed

[GitHub] spark pull request: [SPARK-2737] Add retag() method for changing R...

2014-07-30 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1639#issuecomment-50577126 QA results for PR 1639:br- This patch PASSES unit tests.br- This patch merges cleanlybr- This patch adds no public classesbrbrFor more information see test

[GitHub] spark pull request: [SPARK-2024] Add saveAsSequenceFile to PySpark

2014-07-30 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1338#issuecomment-50577245 QA results for PR 1338:br- This patch FAILED unit tests.br- This patch merges cleanlybr- This patch adds no public classesbrbrFor more information see test

[GitHub] spark pull request: [SPARK-2585] Remove special handling of Hadoop...

2014-07-30 Thread rxin
Github user rxin commented on the pull request: https://github.com/apache/spark/pull/1648#issuecomment-50577269 Jenkins, what are you doing ... --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-2314][SQL] Override collect and take in...

2014-07-30 Thread staple
Github user staple commented on the pull request: https://github.com/apache/spark/pull/1592#issuecomment-50577338 Sure, I’m fine with reworking based on other changes (it seems that some merge conflicts have already cropped up in master since I submitted my PR last week). I think

[GitHub] spark pull request: [SPARK-2737] Add retag() method for changing R...

2014-07-30 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1639#issuecomment-50577408 QA tests have started for PR 1639. This patch merges cleanly. brView progress: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/17426/consoleFull ---

[GitHub] spark pull request: [SPARK-2585] Remove special handling of Hadoop...

2014-07-30 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1648#issuecomment-50577383 QA tests have started for PR 1648. This patch merges cleanly. brView progress: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/17425/consoleFull ---

[GitHub] spark pull request: [SPARK-1981] Add AWS Kinesis streaming support

2014-07-30 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/1434#discussion_r15567741 --- Diff: extras/spark-kinesis-asl/src/main/java/org/apache/spark/examples/streaming/JavaKinesisWordCount.java --- @@ -0,0 +1,310 @@ +/* + * Licensed

[GitHub] spark pull request: SPARK-2632, SPARK-2576. Fixed by only importin...

2014-07-30 Thread ScrapCodes
Github user ScrapCodes commented on the pull request: https://github.com/apache/spark/pull/1635#issuecomment-50577431 Jenkins retest this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-2314][SQL] Override collect and take in...

2014-07-30 Thread marmbrus
Github user marmbrus commented on the pull request: https://github.com/apache/spark/pull/1592#issuecomment-50577608 Yeah, I'm hoping to merge #1346 as soon as it passes Jenkins, so I'd wait for that. I also thought I’d add a couple of notes on what I had in mind with this

[GitHub] spark pull request: SPARK-2632, SPARK-2576. Fixed by only importin...

2014-07-30 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1635#issuecomment-50577707 QA tests have started for PR 1635. This patch merges cleanly. brView progress: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/17427/consoleFull ---

[GitHub] spark pull request: [SPARK-2568] RangePartitioner should run only ...

2014-07-30 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/1562 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request: [SQL] Handle null values in debug()

2014-07-30 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/1646 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request: [SPARK-2314][SQL] Override collect and take in...

2014-07-30 Thread staple
Github user staple commented on the pull request: https://github.com/apache/spark/pull/1592#issuecomment-50577874 Sure, added the notes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request: [SPARK-2054][SQL] Code Generation for Expressi...

2014-07-30 Thread ueshin
Github user ueshin commented on the pull request: https://github.com/apache/spark/pull/993#issuecomment-50578802 Hi @marmbrus, thanks for great work! But it seems to break build. I got the following result when I run `sbt assembly` or `sbt publish-local`: ```

[GitHub] spark pull request: [SPARK-2734][SQL] Remove tables from cache whe...

2014-07-30 Thread yhuai
Github user yhuai commented on a diff in the pull request: https://github.com/apache/spark/pull/1650#discussion_r15568205 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveQl.scala --- @@ -377,6 +378,10 @@ private[hive] object HiveQl { } protected

[GitHub] spark pull request: [SPARK-1981] Add AWS Kinesis streaming support

2014-07-30 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/1434#discussion_r15568250 --- Diff: extras/spark-kinesis-asl/src/main/scala/org/apache/spark/examples/streaming/KinesisWordCount.scala --- @@ -0,0 +1,345 @@ +/* + * Licensed to

[GitHub] spark pull request: [SPARK-1630] Turn Null of Java/Scala into None...

2014-07-30 Thread davies
Github user davies commented on a diff in the pull request: https://github.com/apache/spark/pull/1551#discussion_r15568273 --- Diff: core/src/main/scala/org/apache/spark/api/python/PythonRDD.scala --- @@ -344,7 +345,12 @@ private[spark] object PythonRDD extends Logging {

[GitHub] spark pull request: [SPARK-2544][MLLIB] Improve ALS algorithm reso...

2014-07-30 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/929#issuecomment-50579245 QA results for PR 929:br- This patch PASSES unit tests.br- This patch merges cleanlybr- This patch adds no public classesbrbrFor more information see test

[GitHub] spark pull request: [SPARK-2734][SQL] Remove tables from cache whe...

2014-07-30 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1650#issuecomment-50579517 QA results for PR 1650:br- This patch FAILED unit tests.br- This patch merges cleanlybr- This patch adds the following public classes (experimental):brcase class

[GitHub] spark pull request: Avoid numerical instability

2014-07-30 Thread naftaliharris
GitHub user naftaliharris opened a pull request: https://github.com/apache/spark/pull/1652 Avoid numerical instability This avoids basically doing 1 - 1, for example: ```python from math import exp margin = -40 1 - 1 / (1 + exp(margin)) 0.0

[GitHub] spark pull request: Avoid numerical instability

2014-07-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/1652#issuecomment-50579636 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: Add caching information to rdd.toDebugString

2014-07-30 Thread pwendell
Github user pwendell commented on the pull request: https://github.com/apache/spark/pull/1535#issuecomment-50579659 Hey @nkronenfeld - I traced through the exact function call more closely and I actually think it's fine. The issue I pointed out in the JIRA is orthogonal. So I'm fine

[GitHub] spark pull request: [SPARK-2552][MLLIB] stabilize logistic functio...

2014-07-30 Thread naftaliharris
Github user naftaliharris commented on a diff in the pull request: https://github.com/apache/spark/pull/1493#discussion_r15568446 --- Diff: python/pyspark/mllib/classification.py --- @@ -63,7 +63,10 @@ class LogisticRegressionModel(LinearModel): def predict(self, x):

[GitHub] spark pull request: [SPARK-2521] Broadcast RDD object (instead of ...

2014-07-30 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1498#issuecomment-50579747 QA tests have started for PR 1498. This patch merges cleanly. brView progress: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/17428/consoleFull ---

[GitHub] spark pull request: [SPARK-2260] Fix standalone-cluster mode, whic...

2014-07-30 Thread pwendell
Github user pwendell commented on the pull request: https://github.com/apache/spark/pull/1538#issuecomment-50579956 LGTM - thanks andrew! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request: [SPARK-1981] Add AWS Kinesis streaming support

2014-07-30 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/1434#discussion_r15568742 --- Diff: extras/spark-kinesis-asl/src/main/scala/org/apache/spark/examples/streaming/KinesisWordCount.scala --- @@ -0,0 +1,345 @@ +/* + * Licensed to

[GitHub] spark pull request: [SQL] Fix compiling of catalyst docs.

2014-07-30 Thread marmbrus
GitHub user marmbrus opened a pull request: https://github.com/apache/spark/pull/1653 [SQL] Fix compiling of catalyst docs. You can merge this pull request into a Git repository by running: $ git pull https://github.com/marmbrus/spark fixDocs Alternatively you can review and

[GitHub] spark pull request: [SPARK-2737] Add retag() method for changing R...

2014-07-30 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1639#issuecomment-50580659 QA results for PR 1639:br- This patch FAILED unit tests.br- This patch merges cleanlybr- This patch adds no public classesbrbrFor more information see test

[GitHub] spark pull request: [SPARK-2054][SQL] Code Generation for Expressi...

2014-07-30 Thread marmbrus
Github user marmbrus commented on the pull request: https://github.com/apache/spark/pull/993#issuecomment-50580695 @ueshin thanks for reporting. I've opened #1653 to fix this. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as

[GitHub] spark pull request: [SPARK-2743][SQL] Resolve original attributes ...

2014-07-30 Thread marmbrus
Github user marmbrus commented on the pull request: https://github.com/apache/spark/pull/1647#issuecomment-50580872 test this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request: [SQL] Fix compiling of catalyst docs.

2014-07-30 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1653#issuecomment-50580853 QA tests have started for PR 1653. This patch merges cleanly. brView progress: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/17429/consoleFull ---

[GitHub] spark pull request: [SPARK-2737] Add retag() method for changing R...

2014-07-30 Thread mateiz
Github user mateiz commented on a diff in the pull request: https://github.com/apache/spark/pull/1639#discussion_r15568957 --- Diff: core/src/main/scala/org/apache/spark/rdd/RDD.scala --- @@ -1239,6 +1239,28 @@ abstract class RDD[T: ClassTag]( /** The

[GitHub] spark pull request: [SPARK-2743][SQL] Resolve original attributes ...

2014-07-30 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1647#issuecomment-50581178 QA tests have started for PR 1647. This patch merges cleanly. brView progress: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/17430/consoleFull ---

[GitHub] spark pull request: [SPARK-2737] Add retag() method for changing R...

2014-07-30 Thread mateiz
Github user mateiz commented on a diff in the pull request: https://github.com/apache/spark/pull/1639#discussion_r15568992 --- Diff: core/src/main/scala/org/apache/spark/rdd/RDD.scala --- @@ -1239,6 +1239,28 @@ abstract class RDD[T: ClassTag]( /** The

[GitHub] spark pull request: [SPARK-1981] Add AWS Kinesis streaming support

2014-07-30 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/1434#discussion_r15568964 --- Diff: extras/spark-kinesis-asl/src/main/scala/org/apache/spark/streaming/kinesis/KinesisUtils.scala --- @@ -0,0 +1,151 @@ +/* + * Licensed to the

[GitHub] spark pull request: [SPARK-1981] Add AWS Kinesis streaming support

2014-07-30 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/1434#discussion_r15568976 --- Diff: extras/spark-kinesis-asl/src/main/scala/org/apache/spark/streaming/kinesis/KinesisUtils.scala --- @@ -0,0 +1,151 @@ +/* + * Licensed to the

[GitHub] spark pull request: [SPARK-2737] Add retag() method for changing R...

2014-07-30 Thread mateiz
Github user mateiz commented on the pull request: https://github.com/apache/spark/pull/1639#issuecomment-50581291 In case you don't see the hidden comment above: I don't think mapPartitions would hurt performance here. All you do is pass through the parent's iterator. When you call

[GitHub] spark pull request: [SPARK-2737] Add retag() method for changing R...

2014-07-30 Thread mateiz
Github user mateiz commented on the pull request: https://github.com/apache/spark/pull/1639#issuecomment-50581349 Basically it's a shorter way of writing what you wrote. Take a look at MapPartitionsRDD. --- If your project is set up for it, you can reply to this email and have your

[GitHub] spark pull request: SPARK-2632, SPARK-2576. Fixed by only importin...

2014-07-30 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1635#issuecomment-50581428 QA results for PR 1635:br- This patch PASSES unit tests.br- This patch merges cleanlybr- This patch adds no public classesbrbrFor more information see test

[GitHub] spark pull request: [SPARK-2179][SQL] Public API for DataTypes and...

2014-07-30 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1346#issuecomment-50581580 QA results for PR 1346:br- This patch PASSES unit tests.br- This patch merges cleanlybr- This patch adds no public classesbrbrFor more information see test

[GitHub] spark pull request: [SPARK-1630] Turn Null of Java/Scala into None...

2014-07-30 Thread mateiz
Github user mateiz commented on a diff in the pull request: https://github.com/apache/spark/pull/1551#discussion_r15569151 --- Diff: core/src/main/scala/org/apache/spark/api/python/PythonRDD.scala --- @@ -344,7 +345,12 @@ private[spark] object PythonRDD extends Logging {

[GitHub] spark pull request: SPARK-2632, SPARK-2576. Fixed by only importin...

2014-07-30 Thread marmbrus
Github user marmbrus commented on a diff in the pull request: https://github.com/apache/spark/pull/1635#discussion_r15569238 --- Diff: repl/src/test/scala/org/apache/spark/repl/ReplSuite.scala --- @@ -256,6 +256,33 @@ class ReplSuite extends FunSuite {

[GitHub] spark pull request: [SPARK-2260] Fix standalone-cluster mode, whic...

2014-07-30 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/1538 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request: [SPARK-2179][SQL] Public API for DataTypes and...

2014-07-30 Thread marmbrus
Github user marmbrus commented on the pull request: https://github.com/apache/spark/pull/1346#issuecomment-50582008 Thanks for working on this! Merged to master. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-1630] Turn Null of Java/Scala into None...

2014-07-30 Thread mateiz
Github user mateiz commented on a diff in the pull request: https://github.com/apache/spark/pull/1551#discussion_r15569391 --- Diff: core/src/main/scala/org/apache/spark/api/python/PythonRDD.scala --- @@ -344,7 +345,12 @@ private[spark] object PythonRDD extends Logging {

[GitHub] spark pull request: [SPARK-2521] Broadcast RDD object (instead of ...

2014-07-30 Thread pwendell
Github user pwendell commented on a diff in the pull request: https://github.com/apache/spark/pull/1498#discussion_r15569659 --- Diff: core/src/main/scala/org/apache/spark/Dependency.scala --- @@ -36,20 +38,24 @@ abstract class Dependency[T](val rdd: RDD[T]) extends Serializable

[GitHub] spark pull request: [SPARK-1630] Turn Null of Java/Scala into None...

2014-07-30 Thread davies
Github user davies commented on a diff in the pull request: https://github.com/apache/spark/pull/1551#discussion_r15569680 --- Diff: core/src/main/scala/org/apache/spark/api/python/PythonRDD.scala --- @@ -344,7 +345,12 @@ private[spark] object PythonRDD extends Logging {

[GitHub] spark pull request: SPARK-2646. log4j initialization not quite com...

2014-07-30 Thread pwendell
Github user pwendell commented on a diff in the pull request: https://github.com/apache/spark/pull/1547#discussion_r15569831 --- Diff: core/src/main/scala/org/apache/spark/Logging.scala --- @@ -110,23 +110,26 @@ trait Logging { } private def initializeLogging()

[GitHub] spark pull request: [SPARK-2521] Broadcast RDD object (instead of ...

2014-07-30 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1498#issuecomment-50583518 QA results for PR 1498:br- This patch PASSES unit tests.br- This patch merges cleanlybr- This patch adds the following public classes (experimental):brabstract class

[GitHub] spark pull request: SPARK-2632, SPARK-2576. Fixed by only importin...

2014-07-30 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1635#issuecomment-50583559 QA tests have started for PR 1635. This patch merges cleanly. brView progress: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/17431/consoleFull ---

[GitHub] spark pull request: Broadcast task sql test [DO NOT MERGE]

2014-07-30 Thread rxin
GitHub user rxin opened a pull request: https://github.com/apache/spark/pull/1654 Broadcast task sql test [DO NOT MERGE] This is the same as #1498 but adds a dummy change to sql so I can trigger a SQL test on Jenkins. This is not meant to be merged. You can merge this

[GitHub] spark pull request: SPARK-2632, SPARK-2576. Fixed by only importin...

2014-07-30 Thread ScrapCodes
Github user ScrapCodes commented on a diff in the pull request: https://github.com/apache/spark/pull/1635#discussion_r15569917 --- Diff: repl/src/test/scala/org/apache/spark/repl/ReplSuite.scala --- @@ -256,6 +256,33 @@ class ReplSuite extends FunSuite {

[GitHub] spark pull request: Broadcast task sql test [DO NOT MERGE]

2014-07-30 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1654#issuecomment-50583989 QA tests have started for PR 1654. This patch merges cleanly. brView progress: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/17432/consoleFull ---

[GitHub] spark pull request: SPARK-2646. log4j initialization not quite com...

2014-07-30 Thread pwendell
Github user pwendell commented on a diff in the pull request: https://github.com/apache/spark/pull/1547#discussion_r15570127 --- Diff: core/src/main/scala/org/apache/spark/Logging.scala --- @@ -110,23 +110,26 @@ trait Logging { } private def initializeLogging()

[GitHub] spark pull request: [SPARK-2746] Set SBT_MAVEN_PROFILES only when ...

2014-07-30 Thread rxin
GitHub user rxin opened a pull request: https://github.com/apache/spark/pull/1655 [SPARK-2746] Set SBT_MAVEN_PROFILES only when it is not set explicitly b... ...y the user. You can merge this pull request into a Git repository by running: $ git pull

[GitHub] spark pull request: [SQL] Fix compiling of catalyst docs.

2014-07-30 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1653#issuecomment-50584701 QA results for PR 1653:br- This patch PASSES unit tests.br- This patch merges cleanlybr- This patch adds no public classesbrbrFor more information see test

[GitHub] spark pull request: [SPARK-2746] Set SBT_MAVEN_PROFILES only when ...

2014-07-30 Thread pwendell
Github user pwendell commented on the pull request: https://github.com/apache/spark/pull/1655#issuecomment-50585615 LGTM --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] spark pull request: Feat kryo max buffersize

2014-07-30 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/735 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request: [SPARK-2179][SQL] Public API for DataTypes and...

2014-07-30 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/1346 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request: [SPARK-2743][SQL] Resolve original attributes ...

2014-07-30 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1647#issuecomment-50586522 QA results for PR 1647:br- This patch FAILED unit tests.br- This patch merges cleanlybr- This patch adds no public classesbrbrFor more information see test

[GitHub] spark pull request: SPARK-2098: All Spark processes should support...

2014-07-30 Thread witgo
Github user witgo commented on the pull request: https://github.com/apache/spark/pull/1256#issuecomment-50586734 Jenkins, retest this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: SPARK-2098: All Spark processes should support...

2014-07-30 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1256#issuecomment-50586736 QA tests have started for PR 1256. This patch merges cleanly. brView progress: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/17434/consoleFull ---

[GitHub] spark pull request: [SPARK-2497] Included checks for module symbol...

2014-07-30 Thread pwendell
Github user pwendell commented on the pull request: https://github.com/apache/spark/pull/1463#issuecomment-50586830 In your example, the reason why there is no annotation on the `A` class is that it the class is distinct from the companion object. You only created the companion

[GitHub] spark pull request: SPARK-2632, SPARK-2576. Fixed by only importin...

2014-07-30 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1635#issuecomment-50586938 QA results for PR 1635:br- This patch FAILED unit tests.br- This patch merges cleanlybr- This patch adds no public classesbrbrFor more information see test

[GitHub] spark pull request: SPARK-2098: All Spark processes should support...

2014-07-30 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1256#issuecomment-50587212 QA tests have started for PR 1256. This patch merges cleanly. brView progress: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/17435/consoleFull ---

[GitHub] spark pull request: Broadcast task sql test [DO NOT MERGE]

2014-07-30 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1654#issuecomment-50587953 QA results for PR 1654:br- This patch PASSES unit tests.br- This patch merges cleanlybr- This patch adds the following public classes (experimental):brabstract class

[GitHub] spark pull request: Broadcast task sql test [DO NOT MERGE]

2014-07-30 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1654#issuecomment-50588540 QA results for PR 1654:br- This patch FAILED unit tests.br- This patch merges cleanlybr- This patch adds the following public classes (experimental):brabstract class

[GitHub] spark pull request: [SPARK-2747] git diff --dirstat can miss sql c...

2014-07-30 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1656#issuecomment-50588923 QA tests have started for PR 1656. This patch merges cleanly. brView progress: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/17437/consoleFull ---

[GitHub] spark pull request: [SPARK-2179][SQL] Public API for DataTypes and...

2014-07-30 Thread chenghao-intel
Github user chenghao-intel commented on the pull request: https://github.com/apache/spark/pull/1346#issuecomment-50590014 Thank you @yhuai for the explanation. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: SPARK-2632, SPARK-2576. Fixed by only importin...

2014-07-30 Thread ScrapCodes
Github user ScrapCodes commented on the pull request: https://github.com/apache/spark/pull/1635#issuecomment-50590047 Jenkins retest this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-2497] Included checks for module symbol...

2014-07-30 Thread ScrapCodes
Github user ScrapCodes commented on the pull request: https://github.com/apache/spark/pull/1463#issuecomment-50590574 I changed the patch to first check for presence of annotation on class and then on object using separate `symbol` for both. Previously I was doing that because I did

[GitHub] spark pull request: SPARK-2098: All Spark processes should support...

2014-07-30 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1256#issuecomment-50590948 QA results for PR 1256:br- This patch PASSES unit tests.br- This patch merges cleanlybr- This patch adds the following public classes (experimental):brclass

[GitHub] spark pull request: [MLLIB] [spark-2352] Implementation of an 1-hi...

2014-07-30 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1290#issuecomment-50591442 QA results for PR 1290:br- This patch FAILED unit tests.br- This patch merges cleanlybr- This patch adds the following public classes (experimental):brabstract class

[GitHub] spark pull request: SPARK-2098: All Spark processes should support...

2014-07-30 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1256#issuecomment-50591525 QA results for PR 1256:br- This patch PASSES unit tests.br- This patch merges cleanlybr- This patch adds the following public classes (experimental):brclass

[GitHub] spark pull request: [SPARK-2729] [SQL] Forgot to match Timestamp t...

2014-07-30 Thread chutium
Github user chutium commented on the pull request: https://github.com/apache/spark/pull/1636#issuecomment-50592256 it seems should be this one? NullableColumnBuilderSuite.scala i can do it --- If your project is set up for it, you can reply to this email and have your

[GitHub] spark pull request: [SPARK-2677] BasicBlockFetchIterator#next can ...

2014-07-30 Thread lianhuiwang
Github user lianhuiwang commented on a diff in the pull request: https://github.com/apache/spark/pull/1632#discussion_r15573389 --- Diff: core/src/main/scala/org/apache/spark/storage/BlockFetcherIterator.scala --- @@ -117,31 +121,45 @@ object BlockFetcherIterator { })

[GitHub] spark pull request: [SPARK-2710] [SQL] Build SchemaRDD from a Jdbc...

2014-07-30 Thread chutium
Github user chutium commented on a diff in the pull request: https://github.com/apache/spark/pull/1612#discussion_r15573636 --- Diff: core/src/main/scala/org/apache/spark/rdd/JdbcRDD.scala --- @@ -81,8 +105,10 @@ class JdbcRDD[T: ClassTag]( logInfo(statement fetch size

[GitHub] spark pull request: [SPARK-2710] [SQL] Build SchemaRDD from a Jdbc...

2014-07-30 Thread chutium
Github user chutium commented on a diff in the pull request: https://github.com/apache/spark/pull/1612#discussion_r15573659 --- Diff: core/src/main/scala/org/apache/spark/rdd/JdbcRDD.scala --- @@ -67,6 +69,28 @@ class JdbcRDD[T: ClassTag]( }).toArray } +

[GitHub] spark pull request: [SPARK-2747] git diff --dirstat can miss sql c...

2014-07-30 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1656#issuecomment-50593309 QA results for PR 1656:br- This patch PASSES unit tests.br- This patch merges cleanlybr- This patch adds no public classesbrbrFor more information see test

[GitHub] spark pull request: [SPARK-2497] Included checks for module symbol...

2014-07-30 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1463#issuecomment-50593446 QA tests have started for PR 1463. This patch merges cleanly. brView progress: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/17441/consoleFull ---

[GitHub] spark pull request: SPARK-2641: Fixing how spark arguments are loa...

2014-07-30 Thread kjsingh
GitHub user kjsingh opened a pull request: https://github.com/apache/spark/pull/1657 SPARK-2641: Fixing how spark arguments are loaded from properties file You can merge this pull request into a Git repository by running: $ git pull https://github.com/kjsingh/spark branch-1.0

[GitHub] spark pull request: SPARK-2641: Fixing how spark arguments are loa...

2014-07-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/1657#issuecomment-50594217 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: Avoid numerical instability

2014-07-30 Thread srowen
Github user srowen commented on the pull request: https://github.com/apache/spark/pull/1652#issuecomment-50595894 Y'know, there's a similar issue in `LogisticGradient.scala`, in lines like: `math.log(1 + math.exp(margin))` For -40, this gives 0.0, when really it

[GitHub] spark pull request: Broadcast task sql test [DO NOT MERGE]

2014-07-30 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1654#issuecomment-50596158 QA results for PR 1654:br- This patch PASSES unit tests.br- This patch merges cleanlybr- This patch adds the following public classes (experimental):brabstract class

[GitHub] spark pull request: [SPARK-1812] [WIP] Scala 2.11 support

2014-07-30 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/1649#discussion_r15575515 --- Diff: pom.xml --- @@ -1015,7 +1016,6 @@ idhadoop-2.2/id properties hadoop.version2.2.0/hadoop.version -

[GitHub] spark pull request: [SPARK-1812] [WIP] Scala 2.11 support

2014-07-30 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/1649#discussion_r15575545 --- Diff: pom.xml --- @@ -111,16 +111,16 @@ java.version1.6/java.version sbt.project.namespark/sbt.project.name -

[GitHub] spark pull request: [SPARK-1812] [WIP] Scala 2.11 support

2014-07-30 Thread srowen
Github user srowen commented on the pull request: https://github.com/apache/spark/pull/1649#issuecomment-50597071 I wonder if you can pull out the changes that will also work in Scala 2.10? some of it looks like some syntactic cleanup which could go in regardless of the Scala

[GitHub] spark pull request: [SPARK-2497] Included checks for module symbol...

2014-07-30 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1463#issuecomment-50597662 QA results for PR 1463:br- This patch PASSES unit tests.br- This patch merges cleanlybr- This patch adds no public classesbrbrFor more information see test

[GitHub] spark pull request: SPARK-2646. log4j initialization not quite com...

2014-07-30 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/1547#discussion_r15576553 --- Diff: core/src/main/scala/org/apache/spark/Logging.scala --- @@ -110,23 +110,26 @@ trait Logging { } private def initializeLogging() {

[GitHub] spark pull request: SPARK-2646. log4j initialization not quite com...

2014-07-30 Thread srowen
Github user srowen commented on the pull request: https://github.com/apache/spark/pull/1547#issuecomment-50599689 No problem, all done, including the stripSuffix change. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If

[GitHub] spark pull request: SPARK-2646. log4j initialization not quite com...

2014-07-30 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1547#issuecomment-50600089 QA tests have started for PR 1547. This patch merges cleanly. brView progress: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/17442/consoleFull ---

[GitHub] spark pull request: [SPARK-2532] Consolidated shuffle fixes

2014-07-30 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1609#issuecomment-50600899 QA tests have started for PR 1609. This patch merges cleanly. brView progress: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/17443/consoleFull ---

[GitHub] spark pull request: Generic Binary File Support in Spark

2014-07-30 Thread kmader
GitHub user kmader opened a pull request: https://github.com/apache/spark/pull/1658 Generic Binary File Support in Spark The additions add the abstract BinaryFileInputFormat and BinaryRecordReader classes for reading in data as a byte stream and converting it to another format

[GitHub] spark pull request: SPARK-2646. log4j initialization not quite com...

2014-07-30 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1547#issuecomment-50603304 QA results for PR 1547:br- This patch FAILED unit tests.br- This patch merges cleanlybr- This patch adds no public classesbrbrFor more information see test

[GitHub] spark pull request: Generic Binary File Support in Spark

2014-07-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/1658#issuecomment-50603528 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: SPARK-2748 [MLLIB] [GRAPHX] Loss of precision ...

2014-07-30 Thread srowen
GitHub user srowen opened a pull request: https://github.com/apache/spark/pull/1659 SPARK-2748 [MLLIB] [GRAPHX] Loss of precision for small arguments to Math.exp, Math.log In a few places in MLlib, an expression of the form `log(1.0 + p)` is evaluated. When p is so small that `1.0

[GitHub] spark pull request: Avoid numerical instability

2014-07-30 Thread srowen
Github user srowen commented on the pull request: https://github.com/apache/spark/pull/1652#issuecomment-50603835 See also https://issues.apache.org/jira/browse/SPARK-2748 and https://github.com/apache/spark/pull/1659 . This could be considered part of SPARK-2748. --- If your

  1   2   3   4   5   6   7   >