[GitHub] spark pull request: [SPARK-1266] persist factors in implicit ALS

2014-03-18 Thread mengxr
Github user mengxr commented on the pull request: https://github.com/apache/spark/pull/165#issuecomment-37902246 Thanks a lot! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark pull request: [SPARK-1265] Fix 404 error from Jetty 9 upgrad...

2014-03-18 Thread witgo
Github user witgo commented on the pull request: https://github.com/apache/spark/pull/163#issuecomment-37902518 #150 the PR can add filters to static resources. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-1265] Fix 404 error from Jetty 9 upgrad...

2014-03-18 Thread pwendell
Github user pwendell commented on the pull request: https://github.com/apache/spark/pull/163#issuecomment-37903185 Hey @witgo thanks! Just wondering - why did you close #150? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as

[GitHub] spark pull request: Fix SPARK-1256: Master web UI and Worker web U...

2014-03-18 Thread witgo
Github user witgo commented on the pull request: https://github.com/apache/spark/pull/150#issuecomment-37903246 High degree of overlap with #163 . Once #163 merged into master to see whether to reopen --- If your project is set up for it, you can reply to this email and have your

[GitHub] spark pull request: [SPARK-1133] add small files input in MLlib

2014-03-18 Thread mengxr
Github user mengxr commented on a diff in the pull request: https://github.com/apache/spark/pull/164#discussion_r10692101 --- Diff: mllib/src/main/java/org/apache/spark/mllib/util/BatchFileRecordReader.java --- @@ -0,0 +1,117 @@ +/* + * Licensed to the Apache Software

[GitHub] spark pull request: [SPARK-1132] Persisting Web UI through refacto...

2014-03-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/42#issuecomment-37903798 All automated tests passed. Refer to this link for build results: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/13227/ --- If your project

[GitHub] spark pull request: Revert SPARK-1236 - Upgrade Jetty to 9.1.3.v2...

2014-03-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/167#issuecomment-37903945 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: Bugfixes/improvements to scheduler

2014-03-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/159#issuecomment-37903952 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: Fix SPARK-1256: Master web UI and Worker web U...

2014-03-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/150#issuecomment-37903960 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [WIP] [MLLIB-28] An optimized GradientDescent ...

2014-03-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/166#issuecomment-37903948 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: Revert SPARK-1236 - Upgrade Jetty to 9.1.3.v2...

2014-03-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/167#issuecomment-37903946 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [WIP] [MLLIB-28] An optimized GradientDescent ...

2014-03-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/166#issuecomment-37903949 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: Principal Component Analysis

2014-03-18 Thread rezazadeh
Github user rezazadeh commented on a diff in the pull request: https://github.com/apache/spark/pull/88#discussion_r10692389 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/linalg/PCA.scala --- @@ -0,0 +1,153 @@ +/* + * Licensed to the Apache Software Foundation (ASF)

[GitHub] spark pull request: SPARK-1236 - Upgrade Jetty to 9.1.3.v20140225.

2014-03-18 Thread rxin
Github user rxin commented on the pull request: https://github.com/apache/spark/pull/113#issuecomment-37904304 We are reverting this pull request in #167 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark pull request: Principal Component Analysis

2014-03-18 Thread rezazadeh
Github user rezazadeh commented on a diff in the pull request: https://github.com/apache/spark/pull/88#discussion_r10692407 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/linalg/PCA.scala --- @@ -0,0 +1,153 @@ +/* + * Licensed to the Apache Software Foundation (ASF)

[GitHub] spark pull request: [SPARK-1133] add small files input in MLlib

2014-03-18 Thread yinxusen
Github user yinxusen commented on a diff in the pull request: https://github.com/apache/spark/pull/164#discussion_r10692570 --- Diff: mllib/src/main/java/org/apache/spark/mllib/util/BatchFileRecordReader.java --- @@ -0,0 +1,117 @@ +/* + * Licensed to the Apache Software

[GitHub] spark pull request: Spark 1095 : Adding explicit return types to a...

2014-03-18 Thread NirmalReddy
GitHub user NirmalReddy opened a pull request: https://github.com/apache/spark/pull/168 Spark 1095 : Adding explicit return types to all public methods Excluded those that are self-evident and the cases that are discussed in the mailing list. You can merge this pull request into a

[GitHub] spark pull request: Revert SPARK-1236 - Upgrade Jetty to 9.1.3.v2...

2014-03-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/167#issuecomment-37906511 Merged build finished. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-1266] persist factors in implicit ALS

2014-03-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/165#issuecomment-37906540 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: Revert SPARK-1236 - Upgrade Jetty to 9.1.3.v2...

2014-03-18 Thread rxin
Github user rxin commented on the pull request: https://github.com/apache/spark/pull/167#issuecomment-37906590 Ok I merged this. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark pull request: Spark 1246 add min max to stat counter

2014-03-18 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/144 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request: Revert SPARK-1236 - Upgrade Jetty to 9.1.3.v2...

2014-03-18 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/167 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request: Spark 1095 : Adding explicit return types to a...

2014-03-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/168#issuecomment-37909968 All automated tests passed. Refer to this link for build results: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/13231/ --- If your project

[GitHub] spark pull request: [SPARK-1266] persist factors in implicit ALS

2014-03-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/165#issuecomment-37909964 All automated tests passed. Refer to this link for build results: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/13232/ --- If your project

[GitHub] spark pull request: [SPARK-1266] persist factors in implicit ALS

2014-03-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/165#issuecomment-37909963 Merged build finished. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: Spark 1095 : Adding explicit return types to a...

2014-03-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/168#issuecomment-37909967 Merged build finished. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: Principal Component Analysis

2014-03-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/88#issuecomment-37910602 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: Principal Component Analysis

2014-03-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/88#issuecomment-37910603 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: Principal Component Analysis

2014-03-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/88#issuecomment-37910689 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: Principal Component Analysis

2014-03-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/88#issuecomment-37910690 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-1133] add small files input in MLlib

2014-03-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/164#issuecomment-37910847 One or more automated tests failed Refer to this link for build results: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/13234/ --- If your

[GitHub] spark pull request: [SPARK-1133] add small files input in MLlib

2014-03-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/164#issuecomment-37910845 Merged build finished. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: Principal Component Analysis

2014-03-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/88#issuecomment-37914314 Merged build finished. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: Principal Component Analysis

2014-03-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/88#issuecomment-37914315 All automated tests passed. Refer to this link for build results: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/13237/ --- If your project

[GitHub] spark pull request: Principal Component Analysis

2014-03-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/88#issuecomment-37914316 One or more automated tests failed Refer to this link for build results: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/13236/ --- If your

[GitHub] spark pull request: [WIP] [MLLIB-28] An optimized GradientDescent ...

2014-03-18 Thread liancheng
Github user liancheng commented on a diff in the pull request: https://github.com/apache/spark/pull/166#discussion_r10695930 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/optimization/GradientDescent.scala --- @@ -32,10 +32,10 @@ import

[GitHub] spark pull request: [WIP] [MLLIB-28] An optimized GradientDescent ...

2014-03-18 Thread liancheng
Github user liancheng commented on a diff in the pull request: https://github.com/apache/spark/pull/166#discussion_r10695946 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/optimization/GradientDescent.scala --- @@ -32,10 +32,10 @@ import

[GitHub] spark pull request: [WIP] [MLLIB-28] An optimized GradientDescent ...

2014-03-18 Thread liancheng
Github user liancheng commented on a diff in the pull request: https://github.com/apache/spark/pull/166#discussion_r10695955 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/optimization/GradientDescentWithLocalUpdate.scala --- @@ -0,0 +1,147 @@ +/* + * Licensed to

[GitHub] spark pull request: [WIP] [MLLIB-28] An optimized GradientDescent ...

2014-03-18 Thread liancheng
Github user liancheng commented on a diff in the pull request: https://github.com/apache/spark/pull/166#discussion_r10696002 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/optimization/GradientDescentWithLocalUpdate.scala --- @@ -0,0 +1,147 @@ +/* + * Licensed to

[GitHub] spark pull request: [WIP] [MLLIB-28] An optimized GradientDescent ...

2014-03-18 Thread liancheng
Github user liancheng commented on a diff in the pull request: https://github.com/apache/spark/pull/166#discussion_r10696089 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/optimization/GradientDescentWithLocalUpdate.scala --- @@ -0,0 +1,147 @@ +/* + * Licensed to

[GitHub] spark pull request: [WIP] [MLLIB-28] An optimized GradientDescent ...

2014-03-18 Thread liancheng
Github user liancheng commented on a diff in the pull request: https://github.com/apache/spark/pull/166#discussion_r10696461 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/optimization/GradientDescentWithLocalUpdate.scala --- @@ -0,0 +1,147 @@ +/* + * Licensed to

[GitHub] spark pull request: [WIP] [MLLIB-28] An optimized GradientDescent ...

2014-03-18 Thread liancheng
Github user liancheng commented on a diff in the pull request: https://github.com/apache/spark/pull/166#discussion_r10696552 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/optimization/GradientDescentWithLocalUpdate.scala --- @@ -0,0 +1,147 @@ +/* + * Licensed to

[GitHub] spark pull request: [WIP] [MLLIB-28] An optimized GradientDescent ...

2014-03-18 Thread liancheng
Github user liancheng commented on a diff in the pull request: https://github.com/apache/spark/pull/166#discussion_r10696719 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/optimization/GradientDescentWithLocalUpdate.scala --- @@ -0,0 +1,147 @@ +/* + * Licensed to

[GitHub] spark pull request: [WIP] [MLLIB-28] An optimized GradientDescent ...

2014-03-18 Thread liancheng
Github user liancheng commented on a diff in the pull request: https://github.com/apache/spark/pull/166#discussion_r10696799 --- Diff: mllib/src/test/scala/org/apache/spark/mllib/optimization/GradientDescentWithLocalUpdateSuite.scala --- @@ -0,0 +1,71 @@ +/* + * Licensed

[GitHub] spark pull request: [WIP] [MLLIB-28] An optimized GradientDescent ...

2014-03-18 Thread srowen
Github user srowen commented on the pull request: https://github.com/apache/spark/pull/166#issuecomment-37925067 Broad question: can this simply replace the existing implementation, if it's better? I'd suggest it is important to not let a bunch of different implementations

[GitHub] spark pull request: [SPARK-1103] [WIP] Automatic garbage collectio...

2014-03-18 Thread mridulm
Github user mridulm commented on a diff in the pull request: https://github.com/apache/spark/pull/126#discussion_r10701708 --- Diff: core/src/main/scala/org/apache/spark/ContextCleaner.scala --- @@ -0,0 +1,135 @@ +/* + * Licensed to the Apache Software Foundation (ASF)

[GitHub] spark pull request: [WIP] [MLLIB-28] An optimized GradientDescent ...

2014-03-18 Thread yinxusen
Github user yinxusen commented on the pull request: https://github.com/apache/spark/pull/166#issuecomment-37930965 In fact, if we set the `numInnerIteration = 1`, which is the default setting, then the `GradientDescentWithLocalUpdate` is identical to `GradientDescent`. However, I

[GitHub] spark pull request: [SPARK-1103] [WIP] Automatic garbage collectio...

2014-03-18 Thread mridulm
Github user mridulm commented on a diff in the pull request: https://github.com/apache/spark/pull/126#discussion_r10701805 --- Diff: core/src/main/scala/org/apache/spark/ContextCleaner.scala --- @@ -0,0 +1,135 @@ +/* + * Licensed to the Apache Software Foundation (ASF)

[GitHub] spark pull request: [SPARK-1103] [WIP] Automatic garbage collectio...

2014-03-18 Thread mridulm
Github user mridulm commented on a diff in the pull request: https://github.com/apache/spark/pull/126#discussion_r10702886 --- Diff: core/src/main/scala/org/apache/spark/rdd/RDD.scala --- @@ -1025,6 +1025,14 @@ abstract class RDD[T: ClassTag](

[GitHub] spark pull request: SPARK-1102: Create a saveAsNewAPIHadoopDataset...

2014-03-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12#issuecomment-37933870 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: SPARK-1252. On YARN, use container-log4j.prope...

2014-03-18 Thread tgravescs
Github user tgravescs commented on the pull request: https://github.com/apache/spark/pull/148#issuecomment-37934593 I agree it could be an improvement. I just don't want to cause confusion when their logs go to a different file when they turn debug on. I'm not sure I have any other

[GitHub] spark pull request: bugfix: Wrong Duration in Active Stages in...

2014-03-18 Thread BlackNiuza
Github user BlackNiuza closed the pull request at: https://github.com/apache/spark/pull/169 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request: bugfix: Wrong Duration in Active Stages in...

2014-03-18 Thread BlackNiuza
GitHub user BlackNiuza opened a pull request: https://github.com/apache/spark/pull/170 bugfix: Wrong Duration in Active Stages in stages page If a stage which has completed once loss parts of data, it will be resubmitted. At this time, it appears that stage.completionTime

[GitHub] spark pull request: bugfix: Wrong Duration in Active Stages in...

2014-03-18 Thread BlackNiuza
Github user BlackNiuza commented on the pull request: https://github.com/apache/spark/pull/169#issuecomment-37936861 Sorry for use my master branch. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does

[GitHub] spark pull request: Added doctest for map function in rdd.py

2014-03-18 Thread jyotiska
GitHub user jyotiska opened a pull request: https://github.com/apache/spark/pull/171 Added doctest for map function in rdd.py You can merge this pull request into a Git repository by running: $ git pull https://github.com/jyotiska/spark pyspark_rdd_map_doctest Alternatively

[GitHub] spark pull request: [WIP] [MLLIB-28] An optimized GradientDescent ...

2014-03-18 Thread liancheng
Github user liancheng commented on the pull request: https://github.com/apache/spark/pull/166#issuecomment-37939348 Left some comments on minor issues like formatting. LGTM otherwise. @srowen According to [previous

[GitHub] spark pull request: [SPARK-1103] [WIP] Automatic garbage collectio...

2014-03-18 Thread mridulm
Github user mridulm commented on the pull request: https://github.com/apache/spark/pull/126#issuecomment-37938604 Would be lovely to see this patch in ! We have a whole bunch of hacks just to avoid memory and resource issues currently : this would alleviate a lot of them. --- If

[GitHub] spark pull request: SPARK-1102: Create a saveAsNewAPIHadoopDataset...

2014-03-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12#issuecomment-37940641 All automated tests passed. Refer to this link for build results: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/13238/ --- If your project

[GitHub] spark pull request: SPARK-1102: Create a saveAsNewAPIHadoopDataset...

2014-03-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12#issuecomment-37940639 Merged build finished. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: Added doctest for map function in rdd.py

2014-03-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/171#issuecomment-37940732 Build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request: bugfix: Wrong Duration in Active Stages in...

2014-03-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/170#issuecomment-37940735 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: bugfix: Wrong Duration in Active Stages in...

2014-03-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/170#issuecomment-37940734 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: Added doctest for map function in rdd.py

2014-03-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/171#issuecomment-37940731 Build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request: bugfix: Wrong Duration in Active Stages in...

2014-03-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/170#issuecomment-37940877 One or more automated tests failed Refer to this link for build results: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/13240/ --- If your

[GitHub] spark pull request: bugfix: Wrong Duration in Active Stages in...

2014-03-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/170#issuecomment-37940876 Merged build finished. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [MLLIB-18] [WIP] Adding sparse data support an...

2014-03-18 Thread martinjaggi
Github user martinjaggi commented on a diff in the pull request: https://github.com/apache/spark/pull/117#discussion_r10706373 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/linalg/Vectors.scala --- @@ -0,0 +1,136 @@ +/* + * Licensed to the Apache Software

[GitHub] spark pull request: [SPARK-1268] Adding XOR and AND-NOT operations...

2014-03-18 Thread petko-nikolov
GitHub user petko-nikolov opened a pull request: https://github.com/apache/spark/pull/172 [SPARK-1268] Adding XOR and AND-NOT operations to spark.util.collection.BitSet Symmetric difference (xor) in particular is useful for computing some distance metrics (e.g. Hamming). Unit

[GitHub] spark pull request: Fix SPARK-1256: Master web UI and Worker web U...

2014-03-18 Thread witgo
Github user witgo commented on the pull request: https://github.com/apache/spark/pull/150#issuecomment-37947800 This is a necessary improvement. This PR can work under jetty 7 and jetty 9 --- If your project is set up for it, you can reply to this email and have your reply appear on

[GitHub] spark pull request: [SPARK-1268] Adding XOR and AND-NOT operations...

2014-03-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/172#issuecomment-37949024 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: Added doctest for map function in rdd.py

2014-03-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/171#issuecomment-37948619 All automated tests passed. Refer to this link for build results: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/13239/ --- If your project

[GitHub] spark pull request: Added doctest for map function in rdd.py

2014-03-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/171#issuecomment-37948618 Build finished. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request: Update CommandUtils.scala

2014-03-18 Thread baishuo
Github user baishuo commented on the pull request: https://github.com/apache/spark/pull/157#issuecomment-37954039 Oh,I see,please let me do that --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does

[GitHub] spark pull request: SPARK-1251 Support for optimizing and executin...

2014-03-18 Thread marmbrus
Github user marmbrus commented on a diff in the pull request: https://github.com/apache/spark/pull/146#discussion_r10711356 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/dsl/package.scala --- @@ -0,0 +1,223 @@ +/* + * Licensed to the Apache Software

[GitHub] spark pull request: [SPARK-1103] [WIP] Automatic garbage collectio...

2014-03-18 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/126#discussion_r10711505 --- Diff: core/src/main/scala/org/apache/spark/MapOutputTracker.scala --- @@ -181,15 +178,50 @@ private[spark] class MapOutputTracker(conf: SparkConf) extends

[GitHub] spark pull request: [SPARK-1103] [WIP] Automatic garbage collectio...

2014-03-18 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/126#discussion_r10711579 --- Diff: core/src/main/scala/org/apache/spark/ContextCleaner.scala --- @@ -0,0 +1,135 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under

[GitHub] spark pull request: [SPARK-1103] [WIP] Automatic garbage collectio...

2014-03-18 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/126#discussion_r10711629 --- Diff: core/src/main/scala/org/apache/spark/ContextCleaner.scala --- @@ -0,0 +1,135 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under

[GitHub] spark pull request: [SPARK-1103] [WIP] Automatic garbage collectio...

2014-03-18 Thread tdas
Github user tdas closed the pull request at: https://github.com/apache/spark/pull/126 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request: [SPARK-1103] [WIP] Automatic garbage collectio...

2014-03-18 Thread tdas
GitHub user tdas reopened a pull request: https://github.com/apache/spark/pull/126 [SPARK-1103] [WIP] Automatic garbage collection of RDD, shuffle and broadcast data This PR allows Spark to automatically cleanup metadata and data related to persisted RDDs, shuffles and broadcast

[GitHub] spark pull request: bugfix: Wrong Duration in Active Stages in...

2014-03-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/170#issuecomment-37957058 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: bugfix: Wrong Duration in Active Stages in...

2014-03-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/170#issuecomment-37957056 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-1103] [WIP] Automatic garbage collectio...

2014-03-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/126#issuecomment-37957084 Build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request: [SPARK-1103] [WIP] Automatic garbage collectio...

2014-03-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/126#issuecomment-37957083 Build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request: [MLLIB-18] [WIP] Adding sparse data support an...

2014-03-18 Thread mengxr
Github user mengxr commented on a diff in the pull request: https://github.com/apache/spark/pull/117#discussion_r10713190 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/linalg/Vectors.scala --- @@ -0,0 +1,136 @@ +/* + * Licensed to the Apache Software Foundation

[GitHub] spark pull request: [SPARK-1265] Fix 404 error from Jetty 9 upgrad...

2014-03-18 Thread andrewor14
Github user andrewor14 closed the pull request at: https://github.com/apache/spark/pull/163 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request: Adding an option to persist Spark RDD blocks ...

2014-03-18 Thread nicklan
Github user nicklan commented on a diff in the pull request: https://github.com/apache/spark/pull/158#discussion_r10713987 --- Diff: core/src/main/scala/org/apache/spark/storage/BlockManager.scala --- @@ -97,6 +101,23 @@ private[spark] class BlockManager( var

[GitHub] spark pull request: bugfix: Wrong Duration in Active Stages in...

2014-03-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/170#issuecomment-37964396 One or more automated tests failed Refer to this link for build results: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/13241/ --- If your

[GitHub] spark pull request: [SPARK-1103] [WIP] Automatic garbage collectio...

2014-03-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/126#issuecomment-37964411 Build finished. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request: SPARK-1102: Create a saveAsNewAPIHadoopDataset...

2014-03-18 Thread mateiz
Github user mateiz commented on the pull request: https://github.com/apache/spark/pull/12#issuecomment-37967135 Thanks Nan, I've merged this in. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-1268] Adding XOR and AND-NOT operations...

2014-03-18 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/172#discussion_r10718417 --- Diff: core/src/main/scala/org/apache/spark/util/collection/BitSet.scala --- @@ -88,6 +88,53 @@ class BitSet(numBits: Int) extends Serializable {

[GitHub] spark pull request: [SPARK-1268] Adding XOR and AND-NOT operations...

2014-03-18 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/172#discussion_r10718405 --- Diff: core/src/main/scala/org/apache/spark/util/collection/BitSet.scala --- @@ -88,6 +88,53 @@ class BitSet(numBits: Int) extends Serializable {

[GitHub] spark pull request: [SPARK-1268] Adding XOR and AND-NOT operations...

2014-03-18 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/172#discussion_r10718473 --- Diff: core/src/main/scala/org/apache/spark/util/collection/BitSet.scala --- @@ -88,6 +88,53 @@ class BitSet(numBits: Int) extends Serializable {

[GitHub] spark pull request: [SPARK-1133] add small files input in MLlib

2014-03-18 Thread mengxr
Github user mengxr commented on the pull request: https://github.com/apache/spark/pull/164#issuecomment-37970513 @yinxusen I'm still confused by the implementation. You want to make the entire content of a text file as the value returned by smallTextFiles. In the

[GitHub] spark pull request: [SPARK-1268] Adding XOR and AND-NOT operations...

2014-03-18 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/172#discussion_r10718582 --- Diff: core/src/main/scala/org/apache/spark/util/collection/BitSet.scala --- @@ -88,6 +88,53 @@ class BitSet(numBits: Int) extends Serializable {

[GitHub] spark pull request: [SPARK-1268] Adding XOR and AND-NOT operations...

2014-03-18 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/172#discussion_r10718634 --- Diff: core/src/test/scala/org/apache/spark/util/collection/BitSetSuite.scala --- @@ -69,4 +69,45 @@ class BitSetSuite extends FunSuite {

[GitHub] spark pull request: [SPARK-1268] Adding XOR and AND-NOT operations...

2014-03-18 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/172#discussion_r10718623 --- Diff: core/src/test/scala/org/apache/spark/util/collection/BitSetSuite.scala --- @@ -69,4 +69,45 @@ class BitSetSuite extends FunSuite {

[GitHub] spark pull request: [SPARK-1268] Adding XOR and AND-NOT operations...

2014-03-18 Thread rxin
Github user rxin commented on the pull request: https://github.com/apache/spark/pull/172#issuecomment-37970799 Hi @petko-nikolov, Thanks a lot for contributing this patch! I left some comments to help the code conform to Spark coding style, and on test coverage. It would be

[GitHub] spark pull request: [SPARK-1266] persist factors in implicit ALS

2014-03-18 Thread mengxr
Github user mengxr closed the pull request at: https://github.com/apache/spark/pull/165 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request: [SPARK-1266] persist factors in implicit ALS

2014-03-18 Thread mengxr
GitHub user mengxr reopened a pull request: https://github.com/apache/spark/pull/165 [SPARK-1266] persist factors in implicit ALS In implicit ALS computation, the user or product factor is used twice in each iteration. Caching can certainly help accelerate the computation. I saw

[GitHub] spark pull request: [MLLIB-18] [WIP] Adding sparse data support an...

2014-03-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/117#issuecomment-37973000 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [MLLIB-18] [WIP] Adding sparse data support an...

2014-03-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/117#issuecomment-37973290 Merged build finished. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

  1   2   3   >