[GitHub] spark pull request: [SPARK-4614][MLLIB] Slight API changes in Matr...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3468#issuecomment-64526546 [Test build #23884 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/23884/consoleFull) for PR 3468 at commit [`3b0e4e2`](https://github.com/apache/spark/commit/3b0e4e213d407b5fb42a4fd06bb157e3d2d10941). * This patch merges cleanly. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-4611][MLlib] Implement the efficient ve...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3462#issuecomment-64527790 [Test build #23886 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/23886/consoleFull) for PR 3462 at commit [`0c3637f`](https://github.com/apache/spark/commit/0c3637f05fafc4be89a46816eabfc30274be7759). * This patch merges cleanly. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-4611][MLlib] Implement the efficient ve...
Github user srowen commented on a diff in the pull request:
https://github.com/apache/spark/pull/3462#discussion_r20921760
--- Diff: mllib/src/main/scala/org/apache/spark/mllib/linalg/Vectors.scala
---
@@ -261,6 +261,57 @@ object Vectors {
sys.error(Unsupported Breeze vector type: + v.getClass.getName)
}
}
+
+ /**
+ * Returns the p-norm of this vector.
+ * @param vector input vector.
+ * @param p norm.
+ * @return norm in L^p^ space.
+ */
+ private[spark] def norm(vector: Vector, p: Double): Double = {
+require(p = 1.0)
+val values = vector match {
+ case dv: DenseVector = dv.values
+ case sv: SparseVector = sv.values
+ case v = throw new IllegalArgumentException(Do not support vector
type + v.getClass)
+}
+val size = values.size
+
+if (p == 1) {
--- End diff --
How about `p match { ...` here? with `@switch` to ensure it's just a
lookup? should be faster even than `if`s.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-4611][MLlib] Implement the efficient ve...
Github user srowen commented on a diff in the pull request:
https://github.com/apache/spark/pull/3462#discussion_r20921781
--- Diff: mllib/src/main/scala/org/apache/spark/mllib/linalg/Vectors.scala
---
@@ -261,6 +261,57 @@ object Vectors {
sys.error(Unsupported Breeze vector type: + v.getClass.getName)
}
}
+
+ /**
+ * Returns the p-norm of this vector.
+ * @param vector input vector.
+ * @param p norm.
+ * @return norm in L^p^ space.
+ */
+ private[spark] def norm(vector: Vector, p: Double): Double = {
+require(p = 1.0)
+val values = vector match {
+ case dv: DenseVector = dv.values
+ case sv: SparseVector = sv.values
+ case v = throw new IllegalArgumentException(Do not support vector
type + v.getClass)
+}
+val size = values.size
+
+if (p == 1) {
+ var sum = 0.0
+ var i = 0
+ while (i size) {
+sum += math.abs(values(i))
+i += 1
+ }
+ sum
+} else if (p == 2) {
+ var sum = 0.0
+ var i = 0
+ while (i size) {
+sum += values(i) * values(i)
--- End diff --
Since this is a common case and tight loop, avoid the duplicated value
lookup?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-4619][Storage]delete redundant time suf...
GitHub user maji2014 opened a pull request: https://github.com/apache/spark/pull/3475 [SPARK-4619][Storage]delete redundant time suffix Time suffix exists in Utils.getUsedTimeMs(startTime), no need to append again, delete that You can merge this pull request into a Git repository by running: $ git pull https://github.com/maji2014/spark SPARK-4619 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/spark/pull/3475.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3475 commit df0da4edec947b7aab4fe3331878641a4ab65d37 Author: maji2014 ma...@asiainfo.com Date: 2014-11-26T08:29:26Z delete redundant time suffix --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-4611][MLlib] Implement the efficient ve...
Github user dbtsai commented on a diff in the pull request:
https://github.com/apache/spark/pull/3462#discussion_r20921838
--- Diff: mllib/src/main/scala/org/apache/spark/mllib/linalg/Vectors.scala
---
@@ -261,6 +261,57 @@ object Vectors {
sys.error(Unsupported Breeze vector type: + v.getClass.getName)
}
}
+
+ /**
+ * Returns the p-norm of this vector.
+ * @param vector input vector.
+ * @param p norm.
+ * @return norm in L^p^ space.
+ */
+ private[spark] def norm(vector: Vector, p: Double): Double = {
+require(p = 1.0)
+val values = vector match {
+ case dv: DenseVector = dv.values
+ case sv: SparseVector = sv.values
+ case v = throw new IllegalArgumentException(Do not support vector
type + v.getClass)
+}
+val size = values.size
+
+if (p == 1) {
+ var sum = 0.0
+ var i = 0
+ while (i size) {
+sum += math.abs(values(i))
+i += 1
+ }
+ sum
+} else if (p == 2) {
+ var sum = 0.0
+ var i = 0
+ while (i size) {
+sum += values(i) * values(i)
--- End diff --
I tried that, and JVM will optimize it so no performance difference.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-4619][Storage]delete redundant time suf...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/3475#issuecomment-64530371 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-4611][MLlib] Implement the efficient ve...
Github user dbtsai commented on a diff in the pull request:
https://github.com/apache/spark/pull/3462#discussion_r20921892
--- Diff: mllib/src/main/scala/org/apache/spark/mllib/linalg/Vectors.scala
---
@@ -261,6 +261,57 @@ object Vectors {
sys.error(Unsupported Breeze vector type: + v.getClass.getName)
}
}
+
+ /**
+ * Returns the p-norm of this vector.
+ * @param vector input vector.
+ * @param p norm.
+ * @return norm in L^p^ space.
+ */
+ private[spark] def norm(vector: Vector, p: Double): Double = {
+require(p = 1.0)
+val values = vector match {
+ case dv: DenseVector = dv.values
+ case sv: SparseVector = sv.values
+ case v = throw new IllegalArgumentException(Do not support vector
type + v.getClass)
+}
+val size = values.size
+
+if (p == 1) {
+ var sum = 0.0
+ var i = 0
+ while (i size) {
+sum += math.abs(values(i))
+i += 1
+ }
+ sum
+} else if (p == 2) {
+ var sum = 0.0
+ var i = 0
+ while (i size) {
+sum += values(i) * values(i)
--- End diff --
will try to see if scala generates the same bytecode tomorrow. Maybe scala
compiler optimizes it.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-4611][MLlib] Implement the efficient ve...
Github user dbtsai commented on a diff in the pull request:
https://github.com/apache/spark/pull/3462#discussion_r20921916
--- Diff: mllib/src/main/scala/org/apache/spark/mllib/linalg/Vectors.scala
---
@@ -261,6 +261,57 @@ object Vectors {
sys.error(Unsupported Breeze vector type: + v.getClass.getName)
}
}
+
+ /**
+ * Returns the p-norm of this vector.
+ * @param vector input vector.
+ * @param p norm.
+ * @return norm in L^p^ space.
+ */
+ private[spark] def norm(vector: Vector, p: Double): Double = {
+require(p = 1.0)
+val values = vector match {
+ case dv: DenseVector = dv.values
+ case sv: SparseVector = sv.values
+ case v = throw new IllegalArgumentException(Do not support vector
type + v.getClass)
+}
+val size = values.size
+
+if (p == 1) {
--- End diff --
Interesting, will try tomorrow. But I don't expect too much difference.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: Removing confusing TripletFields
Github user rxin commented on the pull request: https://github.com/apache/spark/pull/3472#issuecomment-64532086 Merging in master branch-1.2. Thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-4611][MLlib] Implement the efficient ve...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/3462#issuecomment-64532440 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/23881/ Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-4611][MLlib] Implement the efficient ve...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3462#issuecomment-64532433 [Test build #23881 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/23881/consoleFull) for PR 3462 at commit [`9b7cb56`](https://github.com/apache/spark/commit/9b7cb563d0c0521d2e6797786a9256a431abb941). * This patch **passes all tests**. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-4620] Add unpersist in Graph and GraphI...
GitHub user maropu opened a pull request: https://github.com/apache/spark/pull/3476 [SPARK-4620] Add unpersist in Graph and GraphImpl Add an IF to uncache both vertices and edges of Graph/GraphImpl. This IF is useful when iterative graph operations build a new graph in each iteration, and the vertices and edges of previous iterations are no longer needed for following iterations. You can merge this pull request into a Git repository by running: $ git pull https://github.com/maropu/spark UnpersistInGraphSpike Alternatively you can review and apply these changes as the patch at: https://github.com/apache/spark/pull/3476.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3476 commit 77a006a77889a2f847dc0a6ad2e8e15e329b9137 Author: Takeshi Yamamuro linguin@gmail.com Date: 2014-11-26T08:51:48Z Add unpersist in Graph and GraphImpl --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-4620] Add unpersist in Graph and GraphI...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/3476#issuecomment-64532938 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [BRANCH-1.2][SPARK-4604][MLLIB] make MatrixFac...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/3473#issuecomment-64534413 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/23882/ Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [BRANCH-1.2][SPARK-4604][MLLIB] make MatrixFac...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3473#issuecomment-64534404 [Test build #23882 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/23882/consoleFull) for PR 3473 at commit [`fcdc2df`](https://github.com/apache/spark/commit/fcdc2df50f819e1b81bd2c444ab8b52d72507ef2). * This patch **passes all tests**. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-4611][MLlib] Implement the efficient ve...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/3462#issuecomment-64535783 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/23885/ Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-4611][MLlib] Implement the efficient ve...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3462#issuecomment-64535777 [Test build #23885 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/23885/consoleFull) for PR 3462 at commit [`6fa616c`](https://github.com/apache/spark/commit/6fa616cd6e68e0e1a8c87551925e8b2d2af7c11b). * This patch **passes all tests**. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-4546] Improve HistoryServer first time ...
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/3411 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-4196][SPARK-4602][Streaming] Fix serial...
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/3457 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-4516] Avoid allocating Netty PooledByte...
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/3465 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-4604][MLLIB] make MatrixFactorizationMo...
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/3459 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-4614][MLLIB] Slight API changes in Matr...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3468#issuecomment-64536426 [Test build #23884 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/23884/consoleFull) for PR 3468 at commit [`3b0e4e2`](https://github.com/apache/spark/commit/3b0e4e213d407b5fb42a4fd06bb157e3d2d10941). * This patch **passes all tests**. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-4612] Reduce task latency and increase ...
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/3463 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: Removing confusing TripletFields
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/3472 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [BRANCH-1.2][SPARK-4583][MLLIB] LogLoss for Gr...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/3474#issuecomment-64536497 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/23883/ Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-4583] [mllib] LogLoss for GradientBoost...
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/3439 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: Fix SPARK-4471: blockManagerIdFromJson functio...
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/3340 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-4592] Avoid duplicate worker registrati...
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/3447 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-4516] Cap default number of Netty threa...
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/3469 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-4611][MLlib] Implement the efficient ve...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/3462#issuecomment-64537183 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/23886/ Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-4611][MLlib] Implement the efficient ve...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3462#issuecomment-64537174 [Test build #23886 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/23886/consoleFull) for PR 3462 at commit [`0c3637f`](https://github.com/apache/spark/commit/0c3637f05fafc4be89a46816eabfc30274be7759). * This patch **passes all tests**. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-4537][Streaming] Expand StreamingSource...
Github user maasg commented on a diff in the pull request:
https://github.com/apache/spark/pull/3466#discussion_r20924293
--- Diff:
streaming/src/main/scala/org/apache/spark/streaming/StreamingSource.scala ---
@@ -70,4 +78,14 @@ private[streaming] class StreamingSource(ssc:
StreamingContext) extends Source {
_.lastCompletedBatch.flatMap(_.processingStartTime).getOrElse(-1L),
-1L)
registerGauge(lastReceivedBatch_processEndTime,
_.lastCompletedBatch.flatMap(_.processingEndTime).getOrElse(-1L), -1L)
+
+ // Gauge for last received batch records and total received batch
records.
+ private var totalReceivedBatchRecords: Long = 0L
+ def getTotalReceivedBatchRecords(listener:
StreamingJobProgressListener): Long = {
+totalReceivedBatchRecords +=
listener.lastReceivedBatchRecords.values.sum
--- End diff --
Will this counter work? I think that gauges are collected only on request
of a source, so if nobody is consuming the metric or consuming it too often, we
will have a wrong count.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-4193][BUILD] Disable doclint in Java 8 ...
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/3058#discussion_r20924923 --- Diff: pom.xml --- @@ -1242,6 +1242,25 @@ /profile +profile + iddoclint-java8-disable/id + activation +jdk[1.8,)/jdk + /activation + + build +plugins + plugin +groupIdorg.apache.maven.plugins/groupId +artifactIdmaven-javadoc-plugin/artifactId --- End diff -- @ueshin One last tiny thing. I think this plugin should be declared in `pluginManagement`, and should add `version2.10.1/version`, but would not have `configuration` of course. Then this is left exactly as-is. All -- I think this may help the problem in SPARK-4543? It is good to commit this fix anyway but may also simply silence Java 8's warnings on missing javadoc as desired. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-4543] Javadoc failure for network-commo...
Github user srowen commented on the pull request: https://github.com/apache/spark/pull/3405#issuecomment-64539308 I think we should add @ueshin 's patch at https://github.com/apache/spark/pull/3058 which should properly disable javadoc 8 missing javadoc warnings anyway. I think this is indeed Java 8-specific after all; I'm running Java 8. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SparkSQL][SPARK-4528] add comment support for...
GitHub user tsingfu opened a pull request: https://github.com/apache/spark/pull/3477 [SparkSQL][SPARK-4528] add comment support for Spark SQL CLI spark-sql while using *spark-sql*, found it does not support comment while write sqls. **Comment support** is widely using in many projects, it is a necessary to add this feature. this implitation can be archived in the source org.apache.spark.sql.hive.thriftserver.SparkSQLCLIDriver.scala. # three comment styles supported: 1. From a '#' character to the end of the line. 2. From a '--' sequence to the end of the line 3. From a /* sequence to the following */ sequence, as in the C programming language. This syntax allows a comment to extend over multiple lines because the beginning and closing sequences need not be on the same line. You can merge this pull request into a Git repository by running: $ git pull https://github.com/tsingfu/spark SPARK-4528 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/spark/pull/3477.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3477 commit 390aaf2b348bdc70bcfd1fa2a64c8b481deafa6f Author: tsingfu yangfuq...@gmail.com Date: 2014-11-26T10:01:24Z [SparkSQL][SPARK-4528] add comment support for Spark SQL CLI spark-sql --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SparkSQL][SPARK-4528] add comment support for...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/3477#issuecomment-64541553 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-4613][Core] Java API for JdbcRDD
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3478#issuecomment-64545160 [Test build #23888 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/23888/consoleFull) for PR 3478 at commit [`ffcdf2e`](https://github.com/apache/spark/commit/ffcdf2ed4d7c15c2d41b99bd99583edcc4e06248). * This patch merges cleanly. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-4193][BUILD] Disable doclint in Java 8 ...
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/3058#discussion_r20927574 --- Diff: pom.xml --- @@ -1242,6 +1242,25 @@ /profile +profile + iddoclint-java8-disable/id + activation +jdk[1.8,)/jdk + /activation + + build +plugins + plugin +groupIdorg.apache.maven.plugins/groupId +artifactIdmaven-javadoc-plugin/artifactId --- End diff -- @srowen Thank you for your suggestion and I agree with it. I'll add it to the `pluginManagement`. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-4193][BUILD] Disable doclint in Java 8 ...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3058#issuecomment-64546025 [Test build #23889 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/23889/consoleFull) for PR 3058 at commit [`e096bb1`](https://github.com/apache/spark/commit/e096bb13ffc13041b05a5f3cd640900383b0010c). * This patch merges cleanly. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [BRANCH-1.2][SPARK-4583][MLLIB] LogLoss for Gr...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3474#issuecomment-64547142 [Test build #23890 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/23890/consoleFull) for PR 3474 at commit [`aca2abb`](https://github.com/apache/spark/commit/aca2abbbcf0ad14116f1b54ddd1b61e96803e38c). * This patch merges cleanly. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [BRANCH-1.2][SPARK-4604][MLLIB] make MatrixFac...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3473#issuecomment-64547174 [Test build #23891 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/23891/consoleFull) for PR 3473 at commit [`a7638a5`](https://github.com/apache/spark/commit/a7638a5ca0190598a408da3af0bc8c557ed6776b). * This patch merges cleanly. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-4613][Core] Java API for JdbcRDD
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/3478#issuecomment-64579267 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/23888/ Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-4613][Core] Java API for JdbcRDD
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3478#issuecomment-64579183 [Test build #23888 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/23888/consoleFull) for PR 3478 at commit [`ffcdf2e`](https://github.com/apache/spark/commit/ffcdf2ed4d7c15c2d41b99bd99583edcc4e06248). * This patch **fails Spark unit tests**. * This patch merges cleanly. * This patch adds the following public classes _(experimental)_: * ` trait ConnectionFactory extends Serializable ` --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-4608][Streaming] Reorganize StreamingCo...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/3464#issuecomment-64581017 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/23887/ Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-4608][Streaming] Reorganize StreamingCo...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3464#issuecomment-64580965 [Test build #23887 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/23887/consoleFull) for PR 3464 at commit [`f74c190`](https://github.com/apache/spark/commit/f74c190b0a28ac81c1a2059724a69012e342). * This patch **passes all tests**. * This patch merges cleanly. * This patch adds the following public classes _(experimental)_: * `class MatrixFactorizationModel(` --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-4193][BUILD] Disable doclint in Java 8 ...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3058#issuecomment-64596000 [Test build #23889 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/23889/consoleFull) for PR 3058 at commit [`e096bb1`](https://github.com/apache/spark/commit/e096bb13ffc13041b05a5f3cd640900383b0010c). * This patch **passes all tests**. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-4193][BUILD] Disable doclint in Java 8 ...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/3058#issuecomment-64596006 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/23889/ Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-4537][Streaming] Expand StreamingSource...
Github user jerryshao commented on a diff in the pull request:
https://github.com/apache/spark/pull/3466#discussion_r20931831
--- Diff:
streaming/src/main/scala/org/apache/spark/streaming/StreamingSource.scala ---
@@ -70,4 +78,14 @@ private[streaming] class StreamingSource(ssc:
StreamingContext) extends Source {
_.lastCompletedBatch.flatMap(_.processingStartTime).getOrElse(-1L),
-1L)
registerGauge(lastReceivedBatch_processEndTime,
_.lastCompletedBatch.flatMap(_.processingEndTime).getOrElse(-1L), -1L)
+
+ // Gauge for last received batch records and total received batch
records.
+ private var totalReceivedBatchRecords: Long = 0L
+ def getTotalReceivedBatchRecords(listener:
StreamingJobProgressListener): Long = {
+totalReceivedBatchRecords +=
listener.lastReceivedBatchRecords.values.sum
--- End diff --
Aha, you're right, so it is hard to collect the total batch records without
modifying the `StreamingJobProgressListener` code.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [BRANCH-1.2][SPARK-4583][MLLIB] LogLoss for Gr...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3474#issuecomment-64596891 [Test build #23890 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/23890/consoleFull) for PR 3474 at commit [`aca2abb`](https://github.com/apache/spark/commit/aca2abbbcf0ad14116f1b54ddd1b61e96803e38c). * This patch **fails Spark unit tests**. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [BRANCH-1.2][SPARK-4583][MLLIB] LogLoss for Gr...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/3474#issuecomment-64596901 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/23890/ Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-4537][Streaming] Expand StreamingSource...
Github user jerryshao commented on a diff in the pull request:
https://github.com/apache/spark/pull/3466#discussion_r20932324
--- Diff:
streaming/src/main/scala/org/apache/spark/streaming/StreamingSource.scala ---
@@ -70,4 +78,14 @@ private[streaming] class StreamingSource(ssc:
StreamingContext) extends Source {
_.lastCompletedBatch.flatMap(_.processingStartTime).getOrElse(-1L),
-1L)
registerGauge(lastReceivedBatch_processEndTime,
_.lastCompletedBatch.flatMap(_.processingEndTime).getOrElse(-1L), -1L)
+
+ // Gauge for last received batch records and total received batch
records.
+ private var totalReceivedBatchRecords: Long = 0L
+ def getTotalReceivedBatchRecords(listener:
StreamingJobProgressListener): Long = {
+totalReceivedBatchRecords +=
listener.lastReceivedBatchRecords.values.sum
--- End diff --
I will fix this, thanks a lot.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [BRANCH-1.2][SPARK-4604][MLLIB] make MatrixFac...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/3473#issuecomment-64598815 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/23891/ Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [BRANCH-1.2][SPARK-4604][MLLIB] make MatrixFac...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3473#issuecomment-64598806 [Test build #23891 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/23891/consoleFull) for PR 3473 at commit [`a7638a5`](https://github.com/apache/spark/commit/a7638a5ca0190598a408da3af0bc8c557ed6776b). * This patch **passes all tests**. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-4623]Add the some error infomation if u...
GitHub user SaintBacchus opened a pull request: https://github.com/apache/spark/pull/3479 [SPARK-4623]Add the some error infomation if using spark-sql in yarn-cluster mode If using spark-sql in yarn-cluster mode, print an error infomation just as the spark shell in yarn-cluster mode. You can merge this pull request into a Git repository by running: $ git pull https://github.com/SaintBacchus/spark sparkSqlShell Alternatively you can review and apply these changes as the patch at: https://github.com/apache/spark/pull/3479.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3479 commit 4fad75ad0c2f906cf7c8477321b906ed5b82463a Author: carlmartin carlmartin...@gmail.com Date: 2014-11-26T12:04:16Z Add the Error infomation using spark-sql in yarn-cluster mode commit 7bcecc296c4745c0a19a8ca8be67b992230dca62 Author: carlmartin carlmartin...@gmail.com Date: 2014-11-26T12:06:58Z Merge branch 'master' of https://github.com/apache/spark into codereview commit ec957bc3d065c2db70838568da59f1b2dd2bf211 Author: carlmartin carlmartin...@gmail.com Date: 2014-11-26T12:30:11Z Add the some error infomation if using spark-sql in yarn-cluster mode --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-4623]Add the some error infomation if u...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/3479#issuecomment-64599810 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-2458] Make failed application log visib...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3467#issuecomment-64601739 [Test build #23892 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/23892/consoleFull) for PR 3467 at commit [`e8cdf7b`](https://github.com/apache/spark/commit/e8cdf7b7ab72f8b441496156dfb0dce6acdcc5a8). * This patch merges cleanly. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-732][SPARK-3628][CORE][RESUBMIT] elimin...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2524#issuecomment-64603187 [Test build #23893 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/23893/consoleFull) for PR 2524 at commit [`b233737`](https://github.com/apache/spark/commit/b23373746407de3a7c05fbeaf00f67fe78860623). * This patch merges cleanly. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: SPARK-2192 [BUILD] Examples Data Not in Binary...
GitHub user srowen opened a pull request: https://github.com/apache/spark/pull/3480 SPARK-2192 [BUILD] Examples Data Not in Binary Distribution Simply, add data/ to distributions. This adds about 291KB (compressed) to the tarball, FYI. You can merge this pull request into a Git repository by running: $ git pull https://github.com/srowen/spark SPARK-2192 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/spark/pull/3480.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3480 commit 47688f132bd342bce1096480b83b1d27aa8bfd04 Author: Sean Owen so...@cloudera.com Date: 2014-11-26T13:33:14Z Add data/ to distributions --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: SPARK-2192 [BUILD] Examples Data Not in Binary...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3480#issuecomment-64608592 [Test build #23894 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/23894/consoleFull) for PR 3480 at commit [`47688f1`](https://github.com/apache/spark/commit/47688f132bd342bce1096480b83b1d27aa8bfd04). * This patch merges cleanly. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-4623]Add the some error infomation if u...
Github user WangTaoTheTonic commented on a diff in the pull request:
https://github.com/apache/spark/pull/3479#discussion_r20936956
--- Diff: core/src/main/scala/org/apache/spark/deploy/SparkSubmit.scala ---
@@ -142,6 +142,8 @@ object SparkSubmit {
printErrorAndExit(Cluster deploy mode is currently not supported
for python applications.)
case (_, CLUSTER) if isShell(args.primaryResource) =
printErrorAndExit(Cluster deploy mode is not applicable to Spark
shells.)
+ case (_, CLUSTER) if isSqlShell(args.mainClass) =
+printErrorAndExit(Cluster deploy mode is not applicable to Spark
Sql shells.)
--- End diff --
“shells should be in singular form.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-2458] Make failed application log visib...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3467#issuecomment-64653027 [Test build #23892 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/23892/consoleFull) for PR 3467 at commit [`e8cdf7b`](https://github.com/apache/spark/commit/e8cdf7b7ab72f8b441496156dfb0dce6acdcc5a8). * This patch **passes all tests**. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-2458] Make failed application log visib...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/3467#issuecomment-64653042 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/23892/ Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-4623]Add the some error infomation if u...
Github user SaintBacchus commented on a diff in the pull request:
https://github.com/apache/spark/pull/3479#discussion_r20937437
--- Diff: core/src/main/scala/org/apache/spark/deploy/SparkSubmit.scala ---
@@ -142,6 +142,8 @@ object SparkSubmit {
printErrorAndExit(Cluster deploy mode is currently not supported
for python applications.)
case (_, CLUSTER) if isShell(args.primaryResource) =
printErrorAndExit(Cluster deploy mode is not applicable to Spark
shells.)
+ case (_, CLUSTER) if isSqlShell(args.mainClass) =
+printErrorAndExit(Cluster deploy mode is not applicable to Spark
Sql shells.)
--- End diff --
Maybe it will have an pyspark-sql later just as the pyspark-shell
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: SPARK-3779. yarn spark.yarn.applicationMaster....
Github user WangTaoTheTonic commented on the pull request: https://github.com/apache/spark/pull/3471#issuecomment-64653209 Why we need a `val waitTime = 100`? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-4604][MLLIB] make MatrixFactorizationMo...
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/3459#discussion_r20937749 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/recommendation/MatrixFactorizationModel.scala --- @@ -28,13 +28,16 @@ import org.apache.spark.rdd.RDD /** * Model representing the result of matrix factorization. * + * NB: If you create the model directly using constructor, please be aware that fast prediction --- End diff -- NB = Nota Bene, just note well --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-732][SPARK-3628][CORE][RESUBMIT] elimin...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2524#issuecomment-64654772 [Test build #23893 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/23893/consoleFull) for PR 2524 at commit [`b233737`](https://github.com/apache/spark/commit/b23373746407de3a7c05fbeaf00f67fe78860623). * This patch **fails MiMa tests**. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-732][SPARK-3628][CORE][RESUBMIT] elimin...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/2524#issuecomment-64654780 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/23893/ Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-732][SPARK-3628][CORE][RESUBMIT] elimin...
Github user CodingCat commented on the pull request: https://github.com/apache/spark/pull/2524#issuecomment-64655766 Hey, @mateiz , thank you very much for the review, I addressed all of them except the lastId one, as MIMA wants me to keep that since it's public. also, a question for you, shall I submit the patch to the old version branches, since there are some merge conflicts preventing the patch directly to there --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-732][SPARK-3628][CORE][RESUBMIT] elimin...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2524#issuecomment-64655857 [Test build #23895 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/23895/consoleFull) for PR 2524 at commit [`1433e6f`](https://github.com/apache/spark/commit/1433e6f191e321241a1daef3f70dcbf329ed7079). * This patch merges cleanly. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: SPARK-2192 [BUILD] Examples Data Not in Binary...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3480#issuecomment-64657490 [Test build #23894 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/23894/consoleFull) for PR 3480 at commit [`47688f1`](https://github.com/apache/spark/commit/47688f132bd342bce1096480b83b1d27aa8bfd04). * This patch **passes all tests**. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: SPARK-2192 [BUILD] Examples Data Not in Binary...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/3480#issuecomment-64657504 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/23894/ Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-4625] [SQL] Add sort by for DSL Simpl...
GitHub user chenghao-intel opened a pull request: https://github.com/apache/spark/pull/3481 [SPARK-4625] [SQL] Add sort by for DSL SimpleSqlParser Add `sort by` support for both DSL SqlParser. This PR is relevant with #3386, once either one merged, the other need to be rebased. You can merge this pull request into a Git repository by running: $ git pull https://github.com/chenghao-intel/spark sortby Alternatively you can review and apply these changes as the patch at: https://github.com/apache/spark/pull/3481.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3481 commit 041004fe506435934a785f33ac1671e483df87e5 Author: Cheng Hao hao.ch...@intel.com Date: 2014-11-26T15:03:44Z Add sort by for DSL SimpleSqlParser --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-4625] [SQL] Add sort by for DSL Simpl...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3481#issuecomment-64659416 [Test build #23896 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/23896/consoleFull) for PR 3481 at commit [`041004f`](https://github.com/apache/spark/commit/041004fe506435934a785f33ac1671e483df87e5). * This patch merges cleanly. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-4612] Reduce task latency and increase ...
Github user tdas commented on the pull request: https://github.com/apache/spark/pull/3463#issuecomment-64659830 Just an addition note. This issue was caught by @jl982. Thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-4613][Core] Java API for JdbcRDD
Github user pwendell commented on the pull request: https://github.com/apache/spark/pull/3478#issuecomment-64665984 @liancheng this appears to have failed its own test :P --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-732][SPARK-3628][CORE][RESUBMIT] elimin...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2524#issuecomment-64668708 [Test build #23895 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/23895/consoleFull) for PR 2524 at commit [`1433e6f`](https://github.com/apache/spark/commit/1433e6f191e321241a1daef3f70dcbf329ed7079). * This patch **passes all tests**. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-732][SPARK-3628][CORE][RESUBMIT] elimin...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/2524#issuecomment-64668716 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/23895/ Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-4625] [SQL] Add sort by for DSL Simpl...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3481#issuecomment-64669806 [Test build #23896 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/23896/consoleFull) for PR 3481 at commit [`041004f`](https://github.com/apache/spark/commit/041004fe506435934a785f33ac1671e483df87e5). * This patch **passes all tests**. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-4625] [SQL] Add sort by for DSL Simpl...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/3481#issuecomment-64669816 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/23896/ Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [BRANCH-1.2][SPARK-4583][MLLIB] LogLoss for Gr...
Github user mengxr commented on the pull request: https://github.com/apache/spark/pull/3474#issuecomment-64670254 test this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [BRANCH-1.2][SPARK-4583][MLLIB] LogLoss for Gr...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3474#issuecomment-64671010 [Test build #23897 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/23897/consoleFull) for PR 3474 at commit [`aca2abb`](https://github.com/apache/spark/commit/aca2abbbcf0ad14116f1b54ddd1b61e96803e38c). * This patch merges cleanly. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-4614][MLLIB] Slight API changes in Matr...
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/3468 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-4614][MLLIB] Slight API changes in Matr...
Github user mengxr commented on the pull request: https://github.com/apache/spark/pull/3468#issuecomment-64671123 @brkyvz Thanks! I've merged this into master. Will submit another PR for branch-1.2. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [BRANCH-1.2][SPARK-4614][MLLIB] Slight API cha...
GitHub user mengxr opened a pull request: https://github.com/apache/spark/pull/3482 [BRANCH-1.2][SPARK-4614][MLLIB] Slight API changes in Matrix and Matrices This is #3468 for branch-1.2, same content except mima excludes. You can merge this pull request into a Git repository by running: $ git pull https://github.com/mengxr/spark SPARK-4614-1.2 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/spark/pull/3482.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3482 commit ea4f08dbc021eeb6bbe24b0cb0297aee9cc966e7 Author: Xiangrui Meng m...@databricks.com Date: 2014-11-26T03:32:13Z hide transposeMultiply; add rng to rand and randn; add unit tests --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [BRANCH-1.2][SPARK-4604][MLLIB] make MatrixFac...
Github user mengxr closed the pull request at: https://github.com/apache/spark/pull/3473 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [BRANCH-1.2][SPARK-4614][MLLIB] Slight API cha...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3482#issuecomment-64672216 [Test build #23898 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/23898/consoleFull) for PR 3482 at commit [`ea4f08d`](https://github.com/apache/spark/commit/ea4f08dbc021eeb6bbe24b0cb0297aee9cc966e7). * This patch merges cleanly. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-4293][SQL] Make Cast be able to handle ...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3150#issuecomment-64673009 [Test build #23899 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/23899/consoleFull) for PR 3150 at commit [`e935939`](https://github.com/apache/spark/commit/e935939ac829ecaa887df4bcbb6c65027876a210). * This patch merges cleanly. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-4613][Core] Java API for JdbcRDD
Github user liancheng commented on the pull request: https://github.com/apache/spark/pull/3478#issuecomment-64675566 @pwendell Yea, I don't see why it can't find Derby driver, and I can't reproduce it locally :( --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-4409][MLlib] Additional Linear Algebra ...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3319#issuecomment-64681676 [Test build #23900 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/23900/consoleFull) for PR 3319 at commit [`a8120d2`](https://github.com/apache/spark/commit/a8120d2a83720b621b36942add3a98aa4b96bcc3). * This patch merges cleanly. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-4293][SQL] Make Cast be able to handle ...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3150#issuecomment-64682672 [Test build #23899 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/23899/consoleFull) for PR 3150 at commit [`e935939`](https://github.com/apache/spark/commit/e935939ac829ecaa887df4bcbb6c65027876a210). * This patch **passes all tests**. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-4626] Kill a task only if the executorI...
GitHub user roxchkplusony opened a pull request: https://github.com/apache/spark/pull/3483 [SPARK-4626] Kill a task only if the executorId is (still) registered with the scheduler You can merge this pull request into a Git repository by running: $ git pull https://github.com/Paxata/spark bugfix/4626 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/spark/pull/3483.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3483 commit 97c4efb4dd7972c3aae0c6f496b8d1a5984da4d7 Author: roxchkplusony roxchkplus...@gmail.com Date: 2014-11-26T17:37:00Z [SPARK-4626] Kill a task only if the executorId is (still) registered with the scheduler --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-4293][SQL] Make Cast be able to handle ...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/3150#issuecomment-64682678 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/23899/ Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [BRANCH-1.2][SPARK-4583][MLLIB] LogLoss for Gr...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/3474#issuecomment-64682826 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/23897/ Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [BRANCH-1.2][SPARK-4583][MLLIB] LogLoss for Gr...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3474#issuecomment-64682818 [Test build #23897 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/23897/consoleFull) for PR 3474 at commit [`aca2abb`](https://github.com/apache/spark/commit/aca2abbbcf0ad14116f1b54ddd1b61e96803e38c). * This patch **fails Spark unit tests**. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-4626] Kill a task only if the executorI...
Github user roxchkplusony commented on the pull request: https://github.com/apache/spark/pull/3483#issuecomment-64683263 I largely stole the structure from the status update message handler. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-4626] Kill a task only if the executorI...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/3483#issuecomment-64683431 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [BRANCH-1.2][SPARK-4583][MLLIB] LogLoss for Gr...
Github user mengxr commented on the pull request: https://github.com/apache/spark/pull/3474#issuecomment-64683872 @tdas I saw this test also failed in branch-1.2: org.apache.spark.streaming.CheckpointSuite.recovery with file input stream test this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [BRANCH-1.2][SPARK-4614][MLLIB] Slight API cha...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/3482#issuecomment-64688375 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/23898/ Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [BRANCH-1.2][SPARK-4614][MLLIB] Slight API cha...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3482#issuecomment-64688366 [Test build #23898 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/23898/consoleFull) for PR 3482 at commit [`ea4f08d`](https://github.com/apache/spark/commit/ea4f08dbc021eeb6bbe24b0cb0297aee9cc966e7). * This patch **passes all tests**. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
