[GitHub] spark pull request: [SPARK-10049][SPARKR] Support collecting data ...
Github user shivaram commented on the pull request: https://github.com/apache/spark/pull/8458#issuecomment-135648633 @sun-rui I'll take a look at this tomorrow --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-10227] fatal warnings with sbt on Scala...
Github user rxin commented on the pull request: https://github.com/apache/spark/pull/8433#issuecomment-135648501 What's the meaning of param and field? The scala documentation doesn't really say anything: http://www.scala-lang.org/api/2.10.4/index.html#scala.annotation.meta.param Consult the documentation in package scala.annotation.meta. But scala.annotation.meta doesn't say much either. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: SPARK-6548 Adding stddev to DataFrame function...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/6297#issuecomment-135655318 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: SPARK-6548 Adding stddev to DataFrame function...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/6297#issuecomment-135655321 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/41730/ Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: SPARK-6548 Adding stddev to DataFrame function...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/6297#issuecomment-135655316 [Test build #41730 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/41730/console) for PR 6297 at commit [`25425ac`](https://github.com/apache/spark/commit/25425ac742e231bd45b62ee31bf217a94e568e55). * This patch **fails Scala style tests**. * This patch merges cleanly. * This patch adds the following public classes _(experimental)_: * `case class Stddev(child: Expression) extends StddevAgg(child) ` * `case class StddevPop(child: Expression) extends StddevAgg(child) ` * `case class StddevSamp(child: Expression) extends StddevAgg(child) ` * `abstract class StddevAgg(child: Expression) extends AlgebraicAggregate ` * `abstract class StddevAgg1(child: Expression) extends UnaryExpression with PartialAggregate1 ` * `case class Stddev(child: Expression) extends StddevAgg1(child) ` * `case class StddevPop(child: Expression) extends StddevAgg1(child) ` * `case class StddevSamp(child: Expression) extends StddevAgg1(child) ` * `case class ComputePartialStd(child: Expression) extends UnaryExpression with AggregateExpression1 ` * `case class ComputePartialStdFunction (` * `case class MergePartialStd(child: Expression, isSample: Boolean) extends UnaryExpression with AggregateExpression1 ` * `case class MergePartialStdFunction(` * `case class StddevFunction(` --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-10188] [Pyspark] Pyspark CrossValidator...
Github user noel-smith commented on the pull request: https://github.com/apache/spark/pull/8399#issuecomment-135655500 That would be great - I've just messaged him. If there are any other changes you need to get this into 1.5 I'll get them in ASAP today. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-9986][SPARK-9991][SPARK-9993][SQL]Creat...
Github user zsxwing commented on a diff in the pull request:
https://github.com/apache/spark/pull/8464#discussion_r38174902
--- Diff:
sql/core/src/test/scala/org/apache/spark/sql/execution/local/FilterNodeSuite.scala
---
@@ -0,0 +1,41 @@
+/*
+* Licensed to the Apache Software Foundation (ASF) under one or more
+* contributor license agreements. See the NOTICE file distributed with
+* this work for additional information regarding copyright ownership.
+* The ASF licenses this file to You under the Apache License, Version 2.0
+* (the License); you may not use this file except in compliance with
+* the License. You may obtain a copy of the License at
+*
+*http://www.apache.org/licenses/LICENSE-2.0
+*
+* Unless required by applicable law or agreed to in writing, software
+* distributed under the License is distributed on an AS IS BASIS,
+* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+* See the License for the specific language governing permissions and
+* limitations under the License.
+*/
+
+package org.apache.spark.sql.execution.local
+
+import org.apache.spark.sql.test.SharedSQLContext
+
+class FilterNodeSuite extends LocalNodeTest with SharedSQLContext {
--- End diff --
I just want to reuse `SQLTestData`. It would be easy if we could use
DataFrame to test the local stuff. I feel if we make sure not using SQLContext
in the main codes of LocalNodes, we don't need to get rid of SQLContext in the
test cases.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-10065][SQL] Avoid triple copying of var...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8484#issuecomment-135664316 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-10310][SQL]Using \t as the field delime...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8476#issuecomment-135664293 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-10065][SQL] Avoid triple copying of var...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8484#issuecomment-135664287 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-10310][SQL]Using \t as the field delime...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8476#issuecomment-135664315 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-10226][SQL] Fix exclamation mark issue ...
Github user dragos commented on the pull request: https://github.com/apache/spark/pull/8420#issuecomment-135664504 Sorry, I missed the Github notification. LGTM. The Scala REPL does the same thing. Probably the ancient version of jline that was used before didn't perform event expansion. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-10188] [Pyspark] Pyspark CrossValidator...
Github user jkbradley commented on the pull request: https://github.com/apache/spark/pull/8399#issuecomment-135656766 No, this LGTM. I'll merge this with branch-1.5 and master now. Thanks very much! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-9986][SPARK-9991][SPARK-9993][SQL]Creat...
Github user rxin commented on a diff in the pull request:
https://github.com/apache/spark/pull/8464#discussion_r38175042
--- Diff:
sql/core/src/test/scala/org/apache/spark/sql/execution/local/FilterNodeSuite.scala
---
@@ -0,0 +1,41 @@
+/*
+* Licensed to the Apache Software Foundation (ASF) under one or more
+* contributor license agreements. See the NOTICE file distributed with
+* this work for additional information regarding copyright ownership.
+* The ASF licenses this file to You under the Apache License, Version 2.0
+* (the License); you may not use this file except in compliance with
+* the License. You may obtain a copy of the License at
+*
+*http://www.apache.org/licenses/LICENSE-2.0
+*
+* Unless required by applicable law or agreed to in writing, software
+* distributed under the License is distributed on an AS IS BASIS,
+* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+* See the License for the specific language governing permissions and
+* limitations under the License.
+*/
+
+package org.apache.spark.sql.execution.local
+
+import org.apache.spark.sql.test.SharedSQLContext
+
+class FilterNodeSuite extends LocalNodeTest with SharedSQLContext {
--- End diff --
Note that SQLTestData is going away though
https://github.com/apache/spark/pull/7406
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-10227] fatal warnings with sbt on Scala...
Github user skyluc commented on the pull request: https://github.com/apache/spark/pull/8433#issuecomment-135661207 It indicates on which of the elements generated for the class parameter to apply the annotation. There are 4 possible elements generated for a class parameter: the constructor parameter, the field, the getter and the setter (and more for beans). I would expect `@(transient @field)` to be enough, for the cases when a field is actually created, but the warning is not resolve until `@param` is added too. I haven't dig on why the compiler complain in this case. The base question was mostly: do we want to remove the `@transient` annotation which have no effect at all (no field generated by the compiler). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-8952] [SPARKR] - Wrap normalizePath cal...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/8343#issuecomment-135664705 [Test build #41735 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/41735/consoleFull) for PR 8343 at commit [`472c767`](https://github.com/apache/spark/commit/472c76714c25b909e281d8079b7ead6c152d4512). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-10065][SQL] Avoid triple copying of var...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/8484#issuecomment-135664712 [Test build #41736 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/41736/consoleFull) for PR 8484 at commit [`fe7b7bf`](https://github.com/apache/spark/commit/fe7b7bfcd848c9b1d998bb2c3b7194900f021c4f). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-8952] [SPARKR] - Wrap normalizePath cal...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8343#issuecomment-135663625 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-8952] [SPARKR] - Wrap normalizePath cal...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8343#issuecomment-135663634 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-8952] [SPARKR] - Wrap normalizePath cal...
Github user shivaram commented on the pull request: https://github.com/apache/spark/pull/8343#issuecomment-135663528 Jenkins, retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-10260][ML] Add @Since annotation to ml....
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/8455 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-10004] [shuffle] Perform auth checks wh...
Github user rxin commented on the pull request: https://github.com/apache/spark/pull/8218#issuecomment-135643130 Looks alright to me - would be good if @aarondav takes a look too. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-10260][ML] Add @Since annotation to ml....
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8455#issuecomment-135643024 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/41729/ Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-9671] [MLLIB] re-org user guide and add...
Github user mengxr commented on a diff in the pull request: https://github.com/apache/spark/pull/8498#discussion_r38176683 --- Diff: docs/mllib-migration-guides.md --- @@ -7,6 +7,25 @@ description: MLlib migration guides from before Spark SPARK_VERSION_SHORT The migration guide for the current Spark version is kept on the [MLlib Programming Guide main page](mllib-guide.html#migration-guide). +## From 1.3 to 1.4 --- End diff -- No content change here. Just moved the paragraphs from `mllib-guide` and `ml-guide`. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARKR] [SPARK-10328] Fix generic for na.omit
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8495#issuecomment-135656017 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/41724/ Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARKR] [SPARK-10328] Fix generic for na.omit
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/8495#issuecomment-135655957 [Test build #41724 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/41724/console) for PR 8495 at commit [`86e8f78`](https://github.com/apache/spark/commit/86e8f78d8ff48b18a78712fb109fe9413f420bb2). * This patch **passes all tests**. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: SPARK-6548 Adding stddev to DataFrame function...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/6297#issuecomment-135657157 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: typo in comment
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8497#issuecomment-135657150 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-10323] [SQL] fix nullability of In/InSe...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8492#issuecomment-135657152 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-10188] [Pyspark] Pyspark CrossValidator...
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/8399 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: typo in comment
GitHub user dharmeshkakadia opened a pull request: https://github.com/apache/spark/pull/8497 typo in comment You can merge this pull request into a Git repository by running: $ git pull https://github.com/dharmeshkakadia/spark patch-2 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/spark/pull/8497.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #8497 commit 946be29b04cf540a330c21d1bfab8506ae55338f Author: Dharmesh Kakadia dharmeshkaka...@users.noreply.github.com Date: 2015-08-28T07:00:22Z typo in comment --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-10323] [SQL] fix nullability of In/InSe...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8492#issuecomment-135657175 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-9671] [MLLIB] re-org user guide and add...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/8498#issuecomment-135663172 [Test build #41734 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/41734/consoleFull) for PR 8498 at commit [`2790270`](https://github.com/apache/spark/commit/279027032b21f98170d7729050bdcc697b91fb5d). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARKR] [SPARK-10328] Fix generic for na.omit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/8495 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-10260][ML] Add @Since annotation to ml....
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8455#issuecomment-135643023 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-10260][ML] Add @Since annotation to ml....
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/8455#issuecomment-135642972 [Test build #41729 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/41729/console) for PR 8455 at commit [`f4ad585`](https://github.com/apache/spark/commit/f4ad585561bba37ac2050d80d9c759b0392282b2). * This patch **passes all tests**. * This patch merges cleanly. * This patch adds the following public classes _(experimental)_: * `class KMeans @Since(1.5.0) (` --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-10004] [shuffle] Perform auth checks wh...
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/8218#discussion_r38173030 --- Diff: network/common/src/main/java/org/apache/spark/network/client/TransportClient.java --- @@ -70,6 +70,7 @@ private final Channel channel; private final TransportResponseHandler handler; + private String clientId; --- End diff -- Can we use an Optional here, or annotate it as @Nullable? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-10004] [shuffle] Perform auth checks wh...
Github user rxin commented on a diff in the pull request:
https://github.com/apache/spark/pull/8218#discussion_r38173055
--- Diff:
network/common/src/main/java/org/apache/spark/network/client/TransportClient.java
---
@@ -85,6 +86,25 @@ public SocketAddress getSocketAddress() {
}
/**
+ * Returns the ID used by the client to authenticate itself when
authentication is enabled.
+ *
+ * @return The client ID, or null if authentication is disabled.
--- End diff --
this can also be null if clientId hasn't been set yet right?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-10049][SPARKR] Support collecting data ...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/8458#issuecomment-135647373 [Test build #41723 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/41723/console) for PR 8458 at commit [`2bc97ad`](https://github.com/apache/spark/commit/2bc97adc8a081301e0bc7394d35dd617a9ae49a5). * This patch **passes all tests**. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-10049][SPARKR] Support collecting data ...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8458#issuecomment-135647454 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/41723/ Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-10049][SPARKR] Support collecting data ...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8458#issuecomment-135647453 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-10323] [SQL] fix nullability of In/InSe...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/8492#issuecomment-135654582 [Test build #1701 has finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/1701/console) for PR 8492 at commit [`4328e46`](https://github.com/apache/spark/commit/4328e46d479c21fd616b101d29dd7653c64a3e21). * This patch **fails Spark unit tests**. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-10065][SQL] avoid the extra copy when g...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/8496#issuecomment-135655198 [Test build #41731 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/41731/consoleFull) for PR 8496 at commit [`79cfc44`](https://github.com/apache/spark/commit/79cfc445e3e4ae87ab5d4e907c27d57267c37626). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: SPARK-6548 Adding stddev to DataFrame function...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/6297#issuecomment-135654368 [Test build #41730 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/41730/consoleFull) for PR 6297 at commit [`25425ac`](https://github.com/apache/spark/commit/25425ac742e231bd45b62ee31bf217a94e568e55). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-10323] [SQL] fix nullability of In/InSe...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/8492#issuecomment-135657968 [Test build #41733 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/41733/consoleFull) for PR 8492 at commit [`5ca7823`](https://github.com/apache/spark/commit/5ca7823dea93b2341fdb2af6bf548812471ba48b). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-9986][SPARK-9991][SPARK-9993][SQL]Creat...
Github user zsxwing commented on a diff in the pull request:
https://github.com/apache/spark/pull/8464#discussion_r38175373
--- Diff:
sql/core/src/test/scala/org/apache/spark/sql/execution/local/FilterNodeSuite.scala
---
@@ -0,0 +1,41 @@
+/*
+* Licensed to the Apache Software Foundation (ASF) under one or more
+* contributor license agreements. See the NOTICE file distributed with
+* this work for additional information regarding copyright ownership.
+* The ASF licenses this file to You under the Apache License, Version 2.0
+* (the License); you may not use this file except in compliance with
+* the License. You may obtain a copy of the License at
+*
+*http://www.apache.org/licenses/LICENSE-2.0
+*
+* Unless required by applicable law or agreed to in writing, software
+* distributed under the License is distributed on an AS IS BASIS,
+* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+* See the License for the specific language governing permissions and
+* limitations under the License.
+*/
+
+package org.apache.spark.sql.execution.local
+
+import org.apache.spark.sql.test.SharedSQLContext
+
+class FilterNodeSuite extends LocalNodeTest with SharedSQLContext {
--- End diff --
Note that SQLTestData is going away though #7406
Didn't notice that. I will add test data for each test case manually.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-9671] [MLLIB] re-org user guide and add...
GitHub user mengxr opened a pull request: https://github.com/apache/spark/pull/8498 [SPARK-9671] [MLLIB] re-org user guide and add migration guide This PR updates the MLlib user guide and adds migration guide for 1.4-1.5. * merge migration guide for `spark.mllib` and `spark.ml` packages * remove dependency section from `spark.ml` guide * move the paragraph about `spark.mllib` and `spark.ml` to the top and recommend `spark.ml` * move Sam's talk to footnote to make the section focus on dependencies @jkbradley @feynmanliang You can merge this pull request into a Git repository by running: $ git pull https://github.com/mengxr/spark SPARK-9671 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/spark/pull/8498.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #8498 commit 279027032b21f98170d7729050bdcc697b91fb5d Author: Xiangrui Meng m...@databricks.com Date: 2015-08-28T07:13:15Z re-org user guide and add migration guide --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-9671] [MLLIB] re-org user guide and add...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8498#issuecomment-135662135 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-9671] [MLLIB] re-org user guide and add...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8498#issuecomment-135662156 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-10226][SQL] Fix exclamation mark issue ...
Github user watermen commented on the pull request: https://github.com/apache/spark/pull/8420#issuecomment-135648272 LGTM @dragos --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-10065][SQL] Avoid triple copying of var...
Github user cloud-fan commented on the pull request: https://github.com/apache/spark/pull/8484#issuecomment-135655783 Hi @viirya , thanks for looking into this! The reason for this extra copy is that we iterate the array twice: calculate elements data size and copy elements to array buffer. A simple solution is to follow `createCodeForStruct`, we can dynamicly grow the buffer when needed and thus don't need to know the data size ahead. I have opened https://github.com/apache/spark/pull/8496 to fix it, do you mind take a look? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-10065][SQL] avoid the extra copy when g...
Github user cloud-fan commented on the pull request: https://github.com/apache/spark/pull/8496#issuecomment-135655886 cc @davis @viirya --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: SPARK-6548 Adding stddev to DataFrame function...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/6297#issuecomment-135657176 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-10065][SQL] avoid the extra copy when g...
GitHub user cloud-fan opened a pull request: https://github.com/apache/spark/pull/8496 [SPARK-10065][SQL] avoid the extra copy when generate unsafe array The reason for this extra copy is that we iterate the array elements twice: one for calculate size, one for write to `UnsafeArray`. A simple solution is using the same skill in generating unsafe row: grow the byte[] buffer when needed. Then we don't need to know the data size ahead and only need to iterate once. You can merge this pull request into a Git repository by running: $ git pull https://github.com/cloud-fan/spark avoid-copy Alternatively you can review and apply these changes as the patch at: https://github.com/apache/spark/pull/8496.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #8496 commit 56f226842f9668ccfde947fafd07a4400dc38a5a Author: Wenchen Fan cloud0...@outlook.com Date: 2015-08-28T05:44:52Z make sure input.primitive is always variable name not code commit 79cfc445e3e4ae87ab5d4e907c27d57267c37626 Author: Wenchen Fan cloud0...@outlook.com Date: 2015-08-28T06:28:25Z avoid the extra copy when generate unsafe array --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-10065][SQL] avoid the extra copy when g...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8496#issuecomment-135653469 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-10065][SQL] avoid the extra copy when g...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8496#issuecomment-135653522 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARKR] [SPARK-10328] Fix generic for na.omit
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8495#issuecomment-135656016 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-10065][SQL] Avoid triple copying of var...
Github user viirya commented on a diff in the pull request:
https://github.com/apache/spark/pull/8484#discussion_r38177238
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/codegen/GenerateUnsafeProjection.scala
---
@@ -323,6 +309,7 @@ object GenerateUnsafeProjection extends
CodeGenerator[Seq[Expression], UnsafePro
int $cursor = $fixedSize;
for (int $index = 0; $index $numElements; $index++) {
+${convertedElement.code}
--- End diff --
Thanks for reviewing. I've updated this patch to use the dynamic buffer
approach. I think it should be better now.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: SPARK-6548 Adding stddev to DataFrame function...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/6297#issuecomment-135650454 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: SPARK-6548 Adding stddev to DataFrame function...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/6297#issuecomment-135650541 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-10065][SQL] Avoid triple copying of var...
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/8484#discussion_r38174176
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/codegen/GenerateUnsafeProjection.scala
---
@@ -323,6 +309,7 @@ object GenerateUnsafeProjection extends
CodeGenerator[Seq[Expression], UnsafePro
int $cursor = $fixedSize;
for (int $index = 0; $index $numElements; $index++) {
+${convertedElement.code}
--- End diff --
we do an extra conversion here, I think it's worse than an extra copy...
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-9986][SPARK-9991][SPARK-9993][SQL]Creat...
Github user zsxwing commented on a diff in the pull request:
https://github.com/apache/spark/pull/8464#discussion_r38175256
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/local/UnionNode.scala ---
@@ -0,0 +1,75 @@
+/*
+* Licensed to the Apache Software Foundation (ASF) under one or more
+* contributor license agreements. See the NOTICE file distributed with
+* this work for additional information regarding copyright ownership.
+* The ASF licenses this file to You under the Apache License, Version 2.0
+* (the License); you may not use this file except in compliance with
+* the License. You may obtain a copy of the License at
+*
+*http://www.apache.org/licenses/LICENSE-2.0
+*
+* Unless required by applicable law or agreed to in writing, software
+* distributed under the License is distributed on an AS IS BASIS,
+* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+* See the License for the specific language governing permissions and
+* limitations under the License.
+*/
+
+package org.apache.spark.sql.execution.local
+
+import org.apache.spark.sql.catalyst.InternalRow
+import org.apache.spark.sql.catalyst.expressions.Attribute
+
+case class UnionNode(children: Seq[LocalNode]) extends LocalNode {
--- End diff --
It overrides `TreeNode.children` which uses `Seq`.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: SPARK-6548 Adding stddev to DataFrame function...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/6297#issuecomment-135657532 [Test build #41732 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/41732/consoleFull) for PR 6297 at commit [`f4c725c`](https://github.com/apache/spark/commit/f4c725c47d179f66971ac63b18fe2c35f7432ffa). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-10323] [SQL] fix nullability of In/InSe...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/8492#issuecomment-135658488 [Test build #41727 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/41727/console) for PR 8492 at commit [`3be8d05`](https://github.com/apache/spark/commit/3be8d054245a953a3db5314ce5edaa9ce8e97852). * This patch **fails Spark unit tests**. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-10323] [SQL] fix nullability of In/InSe...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8492#issuecomment-135658527 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/41727/ Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-10323] [SQL] fix nullability of In/InSe...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8492#issuecomment-135658526 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-9890] [Doc] [ML] User guide for CountVe...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8487#issuecomment-135668992 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-10325] Override hashCode() for public R...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8500#issuecomment-135668988 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: Fix DynamodDB/DynamoDB typo in Kinesis Integra...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8501#issuecomment-135668985 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-10325] Override hashCode() for public R...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8500#issuecomment-135669004 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-9890] [Doc] [ML] User guide for CountVe...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8487#issuecomment-135669129 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-9890] [Doc] [ML] User guide for CountVe...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8487#issuecomment-135681880 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/41740/ Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-9890] [Doc] [ML] User guide for CountVe...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8487#issuecomment-135681877 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: typo in comment
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/8497 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: SPARK-6548 Adding stddev to DataFrame function...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/6297#issuecomment-135693383 [Test build #41732 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/41732/console) for PR 6297 at commit [`f4c725c`](https://github.com/apache/spark/commit/f4c725c47d179f66971ac63b18fe2c35f7432ffa). * This patch **fails Spark unit tests**. * This patch merges cleanly. * This patch adds the following public classes _(experimental)_: * `case class Stddev(child: Expression) extends StddevAgg(child) ` * `case class StddevPop(child: Expression) extends StddevAgg(child) ` * `case class StddevSamp(child: Expression) extends StddevAgg(child) ` * `abstract class StddevAgg(child: Expression) extends AlgebraicAggregate ` * `abstract class StddevAgg1(child: Expression) extends UnaryExpression with PartialAggregate1 ` * `case class Stddev(child: Expression) extends StddevAgg1(child) ` * `case class StddevPop(child: Expression) extends StddevAgg1(child) ` * `case class StddevSamp(child: Expression) extends StddevAgg1(child) ` * `case class ComputePartialStd(child: Expression) extends UnaryExpression with AggregateExpression1 ` * `case class ComputePartialStdFunction (` * `case class MergePartialStd(` * `case class MergePartialStdFunction(` * `case class StddevFunction(` --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-10330] Use SparkHadoopUtil TaskAttemptC...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8499#issuecomment-135723505 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/41738/ Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-10330] Use SparkHadoopUtil TaskAttemptC...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/8499#issuecomment-135723410 [Test build #41738 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/41738/console) for PR 8499 at commit [`28d213d`](https://github.com/apache/spark/commit/28d213d1ba73a75c98ec988981d421825b50973f). * This patch **passes all tests**. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-10330] Use SparkHadoopUtil TaskAttemptC...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8499#issuecomment-135723504 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-9671] [MLLIB] re-org user guide and add...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/8498#issuecomment-135735710 [Test build #41734 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/41734/console) for PR 8498 at commit [`2790270`](https://github.com/apache/spark/commit/279027032b21f98170d7729050bdcc697b91fb5d). * This patch **passes all tests**. * This patch merges cleanly. * This patch adds the following public classes _(experimental)_: * `* *(Breaking change)* The `apply` and `copy` methods for the case class [`BoostingStrategy`](api/scala/index.html#org.apache.spark.mllib.tree.configuration.BoostingStrategy) have been changed because of a modification to the case class fields. This could be an issue for users who use `BoostingStrategy` to set GBT parameters.` * `* *(Breaking change)* The return value of [`LDA.run`](api/scala/index.html#org.apache.spark.mllib.clustering.LDA) has changed. It now returns an abstract class `LDAModel` instead of the concrete class `DistributedLDAModel`. The object of type `LDAModel` can still be cast to the appropriate concrete type, which depends on the optimization algorithm.` * `* The `scoreCol` output column (with default value score) was renamed to be `probabilityCol` (with default value probability). The type was originally `Double` (for the probability of class 1.0), but it is now `Vector` (for the probability of each class, to support multiclass classification in the future).` --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-9671] [MLLIB] re-org user guide and add...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8498#issuecomment-135735979 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/41734/ Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-9671] [MLLIB] re-org user guide and add...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8498#issuecomment-135735975 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-10172][core]Fix for HistoryServer webUI...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8506#issuecomment-135745326 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-10172][core]Fix for HistoryServer webUI...
GitHub user josiahsams opened a pull request: https://github.com/apache/spark/pull/8506 [SPARK-10172][core]Fix for HistoryServer webUI mess up during sorting This pull request is to address the JIRA SPARK-10172 (History Server web UI gets messed up when sorting on any column). The fix will avoid the use of rowspan during table creation in the web UI, when there are multiple attempts for a job. By avoiding the rowspan, the table won't get messed up during the column sort. You can merge this pull request into a Git repository by running: $ git pull https://github.com/ibmsoe/spark SPARK-10172 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/spark/pull/8506.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #8506 commit 521f1d6130261e84ba37d6c96f459f6219a3973d Author: Josiah Samuel josiah_s...@in.ibm.com Date: 2015-08-26T11:08:46Z [SPARK-10172][core]Fix for HistoryServer webUI mess up during sorting --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-10310][SQL]Using \t as the field delime...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8476#issuecomment-135666371 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/41737/ Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-10310][SQL]Using \t as the field delime...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8476#issuecomment-135666370 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-8952] [SPARKR] - Wrap normalizePath cal...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8343#issuecomment-135667676 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: SPARK-9545, SPARK-9547: Use Maven in PRB if ti...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/7878#issuecomment-135667717 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: SPARK-9545, SPARK-9547: Use Maven in PRB if ti...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/7878#issuecomment-135667613 [Test build #41726 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/41726/console) for PR 7878 at commit [`cf58c49`](https://github.com/apache/spark/commit/cf58c49c3be31c8e33639ba68eca16398f98c7f6). * This patch **passes all tests**. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: SPARK-9545, SPARK-9547: Use Maven in PRB if ti...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/7878#issuecomment-135667719 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/41726/ Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-8952] [SPARKR] - Wrap normalizePath cal...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8343#issuecomment-135667678 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/41735/ Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-10035][SQL] Parquet filters does not pr...
Github user liancheng commented on the pull request: https://github.com/apache/spark/pull/8275#issuecomment-135672199 I backported this PR to branch-1.5 to fix 1.5 Jenkins build failures. Didn't merge this PR to branch-1.5 before because it introduced new feature after branch-1.5 being cut. This PR also refactored `ParquetFilterSuite` and cleaned up outdated code path used by the old Parquet support which has already been removed. The problem is that, PR #8403 merged later depends on the test suite refactoring to pass the build. That's why #8403 passes for master but breaks branch-1.5. This backporting should be safe, since the `EqualSafeNull` support has already been merged into branch-1.5 by #8403 (probably done by Git automatically and unintentionally). So this backporting only introduces test suite changes. See [changed files] [1] of this backporting commit. cc @pwendell @rxin @JoshRosen [1]: https://github.com/apache/spark/commit/f0c4470d43e833d2382a8b98bf4aa21ae9451d00 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-10295] [CORE] Dynamic allocation in Mes...
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/8489 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-8472] [ML] [PySpark] Python API for DCT
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8485#issuecomment-135705308 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-8472] [ML] [PySpark] Python API for DCT
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8485#issuecomment-135705261 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-8472] [ML] [PySpark] Python API for DCT
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/8485#issuecomment-135709399 [Test build #41742 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/41742/consoleFull) for PR 8485 at commit [`d5d5270`](https://github.com/apache/spark/commit/d5d5270630f3e6524d7e5f065b1b5bbf9ed9e78c). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-10288][Yarn][WIP] Add REST client for S...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8504#issuecomment-135722122 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-10260][ML] Add @Since annotation to ml....
Github user yu-iskw commented on the pull request: https://github.com/apache/spark/pull/8455#issuecomment-135665306 Thank you for merging it! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-10326] [yarn] Fix app submission on win...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/8493#issuecomment-135683072 [Test build #1702 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/1702/consoleFull) for PR 8493 at commit [`a14dba5`](https://github.com/apache/spark/commit/a14dba5233526f844a68d77c5d765d98b0534e2a). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-10323] [SQL] fix nullability of In/InSe...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8492#issuecomment-135711841 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/41733/ Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-10323] [SQL] fix nullability of In/InSe...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/8492#issuecomment-135711152 [Test build #41733 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/41733/console) for PR 8492 at commit [`5ca7823`](https://github.com/apache/spark/commit/5ca7823dea93b2341fdb2af6bf548812471ba48b). * This patch **passes all tests**. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
