[GitHub] spark pull request: [SPARK-15152][DOC][MINOR] Scaladoc and Code st...
Github user srowen commented on the pull request: https://github.com/apache/spark/pull/12928#issuecomment-217390333 Weird, is that a scala 2.10 vs 2.11 thing? some other implicit conversions are available by default in 2.11? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-15152][DOC][MINOR] Scaladoc and Code st...
Github user andrewor14 commented on the pull request: https://github.com/apache/spark/pull/12928#issuecomment-217312688 I fixed it in https://github.com/apache/spark/commit/7f5922aa4a810a0b9cc783956a8b7aa3dad86a0a. I'm puzzled as to how it passed tests here? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-15152][DOC][MINOR] Scaladoc and Code st...
Github user tdas commented on the pull request: https://github.com/apache/spark/pull/12928#issuecomment-217312115 This patch seems to have broken Spark build - https://amplab.cs.berkeley.edu/jenkins/view/Spark%20QA%20Compile/job/spark-master-compile-maven-scala-2.10/1222/console --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-15152][DOC][MINOR] Scaladoc and Code st...
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/12928 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-15152][DOC][MINOR] Scaladoc and Code st...
Github user andrewor14 commented on the pull request: https://github.com/apache/spark/pull/12928#issuecomment-217309796 Looks good, pretty straightforward. Merging into master 2.0 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-15152][DOC][MINOR] Scaladoc and Code st...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12928#issuecomment-217308577 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-15152][DOC][MINOR] Scaladoc and Code st...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12928#issuecomment-217308579 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/57924/ Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-15152][DOC][MINOR] Scaladoc and Code st...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12928#issuecomment-217308403 **[Test build #57924 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/57924/consoleFull)** for PR 12928 at commit [`b1e1768`](https://github.com/apache/spark/commit/b1e176837f81c5c221704a154dfecf0c4648d9df). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-15152][DOC][MINOR] Scaladoc and Code st...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12928#issuecomment-217286317 **[Test build #57924 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/57924/consoleFull)** for PR 12928 at commit [`b1e1768`](https://github.com/apache/spark/commit/b1e176837f81c5c221704a154dfecf0c4648d9df). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-15152][DOC][MINOR] Scaladoc and Code st...
Github user andrewor14 commented on the pull request: https://github.com/apache/spark/pull/12928#issuecomment-217285787 add to whitelist --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-15152][DOC][MINOR] Scaladoc and Code st...
Github user srowen commented on a diff in the pull request:
https://github.com/apache/spark/pull/12928#discussion_r62176096
--- Diff: core/src/main/scala/org/apache/spark/scheduler/Pool.scala ---
@@ -98,13 +97,12 @@ private[spark] class Pool(
}
override def getSortedTaskSetQueue: ArrayBuffer[TaskSetManager] = {
-var sortedTaskSetQueue = new ArrayBuffer[TaskSetManager]
-val sortedSchedulableQueue =
-
schedulableQueue.asScala.toSeq.sortWith(taskSetSchedulingAlgorithm.comparator)
-for (schedulable <- sortedSchedulableQueue) {
- sortedTaskSetQueue ++= schedulable.getSortedTaskSetQueue
-}
-sortedTaskSetQueue
+schedulableQueue
--- End diff --
It compiles because of the cast, which might fail at runtime. Then maybe
the tests didn't run? or didn't cover this code path actually
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-15152][DOC][MINOR] Scaladoc and Code st...
Github user jaceklaskowski commented on a diff in the pull request:
https://github.com/apache/spark/pull/12928#discussion_r62170912
--- Diff: core/src/main/scala/org/apache/spark/scheduler/Pool.scala ---
@@ -98,13 +97,12 @@ private[spark] class Pool(
}
override def getSortedTaskSetQueue: ArrayBuffer[TaskSetManager] = {
-var sortedTaskSetQueue = new ArrayBuffer[TaskSetManager]
-val sortedSchedulableQueue =
-
schedulableQueue.asScala.toSeq.sortWith(taskSetSchedulingAlgorithm.comparator)
-for (schedulable <- sortedSchedulableQueue) {
- sortedTaskSetQueue ++= schedulable.getSortedTaskSetQueue
-}
-sortedTaskSetQueue
+schedulableQueue
--- End diff --
You're right! I have overlooked that. I'll revert the change. Sorry.
(Hmmm, how did it work then?! #confused)
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-15152][DOC][MINOR] Scaladoc and Code st...
Github user srowen commented on a diff in the pull request:
https://github.com/apache/spark/pull/12928#discussion_r62170731
--- Diff: yarn/src/main/scala/org/apache/spark/deploy/yarn/Client.scala ---
@@ -168,7 +168,7 @@ private[spark] class Client(
val appContext = createApplicationSubmissionContext(newApp,
containerContext)
// Finally, submit and monitor the application
- logInfo(s"Submitting application ${appId.getId} to ResourceManager")
+ logInfo(s"Submitting application $appId to ResourceManager")
--- End diff --
OK that seems fine
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-15152][DOC][MINOR] Scaladoc and Code st...
Github user jaceklaskowski commented on a diff in the pull request:
https://github.com/apache/spark/pull/12928#discussion_r62170253
--- Diff: yarn/src/main/scala/org/apache/spark/deploy/yarn/Client.scala ---
@@ -168,7 +168,7 @@ private[spark] class Client(
val appContext = createApplicationSubmissionContext(newApp,
containerContext)
// Finally, submit and monitor the application
- logInfo(s"Submitting application ${appId.getId} to ResourceManager")
+ logInfo(s"Submitting application $appId to ResourceManager")
--- End diff --
It's inconsistent with the other log messages where `appId` is printed out
instead. It then makes tracing the calls slightly harder - cf.
https://github.com/apache/spark/blob/master/core/src/main/scala/org/apache/spark/deploy/client/AppClient.scala#L180-L182
and
https://github.com/apache/spark/blob/master/core/src/main/scala/org/apache/spark/deploy/client/AppClient.scala#L174-L175
and few other places.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-15152][DOC][MINOR] Scaladoc and Code st...
Github user jaceklaskowski commented on a diff in the pull request: https://github.com/apache/spark/pull/12928#discussion_r62169812 --- Diff: yarn/src/main/scala/org/apache/spark/deploy/yarn/Client.scala --- @@ -155,10 +155,10 @@ private[spark] class Client( // Get a new application from our RM val newApp = yarnClient.createApplication() - val newAppResponse = newApp.getNewApplicationResponse() - appId = newAppResponse.getApplicationId() + val newAppResponse = newApp.getNewApplicationResponse --- End diff -- Yes, there are, but IDEA kept bugging me all the time about calling getters with `()` as if mutation/computation happened. I'll revert it. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-15152][DOC][MINOR] Scaladoc and Code st...
Github user srowen commented on a diff in the pull request:
https://github.com/apache/spark/pull/12928#discussion_r62169130
--- Diff: core/src/main/scala/org/apache/spark/scheduler/Pool.scala ---
@@ -98,13 +97,12 @@ private[spark] class Pool(
}
override def getSortedTaskSetQueue: ArrayBuffer[TaskSetManager] = {
-var sortedTaskSetQueue = new ArrayBuffer[TaskSetManager]
-val sortedSchedulableQueue =
-
schedulableQueue.asScala.toSeq.sortWith(taskSetSchedulingAlgorithm.comparator)
-for (schedulable <- sortedSchedulableQueue) {
- sortedTaskSetQueue ++= schedulable.getSortedTaskSetQueue
-}
-sortedTaskSetQueue
+schedulableQueue
--- End diff --
This isn't a minor typo change. I'm not clear this will work? before it was
producing a collection from the individual schedulable's queues but that has
disappeared. Is this even of the right type?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-15152][DOC][MINOR] Scaladoc and Code st...
Github user srowen commented on a diff in the pull request:
https://github.com/apache/spark/pull/12928#discussion_r62168794
--- Diff: yarn/src/main/scala/org/apache/spark/deploy/yarn/Client.scala ---
@@ -168,7 +168,7 @@ private[spark] class Client(
val appContext = createApplicationSubmissionContext(newApp,
containerContext)
// Finally, submit and monitor the application
- logInfo(s"Submitting application ${appId.getId} to ResourceManager")
+ logInfo(s"Submitting application $appId to ResourceManager")
--- End diff --
not sure this is the intent of the log? log the ID right?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-15152][DOC][MINOR] Scaladoc and Code st...
Github user srowen commented on a diff in the pull request:
https://github.com/apache/spark/pull/12928#discussion_r62168705
--- Diff:
streaming/src/main/scala/org/apache/spark/streaming/Checkpoint.scala ---
@@ -125,7 +128,7 @@ object Checkpoint extends Logging {
Seq.empty
}
} else {
- logInfo("Checkpoint directory " + path + " does not exist")
+ logWarning("Checkpoint directory " + path + " does not exist")
--- End diff --
You could go ahead and use interpolation here
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-15152][DOC][MINOR] Scaladoc and Code st...
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/12928#discussion_r62168766 --- Diff: yarn/src/main/scala/org/apache/spark/deploy/yarn/Client.scala --- @@ -155,10 +155,10 @@ private[spark] class Client( // Get a new application from our RM val newApp = yarnClient.createApplication() - val newAppResponse = newApp.getNewApplicationResponse() - appId = newAppResponse.getApplicationId() + val newAppResponse = newApp.getNewApplicationResponse --- End diff -- These are Java methods? I think it's canonical to call them with empty args. Not worth changing anyway --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-15152][DOC][MINOR] Scaladoc and Code st...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12928#issuecomment-217117345 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-15152][DOC][MINOR] Scaladoc and Code st...
GitHub user jaceklaskowski opened a pull request: https://github.com/apache/spark/pull/12928 [SPARK-15152][DOC][MINOR] Scaladoc and Code style Improvements ## What changes were proposed in this pull request? Minor doc and code style fixes ## How was this patch tested? local build You can merge this pull request into a Git repository by running: $ git pull https://github.com/jaceklaskowski/spark SPARK-15152 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/spark/pull/12928.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #12928 commit a3449278bc04b1e40cf782779066d58df899ea06 Author: Jacek Laskowski Date: 2016-05-05T10:00:28Z [DOC][MINOR] SPARK-15152 Scaladoc and Code style Improvements --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
