[GitHub] spark pull request #14433: [SPARK-16829][SparkR]:sparkR sc.setLogLevel doesn...
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/14433 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request #14433: [SPARK-16829][SparkR]:sparkR sc.setLogLevel doesn...
Github user shivaram commented on a diff in the pull request:
https://github.com/apache/spark/pull/14433#discussion_r77443065
--- Diff: core/src/main/scala/org/apache/spark/internal/Logging.scala ---
@@ -135,7 +135,8 @@ private[spark] trait Logging {
val replLevel = Option(replLogger.getLevel()).getOrElse(Level.WARN)
if (replLevel != rootLogger.getEffectiveLevel()) {
System.err.printf("Setting default log level to \"%s\".\n",
replLevel)
- System.err.println("To adjust logging level use
sc.setLogLevel(newLevel).")
+ System.err.println("To adjust logging level use
sc.setLogLevel(newLevel). " +
+ "For SparkR, use setLogLevel(newLevel).")
--- End diff --
I fixed this up during the merge
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request #14433: [SPARK-16829][SparkR]:sparkR sc.setLogLevel doesn...
Github user srowen commented on a diff in the pull request:
https://github.com/apache/spark/pull/14433#discussion_r77433830
--- Diff: core/src/main/scala/org/apache/spark/internal/Logging.scala ---
@@ -135,7 +135,8 @@ private[spark] trait Logging {
val replLevel = Option(replLogger.getLevel()).getOrElse(Level.WARN)
if (replLevel != rootLogger.getEffectiveLevel()) {
System.err.printf("Setting default log level to \"%s\".\n",
replLevel)
- System.err.println("To adjust logging level use
sc.setLogLevel(newLevel).")
+ System.err.println("To adjust logging level use
sc.setLogLevel(newLevel). " +
+ "For SparkR, use setLogLevel(newLevel).")
--- End diff --
While I'd indent this continuation line, I don't think it's worth worrying
about. LGTM
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request #14433: [SPARK-16829][SparkR]:sparkR sc.setLogLevel doesn...
Github user shivaram commented on a diff in the pull request:
https://github.com/apache/spark/pull/14433#discussion_r77261262
--- Diff: core/src/main/scala/org/apache/spark/internal/Logging.scala ---
@@ -135,7 +136,12 @@ private[spark] trait Logging {
val replLevel = Option(replLogger.getLevel()).getOrElse(Level.WARN)
if (replLevel != rootLogger.getEffectiveLevel()) {
System.err.printf("Setting default log level to \"%s\".\n",
replLevel)
- System.err.println("To adjust logging level use
sc.setLogLevel(newLevel).")
+ if (SparkSubmit.isRShell) {
--- End diff --
Sounds good to me
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request #14433: [SPARK-16829][SparkR]:sparkR sc.setLogLevel doesn...
Github user wangmiao1981 commented on a diff in the pull request:
https://github.com/apache/spark/pull/14433#discussion_r77260947
--- Diff: core/src/main/scala/org/apache/spark/internal/Logging.scala ---
@@ -135,7 +136,12 @@ private[spark] trait Logging {
val replLevel = Option(replLogger.getLevel()).getOrElse(Level.WARN)
if (replLevel != rootLogger.getEffectiveLevel()) {
System.err.printf("Setting default log level to \"%s\".\n",
replLevel)
- System.err.println("To adjust logging level use
sc.setLogLevel(newLevel).")
+ if (SparkSubmit.isRShell) {
--- End diff --
How about I change the log like `To adjust logging level use
sc.setLogLevel(newLevel). For SparkR, use setLogLevel(newLevel)`? Thanks!
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request #14433: [SPARK-16829][SparkR]:sparkR sc.setLogLevel doesn...
Github user shivaram commented on a diff in the pull request:
https://github.com/apache/spark/pull/14433#discussion_r77205301
--- Diff: core/src/main/scala/org/apache/spark/internal/Logging.scala ---
@@ -135,7 +136,12 @@ private[spark] trait Logging {
val replLevel = Option(replLogger.getLevel()).getOrElse(Level.WARN)
if (replLevel != rootLogger.getEffectiveLevel()) {
System.err.printf("Setting default log level to \"%s\".\n",
replLevel)
- System.err.println("To adjust logging level use
sc.setLogLevel(newLevel).")
+ if (SparkSubmit.isRShell) {
--- End diff --
+1 -- I personally find it odd that we would import `deploy.SparkSubmit`
into `Logging` which is a base class used throughout the code base. We could
also change the log message to be more vague. Something like `To adjust logging
level please call setLogLevel(newLevel) using SparkContext` might work for all
languages ?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request #14433: [SPARK-16829][SparkR]:sparkR sc.setLogLevel doesn...
Github user srowen commented on a diff in the pull request:
https://github.com/apache/spark/pull/14433#discussion_r77125115
--- Diff: core/src/main/scala/org/apache/spark/internal/Logging.scala ---
@@ -135,7 +136,12 @@ private[spark] trait Logging {
val replLevel = Option(replLogger.getLevel()).getOrElse(Level.WARN)
if (replLevel != rootLogger.getEffectiveLevel()) {
System.err.printf("Setting default log level to \"%s\".\n",
replLevel)
- System.err.println("To adjust logging level use
sc.setLogLevel(newLevel).")
+ if (SparkSubmit.isRShell) {
--- End diff --
I personally think it's a bit ugly to pipe through this info with extra
methods and so on. It seems simpler just to amend the message to also state the
right command for sparkr.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request #14433: [SPARK-16829][SparkR]:sparkR sc.setLogLevel doesn...
Github user wangmiao1981 commented on a diff in the pull request:
https://github.com/apache/spark/pull/14433#discussion_r74981188
--- Diff: core/src/main/scala/org/apache/spark/internal/Logging.scala ---
@@ -18,9 +18,9 @@
package org.apache.spark.internal
import org.apache.log4j.{Level, LogManager, PropertyConfigurator}
+import org.apache.spark.deploy.{SparkShellType, SparkSubmit}
import org.slf4j.{Logger, LoggerFactory}
import org.slf4j.impl.StaticLoggerBinder
-
import org.apache.spark.util.Utils
--- End diff --
@felixcheung It seems that you saw my old commit for some reason. I think I
changed the import order in the latest commit. Thanks!
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request #14433: [SPARK-16829][SparkR]:sparkR sc.setLogLevel doesn...
Github user felixcheung commented on a diff in the pull request:
https://github.com/apache/spark/pull/14433#discussion_r74883390
--- Diff: core/src/main/scala/org/apache/spark/internal/Logging.scala ---
@@ -18,9 +18,9 @@
package org.apache.spark.internal
import org.apache.log4j.{Level, LogManager, PropertyConfigurator}
+import org.apache.spark.deploy.{SparkShellType, SparkSubmit}
import org.slf4j.{Logger, LoggerFactory}
import org.slf4j.impl.StaticLoggerBinder
-
import org.apache.spark.util.Utils
--- End diff --
as per
[this](https://cwiki.apache.org/confluence/display/SPARK/Spark+Code+Style+Guide#SparkCodeStyleGuide-Imports)
`import org.apache.spark.deploy` should go here
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request #14433: [SPARK-16829][SparkR]:sparkR sc.setLogLevel doesn...
GitHub user wangmiao1981 opened a pull request:
https://github.com/apache/spark/pull/14433
[SPARK-16829][SparkR]:sparkR sc.setLogLevel doesn't work
## What changes were proposed in this pull request?
(Please fill in changes proposed in this fix)
./bin/sparkR
Launching java with spark-submit command
/Users/mwang/spark_ws_0904/bin/spark-submit "sparkr-shell"
/var/folders/s_/83b0sgvj2kl2kwq4stvft_pmgn/T//RtmpQxJGiZ/backend_porte9474603ed1e
Using Spark's default log4j profile:
org/apache/spark/log4j-defaults.properties
Setting default log level to "WARN".
To adjust logging level use sc.setLogLevel(newLevel).
> sc.setLogLevel("INFO")
Error: could not find function "sc.setLogLevel"
sc.setLogLevel doesn't exist.
R has a function setLogLevel.
I rename the setLogLevel function to sc.setLogLevel.
## How was this patch tested?
(Please explain how this patch was tested. E.g. unit tests, integration
tests, manual tests)
Change unit test. Run unit tests.
Manually tested it in sparkR shell.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/wangmiao1981/spark sc
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/14433.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #14433
commit 9d3d8e52d3690cff4f6d0353e6198df918bc3f5f
Author: wm...@hotmail.com
Date: 2016-08-01T06:10:29Z
rename setLogLevel
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
