[jira] [Commented] (SPARK-7977) Disallow println
[
https://issues.apache.org/jira/browse/SPARK-7977?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14991408#comment-14991408
]
Sean Owen commented on SPARK-7977:
--
This has nothing to do with this JIRA, and is a question for
u...@spark.apache.org.
> Disallow println
>
>
> Key: SPARK-7977
> URL: https://issues.apache.org/jira/browse/SPARK-7977
> Project: Spark
> Issue Type: Sub-task
> Components: Project Infra
>Reporter: Reynold Xin
>Assignee: Jon Alter
> Labels: starter
> Fix For: 1.5.0
>
>
> Very often we see pull requests that added println from debugging, but the
> author forgot to remove it before code review.
> We can use the regex checker to disallow println. For legitimate use of
> println, we can then disable the rule where they are used.
> Add to scalastyle-config.xml file:
> {code}
>class="org.scalastyle.scalariform.TokenChecker" enabled="true">
> ^println$
>
>
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-7977) Disallow println
[
https://issues.apache.org/jira/browse/SPARK-7977?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14991399#comment-14991399
]
sparkerjin commented on SPARK-7977:
---
Hi,
I run the spark-submit with cluster mode, but there was not any output about
the driverID and the status of the driver. Is
The descriptions are as follows:
1. run the spark-submit with cluster mode:
[root@jasonspark02 spark-1.5.1-bin-hadoop2.4]# bin/spark-submit --deploy-mode
cluster --class org.apache.spark.examples.SparkPi ./lib/spark-
examples-1.5.1-hadoop2.4.0.jar
Running Spark using the REST application submission protocol.
15/11/05 02:13:43 INFO rest.RestSubmissionClient: Submitting a request to
launch an application in spark://jasonspark02:7077.
15/11/05 02:13:43 WARN rest.RestSubmissionClient: Unable to connect to server
spark://jasonspark02:7077.
Warning: Master endpoint spark://jasonspark02:7077 was not a REST server.
Falling back to legacy submission gateway instead.
15/11/05 02:13:43 WARN util.NativeCodeLoader: Unable to load native-hadoop
library for your platform... using builtin-java classes where
applicable
[root@jasonspark02 spark-1.5.1-bin-hadoop2.4]#
2. Problem:
we can see that there is not any information about the driverId and the state
of the driver. I think users need to use these information to get more
information about the job.
And running spark-submit cluster mode with --verbose, it also could not get
the info.
3. Reason:
I looked into the code, and found that --verbose or -v was not passed into
the childArgs in spark-submit, and it used the Level.WARN as the default in
Client.scala.
4. Expected:
I think the users should know the driverID and status after he/she submits a
job.
such as:
[root@jasonpark02 spark-1.5.1-bin-hadoop2.4]# bin/spark-submit --deploy-mode
cluster --conf spark.ego.uname=u1 --conf spark.ego.passwd=u1 --class
org.apache.spark.examples.SparkPi ./lib/spark-examples-1.5.1-hadoop2.4.0.jar
Running Spark using the REST application submission protocol.
15/11/05 03:08:43 INFO rest.RestSubmissionClient: Submitting a request to
launch an application in spark://jasonspark02:7077.
15/11/05 03:08:44 WARN rest.RestSubmissionClient: Unable to connect to server
spark://jasonspark02:7077.
Warning: Master endpoint spark://jinspark02:7077 was not a REST server. Falling
back to legacy submission gateway instead.
15/11/05 03:08:44 WARN util.NativeCodeLoader: Unable to load native-hadoop
library for your platform... using builtin-java classes where applicable
15/11/05 03:08:44 INFO spark.SecurityManager: Changing view acls to: root
15/11/05 03:08:44 INFO spark.SecurityManager: Changing modify acls to: root
15/11/05 03:08:44 INFO spark.SecurityManager: SecurityManager: authentication
disabled; ui acls disabled; users with view permissions: Set(root); users with
modify permissions: Set(root)
15/11/05 03:08:45 INFO slf4j.Slf4jLogger: Slf4jLogger started
15/11/05 03:08:45 INFO util.Utils: Successfully started service 'driverClient'
on port 47454.
15/11/05 03:08:45 INFO deploy.ClientEndpoint: Driver successfully submitted as
driver-20151105030845- <-- I think the
info is very important
15/11/05 03:08:45 INFO deploy.ClientEndpoint: ... waiting before polling master
for driver state
15/11/05 03:08:50 INFO deploy.ClientEndpoint: ... polling master for driver
state
15/11/05 03:08:50 INFO deploy.ClientEndpoint: State of
driver-20151105030845-
What do you think of it? Any ideas, please let me know. Thanks.
Jie Hua
> Disallow println
>
>
> Key: SPARK-7977
> URL: https://issues.apache.org/jira/browse/SPARK-7977
> Project: Spark
> Issue Type: Sub-task
> Components: Project Infra
>Reporter: Reynold Xin
>Assignee: Jon Alter
> Labels: starter
> Fix For: 1.5.0
>
>
> Very often we see pull requests that added println from debugging, but the
> author forgot to remove it before code review.
> We can use the regex checker to disallow println. For legitimate use of
> println, we can then disable the rule where they are used.
> Add to scalastyle-config.xml file:
> {code}
>class="org.scalastyle.scalariform.TokenChecker" enabled="true">
> ^println$
>
>
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-7977) Disallow println
[
https://issues.apache.org/jira/browse/SPARK-7977?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14606251#comment-14606251
]
Apache Spark commented on SPARK-7977:
-
User 'jonalter' has created a pull request for this issue:
https://github.com/apache/spark/pull/7093
Disallow println
Key: SPARK-7977
URL: https://issues.apache.org/jira/browse/SPARK-7977
Project: Spark
Issue Type: Sub-task
Components: Project Infra
Reporter: Reynold Xin
Labels: starter
Very often we see pull requests that added println from debugging, but the
author forgot to remove it before code review.
We can use the regex checker to disallow println. For legitimate use of
println, we can then disable the rule where they are used.
Add to scalastyle-config.xml file:
{code}
check customId=println level=error
class=org.scalastyle.scalariform.TokenChecker enabled=true
parametersparameter name=regex^println$/parameter/parameters
customMessage![CDATA[Are you sure you want to println? If yes, wrap
the code block with
// scalastyle:off println
println(...)
// scalastyle:on println]]/customMessage
/check
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-7977) Disallow println
[
https://issues.apache.org/jira/browse/SPARK-7977?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14601827#comment-14601827
]
Jon Alter commented on SPARK-7977:
--
Working on this.
Disallow println
Key: SPARK-7977
URL: https://issues.apache.org/jira/browse/SPARK-7977
Project: Spark
Issue Type: Sub-task
Components: Project Infra
Reporter: Reynold Xin
Labels: starter
Very often we see pull requests that added println from debugging, but the
author forgot to remove it before code review.
We can use the regex checker to disallow println. For legitimate use of
println, we can then disable the rule where they are used.
Add to scalastyle-config.xml file:
{code}
check customId=println level=error
class=org.scalastyle.scalariform.TokenChecker enabled=true
parametersparameter name=regex^println$/parameter/parameters
customMessage![CDATA[Are you sure you want to println? If yes, wrap
the code block with
// scalastyle:off println
println(...)
// scalastyle:on println]]/customMessage
/check
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-7977) Disallow println
[
https://issues.apache.org/jira/browse/SPARK-7977?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14572272#comment-14572272
]
Charles Yeh commented on SPARK-7977:
I'm looking for a starter task, but this was completed.
Disallow println
Key: SPARK-7977
URL: https://issues.apache.org/jira/browse/SPARK-7977
Project: Spark
Issue Type: Sub-task
Components: Project Infra
Reporter: Reynold Xin
Labels: starter
Very often we see pull requests that added println from debugging, but the
author forgot to remove it before code review.
We can use the regex checker to disallow println. For legitimate use of
println, we can then disable the rule.
Add to scalastyle-config.xml file:
{code}
check customId=println level=error
class=org.scalastyle.scalariform.TokenChecker enabled=true
parametersparameter name=regex^println$/parameter/parameters
customMessage![CDATA[Are you sure you want to println? If yes, wrap
the code block with
// scalastyle:off println
println(...)
// scalastyle:on println]]/customMessage
/check
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-7977) Disallow println
[
https://issues.apache.org/jira/browse/SPARK-7977?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14572286#comment-14572286
]
Reynold Xin commented on SPARK-7977:
Yes let's do that.
Disallow println
Key: SPARK-7977
URL: https://issues.apache.org/jira/browse/SPARK-7977
Project: Spark
Issue Type: Sub-task
Components: Project Infra
Reporter: Reynold Xin
Labels: starter
Very often we see pull requests that added println from debugging, but the
author forgot to remove it before code review.
We can use the regex checker to disallow println. For legitimate use of
println, we can then disable the rule where they are used.
Add to scalastyle-config.xml file:
{code}
check customId=println level=error
class=org.scalastyle.scalariform.TokenChecker enabled=true
parametersparameter name=regex^println$/parameter/parameters
customMessage![CDATA[Are you sure you want to println? If yes, wrap
the code block with
// scalastyle:off println
println(...)
// scalastyle:on println]]/customMessage
/check
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-7977) Disallow println
[
https://issues.apache.org/jira/browse/SPARK-7977?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14572283#comment-14572283
]
Charles Yeh commented on SPARK-7977:
scalastyle doesn't seem to have a white-listing method. If I were to add
// scalastyle:off println
// scalastyle:on println
to existing files, is it preferred to have everything in one huge diff?
Disallow println
Key: SPARK-7977
URL: https://issues.apache.org/jira/browse/SPARK-7977
Project: Spark
Issue Type: Sub-task
Components: Project Infra
Reporter: Reynold Xin
Labels: starter
Very often we see pull requests that added println from debugging, but the
author forgot to remove it before code review.
We can use the regex checker to disallow println. For legitimate use of
println, we can then disable the rule where they are used.
Add to scalastyle-config.xml file:
{code}
check customId=println level=error
class=org.scalastyle.scalariform.TokenChecker enabled=true
parametersparameter name=regex^println$/parameter/parameters
customMessage![CDATA[Are you sure you want to println? If yes, wrap
the code block with
// scalastyle:off println
println(...)
// scalastyle:on println]]/customMessage
/check
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
