[
https://issues.apache.org/jira/browse/SPARK-3522?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14151348#comment-14151348
]
Shivaram Venkataraman commented on SPARK-3522:
----------------------------------------------
It would be good, but I think most of the output in spark-ec2 comes from the
shell scripts that install things like HDFS, Spark etc. So this would be less
of a python logging change and more of a change in shell scripts in spark-ec2.
Also the other thing to consider is that the output is often the only way to
figure out what / why things went wrong during cluster launch. So it might be
better to save it to a file (spark-ec2-<cluster-name>-launch.log) as sometimes
re-running spark-ec2 with more logging could be expensive.
> Make spark-ec2 verbosity configurable
> -------------------------------------
>
> Key: SPARK-3522
> URL: https://issues.apache.org/jira/browse/SPARK-3522
> Project: Spark
> Issue Type: Improvement
> Components: EC2
> Reporter: Nicholas Chammas
> Priority: Minor
>
> When launching a cluster, {{spark-ec2}} spits out a lot of stuff that feels
> like debug output. It would be better for the user if {{spark-ec2}} did the
> following:
> * default to info output level
> * allow option to increase verbosity and include debug output
> This will require converting most of the {{print}} statements in the script
> to use Python's {{logging}} module and setting output levels ({{INFO}},
> {{WARN}}, {{DEBUG}}) for each statement.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
