Github user tgravescs commented on the issue:
https://github.com/apache/spark/pull/18547
There is no reason to print out messages that aren't useful to the users.
Many users see Warnings and read them and think there is a problem with their
application or configuration. Most of these are really for advanced users or
devs trying to debug what is going on. So changing them to debug makes sense
to me.
There are also times that shutdown can take a very long time and all its
doing it spewing these log messages. For instance when you have a backlog of
events in the event queue. I have seen multiple minutes of it just throwing
the warnings like: Message $message dropped. ${e.getMessage}
dynamic allocation can also add a lot of warning messages that are
unneeded. When I'm shutting down things should stop quickly and release the
resources for others to use.
I still need to review in detail but these should just be in the shutdown
case, other times I agree that you may want to see these.
We should definitely update the description here and in the jira to have
more details though.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
