[
https://issues.apache.org/jira/browse/SPARK-4878?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14250711#comment-14250711
]
Stephen Haberman commented on SPARK-4878:
-----------------------------------------
Here are some of the log messages in question in the driver output:
{code}
14/12/17 19:38:08 INFO actor.LocalActorRef: Message
[akka.remote.transport.AssociationHandle$Disassociated] from
Actor[akka://sparkDriver/deadLetters] to
Actor[akka://sparkDriver/system/transports/akkaprotocolmanager.tcp0/akkaProtocol-tcp%3A%2F%2FsparkDriver%4010.165.37.184%3A41481-3#1336761453]
was not delivered. [1] dead letters encountered. This logging can be turned
off or adjusted with configuration settings 'akka.log-dead-letters' and
'akka.log-dead-letters-during-shutdown'.
14/12/17 19:38:08 INFO actor.LocalActorRef: Message
[akka.remote.transport.AssociationHandle$Disassociated] from
Actor[akka://sparkDriver/deadLetters] to
Actor[akka://sparkDriver/system/transports/akkaprotocolmanager.tcp0/akkaProtocol-tcp%3A%2F%2FsparkDriver%4010.152.183.137%3A52160-2#-1663053182]
was not delivered. [2] dead letters encountered. This logging can be turned
off or adjusted with configuration settings 'akka.log-dead-letters' and
'akka.log-dead-letters-during-shutdown'.
14/12/17 19:38:08 INFO actor.LocalActorRef: Message
[akka.remote.transport.ActorTransportAdapter$DisassociateUnderlying] from
Actor[akka://sparkDriver/deadLetters] to
Actor[akka://sparkDriver/system/transports/akkaprotocolmanager.tcp0/akkaProtocol-tcp%3A%2F%2FsparkDriver%4010.152.183.137%3A52160-2#-1663053182]
was not delivered. [3] dead letters encountered. This logging can be turned
off or adjusted with configuration settings 'akka.log-dead-letters' and
'akka.log-dead-letters-during-shutdown'.
14/12/17 19:38:08 INFO actor.LocalActorRef: Message
[akka.remote.transport.ActorTransportAdapter$DisassociateUnderlying] from
Actor[akka://sparkDriver/deadLetters] to
Actor[akka://sparkDriver/system/transports/akkaprotocolmanager.tcp0/akkaProtocol-tcp%3A%2F%2FsparkDriver%4010.165.37.184%3A41481-3#1336761453]
was not delivered. [4] dead letters encountered. This logging can be turned
off or adjusted with configuration settings 'akka.log-dead-letters' and
'akka.log-dead-letters-during-shutdown'.
14/12/17 19:38:08 INFO actor.LocalActorRef: Message
[akka.remote.transport.AssociationHandle$Disassociated] from
Actor[akka://sparkDriver/deadLetters] to
Actor[akka://sparkDriver/system/transports/akkaprotocolmanager.tcp0/akkaProtocol-tcp%3A%2F%2FsparkDriver%4010.147.214.241%3A60702-4#-1987258875]
was not delivered. [5] dead letters encountered. This logging can be turned
off or adjusted with configuration settings 'akka.log-dead-letters' and
'akka.log-dead-letters-during-shutdown'.
14/12/17 19:38:08 INFO actor.LocalActorRef: Message
[akka.remote.transport.ActorTransportAdapter$DisassociateUnderlying] from
Actor[akka://sparkDriver/deadLetters] to
Actor[akka://sparkDriver/system/transports/akkaprotocolmanager.tcp0/akkaProtocol-tcp%3A%2F%2FsparkDriver%4010.147.214.241%3A60702-4#-1987258875]
was not delivered. [6] dead letters encountered. This logging can be turned
off or adjusted with configuration settings 'akka.log-dead-letters' and
'akka.log-dead-letters-during-shutdown'.
{code}
> driverPropsFetcher causes spurious Akka disassociate errors
> -----------------------------------------------------------
>
> Key: SPARK-4878
> URL: https://issues.apache.org/jira/browse/SPARK-4878
> Project: Spark
> Issue Type: Bug
> Components: Spark Core
> Affects Versions: 1.2.0
> Reporter: Stephen Haberman
> Priority: Minor
>
> The dedicated Akka system to fetching driver properties seems fine, but it
> leads to very misleading "AssociationHandle$Disassociated", dead letter, etc.
> sort of messages that can lead the user to believe something is wrong with
> the cluster.
> (E.g. personally I thought it was a Spark -rc1/-rc2 bug and spent awhile
> poking around until I saw in the code that driverPropsFetcher is
> purposefully/immediately shutdown.)
> Is there any way to cleanly shutdown that initial akka system so that the
> driver doesn't log these errors?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
