Hi,
Apologies for resurrecting a long-dead thread, but I am facing a similar
problem when using remoting with Akka 2.3.0 with Scala 2.10.2.
I have two actor systems:
- remote-registration-service: Runs on localhost:8245, and "registers"
clients
- registration-client-system-{UUID}: Runs on localhost:2252 and sends
registration request to the "remote-registration-service" using Patterns.ask
I start up the remote-registration-service, and then start up the
registration-client-system. The registration-client-system then uses an
ActorSelection, a resolveOnce, and a Patterns.ask to send a registration
request to, and receive a response from, the remote-registration-service.
Here's the relevant code snippets (Java):
Future<ActorRef> registrarActorRefFuture =
system.actorSelection(registrationServicePath).resolveOne(new
FiniteDuration
(QUEUE_REGISTRATION_RETRY_TIMEOUT_MS,
TimeUnit.MILLISECONDS));
final ActorRef registrarActorRef =
Await.result(registrarActorRefFuture, DURATION);
Future<Object> registrationResultFuture =
Patterns.ask(registrationClientActor, new RegisterMessage(clientId,
priority), DURATION.toMillis());
Object regResult =
Await.result(registrationResultFuture, DURATION);
The registration process executes successfully and I receive a response
back from the remote-registration-service. The registration-client-service
then shuts down.
At this point, the remote-registration-service starts to print out dead
letter messages as shown below. I have marked the messages from the
remote-registration-service in *Bold*
2014-05-01 19:33:19,965 DEBUG: c.c.c.c.s.QueueRegistrationService |
...Registration complete. Starting shutdown of
registration-client-system...
[INFO] [05/01/2014 19:33:19.974]
[registration-client-system0c0be54e-87d8-4920-8725-019b2e8ab0f0-akka.remote.default-remote-dispatcher-6]
[akka.tcp://registration-client-system0c0be54e-87d8-4920-8725-019b2e8ab0f0@127.0.0.1:2552/system/remoting-terminator]
Shutting down remote daemon.
[INFO] [05/01/2014 19:33:19.976]
[registration-client-system0c0be54e-87d8-4920-8725-019b2e8ab0f0-akka.remote.default-remote-dispatcher-6]
[akka.tcp://registration-client-system0c0be54e-87d8-4920-8725-019b2e8ab0f0@127.0.0.1:2552/system/remoting-terminator]
Remote daemon shut down; proceeding with flushing remote transports.
[INFO] [05/01/2014 19:33:19.992]
[registration-client-system0c0be54e-87d8-4920-8725-019b2e8ab0f0-akka.actor.default-dispatcher-3]
[akka://registration-client-system0c0be54e-87d8-4920-8725-019b2e8ab0f0/system/endpointManager/reliableEndpointWriter-akka.tcp%3A%2F%2Fremote-registration-service%40127.0.0.1%3A8245-0/endpointWriter/endpointReader-akka.tcp%3A%2F%2Fremote-registration-service%40127.0.0.1%3A8245-0]
Message [akka.remote.transport.AssociationHandle$Disassociated] from
Actor[akka://registration-client-system0c0be54e-87d8-4920-8725-019b2e8ab0f0/deadLetters]
to
Actor[akka://registration-client-system0c0be54e-87d8-4920-8725-019b2e8ab0f0/system/endpointManager/reliableEndpointWriter-akka.tcp%3A%2F%2Fremote-registration-service%40127.0.0.1%3A8245-0/endpointWriter/endpointReader-akka.tcp%3A%2F%2Fremote-registration-service%40127.0.0.1%3A8245-0#-1326837246]
was not delivered. [1] dead letters encountered. This logging can be turned
off or adjusted with configuration settings 'akka.log-dead-letters' and
'akka.log-dead-letters-during-shutdown'.
[INFO] [05/01/2014 19:33:19.993]
[registration-client-system0c0be54e-87d8-4920-8725-019b2e8ab0f0-akka.actor.default-dispatcher-3]
[akka://registration-client-system0c0be54e-87d8-4920-8725-019b2e8ab0f0/system/transports/akkaprotocolmanager.tcp1/akkaProtocol-tcp%3A%2F%2Fremote-registration-service%40127.0.0.1%3A8245-1]
Message [akka.remote.transport.AssociationHandle$Disassociated] from
Actor[akka://registration-client-system0c0be54e-87d8-4920-8725-019b2e8ab0f0/deadLetters]
to
Actor[akka://registration-client-system0c0be54e-87d8-4920-8725-019b2e8ab0f0/system/transports/akkaprotocolmanager.tcp1/akkaProtocol-tcp%3A%2F%2Fremote-registration-service%40127.0.0.1%3A8245-1#-606871149]
was not delivered. [2] dead letters encountered. This logging can be turned
off or adjusted with configuration settings 'akka.log-dead-letters' and
'akka.log-dead-letters-during-shutdown'.
*[INFO] [05/01/2014 19:33:19.993]
[remote-registration-service-akka.actor.default-dispatcher-2]
[akka://remote-registration-service/system/transports/akkaprotocolmanager.tcp0/akkaProtocol-tcp%3A%2F%2Fremote-registration-service%40127.0.0.1%3A57446-1]
Message [akka.remote.transport.AssociationHandle$Disassociated] from
Actor[akka://remote-registration-service/deadLetters] to
Actor[akka://remote-registration-service/system/transports/akkaprotocolmanager.tcp0/akkaProtocol-tcp%3A%2F%2Fremote-registration-service%40127.0.0.1%3A57446-1#-707785622]
was not delivered. [1] dead letters encountered. This logging can be turned
off or adjusted with configuration settings 'akka.log-dead-letters' and
'akka.log-dead-letters-during-shutdown'.*
*[ERROR] [05/01/2014 19:33:19.998]
[remote-registration-service-akka.remote.default-remote-dispatcher-6]
[akka.tcp://[email protected]:8245/system/endpointManager/reliableEndpointWriter-akka.tcp%3A%2F%2Fregistration-client-system0c0be54e-87d8-4920-8725-019b2e8ab0f0%40127.0.0.1%3A2552-0/endpointWriter]
AssociationError [akka.tcp://[email protected]:8245] <-
[akka.tcp://registration-client-system0c0be54e-87d8-4920-8725-019b2e8ab0f0@127.0.0.1:2552]:
Error [Shut down address:
akka.tcp://registration-client-system0c0be54e-87d8-4920-8725-019b2e8ab0f0@127.0.0.1:2552]
[*
*akka.remote.ShutDownAssociation: Shut down address:
akka.tcp://registration-client-system0c0be54e-87d8-4920-8725-019b2e8ab0f0@127.0.0.1:2552*
*Caused by: akka.remote.transport.Transport$InvalidAssociationException:
The remote system terminated the association because it is shutting down.*
*]*
*[INFO] [05/01/2014 19:33:20.003]
[remote-registration-service-akka.actor.default-dispatcher-3]
[akka://remote-registration-service/system/transports/akkaprotocolmanager.tcp0/akkaProtocol-tcp%3A%2F%2Fremote-registration-service%40127.0.0.1%3A57446-1]
Message
[akka.remote.transport.ActorTransportAdapter$DisassociateUnderlying] from
Actor[akka://remote-registration-service/deadLetters] to
Actor[akka://remote-registration-service/system/transports/akkaprotocolmanager.tcp0/akkaProtocol-tcp%3A%2F%2Fremote-registration-service%40127.0.0.1%3A57446-1#-707785622]
was not delivered. [2] dead letters encountered. This logging can be turned
off or adjusted with configuration settings 'akka.log-dead-letters' and
'akka.log-dead-letters-during-shutdown'.*
[INFO] [05/01/2014 19:33:20.011] [ForkJoinPool-5-worker-15] [Remoting]
Remoting shut down
[INFO] [05/01/2014 19:33:20.011]
[registration-client-system0c0be54e-87d8-4920-8725-019b2e8ab0f0-akka.remote.default-remote-dispatcher-6]
[akka.tcp://registration-client-system0c0be54e-87d8-4920-8725-019b2e8ab0f0@127.0.0.1:2552/system/remoting-terminator]
Remoting shut down.
Then, I restart the registration-client-system. This time, it comes up with
a different actor-system-name because the suffix UUID is different. But
this time, even retrieving the actorref from the ActorSelection fails as
shown in the error logs below:
2014-05-01 19:33:22,084 ERROR: c.c.c.c.s.QueueRegistrationService | FAIL!
akka.actor.ActorNotFound: Actor not found for:
ActorSelection[Anchor(akka.tcp://[email protected]:8245/),
Path(/user/registrar)]
at
akka.actor.ActorSelection$$anonfun$resolveOne$1.apply(ActorSelection.scala:65)
~[akka-actor_2.10-2.3.0.jar:na]
at
akka.actor.ActorSelection$$anonfun$resolveOne$1.apply(ActorSelection.scala:63)
~[akka-actor_2.10-2.3.0.jar:na]
at scala.concurrent.impl.CallbackRunnable.run(Promise.scala:29)
~[scala-library-2.10.2.jar:na]
at
akka.dispatch.BatchingExecutor$Batch$$anonfun$run$1.processBatch$1(BatchingExecutor.scala:67)
~[akka-actor_2.10-2.3.0.jar:na]
at
akka.dispatch.BatchingExecutor$Batch$$anonfun$run$1.apply$mcV$sp(BatchingExecutor.scala:82)
~[akka-actor_2.10-2.3.0.jar:na]
at
akka.dispatch.BatchingExecutor$Batch$$anonfun$run$1.apply(BatchingExecutor.scala:59)
~[akka-actor_2.10-2.3.0.jar:na]
at
akka.dispatch.BatchingExecutor$Batch$$anonfun$run$1.apply(BatchingExecutor.scala:59)
~[akka-actor_2.10-2.3.0.jar:na]
at scala.concurrent.BlockContext$.withBlockContext(BlockContext.scala:72)
~[scala-library-2.10.2.jar:na]
at akka.dispatch.BatchingExecutor$Batch.run(BatchingExecutor.scala:58)
~[akka-actor_2.10-2.3.0.jar:na]
at
akka.dispatch.ExecutionContexts$sameThreadExecutionContext$.unbatchedExecute(Future.scala:74)
~[akka-actor_2.10-2.3.0.jar:na]
at akka.dispatch.BatchingExecutor$class.execute(BatchingExecutor.scala:110)
~[akka-actor_2.10-2.3.0.jar:na]
at
akka.dispatch.ExecutionContexts$sameThreadExecutionContext$.execute(Future.scala:73)
~[akka-actor_2.10-2.3.0.jar:na]
at
scala.concurrent.impl.CallbackRunnable.executeWithValue(Promise.scala:37)
~[scala-library-2.10.2.jar:na]
at
scala.concurrent.impl.Promise$DefaultPromise.tryComplete(Promise.scala:133)
~[scala-library-2.10.2.jar:na]
at akka.pattern.PromiseActorRef.$bang(AskSupport.scala:267)
~[akka-actor_2.10-2.3.0.jar:na]
at akka.remote.DefaultMessageDispatcher.dispatch(Endpoint.scala:86)
~[akka-remote_2.10-2.3.0.jar:na]
at
akka.remote.EndpointReader$$anonfun$receive$2.applyOrElse(Endpoint.scala:804)
~[akka-remote_2.10-2.3.0.jar:na]
at akka.actor.Actor$class.aroundReceive(Actor.scala:465)
~[akka-actor_2.10-2.3.0.jar:na]
at akka.remote.EndpointActor.aroundReceive(Endpoint.scala:408)
~[akka-remote_2.10-2.3.0.jar:na]
at akka.actor.ActorCell.receiveMessage(ActorCell.scala:516)
~[akka-actor_2.10-2.3.0.jar:na]
at akka.actor.ActorCell.invoke(ActorCell.scala:487)
~[akka-actor_2.10-2.3.0.jar:na]
at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:238)
~[akka-actor_2.10-2.3.0.jar:na]
at akka.dispatch.Mailbox.run(Mailbox.scala:220)
~[akka-actor_2.10-2.3.0.jar:na]
at
akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(AbstractDispatcher.scala:393)
~[akka-actor_2.10-2.3.0.jar:na]
at scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)
~[scala-library-2.10.2.jar:na]
at
scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)
~[scala-library-2.10.2.jar:na]
at scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)
~[scala-library-2.10.2.jar:na]
at
scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)
~[scala-library-2.10.2.jar:na]
At this point, the registration-client-system-{UUID} begins its shutdown
process. Then, I see a very interesting error message from the
remote-registration-service which seems to indicate that the
remote-registration-service is aware of the registration-client-system, but
is for some reason not sending any messages back to it!
[ERROR] [05/01/2014 19:33:22.087]
[remote-registration-service-akka.remote.default-remote-dispatcher-6]
[akka.tcp://[email protected]:8245/system/endpointManager/reliableEndpointWriter-akka.tcp%3A%2F%2Fregistration-client-systemfd1fa6e1-5dfb-4a92-81af-089360ff304c%40127.0.0.1%3A2552-1/endpointWriter]
AssociationError [akka.tcp://[email protected]:8245] <-
[akka.tcp://registration-client-systemfd1fa6e1-5dfb-4a92-81af-089360ff304c@127.0.0.1:2552]:
Error [Shut down address:
akka.tcp://registration-client-systemfd1fa6e1-5dfb-4a92-81af-089360ff304c@127.0.0.1:2552]
[
akka.remote.ShutDownAssociation: Shut down address:
akka.tcp://registration-client-systemfd1fa6e1-5dfb-4a92-81af-089360ff304c@127.0.0.1:2552
Caused by: akka.remote.transport.Transport$InvalidAssociationException: The
remote system terminated the association because it is shutting down.
Is there something wrong with our remoting configuration? Could you please
help?
Thanks,
Siddhu
On Saturday, 2 March 2013 14:21:50 UTC, rkuhn wrote:
>
> Hi Daniel,
>
> 1 mar 2013 kl. 14:32 skrev Daniel Krzywicki:
>
> Hi,
>
> I want to write a simple client-server app using akka:
>
> - a server actor awaits for requests in a first actor system
> - a new actor system is created, along with a client actor, which
> submits a task remotedly to the server actor
> - the server actor computes some result and sends it back to the
> client actor
> - the client actor processess the results, then shuts down its actor
> system.
>
>
> The server actor does not retain references to the client. However, when
> the client system shuts down, the server starts
> throwing akka.remote.EndpointException.
>
>
> Thanks for trying out 2.2-M1! The new remoting maps the connection life
> cycle to an actor’s life cycle, and that actor then “fails” when the
> connection is closed. We have a ticket in the pipeline (#2824 - Change
> logging of actor failures) to enable different logging strategies for
> actors according to the application’s needs, and we will apply this to
> sanitize the overly verbose logging of the remoting layer.
>
> Until then please just ignore these exceptions, they are expected when the
> client disconnects.
>
> I’m assuming that afterwards remoting still works, right? If not, then
> please give more information (confirm the Akka version used and include
> full stack traces).
>
> Regards,
>
> Roland
>
>
> How to gracefully disconnect these systems so that the client one can
> shutdown without the server one bothering?
>
> Best regards
>
> --
> >>>>>>>>>> Read the docs: http://akka.io/docs/
> >>>>>>>>>> Check the FAQ: http://akka.io/faq/
> >>>>>>>>>> Search the archives: https://groups.google.com/group/akka-user
> ---
> You received this message because you are subscribed to the Google Groups
> "Akka User List" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to [email protected] <javascript:>.
> To post to this group, send email to [email protected]<javascript:>
> .
> Visit this group at http://groups.google.com/group/akka-user?hl=en.
> For more options, visit https://groups.google.com/groups/opt_out.
>
>
>
>
>
>
> *Dr. Roland Kuhn*
> *Akka Tech Lead*
> Typesafe <http://typesafe.com/> – Empowering professional developers to
> build amazing apps.
> twitter: @rolandkuhn <http://twitter.com/#!/rolandkuhn>
>
>
--
>>>>>>>>>> Read the docs: http://akka.io/docs/
>>>>>>>>>> Check the FAQ:
>>>>>>>>>> http://doc.akka.io/docs/akka/current/additional/faq.html
>>>>>>>>>> Search the archives: https://groups.google.com/group/akka-user
---
You received this message because you are subscribed to the Google Groups "Akka
User List" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To post to this group, send email to [email protected].
Visit this group at http://groups.google.com/group/akka-user.
For more options, visit https://groups.google.com/d/optout.
