Hi,
I'm triggering a Spark job from a service. I need to communicate the
job's computation result back to the service. The service has an actor
system running, anyway, so the idea was to use (remote) actors for this
channel, too. There's no other use for actors on the Spark side, so
basically I have this in the Spark runner's main:
<snip>
val system = ActorSystem("spark-launcher")
// ...
val selection = system.actorSelection(actorUrl)
val actF = selection.resolveOne(3 seconds)
val resF = actF.map(_ ! result).andThen {
case _ => system.terminate()
}
Await.ready(resF, 3 seconds)
</snip>
So far this seemed to work. However, in a test case that duplicates this
exact behavior in a "mock Spark cluster", I have found that the message
sometimes, non-deterministically, does not arrive. Logging/debugging
seems to indicate that the message is actually sent from the
"spark-launcher" system's deadLetters, it just doesn't make it to the
other side. My best guess is that there's a race condition between send
and terminate, terminate "winning" in the failure cases.
I'm aware that Akka does not guarantee message delivery. I just naively
hoped that upon system termination, it would do a best effort to send
out pending messages from its queues.
Is my analysis correct/is this expected behavior? Is there any way to
detect that a message has "really" been sent in order to trigger a
system shutdown afterwards? Does this approach make any sense at all? Is
there a better way?
Thanks and best regards,
Patrick
--
>>>>>>>>>> Read the docs: http://akka.io/docs/
>>>>>>>>>> Check the FAQ:
>>>>>>>>>> http://doc.akka.io/docs/akka/current/additional/faq.html
>>>>>>>>>> Search the archives: https://groups.google.com/group/akka-user
---
You received this message because you are subscribed to the Google Groups "Akka
User List" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To post to this group, send email to [email protected].
Visit this group at https://groups.google.com/group/akka-user.
For more options, visit https://groups.google.com/d/optout.
