Github user zsxwing commented on a diff in the pull request:
https://github.com/apache/spark/pull/11176#discussion_r54036415
--- Diff:
external/akka/src/main/scala/org/apache/spark/streaming/akka/ActorReceiver.scala
---
@@ -162,6 +177,18 @@ abstract class JavaActorReceiver extends UntypedActor {
def store[T](item: T) {
context.parent ! SingleItemData(item)
}
+
+ /**
+ * Store a single item of received data to Spark's memory synchronously.
+ * These single items will be aggregated together into data blocks before
+ * being pushed into Spark's memory.
+ *
+ * As opposed to [[ActorReceiver.store[T]: Unit]], this method allows
flow control
+ * (maxRate, backpressure) to block the input.
+ */
+ def storeSync[T](item: T)(timeout: Timeout) {
+ Await.ready(context.parent.ask(SingleItemDataSync(item))(timeout),
timeout.duration)
--- End diff --
Blocking an Actor may have some potential issue.
Instead of calling `Await.ready`, why not create a similar method like
`ask` and return `Future`? E.g.,
```
def store[T](item: T, timeout: Timeout): Future[Unit] = {
context.parent.ask(AskSingleItem)(timeout).map(_ => {})
}
```
Then the user can call `Await.ready` if they feel it's safe, or use another
`ExecutionContext` to avoid blocking the Actor
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
