mdedetrich commented on code in PR #226:
URL:
https://github.com/apache/incubator-pekko-connectors/pull/226#discussion_r1300169039
##########
kinesis/src/main/scala/org/apache/pekko/stream/connectors/kinesis/impl/KinesisSchedulerSourceStage.scala:
##########
@@ -81,8 +81,14 @@ private[kinesis] class KinesisSchedulerSourceStage(
override def shardRecordProcessor(): ShardRecordProcessor =
new ShardProcessor(newRecordCallback)
})
+ //Run the scheduler loop in a separate thread
+ val thread = new Thread(() => {
+ val result = Try {scheduler.run()}
+ callback.invoke(SchedulerShutdown(result))
+ }, s"KinesisSchedulerSource")
+ thread.setDaemon(true)
+ thread.start()
schedulerOpt = Some(scheduler)
- Future(scheduler.run()).onComplete(result =>
callback.invoke(SchedulerShutdown(result)))
Review Comment:
> That said, I don't think running this "loop" in the default dispatcher
even with "blocking" is a good idea, as it would block a thread for a long
time...
What "loop" are you talking about? And by blocking a thread is the core
point you are making is that it should be asynchronous rather than waiting for
the `scheduler.run` operation to complete?
I am under the impression that the entire point of `Future` is it makes the
enclosing operation asynchronous and what `blocking` does is it makes that
asynchronous operation run on a pool that is designed to handle blocking
operations (i.e. usually a CachedThreadPool of some kind however the way that
`blocking` works is its up to the `ExecutionContext` that is pulled in to
determine how to best handle blocking operations).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
