[
https://issues.apache.org/jira/browse/SPARK-4873?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Shixiong Zhu updated SPARK-4873:
--------------------------------
Description: Use Future.zip instead of Future.flatMap(for-loop). zip
implies these two Futures will run concurrently, while flatMap usually means
one Future depends on the other one. (was: 1. Move the block store work to a
separate thread pool. IO work usually runs longer than CPU work. Running IO
work in `Implicits.global` may block other work that want to run in
`Implicits.global`.
2. Use `Future.zip` instead of `Future.flatMap`(for-loop). `zip` implies these
two Futures will run concurrently, while `flatMap` usually means one Future
depends on the other one.
3. Fix a unit test in `ReceivedBlockTrackerSuite` which may fail occasionally.)
Summary: Use `Future.zip` instead of `Future.flatMap`(for-loop) in
WriteAheadLogBasedBlockHandler (was: WriteAheadLogBasedBlockHandler
improvement)
> Use `Future.zip` instead of `Future.flatMap`(for-loop) in
> WriteAheadLogBasedBlockHandler
> ----------------------------------------------------------------------------------------
>
> Key: SPARK-4873
> URL: https://issues.apache.org/jira/browse/SPARK-4873
> Project: Spark
> Issue Type: Improvement
> Components: Streaming
> Reporter: Shixiong Zhu
> Priority: Minor
>
> Use Future.zip instead of Future.flatMap(for-loop). zip implies these two
> Futures will run concurrently, while flatMap usually means one Future depends
> on the other one.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
