Shixiong Zhu created SPARK-4873:
-----------------------------------
Summary: WriteAheadLogBasedBlockHandler improvement
Key: SPARK-4873
URL: https://issues.apache.org/jira/browse/SPARK-4873
Project: Spark
Issue Type: Improvement
Components: Streaming
Reporter: Shixiong Zhu
Priority: Minor
1. Move the block store work to a separate thread pool. IO work usually runs
longer than CPU work. Running IO work in `Implicits.global` may block other
work that want to run in `Implicits.global`.
2. Use `Future.zip` instead of `Future.flatMap`(for-loop). `zip` implies these
two Futures will run concurrently, while `flatMap` usually means one Future
depends on the other one.
3. Fix a unit test in `ReceivedBlockTrackerSuite` which may fail occasionally.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
