GitHub user HeartSaVioR opened a pull request:
https://github.com/apache/spark/pull/21506
[SPARK-24485][SS] Measure and log elapsed time for filesystem operations in
HDFSBackedStateStoreProvider
## What changes were proposed in this pull request?
This patch measures and logs elapsed time for each operation which
communicate with file system (mostly remote HDFS in production) in
HDFSBackedStateStoreProvider to help investigating any latency issue.
## How was this patch tested?
Manually tested.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/HeartSaVioR/spark SPARK-24485
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/21506.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #21506
----
commit d84f98fc978262f4165f78b3b223b8bb3151f735
Author: Jungtaek Lim <kabhwan@...>
Date: 2018-06-07T14:14:46Z
[SPARK-24485][SS] Measure and log elapsed time for filesystem operations in
HDFSBackedStateStoreProvider
----
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
