Aaron Gresch created STORM-3724:
-----------------------------------
Summary: Use blobstore dir modtime to avoid update lookups by
HDFSBlobstore
Key: STORM-3724
URL: https://issues.apache.org/jira/browse/STORM-3724
Project: Apache Storm
Issue Type: Improvement
Reporter: Aaron Gresch
Assignee: Aaron Gresch
We have multiple storm clusters with 100's of supervisors polling for blob
updates. This causes high load on our Hadoop namenodes that are also used by
multiple other clusters.
An improvement would be for the AsyncLocalizer to check the remote blobstore
last mod time once and then skip checking each individual blob if it was
already checked for the same mod time.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
