[
https://issues.apache.org/jira/browse/STORM-876?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15038675#comment-15038675
]
ASF GitHub Bot commented on STORM-876:
--------------------------------------
Github user revans2 commented on the pull request:
https://github.com/apache/storm/pull/845#issuecomment-161798173
@ptgoetz Thanks for the review. I agree with the follow on items. Because
this is such a big change and @redsanket has been upmerging this regularly I
would like to see this merged in sooner rather then later. I'll try and merge
this in tomorrow assuming no one else has any issues with it between now and
then. The other work can be done in follow on JIRA and I will update the 1.0
wiki to include the new JIRA once they are filed.
Thank you everyone for your help with getting this in.
> Dist Cache: Basic Functionality
> -------------------------------
>
> Key: STORM-876
> URL: https://issues.apache.org/jira/browse/STORM-876
> Project: Apache Storm
> Issue Type: Improvement
> Components: storm-core
> Reporter: Robert Joseph Evans
> Assignee: Robert Joseph Evans
> Attachments: DISTCACHE.md, DistributedCacheDesignDocument.pdf
>
>
> Basic functionality for the Dist Cache feature.
> As part of this a new API should be added to support uploading and
> downloading dist cache items. storm-core.ser, storm-conf.ser and storm.jar
> should be written into the blob store instead of residing locally. We need a
> default implementation of the blob store that does essentially what nimbus
> currently does and does not need anything extra. But having an HDFS backend
> too would be great for scalability and HA.
> The supervisor should provide a way to download and manage these blobs and
> provide a working directory for the worker process with symlinks to the
> blobs. It should also allow the blobs to be updated and switch the symlink
> atomically to point to the new blob once it is downloaded.
> All of this is already done by code internal to Yahoo! we are in the process
> of getting it ready to push back to open source shortly.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
