[
https://issues.apache.org/jira/browse/STORM-2016?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15412780#comment-15412780
]
ASF GitHub Bot commented on STORM-2016:
---------------------------------------
Github user HeartSaVioR commented on a diff in the pull request:
https://github.com/apache/storm/pull/1608#discussion_r73984302
--- Diff: storm-core/src/clj/org/apache/storm/daemon/supervisor.clj ---
@@ -893,29 +891,61 @@
(let [rsrc-file-path (File. (.getFilePath local-rsrc))
key-name (.getName rsrc-file-path)
blob-symlink-target-name (.getName (File.
(.getCurrentSymlinkPath local-rsrc)))
- symlink-name (get-blob-localname (get blobstore-map
key-name) key-name)]
+ symlink-name (fn-symlink-name blobs key-name)]
(create-symlink! tmproot (.getParent rsrc-file-path)
symlink-name
- blob-symlink-target-name))))
+ blob-symlink-target-name))))
(catch AuthorizationException authExp
(log-error authExp))
(catch KeyNotFoundException knf
(log-error knf))))))
+(defn download-blobs-in-blobstore-map-for-topology!
+ "Download all blobs listed in the topology configuration for a given
topology."
+ [conf stormconf-path localizer tmproot]
+ (let [storm-conf (read-supervisor-storm-conf-given-path conf
stormconf-path)
+ blobstore-map (storm-conf TOPOLOGY-BLOBSTORE-MAP)]
+ (download-blobs-for-topology! conf storm-conf localizer tmproot
blobstore-map
+ (fn [blobs]
(blobstore-map-to-localresources blobs))
+ (fn [blobs key-name] (get-blob-localname
(get blobs key-name) key-name)))))
+
+(defn download-dependencies-for-topology!
+ "Download all dependencies blobs listed in the topology configuration
for a given topology."
+ [conf stormconf-path stormcode-path localizer tmproot]
+ (let [storm-conf (read-supervisor-storm-conf-given-path conf
stormconf-path)
+ storm-code (read-supervisor-storm-code-given-path stormcode-path)
+ dependencies (concat (.get_dependency_jars ^StormTopology
storm-code)
+ (.get_dependency_artifacts ^StormTopology
storm-code))]
+ (download-blobs-for-topology! conf storm-conf localizer tmproot
dependencies
+ (fn [blobs] (map #(LocalResource. %
false) blobs))
+ (fn [_ key-name] key-name))))
+
(defn get-blob-file-names
[blobstore-map]
(if blobstore-map
(for [[k, data] blobstore-map]
(get-blob-localname data k))))
(defn download-blobs-for-topology-succeed?
+ [target-dir file-names]
+ (if-not (empty? file-names)
+ (every? #(Utils/checkFileExists target-dir %) file-names)
+ true))
+
+(defn download-blobs-in-blobstore-map-for-topology-succeed?
--- End diff --
I don't mind if it's private or not, since we already ported this to Java
in master branch.
There're other functions which could be private.
> Topology submission improvement: support adding local jars and maven
> artifacts on submission
> --------------------------------------------------------------------------------------------
>
> Key: STORM-2016
> URL: https://issues.apache.org/jira/browse/STORM-2016
> Project: Apache Storm
> Issue Type: Improvement
> Components: storm-core
> Reporter: Jungtaek Lim
> Assignee: Jungtaek Lim
>
> This JIRA tracks actual work on below proposal / design document.
> https://cwiki.apache.org/confluence/display/STORM/A.+Design+doc%3A+adding+jars+and+maven+artifacts+at+submission
> Proposal discussion thread is here:
> http://mail-archives.apache.org/mod_mbox/storm-dev/201608.mbox/%3ccaf5108i9+tjanz0lgrktmkvqel7f+53k9uyzxct6zhsu6oh...@mail.gmail.com%3E
> Let's post on discussion thread if we have any opinions / ideas on this
> instead of leaving comments on this issue.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
