Github user manuzhang commented on a diff in the pull request:
https://github.com/apache/storm/pull/1608#discussion_r73841599
--- Diff: storm-core/src/clj/org/apache/storm/daemon/supervisor.clj ---
@@ -893,29 +891,61 @@
(let [rsrc-file-path (File. (.getFilePath local-rsrc))
key-name (.getName rsrc-file-path)
blob-symlink-target-name (.getName (File.
(.getCurrentSymlinkPath local-rsrc)))
- symlink-name (get-blob-localname (get blobstore-map
key-name) key-name)]
+ symlink-name (fn-symlink-name blobs key-name)]
(create-symlink! tmproot (.getParent rsrc-file-path)
symlink-name
- blob-symlink-target-name))))
+ blob-symlink-target-name))))
(catch AuthorizationException authExp
(log-error authExp))
(catch KeyNotFoundException knf
(log-error knf))))))
+(defn download-blobs-in-blobstore-map-for-topology!
+ "Download all blobs listed in the topology configuration for a given
topology."
+ [conf stormconf-path localizer tmproot]
+ (let [storm-conf (read-supervisor-storm-conf-given-path conf
stormconf-path)
+ blobstore-map (storm-conf TOPOLOGY-BLOBSTORE-MAP)]
+ (download-blobs-for-topology! conf storm-conf localizer tmproot
blobstore-map
+ (fn [blobs]
(blobstore-map-to-localresources blobs))
+ (fn [blobs key-name] (get-blob-localname
(get blobs key-name) key-name)))))
+
+(defn download-dependencies-for-topology!
+ "Download all dependencies blobs listed in the topology configuration
for a given topology."
+ [conf stormconf-path stormcode-path localizer tmproot]
+ (let [storm-conf (read-supervisor-storm-conf-given-path conf
stormconf-path)
+ storm-code (read-supervisor-storm-code-given-path stormcode-path)
+ dependencies (concat (.get_dependency_jars ^StormTopology
storm-code)
+ (.get_dependency_artifacts ^StormTopology
storm-code))]
+ (download-blobs-for-topology! conf storm-conf localizer tmproot
dependencies
+ (fn [blobs] (map #(LocalResource. %
false) blobs))
+ (fn [_ key-name] key-name))))
+
(defn get-blob-file-names
[blobstore-map]
(if blobstore-map
(for [[k, data] blobstore-map]
(get-blob-localname data k))))
(defn download-blobs-for-topology-succeed?
+ [target-dir file-names]
+ (if-not (empty? file-names)
+ (every? #(Utils/checkFileExists target-dir %) file-names)
+ true))
+
+(defn download-blobs-in-blobstore-map-for-topology-succeed?
--- End diff --
It seems these methods could be private ?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
