rzo1 opened a new pull request, #9011: URL: https://github.com/apache/storm/pull/9011
Fixes #7653 / STORM-3871. `Nimbus.rmDependencyJarsInTopology` read only `StormTopology.get_dependency_jars()` and never `get_dependency_artifacts()`, so every blob uploaded through `--artifacts` was left behind when the topology was cleaned up. Of the five places in the tree that read those two fields, this was the only one looking at jars but not artifacts. The blobs are then unrecoverable: a `dep-` key carries no topology id, so once the owning `-stormcode.ser` is gone `topoIdsToClean` can never rediscover them. That matches the report, where blobs from days earlier are still present. Cleanup now collects candidates from both fields and deletes a key only when a cluster-wide scan of the other topologies dependency lists shows nothing else references it. That scan is what makes artifact reclamation safe: before c78ab7462 artifact keys were deterministic and deliberately shared between topologies (STORM-2016: "don t remove artifacts from blobstore since it s shared across topologies"), and a cluster upgraded across that commit still holds such blobs. If the scan itself fails, and one unreadable topology blob is enough since `Utils.deserialize` throws `RuntimeException`, the pass falls back to reclaiming only keys whose shape proves they belong to a single upload (the UUID that `applyUUIDToFileName` stamps), and logs what it kept and why. Without that fallback a single corrupt blob would orphan every dying topology dependencies permanently, which is worse than the behaviour being fixed. Tests cover the success path, a legacy key shared by another live topology, and the degraded path. storm-server: 494 tests, 0 failures. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
