rzo1 opened a new pull request, #9012: URL: https://github.com/apache/storm/pull/9012
`StormTopology.dependency_jars` and `dependency_artifacts` are filled in by the client and nothing on the submit path checks them. Not `validateTopologyName`, `ConfigValidation.validateTopoConf`, `DefaultTopologyValidator.validate`, `validateTopologyWorkerMaxHeapSizeConfigs`, `Utils.validateTopologyBlobStoreMap`, `validateTopologySize`, `StormCommon.validateStructure` or `normalizeTopology`. Nimbus then acts on whatever they contain: it deletes those keys during cleanup, and on gaining leadership it compares them against the blobstore. `LeaderListenerCallback` computes `activeTopologyDependencies - blobStore.listKeys()` and surrenders leadership when the difference is not empty. One dependency key that does not exist, on one active topology, makes every Nimbus win leadership, surrender and re-queue indefinitely, so the cluster does no scheduling, no cleanup and accepts no submissions. Submission now requires every entry to carry the `dep-` prefix and to exist in the blobstore. Existence is checked with `blobStore.getBlobMeta(key, subject)` as the submitting subject, the same way `Utils.validateTopologyBlobStoreMap` does twenty lines below. A key listed twice is looked up once. `StormSubmitter` uploads the blobs, then sets the fields, then submits, and `DependencyUploader` returns a key only after the blob is committed, so a legitimate submission always finds its blobs present. On a replicated blobstore `getBlobMeta` falls through to `checkForBlobOrDownload`, so the lookup also pulls a blob that is missing locally. `BLOB_DEPENDENCIES_PREFIX` becomes public and gains an `isDependencyBlobKey` predicate so the producer and the check use one definition. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
