I think there may be an issue with topology cleanup I ran WordCountTopology for a bit, then killed it. The supervisor seems to keep trying to download the topology jar indefinitely.
2019-08-27 19:49:53.459 o.a.s.l.AsyncLocalizer AsyncLocalizer Executor - 1 [WARN] Failed to download blob LOCAL TOPO BLOB TOPO_JAR word-count-1-1566926844 will try again in 100 ms org.apache.storm.generated.KeyNotFoundException: null at org.apache.storm.generated.Nimbus$getBlobMeta_result$getBlobMeta_resultStandardScheme.read(Nimbus.java:25919) ~[storm-client-2.1.0.jar:2.1.0] at org.apache.storm.generated.Nimbus$getBlobMeta_result$getBlobMeta_resultStandardScheme.read(Nimbus.java:25887) ~[storm-client-2.1.0.jar:2.1.0] at org.apache.storm.generated.Nimbus$getBlobMeta_result.read(Nimbus.java:25818) ~[storm-client-2.1.0.jar:2.1.0] at org.apache.storm.thrift.TServiceClient.receiveBase(TServiceClient.java:88) ~[storm-shaded-deps-2.1.0.jar:2.1.0] at org.apache.storm.generated.Nimbus$Client.recv_getBlobMeta(Nimbus.java:794) ~[storm-client-2.1.0.jar:2.1.0] at org.apache.storm.generated.Nimbus$Client.getBlobMeta(Nimbus.java:781) ~[storm-client-2.1.0.jar:2.1.0] at org.apache.storm.blobstore.NimbusBlobStore.getBlobMeta(NimbusBlobStore.java:85) ~[storm-client-2.1.0.jar:2.1.0] at org.apache.storm.localizer.LocallyCachedTopologyBlob.getRemoteVersion(LocallyCachedTopologyBlob.java:127) ~[storm-server-2.1.0.jar:2.1.0] at org.apache.storm.localizer.AsyncLocalizer.lambda$downloadOrUpdate$10(AsyncLocalizer.java:264) ~[storm-server-2.1.0.jar:2.1.0] at java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1626) [?:1.8.0_144] at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) [?:1.8.0_144] at java.util.concurrent.FutureTask.run(FutureTask.java:266) [?:1.8.0_144] at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180) [?:1.8.0_144] at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) [?:1.8.0_144] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_144] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_144] at java.lang.Thread.run(Thread.java:748) [?:1.8.0_144] These repeat, intermittently broken up by Caused by: java.lang.RuntimeException: Could not download... at org.apache.storm.localizer.AsyncLocalizer.lambda$downloadOrUpdate$10(AsyncLocalizer.java:285) ~[storm-server-2.1.0.jar:2.1.0] at java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1626) ~[?:1.8.0_144] at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) ~[?:1.8.0_144] at java.util.concurrent.FutureTask.run(FutureTask.java:266) ~[?:1.8.0_144] at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180) ~[?:1.8.0_144] at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) ~[?:1.8.0_144] ... 3 more Caused by: org.apache.storm.generated.KeyNotFoundException at org.apache.storm.generated.Nimbus$getBlobMeta_result$getBlobMeta_resultStandardScheme.read(Nimbus.java:25919) ~[storm-client-2.1.0.jar:2.1.0] at org.apache.storm.generated.Nimbus$getBlobMeta_result$getBlobMeta_resultStandardScheme.read(Nimbus.java:25887) ~[storm-client-2.1.0.jar:2.1.0] at org.apache.storm.generated.Nimbus$getBlobMeta_result.read(Nimbus.java:25818) ~[storm-client-2.1.0.jar:2.1.0] at org.apache.storm.thrift.TServiceClient.receiveBase(TServiceClient.java:88) ~[storm-shaded-deps-2.1.0.jar:2.1.0] at org.apache.storm.generated.Nimbus$Client.recv_getBlobMeta(Nimbus.java:794) ~[storm-client-2.1.0.jar:2.1.0] at org.apache.storm.generated.Nimbus$Client.getBlobMeta(Nimbus.java:781) ~[storm-client-2.1.0.jar:2.1.0] at org.apache.storm.blobstore.NimbusBlobStore.getBlobMeta(NimbusBlobStore.java:85) ~[storm-client-2.1.0.jar:2.1.0] at org.apache.storm.localizer.LocallyCachedTopologyBlob.getRemoteVersion(LocallyCachedTopologyBlob.java:127) ~[storm-server-2.1.0.jar:2.1.0] at org.apache.storm.localizer.AsyncLocalizer.lambda$downloadOrUpdate$10(AsyncLocalizer.java:264) ~[storm-server-2.1.0.jar:2.1.0] at java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1626) ~[?:1.8.0_144] at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) ~[?:1.8.0_144] at java.util.concurrent.FutureTask.run(FutureTask.java:266) ~[?:1.8.0_144] at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180) ~[?:1.8.0_144] at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) ~[?:1.8.0_144] ... 3 more Restarting the supervisor seems to fix it. Here's what I did before hitting this issue Verified asc and sha512 files. Built Storm from the zipped source. Extracted zipped binary and started dev-zookeeper, nimbus, supervisor, logviewer, ui. Ran WordCountTopology for a while. Checked rebalance, deactivate/activate, worker log viewing, kill from the UI. Minor things we might want to handle: It might be good to put your key up on a keyserver e.g. http://pgp.mit.edu/, as that is where the https://www.apache.org/info/verification.html page suggests users go to find keys. We should add a note to the Storm downloads page that links to the verification page. Currently users aren't informed how to check signatures and checksums. Den tir. 20. aug. 2019 kl. 23.05 skrev Ethan Li <etha...@apache.org>: > This is a call to vote on releasing Apache Storm 2.1.0 (rc3) > > Full list of changes in this release: > > > https://dist.apache.org/repos/dist/dev/storm/apache-storm-2.1.0-rc3/RELEASE_NOTES.html > > The tag/commit to be voted upon is v2.1.0: > > > https://gitbox.apache.org/repos/asf?p=storm.git;a=tree;h=4288646ef99b97f9737dd9c21b7b6b62c8b17afa;hb=6562d6ee438c2858d92a51bd30ca0a1f2f623c77 > > The source archive being voted upon can be found here: > > > https://dist.apache.org/repos/dist/dev/storm/apache-storm-2.1.0-rc3/apache-storm-2.1.0-src.tar.gz > > Other release files, signatures and digests can be found here: > > https://dist.apache.org/repos/dist/dev/storm/apache-storm-2.1.0-rc3 > > The release artifacts are signed with the following key: > > https://www.apache.org/dist/storm/KEYS > > The Nexus staging repository for this release is: > > https://repository.apache.org/content/repositories/orgapachestorm-1087/ > > Please vote on releasing this package as Apache Storm 2.1.0. > > When voting, please list the actions taken to verify the release. > > This vote will be open for at least 72 hours. > > [ ] +1 Release this package as Apache Storm 2.1.0 > [ ] 0 No opinion > [ ] -1 Do not release this package because... > > Thanks to everyone who contributed to this release. > > Ethan >
