[jira] [Resolved] (STORM-3714) Add rate information for TaskMetrics
[ https://issues.apache.org/jira/browse/STORM-3714?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aaron Gresch resolved STORM-3714. - Fix Version/s: 2.3.0 Resolution: Fixed > Add rate information for TaskMetrics > > > Key: STORM-3714 > URL: https://issues.apache.org/jira/browse/STORM-3714 > Project: Apache Storm > Issue Type: Improvement >Reporter: Aaron Gresch >Assignee: Aaron Gresch >Priority: Minor > Fix For: 2.3.0 > > Time Spent: 3h > Remaining Estimate: 0h > > While converting TaskMetrics to use V2 API, we used Counters over Meters due > to performance implications. We have found we would like to add rate > information as well. > > Ideally we would add some kind of metric that supports a count and rate > without the full performance overhead of the Meter. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (STORM-3727) SUPERVISOR_SLOTS_PORTS could be list of Longs
[
https://issues.apache.org/jira/browse/STORM-3727?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ethan Li resolved STORM-3727.
-
Fix Version/s: 2.3.0
Resolution: Fixed
Thanks [~agresch]. I merged this to master
(ba7f969a3a046c5a312a394026d5d9dd755e04a0)
> SUPERVISOR_SLOTS_PORTS could be list of Longs
> -
>
> Key: STORM-3727
> URL: https://issues.apache.org/jira/browse/STORM-3727
> Project: Apache Storm
> Issue Type: Bug
>Affects Versions: 2.2.0
>Reporter: Aaron Gresch
>Assignee: Aaron Gresch
>Priority: Minor
> Fix For: 2.3.0
>
> Time Spent: 1h 40m
> Remaining Estimate: 0h
>
>
> A user reported:
> There's no guarantee that the {{supervisorConf.getOrDefault}} will be a List
> of Integers.
> Additionally, in ReadClusterState.java, {{.intValue()}} conversion is
> removed. Overall result
>
> {{java.lang.ClassCastException: java.lang.Long cannot be cast to
> java.lang.Integer
> at
> org.apache.storm.daemon.supervisor.ReadClusterState.(ReadClusterState.java:101)
> ~[storm-server-2.2.0.jar:2.2.0]
> at
> org.apache.storm.daemon.supervisor.Supervisor.launch(Supervisor.java:310)
> ~[storm-server-2.2.0.jar:2.2.0]}}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (STORM-3734) IntegerValidator doesn't force the object type to be Integer
[
https://issues.apache.org/jira/browse/STORM-3734?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ethan Li updated STORM-3734:
Description:
The IntegerValidator allows the non-integer object, like Double(1.0).
https://github.com/apache/storm/blob/7bef73a6faa14558ef254efe74cbe4bfef81c2e2/storm-client/src/jvm/org/apache/storm/validation/ConfigValidation.java#L404-L415
It can be reproduced by
{code:java}
IntegerValidator validator = new IntegerValidator();
validator.validateInteger("test", 1.0);
{code}
More details at https://github.com/apache/storm/pull/3365#issuecomment-754775896
was:
The IntegerValidator allows the non-integer object, like Double(1.0).
https://github.com/apache/storm/blob/7bef73a6faa14558ef254efe74cbe4bfef81c2e2/storm-client/src/jvm/org/apache/storm/validation/ConfigValidation.java#L404-L415
It can be reproduced by
{code:java}
IntegerValidator validator = new IntegerValidator();
validator.validateInteger("test", 1.0);
{code}
> IntegerValidator doesn't force the object type to be Integer
>
>
> Key: STORM-3734
> URL: https://issues.apache.org/jira/browse/STORM-3734
> Project: Apache Storm
> Issue Type: Bug
>Reporter: Ethan Li
>Priority: Major
>
> The IntegerValidator allows the non-integer object, like Double(1.0).
> https://github.com/apache/storm/blob/7bef73a6faa14558ef254efe74cbe4bfef81c2e2/storm-client/src/jvm/org/apache/storm/validation/ConfigValidation.java#L404-L415
> It can be reproduced by
> {code:java}
> IntegerValidator validator = new IntegerValidator();
> validator.validateInteger("test", 1.0);
> {code}
> More details at
> https://github.com/apache/storm/pull/3365#issuecomment-754775896
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (STORM-3734) IntegerValidator doesn't force the object type to be Integer
Ethan Li created STORM-3734:
---
Summary: IntegerValidator doesn't force the object type to be
Integer
Key: STORM-3734
URL: https://issues.apache.org/jira/browse/STORM-3734
Project: Apache Storm
Issue Type: Bug
Reporter: Ethan Li
The IntegerValidator allows the non-integer object, like Double(1.0).
https://github.com/apache/storm/blob/7bef73a6faa14558ef254efe74cbe4bfef81c2e2/storm-client/src/jvm/org/apache/storm/validation/ConfigValidation.java#L404-L415
It can be reproduced by
{code:java}
IntegerValidator validator = new IntegerValidator();
validator.validateInteger("test", 1.0);
{code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (STORM-3733) AsyncLocalizer stuck looking for missing topology
Aaron Gresch created STORM-3733:
---
Summary: AsyncLocalizer stuck looking for missing topology
Key: STORM-3733
URL: https://issues.apache.org/jira/browse/STORM-3733
Project: Apache Storm
Issue Type: Bug
Reporter: Aaron Gresch
Assignee: Aaron Gresch
{code:java}
2020-11-09 20:18:12.325 o.a.s.l.AsyncLocalizer AsyncLocalizer Task Executor - 2
[ERROR] Could not update blob, will retry again later
2020-11-09 20:18:43.744 o.a.s.l.AsyncLocalizer AsyncLocalizer Task Executor - 2
[ERROR] Could not update blob, will retry again later
2020-11-09 20:19:14.726 o.a.s.l.AsyncLocalizer AsyncLocalizer Task Executor - 2
[ERROR] Could not update blob, will retry again later
2020-11-09 20:19:46.148 o.a.s.l.AsyncLocalizer AsyncLocalizer Task Executor - 2
[ERROR] Could not update blob, will retry again later
2020-11-09 20:20:16.560 o.a.s.l.AsyncLocalizer AsyncLocalizer Task Executor - 0
[ERROR] Could not update blob, will retry again later
2020-11-09 20:20:47.990 o.a.s.l.AsyncLocalizer AsyncLocalizer Task Executor - 0
[ERROR] Could not update blob, will retry again later
2020-11-09 20:21:19.403 o.a.s.l.AsyncLocalizer AsyncLocalizer Task Executor - 0
[ERROR] Could not update blob, will retry again later
2020-11-09 20:21:50.818 o.a.s.l.AsyncLocalizer AsyncLocalizer Task Executor - 0
[ERROR] Could not update blob, will retry again later
2020-11-09 20:22:21.257 o.a.s.l.AsyncLocalizer AsyncLocalizer Task Executor - 1
[ERROR] Could not update blob, will retry again later
2020-11-09 20:22:52.668 o.a.s.l.AsyncLocalizer AsyncLocalizer Task Executor - 1
[ERROR] Could not update blob, will retry again later
2020-11-09 20:23:24.082 o.a.s.l.AsyncLocalizer AsyncLocalizer Task Executor - 1
[ERROR] Could not update blob, will retry again later
2020-11-09 20:23:55.512 o.a.s.l.AsyncLocalizer AsyncLocalizer Task Executor - 1
[ERROR] Could not update blob, will retry again later
2020-11-09 20:24:25.919 o.a.s.l.AsyncLocalizer AsyncLocalizer Task Executor - 2
[ERROR] Could not update blob, will retry again later
2020-11-09 20:24:57.343 o.a.s.l.AsyncLocalizer AsyncLocalizer Task Executor - 2
[ERROR] Could not update blob, will retry again later
2020-11-09 20:25:28.773 o.a.s.l.AsyncLocalizer AsyncLocalizer Task Executor - 2
[ERROR] Could not update blob, will retry again later
2020-11-09 20:16:09.659 o.a.s.l.AsyncLocalizer AsyncLocalizer Task Executor - 1
[ERROR] Could not update blob, will retry again later
java.util.concurrent.ExecutionException: java.lang.RuntimeException: Could not
download...
at
java.util.concurrent.CompletableFuture.reportGet(CompletableFuture.java:357)
~[?:1.8.0_262]
at
java.util.concurrent.CompletableFuture.get(CompletableFuture.java:1908)
~[?:1.8.0_262]
at
org.apache.storm.localizer.AsyncLocalizer.updateBlobs(AsyncLocalizer.java:333)
~[storm-server-2.3.0.y.jar:2.3.0.y]
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
[?:1.8.0_262]
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
[?:1.8.0_262]
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
[?:1.8.0_262]
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
[?:1.8.0_262]
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
[?:1.8.0_262]
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
[?:1.8.0_262]
at java.lang.Thread.run(Thread.java:748) [?:1.8.0_262]
Caused by: java.lang.RuntimeException: Could not download...
at
org.apache.storm.localizer.AsyncLocalizer.lambda$downloadOrUpdate$10(AsyncLocalizer.java:297)
~[storm-server-2.3.0.y.jar:2.3.0.y]
at
java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1640)
~[?:1.8.0_262]
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
~[?:1.8.0_262]
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
~[?:1.8.0_262]
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
~[?:1.8.0_262]
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
~[?:1.8.0_262]
... 3 more
Caused by: org.apache.storm.utils.WrappedKeyNotFoundException:
testTopology-743-1603821880-stormconf.ser
at
org.apache.storm.hdfs.blobstore.HdfsBlobStore.getStoredBlobMeta(HdfsBlobStore.java:192)
~[storm-hdfs-blobstore-2.3.0.y.jar:2.3.0.y]
at
org.apache.storm.hdfs.blobstore.HdfsBlobStore.getBlobMeta(HdfsBlobStore.java:221)
~[storm-hdfs-blobstore-2.3.0.y.jar:2.3.0.y]
at
[jira] [Created] (STORM-3732) Fix test test-cluster in storm-core: org.apache.storm.scheduler-test
Bipin Prasad created STORM-3732: --- Summary: Fix test test-cluster in storm-core: org.apache.storm.scheduler-test Key: STORM-3732 URL: https://issues.apache.org/jira/browse/STORM-3732 Project: Apache Storm Issue Type: Bug Components: storm-core Reporter: Bipin Prasad Assignee: Bipin Prasad Asserts in TopologyDetails.initConfigs fail the test "test-cluster" in storm-core: org.apache.storm.scheduler-test -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (STORM-3731) Remove unused nashorn import in storm-loadgen:OutputStream.java
[ https://issues.apache.org/jira/browse/STORM-3731?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ethan Li resolved STORM-3731. - Fix Version/s: 2.3.0 Resolution: Fixed Thanks [~bipinprasad]. I merged this to master (5360a15861db8edf5728b9f3eaca92c5470811b0) > Remove unused nashorn import in storm-loadgen:OutputStream.java > --- > > Key: STORM-3731 > URL: https://issues.apache.org/jira/browse/STORM-3731 > Project: Apache Storm > Issue Type: Bug > Components: storm-loadgen >Reporter: Bipin Prasad >Assignee: Bipin Prasad >Priority: Minor > Fix For: 2.3.0 > > Time Spent: 20m > Remaining Estimate: 0h > > Removing unused nashorn import. > This allows mvn build to succeed in JDK16 EA as well. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (STORM-3729) Assigning memory greater and equal than 2048m will make assgin memory for slot values to 1m
[ https://issues.apache.org/jira/browse/STORM-3729?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ethan Li resolved STORM-3729. - Fix Version/s: 2.3.0 Resolution: Fixed Thanks [~Zeahoo]. I merged this to master (f451be2ef81c0821f65a1a4d671b90c221ef99db) > Assigning memory greater and equal than 2048m will make assgin memory for > slot values to 1m > --- > > Key: STORM-3729 > URL: https://issues.apache.org/jira/browse/STORM-3729 > Project: Apache Storm > Issue Type: Bug > Components: storm-client >Affects Versions: 2.0.0, 2.1.0, 2.2.0 >Reporter: ZhihaoZheng >Assignee: ZhihaoZheng >Priority: Minor > Fix For: 2.3.0 > > Time Spent: 3h 40m > Remaining Estimate: 0h > > Hi, everyone. > I set my topology over 2048m, the storm ui shows only 65m, so i found its > error in > [https://github.com/apache/storm/blob/master/storm-client/src/jvm/org/apache/storm/utils/Utils.java] > line 1089 that value cast to int instead of long, It goes wrong if I pass > 2048m and results 1m. > Simply change this line to cast Long can solve this problem.:) -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Assigned] (STORM-3729) Assigning memory greater and equal than 2048m will make assgin memory for slot values to 1m
[ https://issues.apache.org/jira/browse/STORM-3729?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ethan Li reassigned STORM-3729: --- Assignee: ZhihaoZheng > Assigning memory greater and equal than 2048m will make assgin memory for > slot values to 1m > --- > > Key: STORM-3729 > URL: https://issues.apache.org/jira/browse/STORM-3729 > Project: Apache Storm > Issue Type: Bug > Components: storm-client >Affects Versions: 2.0.0, 2.1.0, 2.2.0 >Reporter: ZhihaoZheng >Assignee: ZhihaoZheng >Priority: Minor > Time Spent: 3h 40m > Remaining Estimate: 0h > > Hi, everyone. > I set my topology over 2048m, the storm ui shows only 65m, so i found its > error in > [https://github.com/apache/storm/blob/master/storm-client/src/jvm/org/apache/storm/utils/Utils.java] > line 1089 that value cast to int instead of long, It goes wrong if I pass > 2048m and results 1m. > Simply change this line to cast Long can solve this problem.:) -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (STORM-3729) Assigning memory greater and equal than 2048m will make assgin memory for slot values to 1m
[ https://issues.apache.org/jira/browse/STORM-3729?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ethan Li updated STORM-3729: Affects Version/s: 2.0.0 > Assigning memory greater and equal than 2048m will make assgin memory for > slot values to 1m > --- > > Key: STORM-3729 > URL: https://issues.apache.org/jira/browse/STORM-3729 > Project: Apache Storm > Issue Type: Bug > Components: storm-client >Affects Versions: 2.0.0, 2.1.0, 2.2.0 >Reporter: ZhihaoZheng >Priority: Minor > Time Spent: 3h 40m > Remaining Estimate: 0h > > Hi, everyone. > I set my topology over 2048m, the storm ui shows only 65m, so i found its > error in > [https://github.com/apache/storm/blob/master/storm-client/src/jvm/org/apache/storm/utils/Utils.java] > line 1089 that value cast to int instead of long, It goes wrong if I pass > 2048m and results 1m. > Simply change this line to cast Long can solve this problem.:) -- This message was sent by Atlassian Jira (v8.3.4#803005)
