[
https://issues.apache.org/jira/browse/STORM-3256?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16734943#comment-16734943
]
Stig Rohde Døssing edited comment on STORM-3256 at 1/5/19 4:39 PM:
-------------------------------------------------------------------
You're right. The restriction looks like it has been there at least since Netty
was introduced, and is still there in the current master. Maybe we can change
the message to use an int instead of a short to represent the task id.
If you want to work on this, please let me know and I'll add you to the
contributors list in JIRA. Then you should be able to assign the issue to
yourself.
The person working on this might want to try to account for rolling upgrades,
so the part of the code that reads task messages might need to be able to
handle both int+short and int+int headers.
The part of the code that handles reading the messages is in MessageDecoder.
was (Author: srdo):
You're right. The restriction looks like it has been there at least since Netty
was introduced, and is still there in the current master. Maybe we can change
the message to use an int instead of a short to represent the task id.
If you want to work on this, please let me know and I'll add you to the
contributors list in JIRA. Then you should be able to assign the issue to
yourself.
The person working on this might want to try to account for rolling upgrades,
so the part of the code that reads task messages might need to be able to
handle both int+short and int+int headers.
> If all thread counts exceed 32767, the system will generate errors
> ------------------------------------------------------------------
>
> Key: STORM-3256
> URL: https://issues.apache.org/jira/browse/STORM-3256
> Project: Apache Storm
> Issue Type: Improvement
> Components: storm-client
> Affects Versions: 1.1.0
> Environment: CentOs 7.1
> Reporter: bb_wolf
> Priority: Blocker
> Labels: patch
>
> If all thread counts exceed 32767, the system will generate errors,
> MessageBatch.java Line141.
>
> Storm systems are used for low latency and a linear increase in the number of
> concurrencies as servers increase.
> If you set up four workers per server, it's easy to achieve 4000 degrees of
> parallelism, that is, 4000 threads per server, then 10 servers will exceed
> 32767.
> Supporting only 32767 threads is a disaster for large-scale computing.
> It is hoped that this function can be improved and repaired as soon as
> possible.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
