Github user HeartSaVioR commented on the pull request:
https://github.com/apache/storm/pull/532#issuecomment-158761979
If there's no deadlock when _pendingWrites is bounded queue, we can just
set default value of TOPOLOGY_SHELLBOLT_MAX_PENDING to reasonable and deny set
it to null. In
Github user HeartSaVioR commented on the pull request:
https://github.com/apache/storm/pull/532#issuecomment-158759020
@revans2
I thought about deadlock again, and I couldn't see deadlock you stated.
Let's think about these scenario, subprocess emits tuple and waits for
t
Github user HeartSaVioR commented on the pull request:
https://github.com/apache/storm/pull/532#issuecomment-158578419
@revans2
Sorry to respond late.
I agree that we can set default value of TOPOLOGY_SHELLBOLT_MAX_PENDING to
reasonable value (not off) when deadlock bug you s
Github user HeartSaVioR closed the pull request at:
https://github.com/apache/storm/pull/532
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is
Github user revans2 commented on the pull request:
https://github.com/apache/storm/pull/532#issuecomment-157076377
@HeartSaVioR I would like to see ABP work correctly be default on the Shell
bolt, and have it work correctly when TOPOLOGY_SHELLBOLT_MAX_PENDING is off,
but I don't think
Github user HeartSaVioR commented on the pull request:
https://github.com/apache/storm/pull/532#issuecomment-156864048
@revans2
Oh, I have been forgotten TOPOLOGY_SHELLBOLT_MAX_PENDING when creating this
PR.
I agree that we would be fine with TOPOLOGY_SHELLBOLT_MAX_PENDING si
Github user revans2 commented on the pull request:
https://github.com/apache/storm/pull/532#issuecomment-156129284
I'm not sure I like the delay especially in seconds. I have been playing
around with performance and we could easily process tens of thousands of tuples
in a second. 1
Github user revans2 commented on the pull request:
https://github.com/apache/storm/pull/532#issuecomment-156117703
@HeartSaVioR I am fine either way you want to approach this. I would
prefer not to change the protocol in a non-backwards compatible way, and adding
a required sync feel
Github user HeartSaVioR commented on the pull request:
https://github.com/apache/storm/pull/532#issuecomment-156038940
@revans2
This PR improves subprocess to request ShellBolt for back pressure, but I
left decision when to request to users since I thought it is more flexible.
Github user revans2 commented on the pull request:
https://github.com/apache/storm/pull/532#issuecomment-155917971
@itaifrenkel any update on this? we now have automatic back-pressure like
heron so this would be a nice addition to it too.
---
If your project is set up for it, you ca
Github user HeartSaVioR commented on the pull request:
https://github.com/apache/storm/pull/532#issuecomment-110219954
@itaifrenkel
I see your point.
Spout can skip sending nextTuple to subprocess so that we can save
stdout/stdin traffic and Spout doesn't block or sleep and c
Github user HeartSaVioR commented on the pull request:
https://github.com/apache/storm/pull/532#issuecomment-110216616
@itaifrenkel Actually I read paper about Twitter Heron, and it can
propagate backpressure.
Storm currently doesn't have it, so ShellBolt itself should ensure
bac
Github user HeartSaVioR commented on the pull request:
https://github.com/apache/storm/pull/532#issuecomment-110216352
@itaifrenkel
I think Spout doesn't matter since subprocess has a chance to slow down its
flow.
But Bolt does matter cause ShellBolt sends all tuples to subpro
Github user itaifrenkel commented on the pull request:
https://github.com/apache/storm/pull/532#issuecomment-110206734
AFAIK the spout can just ignore the nextTuple requests. Are you trying to
save on stdout/stdin nextTuple traffic ?
---
If your project is set up for it, you can repl
GitHub user HeartSaVioR opened a pull request:
https://github.com/apache/storm/pull/532
[STORM-756] ShellBolt can delay sending tuples on demand from subprocess
Please refer https://issues.apache.org/jira/browse/STORM-756 to see more
details.
* introduce new multilang proto
Github user HeartSaVioR commented on the pull request:
https://github.com/apache/storm/pull/505#issuecomment-95366034
Hard to upmerge so I close this and make fresh PR.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If y
Github user HeartSaVioR closed the pull request at:
https://github.com/apache/storm/pull/505
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is
GitHub user HeartSaVioR opened a pull request:
https://github.com/apache/storm/pull/505
[STORM-756] ShellBolt can delay sending tuples on demand from subprocess
Please refer https://issues.apache.org/jira/browse/STORM-756 to see more
details.
* introduce new multilang proto
18 matches
Mail list logo
