Adam Lewis created STORM-241:
--------------------------------
Summary: "storm jar" underperforms compared to scp
Key: STORM-241
URL: https://issues.apache.org/jira/browse/STORM-241
Project: Apache Storm (Incubating)
Issue Type: Improvement
Environment: Internet connected dev machine deploying to storm cluster
on AWS. Storm 0.9.0.1
Reporter: Adam Lewis
"storm jar" is very slow at sending the jar file, achieving roughly 10% of the
throughput compared to SCP for the same network configuration. This is
especially problematic when deploying jars from over the Internet.
This appears to be related to the chunk size used to transmit the JAR file over
the thrift protocol. Anecdotally, a 300KB chunk size (compared to the
currently used default of 15KB), achieved performance in line with scp.
Default can be overridden in the BufferFileInputStream ctor:
https://github.com/apache/incubator-storm/blob/master/storm-core/src/jvm/backtype/storm/StormSubmitter.java?source=cc#L154
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
