Hello all, After upgrading the Storm platform our topology is running approximately 100% slower on the same machine and with the same memory and threading settings, i.e. taking twice as long on the same testset.
The topology is processing documents of varying lengths. The documents are split into sentences. Further processing is done by bolts that operate on either ‘document-level’ or ‘sentence-level’. Bolts that process sentences are set to higher parallelism. In Storm version 1.2.0 we found optimal performance when running on 2 workers on a single server, with document based bolts having their parallelism set to 2 and the sentence bolts having parallelism set to 8. Worker-xmx is set to 2048mb. This configuration runs twice as slow on Storm 2.2.0. When running the topology on 1 worker and with all parallelism set to 1 the speed returns to nearly that of 1.2.0. Further performance tuning has also been attempted but to no avail. This is not the behaviour that we expected of the new platform. Can anyone shed some light on this situation or perhaps let us know if our expectations were wrong? Thanks in advance, Trude ---------------------------------- Trude Gentenaar Research&Development ---------------------------------- SemLab Zuidpoolsingel 14-A 2408 ZE Alphen a/d Rijn The Netherlands T: +31 172 494 777 E: [email protected] W: http://www.semlab.nl
