Github user HeartSaVioR commented on the issue:

    https://github.com/apache/storm/pull/2502
  
    Here's my test result: TVL with rate 85000 and max spout pending 5000. 
spouts/splitters/counters are set to same as worker count.
    
    Again, this is just to see if there's clear performance regression from TVL 
which we have been using. Full analysis like [google doc in 
STORM-2306](https://docs.google.com/document/d/1A5k41UjVFY8jZg01BHc1fFxmI0AcFz5jRiKhNjhOj7I/edit?usp=sharing)
 would require so much efforts and resources (dedicated machines). If we could 
update the numbers with latest master vs latest patch of STORM-2306 it would be 
really great, but we should be OK if we postpone measuring numbers just before 
releasing Storm 2.0.0 comparing with latest Storm 1.x release.
    
    > 4 workers
    
    >> master (ab7b4ca)
    
    1.
    
    ```
    (s)  rate(tuple/s)       mean(ms)     99%ile(ms)   99.9%ile(ms)          
cores        mem(MB)         failed
      0     56,660.500      1,482.288      3,808.428      3,942.646          
6.554        329.044              0
     30     82,245.581         10.817         27.607         50.430          
5.093        364.626              0
     61     85,022.500         10.495         21.316         36.405          
5.192        242.701              0
     91     84,994.733         10.365         19.186         29.688          
5.152        399.110              0
    121     85,008.333         10.385         19.595         29.016          
5.195        269.351              0
    151     84,999.867         10.367         19.005         29.000          
5.156        361.585              0
    181     85,006.133         10.361         18.285         24.953          
5.215        319.568              0
    211     85,007.433         10.373         18.547         26.018          
5.227        374.250              0
    241     84,978.300         10.390         19.153         29.852          
5.107        240.743              0
    271     85,025.733         10.351         18.366         28.934          
5.134        308.930              0
    ```
    
    2.
    
    ```
    (s)  rate(tuple/s)       mean(ms)     99%ile(ms)   99.9%ile(ms)          
cores        mem(MB)         failed
      0     24,746.933      2,497.621      3,498.050      3,571.450          
4.175        480.104              0
     30     88,589.100        160.265      1,622.147      1,778.385          
5.883        329.437              0
     60     85,012.833         10.494         20.120         31.326          
5.211        421.895              0
     90     84,993.867         10.394         18.956         26.132          
5.185        434.169              0
    120     85,008.100         10.474         20.201         31.523          
5.194        440.354              0
    150     85,001.067         10.396         18.776         28.574          
5.238        449.158              0
    180     85,005.633         10.477         20.283         35.127          
5.289        441.979              0
    210     84,994.833         10.390         18.678         29.295          
5.136        419.486              0
    240     85,011.167         10.370         18.219         26.051          
5.219        308.977              0
    270     85,007.600         10.438         19.317         27.804          
5.158        403.509              0
    ```
    
    3.
    
    ```
    (s)  rate(tuple/s)       mean(ms)     99%ile(ms)   99.9%ile(ms)          
cores        mem(MB)         failed
      0     56,648.833      1,368.022      3,565.158      3,682.599          
6.476        405.396              0
     30     85,045.067         10.725         23.446         44.007          
5.249        448.851              0
     60     85,002.400         10.527         19.874         28.901          
5.209        452.087              0
     90     84,997.800         10.412         19.694         34.341          
5.136        356.300              0
    120     84,995.033         10.451         20.267         29.802          
5.174        379.923              0
    150     85,019.533         10.413         19.153         30.556          
5.193        398.516              0
    180     85,001.367         10.371         18.448         25.182          
5.191        331.490              0
    210     85,004.400         10.386         18.874         24.101          
5.195        368.889              0
    240     84,991.367         10.368         18.416         25.018          
5.132        392.701              0
    270     85,004.367         10.469         19.808         35.586          
5.147        443.393              0
    ```
    
    >> STORM-2306 (5b54809)
    
    1.
    
    ```
    (s)  rate(tuple/s)       mean(ms)     99%ile(ms)   99.9%ile(ms)          
cores        mem(MB)         failed
      0     28,098.000      1,549.653      3,183.477      3,227.517          
4.041        192.208              0
     30     85,306.133         13.650        270.795        406.585          
8.958        275.171              0
     60     85,007.067          5.466         17.449         28.836          
9.023        287.177              0
     90     92,084.333          5.483         17.269         26.280          
9.004        212.465              0
    120     84,999.400          5.561         17.662         29.458          
9.739        368.272              0
    150     85,005.367          5.593         17.449         37.126          
8.979        293.334              0
    180     92,082.733          5.584         18.219         28.787          
8.981        334.805              0
    210     85,008.000          5.571         18.022         29.213          
8.962        363.184              0
    240     84,992.867          5.604         17.662         39.191          
9.020        271.483              0
    270     92,094.467          5.590         17.351         24.953          
8.939        320.613              0
    ```
    
    2.
    
    ```
    (s)  rate(tuple/s)       mean(ms)     99%ile(ms)   99.9%ile(ms)          
cores        mem(MB)         failed
      0     56,779.467        728.445      2,967.470      3,066.036          
7.032        308.975              0
     30     84,986.933          5.851         21.037         33.489          
9.101        404.124              0
     60     85,021.833          5.593         18.301         29.049          
9.104        377.985              0
     90     85,000.500          5.577         17.465         26.853          
9.101        343.799              0
    120     85,009.300          5.714         18.661         29.196          
9.084        252.083              0
    150     84,998.533          5.532         16.957         26.460          
9.114        252.234              0
    180     85,005.300          5.588         16.810         26.165          
9.102        227.976              0
    210     84,997.633          5.708         19.055         32.096          
9.157        294.213              0
    240     84,994.433          5.692         18.481         30.261          
9.046        365.903              0
    270     85,015.500          5.590         18.022         28.197          
9.080        401.871              0
    ```
    
    3.
    
    ```
    (s)  rate(tuple/s)       mean(ms)     99%ile(ms)   99.9%ile(ms)          
cores        mem(MB)         failed
      0     34,802.167      1,451.699      3,227.517      3,275.751          
4.887        358.051              0
     30     99,803.667         26.173        740.295        880.804          
9.226        349.733              0
     60     92,079.433          5.790         20.398         34.800         
11.521        254.316              0
     90     85,002.100          5.590         17.220         27.361          
9.218        333.207              0
    120     84,998.033          5.750         18.432         28.066          
9.179        325.581              0
    150     84,993.500          5.745         18.792         29.966          
9.190        323.914              0
    180     85,011.500          5.828         17.269         25.444          
9.217        352.495              0
    210     85,005.200          5.761         18.514         32.899          
9.244        338.353              0
    240     84,983.900          5.756         20.546         33.374          
9.165        346.788              0
    270     85,018.133          5.516         17.138         32.915          
9.206        404.615              0
    ```
    
    > 1 worker
    
    >> master (ab7b4ca)
    
    1.
    
    ```
    (s)  rate(tuple/s)       mean(ms)     99%ile(ms)   99.9%ile(ms)          
cores        mem(MB)         failed
      0     56,667.633         96.401        944.767        972.030          
2.220        115.121              0
     30     85,012.833          6.684         21.283         27.427          
2.616         27.433              0
     60     85,003.867          6.715         22.315         28.967          
2.600         48.685              0
     90     85,009.633          7.003         24.855         30.261          
2.650         56.965              0
    120     85,008.167          6.765         22.331         35.750          
2.651         75.872              0
    150     84,997.767          6.788         22.430         31.998          
2.597         99.166              0
    180     85,004.633          6.658         21.905         26.755          
2.574        123.478              0
    210     85,011.633          6.661         21.070         26.460          
2.573         34.217              0
    240     84,961.500          6.658         21.955         29.606          
2.593         38.743              0
    270     85,062.533          6.715         22.790         28.492          
2.597         76.703              0
    ```
    
    2.
    
    ```
    (s)  rate(tuple/s)       mean(ms)     99%ile(ms)   99.9%ile(ms)          
cores        mem(MB)         failed
      0     56,666.067        100.076        956.301        968.884          
2.289         90.811              0
     30     84,985.100          6.886         24.461         30.654          
2.644         43.144              0
     60     85,033.667          6.738         21.725         30.786          
2.613         44.536              0
     90     85,006.833          6.827         24.183         28.361          
2.636        123.622              0
    120     85,003.100          6.767         21.807         27.967          
2.700        111.695              0
    150     85,007.800          6.564         21.365         33.194          
2.574        104.018              0
    180     85,004.067          7.089         24.527         29.688          
2.669         80.106              0
    210     85,009.400          6.673         21.348         27.853          
2.617         53.960              0
    240     84,999.367          7.159         24.216         29.131          
2.668        134.561              0
    270     85,064.100          7.168         24.773         28.393          
2.670        115.301              0
    ```
    
    3.
    
    ```
    (s)  rate(tuple/s)       mean(ms)     99%ile(ms)   99.9%ile(ms)          
cores        mem(MB)         failed
      0     56,651.067        104.108        983.040        999.293          
2.245         94.063              0
     30     82,287.774          7.086         23.364         43.221          
2.574        106.583              0
     61     85,002.000          6.953         23.396         29.213          
2.642        113.673              0
     91     85,013.500          6.913         22.839         29.393          
2.628        129.253              0
    121     85,008.667          6.775         22.348         31.982          
2.650        153.746              0
    151     85,003.467          6.576         19.956         27.017          
2.560         56.378              0
    181     85,015.300          6.837         22.692         31.408          
2.607         79.616              0
    211     85,004.100          6.879         25.346         31.097          
2.581         94.896              0
    241     85,005.867          6.891         23.609         29.278          
2.594        116.510              0
    271     85,010.800          6.760         21.987         28.541          
2.575        125.168              0
    ```
    
    >> STORM-2306 (5b54809)
    
    1.
    
    ```
    (s)  rate(tuple/s)       mean(ms)     99%ile(ms)   99.9%ile(ms)          
cores        mem(MB)         failed
      0     56,666.333         18.022        330.564        346.030          
2.079         46.405              0
     30     85,017.400          3.748         20.791         24.805          
2.686        107.791              0
     60     85,020.833          3.466         20.120         24.068          
2.679         46.481              0
     90     85,009.833          3.470         19.988         23.118          
2.639         63.486              0
    120     85,007.600          3.676         21.250         26.542          
2.708         76.022              0
    150     85,006.733          3.395         20.283         23.904          
2.636         80.490              0
    180     85,009.333          3.632         21.086         26.231          
2.657         78.360              0
    210     85,003.300          3.643         20.840         26.362          
2.670         87.437              0
    240     85,007.733          3.735         21.545         25.788          
2.686         98.075              0
    270     85,008.000          3.675         21.430         29.606          
2.647         93.264              0
    ```
    
    2.
    
    ```
    (s)  rate(tuple/s)       mean(ms)     99%ile(ms)   99.9%ile(ms)          
cores        mem(MB)         failed
      0     56,684.100         15.841        314.835        371.196          
2.051         52.854              0
     30     84,998.900          3.343         20.464         24.134          
2.624         95.248              0
     60     85,014.100          3.600         21.299         24.363          
2.665         43.992              0
     90     85,000.533          3.539         20.726         23.839          
2.650        100.354              0
    120     85,006.567          3.483         20.251         23.347          
2.665         40.798              0
    150     84,992.467          3.562         20.791         26.001          
2.666        102.680              0
    180     82,297.839          3.655         20.890         27.378          
2.590         51.539              0
    211     85,007.333          3.567         21.299         26.640          
2.650        107.193              0
    241     85,012.467          3.604         20.955         27.460          
2.646         50.419              0
    271     85,001.333          3.288         19.726         24.543          
2.604        103.526              0
    ```
    
    3.
    
    ```
    (s)  rate(tuple/s)       mean(ms)     99%ile(ms)   99.9%ile(ms)          
cores        mem(MB)         failed
      0     56,680.200         14.695        299.893        340.263          
2.096        100.239              0
     30     85,012.833          4.007         21.512         27.427          
2.719         67.345              0
     60     85,003.900          3.921         22.233         31.228          
2.709         43.670              0
     90     85,009.467          3.881         22.249         26.722          
2.683         23.378              0
    120     85,027.600          3.955         21.840         25.772          
2.717        108.810              0
    150     85,009.300          4.049         22.479         36.405          
2.724         80.150              0
    180     85,005.933          3.381         20.283         23.478          
2.639         43.224              0
    210     85,007.733          3.798         21.053         24.199          
2.717        115.814              0
    240     85,007.800          4.275         22.266         26.608          
2.751        106.709              0
    270     85,005.867          3.507         20.644         24.297          
2.700         59.038              0
    ```
    
    In overall, this patch shows half of mean latency, whereas 99%ile/99.9%ile 
of latencies are close to. It also shows this patch consumes more CPU. The gap 
is fairly small for single-worker but becomes somewhat huge on 4 workers. I 
might be wrong about testing so it would be really nice if someone also runs 
the multi-workers test and shares the result.
    
    Even my test result is not wrong I'm +1, because this patch introduces 
better design of backpressure which should have been enabled by default but we 
disabled. We could file some follow-up issues if there're something we should 
address.


---

Reply via email to