[GitHub] [arrow] wesm commented on pull request #7516: ARROW-9201: [Archery] More user-friendly console output for benchmark diffs, add repetitions argument, don't build unit tests

2020-06-23 Thread GitBox 


wesm commented on pull request #7516:
URL: https://github.com/apache/arrow/pull/7516#issuecomment-648162680


   +1. The bot changes can't be done here so going to go ahead and merge this 
so I can use it more easily without having to switch branches (to use this 
branch) before running benchmarks



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [arrow] wesm commented on pull request #7516: ARROW-9201: [Archery] More user-friendly console output for benchmark diffs, add repetitions argument, don't build unit tests

2020-06-22 Thread GitBox 


wesm commented on pull request #7516:
URL: https://github.com/apache/arrow/pull/7516#issuecomment-647883524


   I improved the output to show the `state.counters` stuff
   
   ```
 benchmark baselinecontender  change %  
  counters
   40UniqueInt64/56.442 GiB/sec   18.346 GiB/sec   184.782  
{'iterations': 145, 'null_percent': 100.0}
   0UniqueInt64/116.500 GiB/sec   18.364 GiB/sec   182.522  
{'iterations': 145, 'null_percent': 100.0}
   11UniqueUInt8/5  812.047 MiB/sec1.755 GiB/sec   121.298  
{'iterations': 142, 'null_percent': 100.0}
   7 UniqueUInt8/1  683.943 MiB/sec1.253 GiB/sec87.593
{'iterations': 117, 'null_percent': 0.1}
   38UniqueUInt8/4  762.983 MiB/sec  950.521 MiB/sec24.580   
{'iterations': 133, 'null_percent': 99.0}
   29UniqueUInt8/2  659.082 MiB/sec  820.410 MiB/sec24.478
{'iterations': 114, 'null_percent': 1.0}
   5 UniqueInt64/12.656 GiB/sec3.300 GiB/sec24.223 
{'iterations': 60, 'null_percent': 0.1}
   32UniqueInt64/45.627 GiB/sec6.772 GiB/sec20.349   
{'iterations': 119, 'null_percent': 99.0}
   25   UniqueInt64/105.234 GiB/sec6.294 GiB/sec20.254   
{'iterations': 110, 'null_percent': 99.0}
   39  UniqueString100bytes/11   26.815 GiB/sec   31.122 GiB/sec16.061   
{'iterations': 48, 'null_percent': 100.0}
   23UniqueString10bytes/52.691 GiB/sec3.113 GiB/sec15.667   
{'iterations': 48, 'null_percent': 100.0}
   34   UniqueString100bytes/5   26.944 GiB/sec   31.015 GiB/sec15.108   
{'iterations': 48, 'null_percent': 100.0}
   6UniqueString10bytes/112.699 GiB/sec3.096 GiB/sec14.721   
{'iterations': 49, 'null_percent': 100.0}
   21   UniqueString100bytes/71.947 GiB/sec2.217 GiB/sec13.866  
{'iterations': 3, 'null_percent': 0.1}
   28UniqueInt64/22.622 GiB/sec2.904 GiB/sec10.770 
{'iterations': 59, 'null_percent': 1.0}
   13UniqueInt64/32.157 GiB/sec2.343 GiB/sec 8.644
{'iterations': 48, 'null_percent': 10.0}
   33   UniqueString100bytes/4   24.286 GiB/sec   26.030 GiB/sec 7.181
{'iterations': 43, 'null_percent': 99.0}
   22UniqueInt64/72.542 GiB/sec2.707 GiB/sec 6.497 
{'iterations': 56, 'null_percent': 0.1}
   20  UniqueString100bytes/10   22.536 GiB/sec   23.985 GiB/sec 6.432
{'iterations': 40, 'null_percent': 99.0}
   35UniqueString10bytes/1  788.817 MiB/sec  836.008 MiB/sec 5.983 
{'iterations': 14, 'null_percent': 0.1}
   17UniqueString10bytes/7  592.671 MiB/sec  628.054 MiB/sec 5.970 
{'iterations': 10, 'null_percent': 0.1}
   3 UniqueString10bytes/42.515 GiB/sec2.658 GiB/sec 5.687
{'iterations': 45, 'null_percent': 99.0}
   19   UniqueString10bytes/102.402 GiB/sec2.529 GiB/sec 5.269
{'iterations': 42, 'null_percent': 99.0}
   9UniqueString100bytes/13.929 GiB/sec4.077 GiB/sec 3.762  
{'iterations': 7, 'null_percent': 0.1}
   30UniqueString10bytes/8  593.560 MiB/sec  610.253 MiB/sec 2.812 
{'iterations': 10, 'null_percent': 1.0}
   12UniqueString10bytes/2  788.505 MiB/sec  808.396 MiB/sec 2.523 
{'iterations': 14, 'null_percent': 1.0}
   37   UniqueString100bytes/81.965 GiB/sec1.998 GiB/sec 1.697  
{'iterations': 3, 'null_percent': 1.0}
   1UniqueString100bytes/23.984 GiB/sec4.025 GiB/sec 1.028  
{'iterations': 7, 'null_percent': 1.0}
   36   UniqueString100bytes/34.262 GiB/sec4.293 GiB/sec 0.725 
{'iterations': 8, 'null_percent': 10.0}
   8 BuildStringDictionary   85.507 MiB/sec   85.687 MiB/sec 0.211  
   {'iterations': 198}
   16   UniqueString100bytes/92.121 GiB/sec2.111 GiB/sec-0.469 
{'iterations': 4, 'null_percent': 10.0}
   4UniqueString100bytes/62.056 GiB/sec2.043 GiB/sec-0.626  
{'iterations': 4, 'null_percent': 0.0}
   10UniqueUInt8/3  453.281 MiB/sec  448.407 MiB/sec-1.075
{'iterations': 79, 'null_percent': 10.0}
   14   UniqueString100bytes/04.100 GiB/sec4.055 GiB/sec-1.089  
{'iterations': 7, 'null_percent': 0.0}
   24UniqueInt64/82.473 GiB/sec2.443 GiB/sec-1.202 
{'iterations': 55, 'null_percent': 1.0}
   26UniqueString10bytes/9  615.880 MiB/sec  608.453 MiB/sec-1.206
{'iterations': 11, 'null_percent': 10.0}
   42UniqueString10bytes/6  651.430 MiB/sec  640.128 MiB/sec-1.735 
{'iterations': 11, 'null_percent': 0.0}
   27UniqueUInt8/01.775 GiB/sec1.738 GiB/sec-2.063
{'iterations': 318, 'null_percent': 0.0}
   31UniqueInt64/92.076 GiB/sec2.033 GiB/sec-2.067
{'iterations': 46, 'null_percent': 10.0}
   15  B

[GitHub] [arrow] wesm commented on pull request #7516: ARROW-9201: [Archery] More user-friendly console output for benchmark diffs, add repetitions argument, don't build unit tests

2020-06-22 Thread GitBox 


wesm commented on pull request #7516:
URL: https://github.com/apache/arrow/pull/7516#issuecomment-647758158


   I’m sort of -1 on using anything but pandas for data munging and data 
presentation in our tooling. It’s not a very large dependency and has 
everything we need. 



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [arrow] wesm commented on pull request #7516: ARROW-9201: [Archery] More user-friendly console output for benchmark diffs, add repetitions argument, don't build unit tests

2020-06-22 Thread GitBox 


wesm commented on pull request #7516:
URL: https://github.com/apache/arrow/pull/7516#issuecomment-647641007


   @kszucs can you assist me with adapting ursabot for these changes? I think 
we can use pandas's `DataFrame.to_html` to create a colorized table for GitHub, 
too https://pandas.pydata.org/pandas-docs/stable/user_guide/style.html
   
   Changes that would be good to have in `ursabot benchmark`:
   
   * Pass through `--cc` and `--cxx` options
   * Pass through `--repetitions`



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org