westonpace commented on pull request #11616: URL: https://github.com/apache/arrow/pull/11616#issuecomment-988441083
I fixed up the benchmarks somewhat and added a benchmark for cold I/O. Initial results are a bit noisy: ``` ReadRealFile/1/0/real_time_mean +0.1096 +0.0203 882364277 979101124 141562817 144432381 ReadRealFile/4/0/real_time_mean +0.1379 -0.0096 1014012277 1153867283 141160362 139811386 ReadRealFile/16/0/real_time_mean -0.0303 +0.0033 900165884 872914568 143141575 143615607 ReadRealFile/64/0/real_time_mean -0.1049 +0.0007 1154140298 1033116675 140212506 140312232 ReadRealFile/256/0/real_time_mean +0.0990 +0.0239 1036406449 1138971720 149655837 153234293 ReadRealFile/1024/0/real_time_mean +0.0572 +0.0509 841348002 889506548 160709304 168895957 ReadRealFile/4096/0/real_time_mean -0.0244 +0.0147 1040310983 1014902847 269116155 273081737 ReadRealFile/8192/0/real_time_mean +0.1152 +0.0127 1073965563 1197736789 388846023 393793665 ReadRealFile/1/1/real_time_mean -0.0211 -0.0686 852270837 834276010 146015520 136002655 ReadRealFile/4/1/real_time_mean -0.0477 -0.0025 444691103 423473671 53215988 53082698 ReadRealFile/16/1/real_time_mean +1.5411 -0.1796 172459194 438231423 29137200 23905451 ReadRealFile/64/1/real_time_mean -0.1627 -0.0478 1004655612 841218794 35870849 34156124 ReadRealFile/256/1/real_time_mean +0.0982 -0.0181 897612093 985737042 63828576 62673771 ReadRealFile/1024/1/real_time_mean +0.2167 +0.5705 1020315892 1241371726 84622012 132899024 ReadRealFile/4096/1/real_time_mean -0.2797 -0.1929 1209181656 871000618 71425702 57648840 ReadRealFile/8192/1/real_time_mean +0.2138 +0.3822 938647868 1139351095 159262940 220134001 ReadRealFileAsync/1/0/real_time_mean +0.3462 +0.3408 906170613 1219925626 2299242 3082930 ReadRealFileAsync/4/0/real_time_mean -0.1024 +0.3570 1124871244 1009668198 2312991 3138757 ReadRealFileAsync/16/0/real_time_mean +0.2950 +0.5326 846347457 1096059130 2476560 3795647 ReadRealFileAsync/64/0/real_time_mean +0.0606 +0.8470 1225185369 1299444655 2964952 5476138 ReadRealFileAsync/256/0/real_time_mean -0.0583 +1.6426 1091256184 1027624433 4635198 12249101 ReadRealFileAsync/1024/0/real_time_mean +0.2313 +2.2223 925963093 1140141694 11659770 37571048 ReadRealFileAsync/4096/0/real_time_mean +0.2538 +1.0950 981474329 1230572649 46248110 96890904 ReadRealFileAsync/8192/0/real_time_mean -0.2438 +1.0463 1352917947 1023135386 86818642 177659883 ReadRealFileAsync/1/1/real_time_mean +0.2791 +0.7906 856795160 1095885713 2013025 3604501 ReadRealFileAsync/4/1/real_time_mean -0.6628 +7.4180 1022300282 344763011 2321308 19540718 ReadRealFileAsync/16/1/real_time_mean -0.8603 +5.4649 1240246020 173301490 2479652 16030715 ReadRealFileAsync/64/1/real_time_mean -0.1437 +6.2582 852187403 729739446 2156059 15649018 ReadRealFileAsync/256/1/real_time_mean +0.1711 +6.8878 1059134316 1240323095 2993556 23612644 ReadRealFileAsync/1024/1/real_time_mean +0.1014 +8.6551 1091676930 1202395405 4424422 42718098 ReadRealFileAsync/4096/1/real_time_mean +0.5519 +6.7698 1042589081 1618010814 8712906 67697324 ReadRealFileAsync/8192/1/real_time_mean -0.2005 +4.6431 1307820760 1045555603 17304062 97648656 ``` There is a substantial improvement in the partial-column read async case (ReadRealFileAsync/num_cols/1) at 4/16 columns. I'm not sure why that falls away as the number of columns increases so I need to investigate that still. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
