mapleFU commented on PR #41703: URL: https://github.com/apache/arrow/pull/41703#issuecomment-2123079469
On my AMD 3800X: Before: ``` WriteMetadata/num_columns:1/num_row_groups:1 14816 ns 14816 ns 48309 file_size=459 items_per_second=67.4938k/s WriteMetadata/num_columns:1/num_row_groups:100 1087488 ns 1087494 ns 699 file_size=37.383k items_per_second=919.545/s WriteMetadata/num_columns:1/num_row_groups:1000 9939072 ns 9937325 ns 59 file_size=374.885k items_per_second=100.631/s WriteMetadata/num_columns:10/num_row_groups:1 118413 ns 118412 ns 5646 file_size=3.762k items_per_second=8.44508k/s WriteMetadata/num_columns:10/num_row_groups:100 9458300 ns 9458325 ns 69 file_size=358.835k items_per_second=105.727/s WriteMetadata/num_columns:10/num_row_groups:1000 99919203 ns 99917686 ns 7 file_size=3.614M items_per_second=10.0082/s WriteMetadata/num_columns:100/num_row_groups:1 1145205 ns 1145207 ns 624 file_size=37.352k items_per_second=873.204/s WriteMetadata/num_columns:100/num_row_groups:100 101083527 ns 101079183 ns 6 file_size=3.61693M items_per_second=9.89323/s WriteMetadata/num_columns:100/num_row_groups:1000 1027574292 ns 1027570500 ns 1 file_size=36.2887M items_per_second=0.973169/s WriteMetadata/num_columns:1000/num_row_groups:1 10933400 ns 10933348 ns 66 file_size=376.655k items_per_second=91.4633/s WriteMetadata/num_columns:1000/num_row_groups:100 997383234 ns 989469900 ns 1 file_size=36.4815M items_per_second=1.01064/s WriteMetadata/num_columns:10000/num_row_groups:1 113278188 ns 113278583 ns 6 file_size=3.82213M items_per_second=8.82779/s WriteMetadata/num_columns:10000/num_row_groups:100 1.0258e+10 ns 1.0257e+10 ns 1 file_size=369.089M items_per_second=0.0974916/s ReadMetadata/num_columns:1/num_row_groups:1 4286 ns 4286 ns 156757 file_size=459 items_per_second=233.3k/s ReadMetadata/num_columns:1/num_row_groups:100 92293 ns 92293 ns 7323 file_size=37.383k items_per_second=10.8351k/s ReadMetadata/num_columns:1/num_row_groups:1000 874417 ns 874410 ns 838 file_size=374.885k items_per_second=1.14363k/s ReadMetadata/num_columns:10/num_row_groups:1 20860 ns 20860 ns 33415 file_size=3.762k items_per_second=47.9385k/s ReadMetadata/num_columns:10/num_row_groups:100 730042 ns 730043 ns 997 file_size=358.835k items_per_second=1.36978k/s ReadMetadata/num_columns:10/num_row_groups:1000 7307342 ns 7307324 ns 83 file_size=3.614M items_per_second=136.849/s ReadMetadata/num_columns:100/num_row_groups:1 186400 ns 186401 ns 3787 file_size=37.352k items_per_second=5.36479k/s ReadMetadata/num_columns:100/num_row_groups:100 7121296 ns 7121319 ns 85 file_size=3.61693M items_per_second=140.423/s ReadMetadata/num_columns:100/num_row_groups:1000 81473986 ns 81458278 ns 9 file_size=36.2887M items_per_second=12.2762/s ReadMetadata/num_columns:1000/num_row_groups:1 1814784 ns 1814759 ns 400 file_size=376.655k items_per_second=551.037/s ReadMetadata/num_columns:1000/num_row_groups:100 79376073 ns 79376022 ns 9 file_size=36.4815M items_per_second=12.5983/s ReadMetadata/num_columns:10000/num_row_groups:1 20298949 ns 20236894 ns 36 file_size=3.82213M items_per_second=49.4147/s ReadMetadata/num_columns:10000/num_row_groups:100 797135144 ns 797041100 ns 1 file_size=369.089M items_per_second=1.25464/s ``` After: ``` WriteMetadata/num_columns:1/num_row_groups:1 14042 ns 14026 ns 48265 file_size=459 items_per_second=71.2951k/s WriteMetadata/num_columns:1/num_row_groups:100 982543 ns 982545 ns 693 file_size=37.383k items_per_second=1.01776k/s WriteMetadata/num_columns:1/num_row_groups:1000 9236559 ns 9234951 ns 75 file_size=374.885k items_per_second=108.284/s WriteMetadata/num_columns:10/num_row_groups:1 115867 ns 115865 ns 6050 file_size=3.762k items_per_second=8.63075k/s WriteMetadata/num_columns:10/num_row_groups:100 9106303 ns 9106322 ns 77 file_size=358.835k items_per_second=109.814/s WriteMetadata/num_columns:10/num_row_groups:1000 95039480 ns 95039886 ns 7 file_size=3.614M items_per_second=10.5219/s WriteMetadata/num_columns:100/num_row_groups:1 1066471 ns 1066474 ns 648 file_size=37.352k items_per_second=937.67/s WriteMetadata/num_columns:100/num_row_groups:100 92350381 ns 92350900 ns 8 file_size=3.61693M items_per_second=10.8283/s WriteMetadata/num_columns:100/num_row_groups:1000 972198408 ns 971689600 ns 1 file_size=36.2887M items_per_second=1.02914/s WriteMetadata/num_columns:1000/num_row_groups:1 10303438 ns 10302799 ns 68 file_size=376.655k items_per_second=97.061/s WriteMetadata/num_columns:1000/num_row_groups:100 926151272 ns 926026200 ns 1 file_size=36.4815M items_per_second=1.07988/s WriteMetadata/num_columns:10000/num_row_groups:1 109520337 ns 109283500 ns 6 file_size=3.82213M items_per_second=9.15051/s WriteMetadata/num_columns:10000/num_row_groups:100 9607536338 ns 9603598900 ns 1 file_size=369.089M items_per_second=0.104128/s ReadMetadata/num_columns:1/num_row_groups:1 3776 ns 3737 ns 190309 file_size=459 items_per_second=267.588k/s ReadMetadata/num_columns:1/num_row_groups:100 76296 ns 76114 ns 9217 file_size=37.383k items_per_second=13.1382k/s ReadMetadata/num_columns:1/num_row_groups:1000 706469 ns 706463 ns 993 file_size=374.885k items_per_second=1.4155k/s ReadMetadata/num_columns:10/num_row_groups:1 18738 ns 18738 ns 35672 file_size=3.762k items_per_second=53.3679k/s ReadMetadata/num_columns:10/num_row_groups:100 590179 ns 590180 ns 1202 file_size=358.835k items_per_second=1.6944k/s ReadMetadata/num_columns:10/num_row_groups:1000 5821858 ns 5821727 ns 123 file_size=3.614M items_per_second=171.77/s ReadMetadata/num_columns:100/num_row_groups:1 168284 ns 168284 ns 4074 file_size=37.352k items_per_second=5.94234k/s ReadMetadata/num_columns:100/num_row_groups:100 5752814 ns 5752800 ns 118 file_size=3.61693M items_per_second=173.828/s ReadMetadata/num_columns:100/num_row_groups:1000 65674677 ns 65672427 ns 11 file_size=36.2887M items_per_second=15.2271/s ReadMetadata/num_columns:1000/num_row_groups:1 1574680 ns 1574646 ns 444 file_size=376.655k items_per_second=635.063/s ReadMetadata/num_columns:1000/num_row_groups:100 65989678 ns 65988873 ns 11 file_size=36.4815M items_per_second=15.1541/s ReadMetadata/num_columns:10000/num_row_groups:1 16967274 ns 16966876 ns 41 file_size=3.82213M items_per_second=58.9384/s ReadMetadata/num_columns:10000/num_row_groups:100 652885946 ns 652766800 ns 1 file_size=369.089M items_per_second=1.53194/s ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
