pitrou commented on PR #49679:
URL: https://github.com/apache/arrow/pull/49679#issuecomment-4593729317
I took a look at the compiled code size here and we're really generating too
much code:
```
$ ls -lh
/build/build-release/src/arrow/CMakeFiles/arrow_compute_objlib.dir/compute/kernels/*.o
-rw-rw-r-- 1 antoine antoine 7,8M juin 1 14:53
/build/build-release/src/arrow/CMakeFiles/arrow_compute_objlib.dir/compute/kernels/aggregate_basic_avx2.cc.o
-rw-rw-r-- 1 antoine antoine 7,8M juin 1 14:53
/build/build-release/src/arrow/CMakeFiles/arrow_compute_objlib.dir/compute/kernels/aggregate_basic_avx512.cc.o
-rw-rw-r-- 1 antoine antoine 18M juin 1 14:53
/build/build-release/src/arrow/CMakeFiles/arrow_compute_objlib.dir/compute/kernels/aggregate_basic.cc.o
-rw-rw-r-- 1 antoine antoine 6,3M juin 1 14:52
/build/build-release/src/arrow/CMakeFiles/arrow_compute_objlib.dir/compute/kernels/aggregate_mode.cc.o
-rw-rw-r-- 1 antoine antoine 1,7M juin 1 14:51
/build/build-release/src/arrow/CMakeFiles/arrow_compute_objlib.dir/compute/kernels/aggregate_pivot.cc.o
-rw-rw-r-- 1 antoine antoine 5,4M juin 1 14:52
/build/build-release/src/arrow/CMakeFiles/arrow_compute_objlib.dir/compute/kernels/aggregate_quantile.cc.o
-rw-rw-r-- 1 antoine antoine 2,4M juin 1 14:51
/build/build-release/src/arrow/CMakeFiles/arrow_compute_objlib.dir/compute/kernels/aggregate_tdigest.cc.o
-rw-rw-r-- 1 antoine antoine 2,5M juin 1 14:52
/build/build-release/src/arrow/CMakeFiles/arrow_compute_objlib.dir/compute/kernels/aggregate_var_std.cc.o
-rw-rw-r-- 1 antoine antoine 17M juin 1 14:52
/build/build-release/src/arrow/CMakeFiles/arrow_compute_objlib.dir/compute/kernels/hash_aggregate.cc.o
-rw-rw-r-- 1 antoine antoine 16M juin 1 14:52
/build/build-release/src/arrow/CMakeFiles/arrow_compute_objlib.dir/compute/kernels/hash_aggregate_numeric.cc.o
-rw-rw-r-- 1 antoine antoine 2,7M juin 1 14:51
/build/build-release/src/arrow/CMakeFiles/arrow_compute_objlib.dir/compute/kernels/hash_aggregate_pivot.cc.o
-rw-rw-r-- 1 antoine antoine 1,7M juin 1 14:51
/build/build-release/src/arrow/CMakeFiles/arrow_compute_objlib.dir/compute/kernels/pivot_internal.cc.o
-rw-rw-r-- 1 antoine antoine 749K juin 1 14:51
/build/build-release/src/arrow/CMakeFiles/arrow_compute_objlib.dir/compute/kernels/ree_util_internal.cc.o
-rw-rw-r-- 1 antoine antoine 18M juin 1 14:53
/build/build-release/src/arrow/CMakeFiles/arrow_compute_objlib.dir/compute/kernels/scalar_arithmetic.cc.o
-rw-rw-r-- 1 antoine antoine 1,9M juin 1 14:52
/build/build-release/src/arrow/CMakeFiles/arrow_compute_objlib.dir/compute/kernels/scalar_boolean.cc.o
-rw-rw-r-- 1 antoine antoine 11M juin 1 14:52
/build/build-release/src/arrow/CMakeFiles/arrow_compute_objlib.dir/compute/kernels/scalar_compare.cc.o
-rw-rw-r-- 1 antoine antoine 6,3M juin 1 14:52
/build/build-release/src/arrow/CMakeFiles/arrow_compute_objlib.dir/compute/kernels/scalar_if_else.cc.o
-rw-rw-r-- 1 antoine antoine 7,0M juin 1 14:52
/build/build-release/src/arrow/CMakeFiles/arrow_compute_objlib.dir/compute/kernels/scalar_nested.cc.o
-rw-rw-r-- 1 antoine antoine 1,1M juin 1 14:51
/build/build-release/src/arrow/CMakeFiles/arrow_compute_objlib.dir/compute/kernels/scalar_random.cc.o
-rw-rw-r-- 1 antoine antoine 22M juin 1 14:53
/build/build-release/src/arrow/CMakeFiles/arrow_compute_objlib.dir/compute/kernels/scalar_round.cc.o
-rw-rw-r-- 1 antoine antoine 4,6M juin 1 14:52
/build/build-release/src/arrow/CMakeFiles/arrow_compute_objlib.dir/compute/kernels/scalar_set_lookup.cc.o
-rw-rw-r-- 1 antoine antoine 12M juin 1 14:52
/build/build-release/src/arrow/CMakeFiles/arrow_compute_objlib.dir/compute/kernels/scalar_string_ascii.cc.o
-rw-rw-r-- 1 antoine antoine 4,8M juin 1 14:52
/build/build-release/src/arrow/CMakeFiles/arrow_compute_objlib.dir/compute/kernels/scalar_string_utf8.cc.o
-rw-rw-r-- 1 antoine antoine 19M juin 1 14:53
/build/build-release/src/arrow/CMakeFiles/arrow_compute_objlib.dir/compute/kernels/scalar_temporal_binary.cc.o
-rw-rw-r-- 1 antoine antoine 19M juin 1 14:53
/build/build-release/src/arrow/CMakeFiles/arrow_compute_objlib.dir/compute/kernels/scalar_temporal_unary.cc.o
-rw-rw-r-- 1 antoine antoine 2,1M juin 1 14:52
/build/build-release/src/arrow/CMakeFiles/arrow_compute_objlib.dir/compute/kernels/scalar_validity.cc.o
-rw-rw-r-- 1 antoine antoine 627K juin 1 14:52
/build/build-release/src/arrow/CMakeFiles/arrow_compute_objlib.dir/compute/kernels/util_internal.cc.o
-rw-rw-r-- 1 antoine antoine 7,8M juin 1 14:52
/build/build-release/src/arrow/CMakeFiles/arrow_compute_objlib.dir/compute/kernels/vector_array_sort.cc.o
-rw-rw-r-- 1 antoine antoine 8,3M juin 1 14:52
/build/build-release/src/arrow/CMakeFiles/arrow_compute_objlib.dir/compute/kernels/vector_cumulative_ops.cc.o
-rw-rw-r-- 1 antoine antoine 2,3M juin 1 14:52
/build/build-release/src/arrow/CMakeFiles/arrow_compute_objlib.dir/compute/kernels/vector_nested.cc.o
-rw-rw-r-- 1 antoine antoine 1,7M juin 1 14:52
/build/build-release/src/arrow/CMakeFiles/arrow_compute_objlib.dir/compute/kernels/vector_pairwise.cc.o
-rw-rw-r-- 1 antoine antoine 2,7M juin 1 14:52
/build/build-release/src/arrow/CMakeFiles/arrow_compute_objlib.dir/compute/kernels/vector_rank.cc.o
-rw-rw-r-- 1 antoine antoine 5,7M juin 1 14:52
/build/build-release/src/arrow/CMakeFiles/arrow_compute_objlib.dir/compute/kernels/vector_replace.cc.o
-rw-rw-r-- 1 antoine antoine 6,4M juin 1 14:52
/build/build-release/src/arrow/CMakeFiles/arrow_compute_objlib.dir/compute/kernels/vector_run_end_encode.cc.o
-rw-rw-r-- 1 antoine antoine 48M juin 1 14:54
/build/build-release/src/arrow/CMakeFiles/arrow_compute_objlib.dir/compute/kernels/vector_search_sorted.cc.o
-rw-rw-r-- 1 antoine antoine 16M juin 1 14:53
/build/build-release/src/arrow/CMakeFiles/arrow_compute_objlib.dir/compute/kernels/vector_select_k.cc.o
-rw-rw-r-- 1 antoine antoine 13M juin 1 14:53
/build/build-release/src/arrow/CMakeFiles/arrow_compute_objlib.dir/compute/kernels/vector_sort.cc.o
-rw-rw-r-- 1 antoine antoine 3,5M juin 1 14:52
/build/build-release/src/arrow/CMakeFiles/arrow_compute_objlib.dir/compute/kernels/vector_statistics.cc.o
```
48M for the search_sorted kernels alone is almost 20% of the total size of
compute kernels (256M for libarrow_compute):
```
$ ls -lh /build/build-release/relwithdebinfo/libarrow*.so.2400.*
-rwxrwxr-x 1 antoine antoine 256M juin 1 14:54
/build/build-release/relwithdebinfo/libarrow_compute.so.2400.0.0
-rwxrwxr-x 1 antoine antoine 268M juin 1 14:53
/build/build-release/relwithdebinfo/libarrow.so.2400.0.0
-rwxrwxr-x 1 antoine antoine 24M juin 1 14:53
/build/build-release/relwithdebinfo/libarrow_testing.so.2400.0.0
```
This is also confirmed by code sizes (the `text` column below), ignoring
debug information:
```
$ size
/build/build-release/src/arrow/CMakeFiles/arrow_compute_objlib.dir/compute/kernels/*.o
text data bss dec hex filename
335103 13600 0 348703 5521f
/build/build-release/src/arrow/CMakeFiles/arrow_compute_objlib.dir/compute/kernels/aggregate_basic_avx2.cc.o
338903 13600 0 352503 560f7
/build/build-release/src/arrow/CMakeFiles/arrow_compute_objlib.dir/compute/kernels/aggregate_basic_avx512.cc.o
762469 23136 1888 787493 c0425
/build/build-release/src/arrow/CMakeFiles/arrow_compute_objlib.dir/compute/kernels/aggregate_basic.cc.o
227705 752 192 228649 37d29
/build/build-release/src/arrow/CMakeFiles/arrow_compute_objlib.dir/compute/kernels/aggregate_mode.cc.o
43190 976 224 44390 ad66
/build/build-release/src/arrow/CMakeFiles/arrow_compute_objlib.dir/compute/kernels/aggregate_pivot.cc.o
211898 752 224 212874 33f8a
/build/build-release/src/arrow/CMakeFiles/arrow_compute_objlib.dir/compute/kernels/aggregate_quantile.cc.o
71202 1712 384 73298 11e52
/build/build-release/src/arrow/CMakeFiles/arrow_compute_objlib.dir/compute/kernels/aggregate_tdigest.cc.o
131967 2000 704 134671 20e0f
/build/build-release/src/arrow/CMakeFiles/arrow_compute_objlib.dir/compute/kernels/aggregate_var_std.cc.o
691424 10464 1920 703808 abd40
/build/build-release/src/arrow/CMakeFiles/arrow_compute_objlib.dir/compute/kernels/hash_aggregate.cc.o
727769 20160 1376 749305 b6ef9
/build/build-release/src/arrow/CMakeFiles/arrow_compute_objlib.dir/compute/kernels/hash_aggregate_numeric.cc.o
71421 1280 224 72925 11cdd
/build/build-release/src/arrow/CMakeFiles/arrow_compute_objlib.dir/compute/kernels/hash_aggregate_pivot.cc.o
39497 808 0 40305 9d71
/build/build-release/src/arrow/CMakeFiles/arrow_compute_objlib.dir/compute/kernels/pivot_internal.cc.o
20989 208 0 21197 52cd
/build/build-release/src/arrow/CMakeFiles/arrow_compute_objlib.dir/compute/kernels/ree_util_internal.cc.o
1135819 1128 7296 1144243 1175b3
/build/build-release/src/arrow/CMakeFiles/arrow_compute_objlib.dir/compute/kernels/scalar_arithmetic.cc.o
72828 776 1024 74628 12384
/build/build-release/src/arrow/CMakeFiles/arrow_compute_objlib.dir/compute/kernels/scalar_boolean.cc.o
580496 7008 1120 588624 8fb50
/build/build-release/src/arrow/CMakeFiles/arrow_compute_objlib.dir/compute/kernels/scalar_compare.cc.o
346737 2648 544 349929 556e9
/build/build-release/src/arrow/CMakeFiles/arrow_compute_objlib.dir/compute/kernels/scalar_if_else.cc.o
308651 1664 928 311243 4bfcb
/build/build-release/src/arrow/CMakeFiles/arrow_compute_objlib.dir/compute/kernels/scalar_nested.cc.o
18833 688 256 19777 4d41
/build/build-release/src/arrow/CMakeFiles/arrow_compute_objlib.dir/compute/kernels/scalar_random.cc.o
1428882 2816 928 1432626 15dc32
/build/build-release/src/arrow/CMakeFiles/arrow_compute_objlib.dir/compute/kernels/scalar_round.cc.o
186180 3240 512 189932 2e5ec
/build/build-release/src/arrow/CMakeFiles/arrow_compute_objlib.dir/compute/kernels/scalar_set_lookup.cc.o
608678 5336 8352 622366 97f1e
/build/build-release/src/arrow/CMakeFiles/arrow_compute_objlib.dir/compute/kernels/scalar_string_ascii.cc.o
259069 2808 4168 266045 40f3d
/build/build-release/src/arrow/CMakeFiles/arrow_compute_objlib.dir/compute/kernels/scalar_string_utf8.cc.o
962615 744 1736 965095 eb9e7
/build/build-release/src/arrow/CMakeFiles/arrow_compute_objlib.dir/compute/kernels/scalar_temporal_binary.cc.o
941534 2640 4072 948246 e7816
/build/build-release/src/arrow/CMakeFiles/arrow_compute_objlib.dir/compute/kernels/scalar_temporal_unary.cc.o
70576 696 832 72104 119a8
/build/build-release/src/arrow/CMakeFiles/arrow_compute_objlib.dir/compute/kernels/scalar_validity.cc.o
5279 72 0 5351 14e7
/build/build-release/src/arrow/CMakeFiles/arrow_compute_objlib.dir/compute/kernels/util_internal.cc.o
396050 2288 296 398634 6152a
/build/build-release/src/arrow/CMakeFiles/arrow_compute_objlib.dir/compute/kernels/vector_array_sort.cc.o
465965 3592 1376 470933 72f95
/build/build-release/src/arrow/CMakeFiles/arrow_compute_objlib.dir/compute/kernels/vector_cumulative_ops.cc.o
69255 1704 320 71279 1166f
/build/build-release/src/arrow/CMakeFiles/arrow_compute_objlib.dir/compute/kernels/vector_nested.cc.o
39943 800 320 41063 a067
/build/build-release/src/arrow/CMakeFiles/arrow_compute_objlib.dir/compute/kernels/vector_pairwise.cc.o
93968 2880 608 97456 17cb0
/build/build-release/src/arrow/CMakeFiles/arrow_compute_objlib.dir/compute/kernels/vector_rank.cc.o
257344 1576 384 259304 3f4e8
/build/build-release/src/arrow/CMakeFiles/arrow_compute_objlib.dir/compute/kernels/vector_replace.cc.o
288222 704 256 289182 4699e
/build/build-release/src/arrow/CMakeFiles/arrow_compute_objlib.dir/compute/kernels/vector_run_end_encode.cc.o
2058224 2424 168 2060816 1f7210
/build/build-release/src/arrow/CMakeFiles/arrow_compute_objlib.dir/compute/kernels/vector_search_sorted.cc.o
709493 8712 208 718413 af64d
/build/build-release/src/arrow/CMakeFiles/arrow_compute_objlib.dir/compute/kernels/vector_select_k.cc.o
676708 5800 208 682716 a6adc
/build/build-release/src/arrow/CMakeFiles/arrow_compute_objlib.dir/compute/kernels/vector_sort.cc.o
107434 888 192 108514 1a7e2
/build/build-release/src/arrow/CMakeFiles/arrow_compute_objlib.dir/compute/kernels/vector_statistics.cc.o
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
