viirya opened a new pull request, #5901:
URL: https://github.com/apache/datafusion-comet/pull/5901
## Which issue does this PR close?
Closes #5900. Part of #5818; follows the matched normalization/hash
benchmarks in #5822 and singleton optimization in #5887.
## Rationale for this change
Normalizing multi-entry `map<string,int>` batches slices the key array and
allocates Arrow sorting
indices and prefix tuples for every row. Reuse the prefix-tuple buffer
across rows to reduce this
allocation overhead while preserving the exact permutation.
## What changes are included in this PR?
- Specialize multi-entry `Utf8` keys with `Int32` values, reusing `(u32
index, u32 prefix, u64 length)`
scratch storage. Keep Arrow's comparator and tuple layout, including
equal-key permutations.
Scratch capacity grows with the largest row. The existing global index
append and single `take` remain.
- Preserve #5887's singleton eligibility check, batch dispatch and direct
index append. Insert the
multi-entry path into its const-generic helper; other key/value types
retain Arrow validation.
- Fill rebased offsets in bulk for all-empty visible batches while retaining
map reconstruction.
- Add wide/null/Unicode int-value fixtures and four tests for exact
duplicate-key permutations,
mixed singleton/multi-entry slices, physical entries under null maps,
unsupported keys and empty
visible slices. Retain all upstream tests and the existing
null/schema/sorted-flag semantics.
## How are these changes tested?
- All 16 map-sort tests passed (12 upstream tests retained unchanged, 4
added).
- Workspace/all-target Clippy with `-D warnings` passed.
- `make format PROFILES=-Pspark-4.0` passed, using the existing profile that
provides SemanticDB.
- Release workspace/native-library build passed.
- Fresh upstream and integrated release cohorts each completed all 122
cases. All 18 full-run
slowdown flags with CI lower bound above +1% received independent paired
measurements, as did
two wide-map wins. Two fallback cases still above +2% received
reverse-order pairs.
Matched 2–10-entry forward normalization improved about 66% (3x); matched
2–50-entry cases improved
28–39% in the full cohort and 33–39% in fresh pairs. No larger slowdown
persisted across the
confirmations. The int-key 16-entry fallback was +3.50% in a fresh pair and
+1.99% in reverse order;
small effects remain uncertain under observed process drift. The int-key
4-entry case fell from
+6.09% to +0.11% in reverse order. Initial singleton +8–9% flags did not
reproduce: fresh matched
singleton pairs were -0.08% and +1.72%. These are microbenchmarks, not
end-to-end Spark measurements.
### Measurement cohort and limits
Base: `4abfd95114d61ad454f9ee269be1615f469f24e2`, including merged singleton
PR #5887 (`ca223a8d4`).
All 122 baseline measurements completed before the integrated production
source was applied.
Both cohorts used byte-identical benchmark files, dependency manifests and
lockfile.
Apple M4 Max, 64 GB, Rust 1.97.1, release/thin LTO/codegen-units=1, no
RUSTFLAGS override.
Full cohorts: 50 samples, 1 s warmup, 2 s target measurement (Criterion
extends long cases).
Independent pairs: 100 samples, 3 s warmup, 5 s target measurement. No heavy
build or benchmark ran
concurrently. Narrow within-process CIs do not capture desktop/background
and cross-process drift.
All unfavorable original measurements are retained alongside confirmations.
Inputs and correctness checks are untimed. Normalization output
allocation/drop and combined
struct reconstruction/drop are timed. Hash-buffer allocation is untimed and
seed reset is timed.
Hash-only forward/reversed cases use the same normalized input as repeated
controls. Each batch
has 8,192 visible rows; physical entries under null maps count toward entry
rates. No phase subtraction
is used. Hashing implementation and nested-hashing defaults are unchanged.
<details>
<summary>Complete integrated comparison (122 cases)</summary>
Time is ms per 8,192-row batch, with 95% CI; unsupported-key and
buffer-reset cases are per call.
The change column is Criterion's cross-process mean estimate; confirmations
follow below.
| Case | Baseline ms (95% CI) | Integrated ms (95% CI) | Change % (95% CI) |
|---|---:|---:|---:|
| matched_maps/hash_buffer_seed_reset | 0.000297 [0.000290, 0.000303] |
0.000263 [0.000256, 0.000270] | -12.62 [-15.17, -9.89] |
| matched_maps/hash_only/map/0/forward | 0.004978 [0.004940, 0.005014] |
0.004620 [0.004580, 0.004656] | -7.50 [-8.55, -6.38] |
| matched_maps/hash_only/map/0/reversed | 0.004934 [0.004895, 0.004971] |
0.004646 [0.004607, 0.004691] | -5.12 [-5.98, -4.21] |
| matched_maps/hash_only/map/1/forward | 0.032509 [0.032211, 0.032795] |
0.032108 [0.031790, 0.032468] | -1.43 [-3.03, 0.17] |
| matched_maps/hash_only/map/1/reversed | 0.030791 [0.030709, 0.030898] |
0.030644 [0.030586, 0.030703] | -2.11 [-3.12, -1.15] |
| matched_maps/hash_only/map/10/forward | 0.352903 [0.347282, 0.359123] |
0.341396 [0.337814, 0.345680] | -5.95 [-7.30, -4.58] |
| matched_maps/hash_only/map/10/reversed | 0.333073 [0.331511, 0.334519] |
0.360368 [0.355854, 0.364416] | 7.57 [6.15, 8.80] |
| matched_maps/hash_only/map/50/forward | 2.165835 [2.150633, 2.181090] |
2.248646 [2.216507, 2.280922] | 3.10 [1.67, 4.71] |
| matched_maps/hash_only/map/50/reversed | 2.166743 [2.154227, 2.179776] |
2.235745 [2.205941, 2.273575] | 5.81 [3.76, 7.92] |
| matched_maps/hash_only/struct_map_int/0/forward | 0.012248 [0.012107,
0.012398] | 0.011977 [0.011887, 0.012084] | -0.26 [-1.61, 1.14] |
| matched_maps/hash_only/struct_map_int/0/reversed | 0.012756 [0.012627,
0.012881] | 0.012039 [0.011928, 0.012167] | -6.78 [-7.75, -5.67] |
| matched_maps/hash_only/struct_map_int/1/forward | 0.038521 [0.038220,
0.038885] | 0.037884 [0.037690, 0.038159] | -2.68 [-3.61, -1.70] |
| matched_maps/hash_only/struct_map_int/1/reversed | 0.037590 [0.037378,
0.037797] | 0.039951 [0.039543, 0.040325] | 4.79 [3.61, 5.98] |
| matched_maps/hash_only/struct_map_int/10/forward | 0.364487 [0.361603,
0.367084] | 0.367201 [0.363984, 0.370435] | -1.40 [-2.65, -0.07] |
| matched_maps/hash_only/struct_map_int/10/reversed | 0.343979 [0.340997,
0.346947] | 0.357271 [0.353344, 0.361585] | 4.19 [3.00, 5.47] |
| matched_maps/hash_only/struct_map_int/50/forward | 2.213356 [2.195088,
2.232738] | 2.211575 [2.179998, 2.248343] | -0.17 [-1.79, 1.33] |
| matched_maps/hash_only/struct_map_int/50/reversed | 2.139081 [2.130019,
2.148116] | 2.243743 [2.219535, 2.271584] | 7.77 [6.19, 9.52] |
| matched_maps/normalize_hash/map/0/forward | 0.011930 [0.011872, 0.011976]
| 0.007811 [0.007772, 0.007854] | -34.53 [-34.97, -34.11] |
| matched_maps/normalize_hash/map/0/reversed | 0.011863 [0.011806, 0.011928]
| 0.007999 [0.007914, 0.008111] | -30.74 [-31.69, -29.79] |
| matched_maps/normalize_hash/map/1/forward | 0.082434 [0.081518, 0.083587]
| 0.077613 [0.076782, 0.078749] | -2.62 [-4.38, -0.83] |
| matched_maps/normalize_hash/map/1/reversed | 0.097234 [0.096097, 0.098288]
| 0.074317 [0.074074, 0.074615] | -23.75 [-24.44, -23.06] |
| matched_maps/normalize_hash/map/10/forward | 2.082057 [2.070464, 2.093390]
| 0.750409 [0.747988, 0.752879] | -63.81 [-64.06, -63.56] |
| matched_maps/normalize_hash/map/10/reversed | 2.429893 [2.407186,
2.453356] | 1.151531 [1.145534, 1.158021] | -53.15 [-53.53, -52.74] |
| matched_maps/normalize_hash/map/50/forward | 5.477007 [5.436108, 5.520821]
| 3.710618 [3.699270, 3.722581] | -32.25 [-32.82, -31.68] |
| matched_maps/normalize_hash/map/50/reversed | 6.370159 [6.324398,
6.414206] | 4.374540 [4.363505, 4.385494] | -31.33 [-31.83, -30.80] |
| matched_maps/normalize_hash/struct_map_int/0/forward | 0.020538 [0.020326,
0.020813] | 0.015627 [0.015430, 0.015867] | -25.25 [-26.27, -24.23] |
| matched_maps/normalize_hash/struct_map_int/0/reversed | 0.021126
[0.020736, 0.021448] | 0.016097 [0.015938, 0.016238] | -23.01 [-24.11, -21.86] |
| matched_maps/normalize_hash/struct_map_int/1/forward | 0.088329 [0.088211,
0.088437] | 0.082637 [0.082505, 0.082763] | -6.48 [-6.68, -6.26] |
| matched_maps/normalize_hash/struct_map_int/1/reversed | 0.101194
[0.099879, 0.102577] | 0.081745 [0.081589, 0.081884] | -20.14 [-20.98, -19.31] |
| matched_maps/normalize_hash/struct_map_int/10/forward | 2.104292
[2.087008, 2.120388] | 0.762873 [0.760500, 0.765230] | -63.93 [-64.21, -63.66] |
| matched_maps/normalize_hash/struct_map_int/10/reversed | 2.343765
[2.333059, 2.357974] | 1.213060 [1.199996, 1.230424] | -50.03 [-50.75, -49.33] |
| matched_maps/normalize_hash/struct_map_int/50/forward | 5.573334
[5.520074, 5.625929] | 3.727349 [3.714379, 3.742165] | -33.12 [-33.80, -32.43] |
| matched_maps/normalize_hash/struct_map_int/50/reversed | 6.094283
[6.059158, 6.132586] | 4.398988 [4.384451, 4.413022] | -27.82 [-28.32, -27.34] |
| matched_maps/normalize_only/map/0/forward | 0.006960 [0.006917, 0.007002]
| 0.003103 [0.003052, 0.003154] | -55.86 [-56.29, -55.45] |
| matched_maps/normalize_only/map/0/reversed | 0.006939 [0.006907, 0.006970]
| 0.003285 [0.003255, 0.003321] | -52.50 [-53.11, -51.90] |
| matched_maps/normalize_only/map/1/forward | 0.047532 [0.047305, 0.047784]
| 0.051704 [0.050974, 0.052802] | 8.58 [7.03, 10.66] |
| matched_maps/normalize_only/map/1/reversed | 0.046676 [0.046159, 0.047362]
| 0.051078 [0.050758, 0.051435] | 9.46 [8.08, 10.77] |
| matched_maps/normalize_only/map/10/forward | 1.457805 [1.449019, 1.470317]
| 0.492988 [0.489871, 0.496710] | -66.20 [-66.59, -65.69] |
| matched_maps/normalize_only/map/10/reversed | 1.859864 [1.852003,
1.868389] | 0.903327 [0.899307, 0.907138] | -51.74 [-52.04, -51.44] |
| matched_maps/normalize_only/map/50/forward | 3.070098 [3.018114, 3.127601]
| 1.835687 [1.820745, 1.852364] | -38.61 [-39.61, -37.60] |
| matched_maps/normalize_only/map/50/reversed | 3.588305 [3.580530,
3.596033] | 2.556989 [2.532958, 2.579918] | -28.33 [-28.84, -27.84] |
| multi_entry_int_maps/hash_only/map/mixed_dense_null | 0.449289 [0.445058,
0.454586] | 0.402165 [0.400495, 0.403981] | -9.89 [-10.52, -9.29] |
| multi_entry_int_maps/hash_only/map/wide_dense_null | 0.617394 [0.614634,
0.620249] | 0.556214 [0.553946, 0.558295] | -9.70 [-10.19, -9.20] |
| multi_entry_int_maps/hash_only/map/wide_long_unicode | 71.502781
[71.167283, 71.850570] | 64.300277 [64.174908, 64.432286] | -10.07 [-10.53,
-9.61] |
| multi_entry_int_maps/hash_only/map/wide_long_unicode_dense_null |
36.097994 [35.900216, 36.308762] | 32.683120 [32.545423, 32.845109] | -9.46
[-10.12, -8.78] |
| multi_entry_int_maps/hash_only/map/wide_no_null | 1.700726 [1.686073,
1.719461] | 1.486960 [1.482161, 1.492874] | -15.00 [-16.07, -13.91] |
| multi_entry_int_maps/hash_only/map/wide_sparse_null | 1.727546 [1.708216,
1.747878] | 1.511608 [1.504700, 1.518844] | -14.33 [-15.25, -13.38] |
| multi_entry_int_maps/hash_only/struct_map_int/mixed_dense_null | 0.452555
[0.449866, 0.455648] | 0.409470 [0.407853, 0.411273] | -9.53 [-10.37, -8.73] |
| multi_entry_int_maps/hash_only/struct_map_int/wide_dense_null | 0.620534
[0.616507, 0.624194] | 0.566632 [0.564718, 0.568522] | -9.13 [-9.75, -8.48] |
| multi_entry_int_maps/hash_only/struct_map_int/wide_long_unicode |
73.218033 [72.742804, 73.717389] | 64.374067 [64.246127, 64.511935] | -12.08
[-12.70, -11.47] |
| multi_entry_int_maps/hash_only/struct_map_int/wide_long_unicode_dense_null
| 35.743625 [35.499035, 35.991249] | 32.728699 [32.555951, 32.932193] | -8.43
[-9.27, -7.61] |
| multi_entry_int_maps/hash_only/struct_map_int/wide_no_null | 1.686136
[1.670163, 1.702652] | 1.496503 [1.490776, 1.504781] | -9.63 [-10.48, -8.76] |
| multi_entry_int_maps/hash_only/struct_map_int/wide_sparse_null | 1.724639
[1.709869, 1.739987] | 1.516817 [1.512161, 1.520917] | -11.60 [-12.12, -11.09] |
| multi_entry_int_maps/normalize_hash/map/mixed_dense_null | 1.677325
[1.659271, 1.695524] | 0.879415 [0.876691, 0.882169] | -48.38 [-48.97, -47.78] |
| multi_entry_int_maps/normalize_hash/map/wide_dense_null | 3.492575
[3.463381, 3.522336] | 1.770534 [1.764865, 1.775920] | -49.29 [-49.75, -48.83] |
| multi_entry_int_maps/normalize_hash/map/wide_long_unicode | 98.993755
[97.956763, 100.109657] | 85.089966 [84.857972, 85.343834] | -14.05 [-15.03,
-13.11] |
| multi_entry_int_maps/normalize_hash/map/wide_long_unicode_dense_null |
62.323938 [61.782437, 62.892852] | 53.181282 [52.967561, 53.425551] | -14.67
[-15.52, -13.85] |
| multi_entry_int_maps/normalize_hash/map/wide_no_null | 4.377918 [4.356941,
4.400787] | 2.601935 [2.594394, 2.610129] | -40.75 [-41.14, -40.38] |
| multi_entry_int_maps/normalize_hash/map/wide_sparse_null | 4.341314
[4.327330, 4.355079] | 2.719642 [2.711363, 2.728375] | -37.30 [-37.57, -37.03] |
| multi_entry_int_maps/normalize_hash/struct_map_int/mixed_dense_null |
1.591679 [1.577916, 1.607271] | 0.886168 [0.882572, 0.889973] | -44.67 [-45.20,
-44.15] |
| multi_entry_int_maps/normalize_hash/struct_map_int/wide_dense_null |
3.350707 [3.333561, 3.369101] | 1.778045 [1.772774, 1.783062] | -46.87 [-47.19,
-46.54] |
| multi_entry_int_maps/normalize_hash/struct_map_int/wide_long_unicode |
99.514669 [98.494585, 100.582908] | 84.849294 [84.610544, 85.119287] | -14.74
[-15.68, -13.81] |
|
multi_entry_int_maps/normalize_hash/struct_map_int/wide_long_unicode_dense_null
| 62.983163 [62.488513, 63.466262] | 53.196420 [53.022391, 53.392616] | -15.54
[-16.25, -14.80] |
| multi_entry_int_maps/normalize_hash/struct_map_int/wide_no_null | 4.241714
[4.225298, 4.257163] | 2.605669 [2.598808, 2.612900] | -38.36 [-38.64, -38.06] |
| multi_entry_int_maps/normalize_hash/struct_map_int/wide_sparse_null |
4.326263 [4.307835, 4.343826] | 2.755664 [2.742788, 2.768494] | -36.54 [-36.86,
-36.19] |
| multi_entry_int_maps/normalize_only/map/mixed_dense_null | 1.049101
[1.040979, 1.058181] | 0.499449 [0.495715, 0.503455] | -52.68 [-53.16, -52.20] |
| multi_entry_int_maps/normalize_only/map/wide_dense_null | 2.553617
[2.516866, 2.588511] | 1.243265 [1.226862, 1.257867] | -51.60 [-52.10, -51.08] |
| multi_entry_int_maps/normalize_only/map/wide_long_unicode | 22.638445
[22.522588, 22.776090] | 21.227017 [21.140163, 21.322381] | -6.23 [-6.92,
-5.59] |
| multi_entry_int_maps/normalize_only/map/wide_long_unicode_dense_null |
22.743612 [22.675809, 22.808203] | 21.271406 [21.188210, 21.359706] | -6.47
[-6.94, -6.00] |
| multi_entry_int_maps/normalize_only/map/wide_no_null | 2.353425 [2.340957,
2.366710] | 1.125002 [1.116061, 1.136612] | -51.46 [-51.85, -51.11] |
| multi_entry_int_maps/normalize_only/map/wide_sparse_null | 2.451406
[2.431075, 2.468949] | 1.216252 [1.213643, 1.219300] | -50.40 [-50.68, -50.10] |
| regression_maps/hash_only/map/mixed_dense_null | 0.573734 [0.571174,
0.576098] | 0.515212 [0.513514, 0.517152] | -9.99 [-10.50, -9.45] |
| regression_maps/hash_only/map/mixed_long_unicode_dense_null | 39.475333
[39.143922, 39.776078] | 32.243070 [32.172971, 32.317813] | -18.32 [-18.96,
-17.61] |
| regression_maps/hash_only/map/mixed_no_null | 0.955752 [0.951690,
0.959291] | 0.861644 [0.858801, 0.864760] | -8.97 [-9.46, -8.47] |
| regression_maps/hash_only/map/mixed_sparse_null | 0.985686 [0.977372,
0.994366] | 0.881138 [0.878176, 0.883949] | -10.12 [-11.09, -9.17] |
| regression_maps/hash_only/map/singleton_dense_null | 0.012874 [0.012810,
0.012933] | 0.012009 [0.011962, 0.012059] | -7.20 [-7.97, -6.48] |
| regression_maps/hash_only/map/singleton_long_unicode | 5.413250 [5.360072,
5.466480] | 4.777156 [4.749153, 4.808462] | -11.75 [-12.75, -10.70] |
| regression_maps/hash_only/map/singleton_no_null | 0.034010 [0.033710,
0.034319] | 0.031366 [0.031325, 0.031407] | -7.47 [-8.05, -6.91] |
| regression_maps/hash_only/map/singleton_sparse_null | 0.042623 [0.042278,
0.042941] | 0.038719 [0.038635, 0.038820] | -7.62 [-8.26, -7.00] |
| regression_maps/hash_only/struct_map_int/mixed_dense_null | 0.620130
[0.612510, 0.627184] | 0.525686 [0.523824, 0.527695] | -13.82 [-15.11, -12.53] |
| regression_maps/hash_only/struct_map_int/mixed_long_unicode_dense_null |
38.143917 [37.726545, 38.572804] | 32.247516 [32.174597, 32.324397] | -15.46
[-16.42, -14.50] |
| regression_maps/hash_only/struct_map_int/mixed_no_null | 0.961241
[0.949367, 0.975726] | 0.871356 [0.869152, 0.873629] | -10.16 [-10.91, -9.47] |
| regression_maps/hash_only/struct_map_int/mixed_sparse_null | 1.002809
[0.995061, 1.011914] | 0.888110 [0.883873, 0.893059] | -13.26 [-14.24, -12.28] |
| regression_maps/hash_only/struct_map_int/singleton_dense_null | 0.021009
[0.020927, 0.021112] | 0.019654 [0.019601, 0.019726] | -6.74 [-7.34, -6.05] |
| regression_maps/hash_only/struct_map_int/singleton_long_unicode | 5.608737
[5.546328, 5.672366] | 4.744444 [4.718851, 4.775973] | -15.41 [-16.49, -14.30] |
| regression_maps/hash_only/struct_map_int/singleton_no_null | 0.042110
[0.041828, 0.042441] | 0.037817 [0.037783, 0.037849] | -10.15 [-10.58, -9.73] |
| regression_maps/hash_only/struct_map_int/singleton_sparse_null | 0.050949
[0.050603, 0.051328] | 0.044919 [0.044813, 0.045050] | -11.15 [-11.72, -10.64] |
| regression_maps/normalize_hash/map/mixed_dense_null | 1.932876 [1.913055,
1.953200] | 1.757228 [1.751421, 1.763408] | -10.82 [-11.78, -9.83] |
| regression_maps/normalize_hash/map/mixed_long_unicode_dense_null |
46.546517 [46.249115, 46.851328] | 41.980622 [41.879802, 42.081515] | -9.81
[-10.44, -9.19] |
| regression_maps/normalize_hash/map/mixed_no_null | 2.394949 [2.377064,
2.413906] | 2.086912 [2.078818, 2.096521] | -12.62 [-13.45, -11.71] |
| regression_maps/normalize_hash/map/mixed_sparse_null | 2.752169 [2.717999,
2.788445] | 2.406924 [2.398357, 2.416604] | -12.14 [-13.01, -11.28] |
| regression_maps/normalize_hash/map/singleton_dense_null | 0.084054
[0.083345, 0.084878] | 0.076710 [0.076291, 0.077178] | -9.23 [-10.47, -7.91] |
| regression_maps/normalize_hash/map/singleton_long_unicode | 5.707849
[5.679360, 5.739430] | 5.162268 [5.131169, 5.197831] | -9.56 [-10.29, -8.77] |
| regression_maps/normalize_hash/map/singleton_no_null | 0.098735 [0.097811,
0.099723] | 0.093007 [0.092184, 0.093867] | -6.39 [-7.46, -5.38] |
| regression_maps/normalize_hash/map/singleton_sparse_null | 0.137767
[0.136914, 0.138736] | 0.127877 [0.127098, 0.128788] | -7.30 [-7.89, -6.66] |
| regression_maps/normalize_hash/struct_map_int/mixed_dense_null | 1.936508
[1.919340, 1.955266] | 1.774254 [1.763450, 1.785962] | -8.48 [-9.69, -7.27] |
|
regression_maps/normalize_hash/struct_map_int/mixed_long_unicode_dense_null |
47.735977 [47.309989, 48.159968] | 41.824211 [41.712403, 41.938327] | -12.38
[-13.20, -11.57] |
| regression_maps/normalize_hash/struct_map_int/mixed_no_null | 2.393451
[2.370252, 2.415322] | 2.097543 [2.089294, 2.106787] | -13.07 [-13.94, -12.20] |
| regression_maps/normalize_hash/struct_map_int/mixed_sparse_null | 2.663930
[2.653059, 2.675174] | 2.441886 [2.417599, 2.468089] | -8.77 [-9.29, -8.25] |
| regression_maps/normalize_hash/struct_map_int/singleton_dense_null |
0.093762 [0.092784, 0.094827] | 0.084679 [0.084273, 0.085129] | -10.34 [-11.37,
-9.28] |
| regression_maps/normalize_hash/struct_map_int/singleton_long_unicode |
5.848648 [5.796031, 5.900309] | 5.130805 [5.112896, 5.149592] | -12.27 [-13.11,
-11.40] |
| regression_maps/normalize_hash/struct_map_int/singleton_no_null | 0.107473
[0.106733, 0.108489] | 0.098741 [0.098308, 0.099330] | -9.20 [-10.45, -7.81] |
| regression_maps/normalize_hash/struct_map_int/singleton_sparse_null |
0.153384 [0.151849, 0.154817] | 0.134456 [0.134169, 0.134739] | -10.76 [-11.74,
-9.79] |
| regression_maps/normalize_only/map/mixed_dense_null | 1.305613 [1.298508,
1.314473] | 1.364932 [1.346962, 1.381698] | 1.24 [-0.01, 2.62] |
| regression_maps/normalize_only/map/mixed_long_unicode_dense_null |
10.100908 [10.057039, 10.149230] | 10.152993 [10.067440, 10.245531] | 0.52
[-0.41, 1.51] |
| regression_maps/normalize_only/map/mixed_no_null | 1.238506 [1.233877,
1.244083] | 1.298261 [1.287412, 1.308777] | 4.77 [3.68, 5.84] |
| regression_maps/normalize_only/map/mixed_sparse_null | 1.670553 [1.632795,
1.702511] | 1.619063 [1.603978, 1.633042] | -1.04 [-2.56, 0.39] |
| regression_maps/normalize_only/map/singleton_dense_null | 0.065474
[0.065122, 0.065870] | 0.068979 [0.068489, 0.069474] | 5.41 [4.68, 6.14] |
| regression_maps/normalize_only/map/singleton_long_unicode | 0.395832
[0.393017, 0.398785] | 0.415664 [0.411426, 0.420134] | 6.29 [5.03, 7.58] |
| regression_maps/normalize_only/map/singleton_no_null | 0.061801 [0.061540,
0.062139] | 0.067296 [0.066293, 0.068235] | 6.76 [5.52, 7.97] |
| regression_maps/normalize_only/map/singleton_sparse_null | 0.093908
[0.092231, 0.095821] | 0.097075 [0.096292, 0.097936] | 6.11 [4.79, 7.32] |
| spark_map_sort/int_keys/0 | 0.006962 [0.006902, 0.007018] | 0.003254
[0.003215, 0.003289] | -53.25 [-53.78, -52.73] |
| spark_map_sort/int_keys/1 | 0.017960 [0.017881, 0.018045] | 0.019514
[0.019332, 0.019680] | 8.85 [8.08, 9.60] |
| spark_map_sort/int_keys/16 | 1.416743 [1.413385, 1.420491] | 1.538763
[1.525474, 1.552531] | 8.02 [7.03, 8.95] |
| spark_map_sort/int_keys/4 | 0.933755 [0.931875, 0.935690] | 1.010855
[1.005474, 1.016571] | 9.34 [8.70, 9.98] |
| spark_map_sort/int_keys/64 | 1.868838 [1.864112, 1.873809] | 2.045926
[2.001160, 2.093124] | 10.00 [8.59, 11.47] |
| spark_map_sort/string_keys/0 | 0.006997 [0.006971, 0.007029] | 0.003286
[0.003268, 0.003302] | -53.16 [-53.53, -52.78] |
| spark_map_sort/string_keys/1 | 0.035552 [0.035431, 0.035720] | 0.040330
[0.039887, 0.040847] | 12.93 [11.97, 13.89] |
| spark_map_sort/string_keys/16 | 3.999255 [3.988036, 4.012051] | 3.193286
[3.172456, 3.213719] | -20.15 [-20.70, -19.60] |
| spark_map_sort/string_keys/4 | 1.126407 [1.120661, 1.133037] | 0.306869
[0.302937, 0.311142] | -73.38 [-73.72, -73.04] |
| spark_map_sort/string_keys/64 | 4.220635 [4.200750, 4.243675] | 3.287736
[3.268866, 3.306038] | -22.10 [-22.74, -21.54] |
| spark_map_sort/unsupported_singleton_struct_key | 0.000450 [0.000448,
0.000452] | 0.000450 [0.000447, 0.000454] | 0.22 [-0.68, 1.17] |
</details>
<details>
<summary>Fresh independent pairs and reverse-order confirmations</summary>
| Case | Fresh baseline ms (95% CI) | Fresh integrated ms (95% CI) | A/B
change % | B/A change % |
|---|---:|---:|---:|---:|
| matched_maps/hash_only/map/10/reversed | 0.307597 [0.306905, 0.308324] |
0.308063 [0.307327, 0.308836] | 0.15 | — |
| matched_maps/hash_only/map/50/forward | 2.043116 [2.027317, 2.061498] |
2.023694 [2.015670, 2.032417] | -0.95 | — |
| matched_maps/hash_only/map/50/reversed | 2.031379 [2.021587, 2.042563] |
2.033373 [2.025952, 2.041082] | 0.10 | — |
| matched_maps/hash_only/struct_map_int/1/reversed | 0.035797 [0.035662,
0.035952] | 0.035960 [0.035927, 0.035994] | 0.46 | — |
| matched_maps/hash_only/struct_map_int/10/reversed | 0.322155 [0.321483,
0.322876] | 0.323318 [0.322561, 0.324042] | 0.36 | — |
| matched_maps/hash_only/struct_map_int/50/reversed | 2.060097 [2.054241,
2.066084] | 2.059854 [2.051329, 2.069160] | -0.01 | — |
| matched_maps/normalize_only/map/1/forward | 0.044923 [0.044822, 0.045029]
| 0.044885 [0.044768, 0.045009] | -0.08 | — |
| matched_maps/normalize_only/map/1/reversed | 0.045150 [0.045023, 0.045276]
| 0.045925 [0.045612, 0.046272] | 1.72 | — |
| matched_maps/normalize_only/map/50/forward | 2.837147 [2.834211, 2.840324]
| 1.718847 [1.713643, 1.724722] | -39.42 | — |
| matched_maps/normalize_only/map/50/reversed | 3.585526 [3.575871,
3.598663] | 2.419840 [2.417011, 2.422764] | -32.51 | — |
| regression_maps/normalize_only/map/mixed_no_null | 1.202431 [1.194928,
1.210687] | 1.190024 [1.180861, 1.200472] | -1.03 | — |
| regression_maps/normalize_only/map/singleton_dense_null | 0.064859
[0.064683, 0.065039] | 0.064748 [0.064412, 0.065114] | -0.17 | — |
| regression_maps/normalize_only/map/singleton_long_unicode | 0.372372
[0.371965, 0.372786] | 0.373767 [0.373339, 0.374249] | 0.37 | — |
| regression_maps/normalize_only/map/singleton_no_null | 0.061337 [0.061022,
0.061699] | 0.061878 [0.061588, 0.062208] | 0.88 | — |
| regression_maps/normalize_only/map/singleton_sparse_null | 0.092175
[0.091949, 0.092419] | 0.092645 [0.092393, 0.092924] | 0.51 | — |
| spark_map_sort/int_keys/1 | 0.017857 [0.017817, 0.017899] | 0.017066
[0.016999, 0.017147] | -4.43 | — |
| spark_map_sort/int_keys/16 | 1.393109 [1.390020, 1.396776] | 1.441855
[1.439159, 1.444649] | 3.50 | 1.99 |
| spark_map_sort/int_keys/4 | 0.939643 [0.936180, 0.943118] | 0.996864
[0.994855, 0.999169] | 6.09 | 0.11 |
| spark_map_sort/int_keys/64 | 1.859120 [1.853594, 1.865412] | 1.741639
[1.736799, 1.747314] | -6.32 | — |
| spark_map_sort/string_keys/1 | 0.035479 [0.035395, 0.035553] | 0.035450
[0.035257, 0.035684] | -0.08 | — |
</details>
<details>
<summary>Benchmark executable identities (SHA-256)</summary>
- baseline `map_sort`:
`d93a75c7df21c0b57fb994664c6024b8aa07bea85939f08d7b0e6279689c6e8f`
- baseline `hash`:
`05739bd6b3ef97c63ed814943cd994e205212abb9a95c8484c977ebdc77e3207`
- integrated `map_sort`:
`47a0a73bbedfd8c6339c0e3626d1d0af6ffb9e6fdee679c8c26896905147f7a3`
- integrated `hash`:
`ccc68e4d95296c241e222a7c5a552c8a57c30b5365fccbe72ff097a5400e7dd3`
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
