gyanranjanpanda commented on code in PR #9373:
URL: https://github.com/apache/arrow-rs/pull/9373#discussion_r2786453766
##########
arrow-select/src/dictionary.rs:
##########
@@ -275,12 +275,12 @@ pub(crate) fn merge_dictionary_values<K:
ArrowDictionaryKeyType>(
for (value_idx, value) in values {
mapping[value_idx] =
- *interner.intern(value, || match
K::Native::from_usize(indices.len()) {
- Some(idx) => {
- indices.push((dictionary_idx, value_idx));
- Ok(idx)
- }
- None => Err(ArrowError::DictionaryKeyOverflowError),
+ *interner.intern(value, || -> Result<K::Native,
ArrowError> {
Review Comment:
test_concat_u8_dictionary_256_values
Passes with exactly 256 unique values (this previously failed).
test_concat_u8_dictionary_257_values_fails
Correctly returns an error when exceeding the u8 boundary (257 values).
test_concat_u16_dictionary_65536_values
Passes with exactly 65,536 unique values (this previously failed).
test_concat_returns_error_not_panic
Verifies that overflow results in a proper ArrowError instead of a panic.
Let me know if you'd like any additional edge cases covered.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
