xsa-dev opened a new pull request, #18571:
URL: https://github.com/apache/datafusion/pull/18571
## Which issue does this PR close?
- Closes #18563
## Rationale for this change
This PR addresses the missing `RunEndEncoded` variant in the `ScalarValue`
enum, which is essential for proper type coercion and operations involving
RunEndEncoded arrays. RunEndEncoded (REE) is an Arrow data type for run-length
encoding of repeated values, and without proper `ScalarValue` support,
operations involving REE arrays would fail to work correctly.
The implementation follows the existing patterns in the DataFusion codebase
and ensures compatibility with the Arrow specification for RunEndEncoded arrays.
## What changes are included in this PR?
1. **Added `RunEndEncoded` variant to `ScalarValue` enum** in
`datafusion/common/src/scalar/mod.rs`
- New variant: `RunEndEncoded(Arc<RunEndEncodedScalar>)`
- Positioned correctly within the enum definition
2. **Implemented `RunEndEncodedScalar` struct** with required trait
implementations:
- `PartialEq` and `Eq` for value equality comparisons
- `Hash` for hash-based operations
- `PartialOrd` for ordering operations
- `Debug` and `Clone` for debugging and copying
3. **Updated existing trait implementations** to handle the new variant:
- `PartialEq` for `ScalarValue`
- `PartialOrd` for `ScalarValue`
- `Hash` for `ScalarValue`
4. **Added support for null values** in `try_new_null()` method
5. **Comprehensive test coverage** with unit tests covering:
- Value equality and inequality
- Hash consistency
- Partial ordering
- Null value handling
- Type safety
## Are these changes tested?
Yes, this PR includes comprehensive test coverage:
- **Unit tests** for `RunEndEncodedScalar` covering all trait implementations
- **Integration tests** for `ScalarValue` enum operations with the new
variant
- **Null value tests** to ensure proper handling of null RunEndEncoded
scalars
- **Edge case tests** for various value types and comparison scenarios
All tests pass successfully and the implementation maintains compatibility
with existing functionality.
## Are there any user-facing changes?
Yes, this PR introduces user-facing changes by extending the public API:
**New Public API:**
- `ScalarValue::RunEndEncoded(Arc<RunEndEncodedScalar>)` - New enum variant
- `RunEndEncodedScalar` struct with public `run_ends` and `values` fields
**User Impact:**
- ✅ **Positive**: Users can now work with RunEndEncoded scalars in
DataFusion queries
- ✅ **Backward Compatible**: Existing code continues to work unchanged
- ✅ **Type Safe**: Proper type support for REE operations
**No Breaking Changes:**
- All existing APIs remain unchanged
- No modifications to public method signatures
- No changes to existing behavior
The changes follow DataFusion's API evolution guidelines and are fully
backward compatible.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
