pitrou commented on PR #51325:
URL: https://github.com/apache/arrow/pull/51325#issuecomment-5792616634
> As the result of this discussion is that float values should be merged
only when they are identical in their physical representation, I suggest adding
another flag to `arrow::EqualOptions` to compare float values based on their
bit pattern in a separate PR, then applying this comparison to the REE compute
kernel for nested types in another PR, and to the REE builder here.
I'm not sure. I think we can instead special-case all fixed-width types for
the purpose of comparing REE values. This can also make the REE builder faster
than allocating a new Scalar for every logical value.
Something like this in ree_util.h (untested, just a sketch):
```c++
struct PhysicalValue {
friend bool operator==(const PhysicalValue& left, const PhysicalValue&
right) {
return ...;
}
static PhysicalValueForComparison LookupArray(const Array& array, int64_t
index) {
if (array.IsNull(index)) {
return {std::shared_ptr<Scalar>{}};
}
auto typed_lookup = [&](auto* concrete_type) {
using ArrowType = std::decay_t<decltype(*concrete_type)>;
if constexpr(is_fixed_width(ArrowType::id) &&
!is_boolean(ArrowType::id)) {
const int64_t byte_width = concrete_type->byte_width();
const ArrayData& data = *array.data();
return {data.buffers[1]->span_as<uint8_t>.subspan(
(index + data.offset) * byte_width, byte_width)};
} else {
return array.GetScalar(index);
}
};
return VisitType(*array.type(), typed_lookup);
}
// Three possible states:
// - A span of bytes for a non-null non-boolean primitive value
// - A nullptr Scalar for a null value
// - A non-null generic Scalar otherwise
std::variant<std::shared_ptr<Scalar>, std::span<const uint8_t>> payload_;
};
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
