fornwall opened a new issue, #50512:
URL: https://github.com/apache/arrow/issues/50512
### Describe the enhancement requested
The hash-based vector functions do not support `halffloat` input, while
supporting all other numeric types:
```python
>>> import pyarrow as pa
>>> pa.array([1.5, 2.5], type=pa.float16()).dictionary_encode()
ArrowNotImplementedError: Function 'dictionary_encode' has no kernel
matching input types (halffloat)
```
`unique` and `value_counts` fail the same way. Reproduced on current `main`;
platform-independent (missing kernel registration, not a platform issue).
The cause is that `AddHashKernels()` in
`cpp/src/arrow/compute/kernels/vector_hash.cc` registers kernels by iterating
`PrimitiveTypes()`, which excludes float16 (`FloatingPointTypes()` is only
`{float32(), float64()}`), and `GetHashInit()` has no `Type::HALF_FLOAT` case.
This was previously part of the broader GH-43017 ("Make the set of casts and
hash kernels involving float16 consistent with other floating types", closed as
stale); its PR #43018 stalled on the cast/sort parts, not the hash-kernel part.
This issue proposes fixing only the hash kernels, registering float16
explicitly the same way recent merged float16 additions did (#46446 for
if_else/replace, #46286 for run_end_encode), i.e. without widening
`FloatingPointTypes()`/`NumericTypes()`/`PrimitiveTypes()`, whose other
consumers are templated on the `uint16_t` storage type and would miscompute for
half floats.
Like float32 (hashed via `UInt32Type`) and float64 (via `UInt64Type`),
float16 would hash by its raw bit pattern via `UInt16Type`; NaN/`-0.0`
semantics remain as for the other float types (ARROW-4124).
### Component(s)
C++
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
