mkzung opened a new issue, #50474: URL: https://github.com/apache/arrow/issues/50474
### Describe the bug, including details regarding any error messages, version, and platform. When building a dictionary array from Python values, `pa.array` silently ignores an unsigned index type and hands back the signed one instead: ```python import pyarrow as pa dt = pa.dictionary(pa.uint32(), pa.string()) arr = pa.array(["a", "b", "a"], type=dt) print(arr.type) # dictionary<values=string, indices=int32, ordered=0> print(arr.type == dt) # False ``` The signed index types are honoured. The unsigned ones are all quietly downgraded to their signed counterpart: | requested index type | resulting index type | | --- | --- | | int8 / int16 / int32 / int64 | same (honoured) | | uint8 | int8 | | uint16 | int16 | | uint32 | int32 | | uint64 | int64 | Nothing warns or raises, so you end up with an array whose type is not the type you asked for. The knock-on effect is more confusing than the original problem. `pa.chunked_array` takes a type argument too, and since the chunk it builds internally comes back with the signed index type, the type check fails with an error that never mentions dictionaries or index types: ```python >>> pa.chunked_array([["a", "b", "a"]], pa.dictionary(pa.uint32(), pa.string())) ArrowTypeError: Array chunks must all be same type ``` The same call with a signed index type is fine: ```python >>> pa.chunked_array([["a", "b", "a"]], pa.dictionary(pa.int32(), pa.string())).type dictionary<values=string, indices=int32, ordered=0> ``` And casting does honour unsigned index types, so they are clearly supported elsewhere: ```python >>> pa.chunked_array([["a", "b", "a"]]).cast(pa.dictionary(pa.uint32(), pa.string())).type dictionary<values=string, indices=uint32, ordered=0> ``` I would expect `pa.array(values, type=...)` either to give back the index type that was asked for, or to raise if it cannot, rather than silently returning a different one. The `chunked_array` message could be clearer too, but that looks like a symptom rather than the cause. This came up in narwhals, whose `Categorical` dtype maps to `dictionary(uint32, string)`, so building a Categorical column from Python values fails on the PyArrow backend: https://github.com/narwhals-dev/narwhals/issues/3775 Versions: pyarrow 23.0.1, Python 3.10.11, macOS arm64. ### Component(s) Python -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
