[
https://issues.apache.org/jira/browse/ARROW-4492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16833590#comment-16833590
]
Joris Van den Bossche commented on ARROW-4492:
----------------------------------------------
[~gsakkis] What error did you get exactly?
The example is working for me with latest arrow and dask 1.2.0:
{code}
In [29]: dd.read_parquet('../Downloads/slug.pq', categories=['slug'],
engine='pyarrow').compute()['slug'].dtype
Out[29]:
CategoricalDtype(categories=['6c23b4eae7c54e47ba1c9d1e04d0fb4b',
'4c5b2fcbc58e4c6989aeafe38f705a5e',
'8415c60d37214b6aa960ec85a63b3ed4', '74602', '59355',
'baeceb3ab1714bc4998ef7040f3b1581', '17770',
'f5be8182c20245aa9967e920212580d8',
'd03f30f7972447d5869d73bdc5e18edd', '40617',
...
'agltb3B1Yi1pbmNyEAsSB0FjY291bnQYyM2fFQw', '79196',
'1100002442', '1230649452a446e39832917c012abae3',
'1100004740', '5c084c81cac845f883bd47f030785d14',
'dbbd91a00ca0410cbecedc8e2f911644',
'75e19f75f90345c0b6195f7658b09e23',
'63512d58c60d4146919dc792c9c4dd84',
'085cfd8e8f5f11e281c11231392559e4'],
ordered=False)
{code}
> [Python] Failure reading Parquet column as pandas Categorical in 0.12
> ---------------------------------------------------------------------
>
> Key: ARROW-4492
> URL: https://issues.apache.org/jira/browse/ARROW-4492
> Project: Apache Arrow
> Issue Type: Bug
> Components: Python
> Affects Versions: 0.12.0
> Reporter: George Sakkis
> Priority: Major
> Labels: Parquet
> Fix For: 0.14.0
>
> Attachments: slug.pq
>
>
> On pyarrow 0.12.0 some (but not all) columns cannot be read as category
> dtype. Attached is an extracted failing sample.
> {noformat}
> import dask.dataframe as dd
> df = dd.read_parquet('slug.pq', categories=['slug'],
> engine='pyarrow').compute()
> print(len(df['slug'].dtype.categories))
> {noformat}
> This works on pyarrow 0.11.1 (and fastparquet 0.2.1).
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
