[
https://issues.apache.org/jira/browse/ARROW-3391?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17065798#comment-17065798
]
Uwe Korn commented on ARROW-3391:
---------------------------------
Have a look at the failing tests in
[https://github.com/apache/arrow/blob/master/python/pyarrow/tests/test_parquet.py#L1646-L1654]
My problem is that I have a binary column with UUIDs (low entropy), there can
be a zero-byte at any position inside the ID. When I now filter on this ID,
e.g. "a\0dfsgjzdsaf" there were some steps that converted the value to C-style
strings and thus in turn to a simple "a" instead of the whole identifier.
> [Python] Support \0 characters in binary Parquet predicate values
> -----------------------------------------------------------------
>
> Key: ARROW-3391
> URL: https://issues.apache.org/jira/browse/ARROW-3391
> Project: Apache Arrow
> Issue Type: Improvement
> Components: Python
> Reporter: Uwe Korn
> Priority: Major
> Labels: dataset, dataset-parquet-read, parquet
>
> As we convert the predicate values of a Parquet filter in some intermediate
> steps to C-style strings, we currently disallow the use of binary and string
> predicate values that contain {{\0}} bytes as they would otherwise result in
> wrong results.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
