[
https://issues.apache.org/jira/browse/ARROW-3391?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17065864#comment-17065864
]
Joris Van den Bossche commented on ARROW-3391:
----------------------------------------------
Thanks for the clarification!
So for the tests you link to, the expected result is an empty table, right?
(since the filter will not find any matching value)
If I change the test that way, the tests pass when using the Datasets API (see
https://github.com/apache/arrow/pull/6303/commits/0f16f99a84b5f2ec85896d2c78db7eaae71b8b34).
But to properly test this, I should maybe also add a table that actually has
such values with nulls in the table, to check we can also filter _with_ a match.
> [Python] Support \0 characters in binary Parquet predicate values
> -----------------------------------------------------------------
>
> Key: ARROW-3391
> URL: https://issues.apache.org/jira/browse/ARROW-3391
> Project: Apache Arrow
> Issue Type: Improvement
> Components: Python
> Reporter: Uwe Korn
> Priority: Major
> Labels: dataset, dataset-parquet-read, parquet
>
> As we convert the predicate values of a Parquet filter in some intermediate
> steps to C-style strings, we currently disallow the use of binary and string
> predicate values that contain {{\0}} bytes as they would otherwise result in
> wrong results.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
