[
https://issues.apache.org/jira/browse/ARROW-2422?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16447956#comment-16447956
]
ASF GitHub Bot commented on ARROW-2422:
---------------------------------------
jneuff commented on a change in pull request #1861: ARROW-2422: Support more
operators for partition filtering
URL: https://github.com/apache/arrow/pull/1861#discussion_r183355089
##########
File path: python/pyarrow/parquet.py
##########
@@ -864,12 +864,23 @@ def filter_accepts_partition(part_key, filter, level):
if p_column != f_column:
return True
- f_value_index = self.partitions.get_index(level, p_column,
- str(f_value))
+ p_value = (self.partitions
+ .levels[level]
+ .dictionary[p_value_index]
+ .as_py())
+
if op == "=":
- return f_value_index == p_value_index
+ return p_value == f_value
elif op == "!=":
- return f_value_index != p_value_index
+ return p_value != f_value
+ elif op == '<':
+ return p_value < f_value
+ elif op == '>':
+ return p_value > f_value
+ elif op == '<=':
+ return p_value <= f_value
+ elif op == '>=':
+ return p_value >= f_value
else:
return True
Review comment:
I take that as a yes.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Support more filter operators on Hive partitioned Parquet files
> ---------------------------------------------------------------
>
> Key: ARROW-2422
> URL: https://issues.apache.org/jira/browse/ARROW-2422
> Project: Apache Arrow
> Issue Type: Improvement
> Reporter: Julius Neuffer
> Priority: Minor
> Labels: features, pull-request-available
>
> After implementing basic filters ('=', '!=') on Hive partitioned Parquet
> files (ARROW-2401), I'll extend them ('>', '<', '<=', '>=') with a new PR on
> Github.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
