[ 
https://issues.apache.org/jira/browse/ARROW-2401?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16428276#comment-16428276
 ] 

ASF GitHub Bot commented on ARROW-2401:
---------------------------------------

xhochy commented on a change in pull request #1840: ARROW-2401 Support filters 
on Hive partitioned Parquet files
URL: https://github.com/apache/arrow/pull/1840#discussion_r179750293
 
 

 ##########
 File path: python/pyarrow/parquet.py
 ##########
 @@ -849,6 +857,31 @@ def open_file(path, meta=None):
                                    common_metadata=self.common_metadata)
         return open_file
 
+    def _filter(self, filters):
+        def filter_accepts_partition(part_key, filter, level):
+            p_column, p_value_index = part_key
+            f_column, op, f_value = filter
+            if p_column != f_column:
+                return True
+
+            f_value_index = self.partitions.get_index(level, p_column,
+                                                      str(f_value))
+            if op == "=":
 
 Review comment:
   Clarified offline with @jneuff that we will refactor 
`self.partitions.get_index` a bit to support more operation than the two above.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
[email protected]


> Support filters on Hive partitioned Parquet files
> -------------------------------------------------
>
>                 Key: ARROW-2401
>                 URL: https://issues.apache.org/jira/browse/ARROW-2401
>             Project: Apache Arrow
>          Issue Type: New Feature
>          Components: Python
>            Reporter: Julius Neuffer
>            Priority: Minor
>              Labels: features, pull-request-available
>
> I'll open a PR on GitHub to support filtering of a `ParquetDataset` along a 
> Hive partitioned directory structure.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Reply via email to