Ji Xu created ARROW-3947:
----------------------------
Summary: query distinct values of a given partition from a
ParquetDataset
Key: ARROW-3947
URL: https://issues.apache.org/jira/browse/ARROW-3947
Project: Apache Arrow
Issue Type: New Feature
Components: Python
Affects Versions: 0.10.0
Environment: MacOSX, Python 3.6,
Reporter: Ji Xu
Right now the values of a given partition from a `ParquetDataset` is buried
inside `ParquetDataset.pieces`, a bit inconvenient for the user to dig out this
information. A helper function/method to perform this task in `ParquetDataset`
class would be very helpful for the users.
A pure personal opinion on the name of this method:
`ParquetDataset.select_distinct()` with partition_name as the positional arg,
to resemble SQL `SELECT DISTINCT column FROM table`.
I'm not sure how to contribute here on Jira, so I created this [GitHub Gist
|https://gist.github.com/xujiboy/c3fcc47f720ed9adf2260c5d0ba8aed2]as an
possible solution for this problem.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
