Ryan Hafen created ARROW-14149:
----------------------------------
Summary: Support a "modified" hive style directory naming scheme
Key: ARROW-14149
URL: https://issues.apache.org/jira/browse/ARROW-14149
Project: Apache Arrow
Issue Type: Improvement
Reporter: Ryan Hafen
I am working on a project where I need to create and analyze parquet files
using Apache Arrow but the environment I'm working with does not allow "=" in
file paths, which the hive naming convention forces, e.g. "year=2007". While I
can specify the partitioning to not use the hive contention, I then lose the
variable names. This is problematic when I'm sharing the datasets with others
because they will have to specify the partitioning variables when opening the
dataset but they don't know what the partitioning variables are.
Would it be possible to allow a modified hive-style directory naming convention
that still preserves the variable name in the directory name? For example,
allowing a delimiter other than "="?
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
