Niltra opened a new pull request, #55345:
URL: https://github.com/apache/arrow/pull/55345
### Rationale for this change
`HivePartitioning::FormatValues` always URI-encoded partition values, even
when the partitioning was created with `segment_encoding="none"`. The option
therefore only affected parsing: writing `My Fancy Product` still produced
`part=My%20Fancy%20Product`, as reported in GH-34905 and GH-41618.
### What changes are included in this PR?
- `HivePartitioning::FormatValues` now honours `segment_encoding`: `Uri`
keeps the current behaviour, `None` writes values verbatim.
- C++ tests for formatting and for writing a dataset with spaces and `%` in
the partition value.
- Python test for `ds.write_dataset` with both encodings, including
read-back.
- Clarified the `HivePartitioning.segment_encoding` docstring.
The default remains `"uri"`, so existing output is unchanged.
### Are these changes tested?
- `arrow-dataset-partition-test`: 38/38 pass, including
`HivePartitioningFormat` and the new `WriteHiveWithoutUriEncoding`.
- `test_write_dataset_hive_segment_encoding`: 2/2 pass against a local
pyarrow built from this branch and the patched Arrow C++. Against pyarrow
26.0.0 the `"none"` case fails as expected, which reproduces the issue.
- `clang-format` 18.1.8, `cpplint` 1.6.1 and `flake8` pass on the touched
files.
### Are there any user-facing changes?
Yes, a bug fix. This now works, without changing the default:
```python
import pyarrow.dataset as ds
import pyarrow.parquet as pq
partitioning = ds.HivePartitioning(
table.select(partition_cols).schema,
segment_encoding="none",
)
pq.write_to_dataset(table, path, partitioning=partitioning, filesystem=fs)
```
`part=My Fancy Product` is written as-is; the default `"uri"` still writes
`part=My%20Fancy%20Product`.
### Was AI used for this PR?
**PR code and description written by:**
- [x] AI
**Reviewed before submission by:**
- [ ] Human
- [x] AI
- [ ] Not reviewed
The change is limited to the existing `segment_encoding` option. The tests
listed above were run locally against the patched C++ and Python builds before
opening this PR.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
