Will Jones created ARROW-15317:
----------------------------------
Summary: [R] Expose API to create Dataset from Fragments
Key: ARROW-15317
URL: https://issues.apache.org/jira/browse/ARROW-15317
Project: Apache Arrow
Issue Type: Improvement
Components: R
Affects Versions: 6.0.1
Reporter: Will Jones
Third-party packages may define dataset factories for table formats like Delta
Lake and Apache Iceberg. These formats store metadata like schema, file lists,
and file-level statistics on the side, and can construct a dataset without a
discovery process needed. Python exposed enough API to do this successfully for
[a Delta Lake dataset reader
here|https://github.com/delta-io/delta-rs/blob/6a8195d6e3cbdcb0c58a14a3ffccc472dd094de0/python/deltalake/table.py#L267-L280].
I propose adding the following to the R API:
* Expose {{Fragment}} as an R6 object
* Add the {{MakeFragment}} method to various file format objects. It's key
that {{partition_expression}} is included as an argument. ([See Python
equivalent
here|https://github.com/apache/arrow/blob/ab86daf3f7c8a67bee6a175a749575fd40417d27/python/pyarrow/_dataset_parquet.pyx#L209-L210])
* Add a dataset constructor that takes a list of {{Fragments}}
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
