Copilot commented on code in PR #51392:
URL: https://github.com/apache/arrow/pull/51392#discussion_r4047848622
##########
r/R/dataset.R:
##########
@@ -64,6 +64,25 @@
#' column types, as described above. If neither are provided, no partitioning
#' information will be taken from the file paths.
#'
+#' @section Adding the source filename as a column:
+#'
+#' Partitioning only recovers information encoded in directory names. If you
+#' need to know which file each row came from, call [add_filename()] inside a
+#' `dplyr` query on the dataset:
+#'
+#' ```r
+#' open_dataset("nyc-taxi") |>
+#' mutate(file = add_filename()) |>
+#' collect()
+#' ```
+#'
+#' This is useful, for example, when you have opened a subdirectory of a
+#' partitioned dataset directly (so the partition columns are not inferred) and
+#' want to recover the partition values from the path. `add_filename()` can
only
+#' be used inside a query on a `Dataset`, and the resulting column cannot be
+#' referenced by later steps of the same query until you call `compute()` or
+#' `collect()`. See [add_filename()] for details.
Review Comment:
This broadens the documented input from `FileSystemDataset` to the generic
`Dataset`, which also includes `InMemoryDataset` and `UnionDataset`, while the
`add_filename()` help above explicitly restricts the helper to
`FileSystemDataset`. Please keep this restriction consistent (or document the
behavior for the other subclasses) so users are not told that every `Dataset`
can provide a source filename.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
