[
https://issues.apache.org/jira/browse/ARROW-13472?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17388735#comment-17388735
]
Jonathan Keane commented on ARROW-13472:
----------------------------------------
Thanks for this and putting it into these words. I agree that the {{.engine}}
argument while convenient, is awkward and departs from the experience of
{{summarise}} and the rest of the other dplyr verbs like you mention. That was
the major motivation in creating the {{to_duckdb}} function.
Though in that PR (this was in the original description + you can see it in the
commit history, though both are relatively hidden now), I came to the
conclusion that neither {{collect()}} nor {{compute()}} was a good match for
what we were doing where a person could pass an Arrow object to duckdb (or
other system) in the middle of a pipeline. The biggest reason being that both
{{collect()}} and {{compute()}} imply that work is being done / executed at
that point which for this it is not. I toyed with the idea of calling this
{{alchemise()}} (or having a family like {{alchemise_to_duckdb()}} so that one
knows what the function returns from the name + it is consistent), so that one
could do something like the following:
{code:r}
ds <- InMemoryDataset$create(example_data)
ds %>%
alchemize(to = "duckdb") %>%
group_by(lgl) %>%
summarise(mean_int = mean(int, na.rm = TRUE), mean_dbl = mean(dbl, na.rm =
TRUE)) %>%
alchemize(to = "arrow") %>%
collect()
{code}
> [R] Redesign the UX for using the DuckDB engine
> -----------------------------------------------
>
> Key: ARROW-13472
> URL: https://issues.apache.org/jira/browse/ARROW-13472
> Project: Apache Arrow
> Issue Type: Improvement
> Components: R
> Reporter: Ian Cook
> Priority: Major
> Fix For: 6.0.0
>
>
> ARROW-12688 added:
> * A new function {{to_duckdb()}} which registers an Arrow Dataset with
> DuckDB and returns a dbplyr object that can be used in dplyr pipelines
> * An {{.engine = "duckdb"}} option in the {{summarise()}} function which
> calls {{to_duckdb()}} inside {{summarise()}}
> At the moment, the latter is very convenient because {{summarise()}} is not
> yet natively supported for Arrow Datasets.
> However, this {{.engine = "duckdb"}} option is probably not such a great
> design for how users should interact with the arrow package in the longer
> term after native {{summarise()}} support is added. At that point, it will
> seem strange that this one particular dplyr verb has an {{.engine}} option
> while the others do not. Adding the option to all the other dplyr verbs also
> seems like a poor UX design.
> Consider whether we should ultimately have users choose whether to use the
> Arrow C++ engine or the DuckDB engine by passing an {{.engine}} argument to
> the {{collect()}} or {{compute()}} function, as [~jonkeane] suggested in
> these comments. {{collect()}} would return a tibble whereas {{compute()}}
> would return an Arrow Table.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
