[
https://issues.apache.org/jira/browse/ARROW-16632?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17552101#comment-17552101
]
Alessandro Molina commented on ARROW-16632:
-------------------------------------------
Analytics functions in PyArrow all use Acero under the hood (filtering,
joining, grouping/aggrs).
At the moment Acero is never mentioned on purpose in Python and R documentation
because we don't have a clear story about what we actually want to expose to
end users. In both Python and R acero is a private api that users can't access
directly and thus they are undocumented.
The idea is that the interface to Acero in R will be dyplr while in Python
there is the intention to make a DataFrame like API and defer to IBIS the lazy
evaluation use case. In both cases we won't expose Acero as a public API and
the idea is to communicate with Acero only through substrait on long term
(That's already being implemented in R, and will be done for Python in the near
future)
To wrap-up, I think at the moment we could talk about Acero as the engine
powering dyplr in R and Analytics capabilities in Python. And we should talk
about Acero itself only from the C++ point of view as probably the R and Python
bindings will never expose it to the public.
PS: If we want to start promoting Acero to the world, I think we should work on
improving a bit the documentation first. Having a blog post that then redirects
people to a docs that they find hard to read/apply might actually be
counterproductive as it might create a fame of being badly documented.
At the moment the only mention of it is
https://arrow.apache.org/docs/cpp/streaming_execution.html and it's not very
easy to follow (not much explainations, just blocks of code). In comparison if
you look at the compute chapter in Python (
https://arrow.apache.org/docs/dev/python/compute.html ) it's much more
talkative and explains things as it goes.
> [Website] Announce Acero Engine
> -------------------------------
>
> Key: ARROW-16632
> URL: https://issues.apache.org/jira/browse/ARROW-16632
> Project: Apache Arrow
> Issue Type: Improvement
> Components: Website
> Reporter: Will Jones
> Priority: Major
>
> Given consensus on Acero as the name for C++ streaming execution engine, it
> may be time to write a blog post announcing the engine, how it's currently
> available in the ecosystem, and what's happening next with it.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
