[
https://issues.apache.org/jira/browse/ARROW-14027?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17416919#comment-17416919
]
David Li commented on ARROW-14027:
----------------------------------
[~bkietz] and/or [~michalno] do either of you have ideas on how best to handle
scalars in GrouperFastImpl? (GrouperImpl is simple enough to extend.) From what
I understand of the implementation, GrouperFastImpl works with pointers to
array buffers, which would be difficult to do given scalars. We could:
# Update the group by node to give accurate shape information to
Grouper::Make, and have Grouper::Make fall back to the non-fast impl if there
are scalar arguments (for now)
# Expand scalars to arrays and continue processing
# …Slice the input batch into 1-row batches and trying to fake the array
buffer layout from the scalar data? (Not sure if this is even worth it
performance-wise)
Since to me, it looks like directly supporting scalars here will be
difficult/require some invasive refactoring.
> [C++][R] Ensure groupers accept scalar inputs (was: Allow me to group_by +
> summarise() with partitioning fields)
> ----------------------------------------------------------------------------------------------------------------
>
> Key: ARROW-14027
> URL: https://issues.apache.org/jira/browse/ARROW-14027
> Project: Apache Arrow
> Issue Type: Bug
> Components: C++, R
> Reporter: Jonathan Keane
> Assignee: David Li
> Priority: Blocker
> Labels: kernel
> Fix For: 6.0.0
>
>
> If one puts a field that is one of the partitioning variables in
> {{group_by()}} and then summarises, we get a segfault:
> {code:r}
> library(arrow)
> library(dplyr)
> temp <- tempfile()
> write_dataset(mtcars, path = temp, partitioning = "cyl")
> ds <- open_dataset(temp)
> # this works just fine
> ds %>%
> group_by(gear) %>%
> summarise(
> sum(mpg)
> ) %>%
> collect()
> # however this segfaults (regardless of the aggregation, even simply n())
> # *** caught segfault ***
> # address 0x0, cause 'memory not mapped'
> ds %>%
> group_by(cyl) %>%
> summarise(
> sum(mpg)
> ) %>%
> collect()
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
