thisisnic commented on issue #45601:
URL: https://github.com/apache/arrow/issues/45601#issuecomment-2907804636
Another question: what format is the original dataset? I also tried
comparing Arrow IPC (Feather) and Parquet, and it seems that when I supply a
schema which uses the types that Arrow *can* work with, then it works for IPC
files.
I'm not suggesting you move to this format, but just that there might be
more we can look into around getting the functionality more aligned there.
``` r
library(haven)
library(arrow)
library(tibble)
library(dplyr)
d <- tibble(
a = labelled(x = 1:5, label = "example variable a"),
b = labelled(x = 11:15, label = "example variable b")
)
tf1 <- tempfile()
write_feather(d, tf1)
tf2 <- tempfile()
write_parquet(d, tf2)
open_dataset(tf1, format = "feather") %>%
filter(a > 3) %>%
collect()
#> Error in `compute.arrow_dplyr_query()`:
#> ! NotImplemented: Function 'greater' has no kernel matching input types
(<labelled<integer>[0]>: example variable a, <labelled<integer>[0]>: example
variable a)
```
``` r
open_dataset(tf1, format = "feather", schema = schema(a = int32(), b =
int32())) %>%
filter(a > 3) %>%
collect()
#> # A tibble: 2 × 2
#> a b
#> <int> <int>
#> 1 4 14
#> 2 5 15
```
``` r
open_dataset(tf2, format = "parquet") %>%
filter(a > 3) %>%
collect()
#> Error in `compute.arrow_dplyr_query()`:
#> ! NotImplemented: Function 'greater' has no kernel matching input types
(<labelled<integer>[0]>: example variable a, <labelled<integer>[0]>: example
variable a)
```
``` r
open_dataset(tf2, format = "parquet", schema = schema(a = int32(), b =
int32())) %>%
filter(a > 3) %>%
collect()
#> Error in `compute.arrow_dplyr_query()`:
#> ! NotImplemented: Function 'greater_equal' has no kernel matching input
types (<labelled<integer>[0]>: example variable a, <labelled<integer>[0]>:
example variable a)
```
<sup>Created on 2025-05-25 with [reprex
v2.1.1](https://reprex.tidyverse.org)</sup>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
