abduazizR commented on issue #14907:
URL: https://github.com/apache/arrow/issues/14907#issuecomment-1345575760
Hi Nic,
Thanks for responding. I am using the latest version of Arrow 10.0.1
In the example I posted, `ccaei` is a dataset I built using multiple parquet
files. I will attach an example, but it does not show exactly the problem I
faced.
``` r
library(tidyverse)
library(arrow)
#>
#> Attaching package: 'arrow'
#> The following object is masked from 'package:utils':
#>
#> timestamp
library(duckdb)
#> Loading required package: DBI
data1 <- tibble(
id = 1:100,
x = rnorm(100)
)
data2 <- tibble(
id = 52:69,
z = rbinom(18, size = 18, prob = 0.5)
)
data2_arrow <- data2 |> as_arrow_table()
# The weired right_join output
data2_arrow |>
right_join(data1) |> collect() |>
# The ids from data 1 are missing
tail(10)
#> # A tibble: 10 × 3
#> id z x
#> <int> <int> <dbl>
#> 1 NA NA -0.369
#> 2 NA NA 0.470
#> 3 NA NA -0.372
#> 4 NA NA 0.505
#> 5 NA NA 1.47
#> 6 NA NA -0.858
#> 7 NA NA -0.330
#> 8 NA NA 2.02
#> 9 NA NA 1.53
#> 10 NA NA -0.399
# The expected right_join output
data2 |>
right_join(data1) |>
tail(10)
#> Joining, by = "id"
#> # A tibble: 10 × 3
#> id z x
#> <int> <int> <dbl>
#> 1 91 NA -0.369
#> 2 92 NA 0.470
#> 3 93 NA -0.372
#> 4 94 NA 0.505
#> 5 95 NA 1.47
#> 6 96 NA -0.858
#> 7 97 NA -0.330
#> 8 98 NA 2.02
#> 9 99 NA 1.53
#> 10 100 NA -0.399
```
<sup>Created on 2022-12-11 with [reprex
v2.0.2](https://reprex.tidyverse.org)</sup>
With my data, `right_join` is acting as `inner_join`. As a result, I don't
even have rows with `NA`s in the ID column.
Thank you.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
