r2evans commented on issue #40423:
URL: https://github.com/apache/arrow/issues/40423#issuecomment-3705937655
I'm not sure if it is related, but I can reliably reproduce that warning
with named vectors in a column:
```r
mutate(mtcars, cyl = setNames(nm = cyl)) |>
arrow_table() |>
rename_with(.fn = toupper)
# Warning: Invalid metadata$r
# Warning: Invalid metadata$r
# Table (query)
# MPG: double
# CYL: double
# DISP: double
# HP: double
# DRAT: double
# WT: double
# QSEC: double
# VS: double
# AM: double
# GEAR: double
# CARB: double
# See $.data for the source Arrow object
```
The `rename_with(.)` is the causative step, without it there are no warnings
(with or without `collect()`). I had not thought that having the name as a
vector attribute would in effect defeat some of the "lazy load" in that if an
arrow table has 1M+ rows, in its lazy state it is memory-efficient (not all
loaded), but the attributes are always loaded and can be largish. Bummer (not
this issue).
When the data is read in using `open_dataset()` (not in the above example)
then I can override with `..$schema$metadata$r$columns$cyl$attributes$names <-
NULL` and then the `rename_with(.)` works. This overwrite does not appear to
work the same using `arrow_table()` (not sure if that's by-design or just a
feature).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
