thisisnic commented on issue #39548: URL: https://github.com/apache/arrow/issues/39548#issuecomment-5732060940
Analysis below generated by Claude Code, posted by @thisisnic. Took another look at this. It's a different mechanism from the `filter()` failure in #33464, and unlike that one it looks fixable in the R package alone. What's going on: `month()` and a number of other bindings (`substr()`, `is.integer()`, anything that dispatches on the input type) call `type_id()` on the input Expression, which resolves the type against the Schema the R package attached to that Expression: https://github.com/apache/arrow/blob/c07de67a7c9a43c7c958ffeb08407b66c38df5b5/r/R/dplyr-funcs-datetime.R#L186-L187 https://github.com/apache/arrow/blob/c07de67a7c9a43c7c958ffeb08407b66c38df5b5/r/R/dplyr-funcs-type.R#L211-L214 https://github.com/apache/arrow/blob/c07de67a7c9a43c7c958ffeb08407b66c38df5b5/r/R/expression.R#L53-L59 That Schema is set in `arrow_mask()` from the dataset schema, which doesn't contain `__filename`: https://github.com/apache/arrow/blob/c07de67a7c9a43c7c958ffeb08407b66c38df5b5/r/R/dplyr-eval.R#L264-L266 So any Expression whose tree references `__filename` (here `file_date`, derived from `file`) fails to bind with `No match for FieldRef.Name(__filename)`. Bindings that never ask for the type, such as `year()`, `nchar()` or `paste0()`, work fine, which is why the reprex gets as far as it does. After `compute()` the column is a real string column in the Table's schema and everything works. The R package already patches the augmented field into the schema in the two other places it resolves types: https://github.com/apache/arrow/blob/c07de67a7c9a43c7c958ffeb08407b66c38df5b5/r/R/dplyr.R#L122-L124 https://github.com/apache/arrow/blob/c07de67a7c9a43c7c958ffeb08407b66c38df5b5/r/R/dplyr-collect.R#L142-L144 `arrow_mask()` is the third place and doesn't. Adding the same `schema[["__filename"]] <- string()` there is the candidate fix. Not yet tried properly, so it may turn out there's another spot that needs the same treatment. Note that `filter()` on the column will still fail after this, since that's bound in libarrow's scan node (#33464). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
