thisisnic commented on issue #39139: URL: https://github.com/apache/arrow/issues/39139#issuecomment-2030030890
I tried this locally (Ubuntu) on the copy of the NYC taxi dataset I have, but couldn't reproduce; I also tried it with a CSV dataset which I knew usually took a lot longer for `glimpse()` to work on, and found that it was slow, due to `dataset__Scanner__CountRows` taking a while, which makes sense when thinking about Parquet metadata vs. CSV's lack of metadata:  @AngelFelizR I don't suppose you'd mind running your example again, but using `profvis` to track the timing, and then sharing the flame graph like the one I pasted above? Something like: ```r library(dplyr) library(arrow) library(profvis) NycTrips2022 <- here::here("data/trip-data/year=2022") |> open_dataset() profvis(glimpse(NycTrips2022)) ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
