paleolimbot commented on PR #280:
URL: https://github.com/apache/arrow-nanoarrow/pull/280#issuecomment-1683166773
Cool observation!
For what it's worth, I wasn't able to observe a difference on MacOS M1 or
x86/AMD Ryzen 5 (although I'm benchmarking via the R package, which unpacks
into 32-bit integers and not 8-bit integers).
With nanoarrow 0.2.0:
``` r
# install.packages("nanoarrow")
library(nanoarrow)
lgls <- nanoarrow:::vec_gen(logical(), 1e6)
bool_array <- as_nanoarrow_array(lgls)
bool_array_arrow <- arrow::as_arrow_array(bool_array)
bench::mark(
convert_array(bool_array, logical()),
as.vector(bool_array_arrow),
as.logical(lgls)
)
#> # A tibble: 3 × 6
#> expression min median `itr/sec` mem_alloc
`gc/sec`
#> <bch:expr> <bch> <bch:> <dbl> <bch:byt>
<dbl>
#> 1 convert_array(bool_array, logical()) 560µs 735µs 1.33e3 3.82MB
154.
#> 2 as.vector(bool_array_arrow) 568µs 782µs 1.28e3 3.82MB
141.
#> 3 as.logical(lgls) 0 0 4.32e8 0B
0
```
With nanoarrow on main:
``` r
# pak::pak("apache/arrow-nanoarrow/r")
library(nanoarrow)
lgls <- nanoarrow:::vec_gen(logical(), 1e6)
bool_array <- as_nanoarrow_array(lgls)
bool_array_arrow <- arrow::as_arrow_array(bool_array)
bench::mark(
convert_array(bool_array, logical()),
as.vector(bool_array_arrow),
as.logical(lgls)
)
#> # A tibble: 3 × 6
#> expression min median `itr/sec` mem_alloc
`gc/sec`
#> <bch:expr> <bch> <bch:> <dbl> <bch:byt>
<dbl>
#> 1 convert_array(bool_array, logical()) 105µs 316µs 3.08e3 3.82MB
350.
#> 2 as.vector(bool_array_arrow) 558µs 778µs 1.29e3 3.82MB
143.
#> 3 as.logical(lgls) 0 1ns 2.94e8 0B
0
```
With this PR:
``` r
# Local build of this branch
library(nanoarrow)
lgls <- nanoarrow:::vec_gen(logical(), 1e6)
bool_array <- as_nanoarrow_array(lgls)
bool_array_arrow <- arrow::as_arrow_array(bool_array)
bench::mark(
convert_array(bool_array, logical()),
as.vector(bool_array_arrow),
as.logical(lgls)
)
#> # A tibble: 3 × 6
#> expression min median `itr/sec` mem_alloc
`gc/sec`
#> <bch:expr> <bch> <bch:> <dbl> <bch:byt>
<dbl>
#> 1 convert_array(bool_array, logical()) 105µs 316µs 3.08e3 3.82MB
350.
#> 2 as.vector(bool_array_arrow) 558µs 778µs 1.29e3 3.82MB
143.
#> 3 as.logical(lgls) 0 1ns 2.94e8 0B
0
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
