eddelbuettel commented on issue #67:
URL: https://github.com/apache/arrow-nanoarrow/issues/67#issuecomment-1323909187
Absolutely. I am on a call now, and now of what we do is hidden in a branch
of a public repo but another I looked this morning follows the same scheme and
in C(++) I do
```r
for (size_t i=0; i<ncol; i++) {
// this allocates, and properly wraps as external pointers
controlling lifetime
SEXP schemaxp = arch_c_allocate_schema();
SEXP arrayxp = arch_c_allocate_array_data();
// now buf is a shared_ptr to ColumnBuffer
auto buf = sr_data->get()->at(names[i]);
// this is pair of array and schema pointer
auto pp = tdbs::ArrowAdapter::to_arrow(buf);
memcpy((void*) R_ExternalPtrAddr(schemaxp), pp.second.get(),
sizeof(ArrowSchema));
memcpy((void*) R_ExternalPtrAddr(arrayxp), pp.first.get(),
sizeof(ArrowArray));
schlst[i] = schemaxp;
arrlst[i] = arrayxp;
}
struct ArrowArray* array_data_tmp = (struct ArrowArray*)
R_ExternalPtrAddr(arrlst[0]);
int rows = static_cast<int>(array_data_tmp->length);
SEXP sxp = arch_c_schema_xptr_new(Rcpp::wrap("+s"), // format
Rcpp::wrap(""), // name
Rcpp::List(), // metadata
Rcpp::wrap(2), // flags, 2:
unord., nullable, no sorted map
schlst, //
children
R_NilValue); // dictionary
SEXP axp =
arch_c_array_from_sexp(Rcpp::List::create(Rcpp::Named("")=R_NilValue), //
buffers
Rcpp::wrap(rows), // length
Rcpp::wrap(-1), // null
count, -1 means not determined
Rcpp::wrap(0), // offset (in
bytes)
arrlst, // children
R_NilValue); // dictionary
Rcpp::List as = Rcpp::List::create(Rcpp::Named("schema") = sxp,
Rcpp::Named("array_data") = axp);
as.attr("class") = "arch_array";
return as;
```
I called this `arch` to not step on your toes but it is essentially your
`narrow`. In R it is then
```r
dat |> arch::from_arch_array(arrow::RecordBatch) |>
arrow::as_arrow_table() |> dplyr::collect() -> D
```
All this works, in a basic manner, and I did similar exercise for
RecordBatches 'in pieces'. I have not gotten to ALTREP and other extensions we
all would love to have here too. So `nanoarrow` is really exciting -- but I
miss a roadmap (not in the sense of what will come, but in the helping me to
get going).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
