Hi Adam, > This sounds really interesting, how about adding the wasm build (C++) to > the releases?
I think this just needs someone to volunteer to do it and maintain it (at a minimum if it doesn't already exist we need CI for it). We would also need to figure out details of publishing and integrating it into the release process. I've done a lot of asm.js work (different from wasm) in the past, but my > assumption would be that using Rust instead of C++ as source for wasm > should result in smaller wasm binaries. I don't know much about either, but I'm curious why you would expect this to be the case? On Tue, Jul 14, 2020 at 8:07 AM Adam Lippai <a...@rigo.sk> wrote: > This sounds really interesting, how about adding the wasm build (C++) to > the releases? > I've done a lot of asm.js work (different from wasm) in the past, but my > assumption would be that using Rust instead of C++ as source for wasm > should result in smaller wasm binaries. > Rust Arrow doesn't really use exotic solutions, eg. simd or tokio > dependency can be turned off. > > Having DataFusion + some performant data access in browsers or even in > node.js would be useful. > Not needing to build fancy HTTP/GraphQL API over the Rust/C++ impl. but > moving the data processing code to the client is viable for "small" > workloads. > Ofc if JS Arrow lands Flight support this may become less of an issue, but > AFAIK it's gRPC based which would need setting up a gRPC reverse proxy for > C++/Rust Arrow. > Overall both the code-duplication and feature fragmentation would decrease > by using a single source (like you don't have a full Python impl. for > obvious reasons) > > Best regards, > Adam Lippai > > On Tue, Jul 14, 2020 at 4:27 PM Micah Kornfield <emkornfi...@gmail.com> > wrote: > >> Fwiw, I believe at least the core c++ library already can be compiled to >> wasm. I think perspective does this [1] >> >> >> I'm curious What are you hoping to achieve with embedded wasm in spark? >> >> Thanks, >> Micah >> >> [1] https://perspective.finos.org/ >> >> On Tuesday, July 14, 2020, Brian Hulette <hulet...@gmail.com> wrote: >> >> > That sounds great! I'd like to have some support for using the rust >> and/or >> > C++ libraries in the browser via wasm as well. >> > As long as the community is ok with your overall approach "to add >> compiler >> > conditionals around any I/O features and libc dependent features of >> these >> > two libraries," I think it may be best to start with a PR and discuss >> > specifics from there. >> > >> > Do any rust contributors have objections to this? >> > >> > Brian >> > >> > On Mon, Jul 13, 2020 at 9:42 PM RJ Atwal <atwal...@gmail.com> wrote: >> > >> > > Hi all, >> > > >> > > Looking for guidance on how to submit a design and PR to add WASM32 >> > support >> > > to apache arrow's rust libraries. >> > > >> > > I am looking to use the arrow library to pass data in arrow format >> > between >> > > the host spark environment and UDFs defined in WASM . >> > > >> > > I created the following JIRA ticket to capture the work >> > > https://issues.apache.org/jira/browse/ARROW-9453 >> > > >> > > Thanks, >> > > RJ >> > > >> > >> >
