Ok yeah, that actually make sense. :)
Two warnings about using memory views here: 1) As with sending a raw pointer, lifetime management is left up to you. Make sure the underlying object doesn't change or get deallocated before you've used the array contents, or your data may be corrupted. 2) Beware of the difference between *typed arrays* and *array buffers* -- a typed array is always a view into an ArrayBuffer object, and when returning data as a memory_view that means you've got a Uint8Array (or other typed array subtype) that is associated with an ArrayBuffer that is the entire, live emscripten heap. So if the API you're passing data to accepts a Uint8Array directly, memory_view is perfect. This is good for, say, uploading WebGL textures. However if your target API actually takes an ArrayBuffer as a single data chunk, then you may need to manually slice out a copy of that portion of the buffer before you can pass it in... Documentation on MDN seems to indicate that XHR.send() should accept a typed array view directly < https://developer.mozilla.org/en-US/docs/Web/API/XMLHttpRequest#send%28%29> but the compatibility table is a bit spotty. Double-check that it works in your target browsers. :) If it does turns out that you need to copy into a separate ArrayBuffer, you can do that on the JS side something like this: var arr = myobj.myfunc(); var buf = arr.buffer.slice(arr.byteOffset, arr.byteOffset + arr.byteLength); xhr.send(buf); Should be able to do the equivalent via emscripten::val directly in your binding function as well, though it's probably a bit verbose... -- brion On Sat, Aug 22, 2015 at 3:16 PM, Matthew Tamayo <[email protected]> wrote: > Thanks, Brion! > > To add a little more context we have POD types that are contiguous blocks > of memory representing multivariate polynomial functions. On the server > side we are martially the raw byte buffers through Java into JNI back into > native C++ data type. > > We're thinking the general flow should be: > > 1. Grab binary buffer view of POD data type through a memory view or > some kind of object binding. > 2. Post it as an arraybuffer type using XMLHttpRequest2 to a Java web > service > 3. Rehydrate on the server by taking byte[] -> jbyteArray (JNI) -> > jByte * -> reinterpret_cast -> POD * > > For security reasons we have validations on the size of the byte arrays > server side. It's a self-describing data structure and is generated by > filling with random data in the first place so as long size is correct we > are safe. > > These objects can get pretty big (100K -> 7 MB) so having zero cost > serialization of the binary form would be ideal. > > Given this additional context is memory_view the correct approach? > > -mtr > > On Fri, Aug 21, 2015 at 5:44 PM, Brion Vibber <[email protected]> wrote: > >> Bindings for a class instance should typically use a class or value >> object binding, not a raw memory view -- that mainly makes sense for binary >> buffers. >> >> See >> https://kripken.github.io/emscripten-site/docs/porting/connecting_cpp_and_javascript/embind.html#classes >> and >> https://kripken.github.io/emscripten-site/docs/porting/connecting_cpp_and_javascript/embind.html#value-types >> for the more typical bindings. >> >> >> If you really need to pass raw object instance variable contents as typed >> array views into JavaScript and then send them back... well it probably >> works to receive the array in your binding as a std::string (there's a >> standard incoming mapping from Uint8Array to std::string), then get the >> string's data pointer and reinterpret_cast<> it. >> >> -- brion >> >> >> On Fri, Aug 21, 2015 at 4:21 PM, Peng Hui How <[email protected]> >> wrote: >> >>> Hi Brion, >>> >>> Please ignore the previous message as it was full of typos. >>> >>> The question is: >>> Suppose we have an instance of class X in C++, converted it into >>> emscripten val using the method you described previously, >>> how do we retrieve it as an instance of class X in C++? >>> >>> It would be super helpful if you can look into this. Thanks a lot! >>> >>> Best, >>> Peng Hui >>> >>> On Tuesday, August 18, 2015 at 7:17:40 PM UTC-7, Matthew Tamayo wrote: >>>> >>>> Thanks Brion! This is super useful. >>>> >>>> -mtr >>>> ------------------------------ >>>> From: Brion Vibber >>>> Sent: 8/15/2015 5:53 AM >>>> To: emscripten Mailing List >>>> Subject: Re: Marshalling Data From C++ to JS Efficiently >>>> >>>> If you declare your C++ function to return an emscripten::val, a memory >>>> view should correctly marshal out and return a Uint8Array (or whichever >>>> type you selected). >>>> >>>> Here's an example in a getter on a value object, but it should work >>>> fine on regular object methods too: >>>> >>>> >>>> #include <emscripten/bind.h> >>>> using namespace emscripten; >>>> >>>> ... >>>> >>>> struct H264Plane { >>>> unsigned char *data; >>>> int stride; >>>> int height; >>>> ... >>>> val data_getter() const; >>>> void data_setter(val v); >>>> }; >>>> >>>> ... >>>> >>>> val H264Plane::data_getter() const >>>> { >>>> return val(memory_view<unsigned char>(stride * height, >>>> (unsigned char *)data)); >>>> } >>>> >>>> ... >>>> >>>> EMSCRIPTEN_BINDINGS(h264_decoder) { >>>> >>>> value_object<H264Plane>("H264Plane") >>>> .field("data", &H264Plane::data_getter, >>>> &H264Plane::data_setter) >>>> ... >>>> ; >>>> ... >>>> } >>>> >>>> -- brion >>>> >>>> >>>> On Fri, Aug 14, 2015 at 11:22 PM, Matthew Tamayo < >>>> [email protected]> wrote: >>>> >>>>> I've been looking around Emscripten to try and figure out how to >>>>> marshall large binary objects from C++. >>>>> >>>>> We want to: >>>>> >>>>> 1. pass binaryArray = (unsigned *)&bigObject as a Uint8Array >>>>> 2. Use XMLHttpRequest2 to send Uint8Array to server >>>>> 3. Read in binary data on server (BigObject*) binaryArray in C++ >>>>> >>>>> The problem we're having is figuring out the most efficient way to get >>>>> a Uint8Array of emscripten using embind. I've seen some threads that >>>>> recommend emscripten::memory_view(output_size, output_ptr), but I don't >>>>> know if it requires passing in a JS function via emscripten::val or if >>>>> returning a memory view will get correctly marshalled. >>>>> >>>>> Does this seem like sane approach? Any easier ways in the most recent >>>>> version of emscripten? >>>>> >>>>> -mtr >>>>> >>>>> -- >>>>> You received this message because you are subscribed to the Google >>>>> Groups "emscripten-discuss" group. >>>>> To unsubscribe from this group and stop receiving emails from it, send >>>>> an email to [email protected]. >>>>> For more options, visit https://groups.google.com/d/optout. >>>>> >>>> >>>> -- >>>> You received this message because you are subscribed to a topic in the >>>> Google Groups "emscripten-discuss" group. >>>> To unsubscribe from this topic, visit >>>> https://groups.google.com/d/topic/emscripten-discuss/l8GkOxZ79Ks/unsubscribe >>>> . >>>> To unsubscribe from this group and all its topics, send an email to >>>> [email protected]. >>>> For more options, visit https://groups.google.com/d/optout. >>>> >>> -- >>> You received this message because you are subscribed to the Google >>> Groups "emscripten-discuss" group. >>> To unsubscribe from this group and stop receiving emails from it, send >>> an email to [email protected]. >>> For more options, visit https://groups.google.com/d/optout. >>> >> >> -- >> You received this message because you are subscribed to a topic in the >> Google Groups "emscripten-discuss" group. >> To unsubscribe from this topic, visit >> https://groups.google.com/d/topic/emscripten-discuss/l8GkOxZ79Ks/unsubscribe >> . >> To unsubscribe from this group and all its topics, send an email to >> [email protected]. >> For more options, visit https://groups.google.com/d/optout. >> > > -- > You received this message because you are subscribed to the Google Groups > "emscripten-discuss" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > For more options, visit https://groups.google.com/d/optout. > -- You received this message because you are subscribed to the Google Groups "emscripten-discuss" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/d/optout.
