Are there any gotchas with a std::string expecting something that is null terminated?
-mtr On Mon, Aug 24, 2015 at 11:41 AM, Brion Vibber <[email protected]> wrote: > When using embind, the simplest way to accept an incoming Uint8Array is to > declare a binding function that takes a std::string -- the standard type > conversions in the bindings will take care of importing the buffer into the > emscripten heap space and you'll get a std::string instance that you can > get a pointer out of, then do whatever casts are needed to get your C++ > object. > > I think this will allocate the std::string on the stack... not sure > offhand if the actual buffer is inline or separately heap-allocated; should > only make a difference if you need to keep the rehydrated object around > after your function exits. > > > If you do need to retain the object in the emscripten heap after your > function exits and want to minimize data copies, you may wish to control > the binding more tightly. > > I've tended to do this more with manual JS bindings wrapping C functions: > * allocate a heap buffer by calling Module._malloc() or other appropriate > allocator > * copy the data in; you can do this manually by calling > Module.HEAPU8.set() or through a nice wrapper Module.writeArrayToMemory(): > https://kripken.github.io/emscripten-site/docs/api_reference/preamble.js.html#writeArrayToMemory > * pass the pointer in to the C++ function and let C++ manage the pointer > lifetime & deallocation > > Should be able to do roughly the same purely within embind by using > emscripten::val calls. > > -- brion > > > On Mon, Aug 24, 2015 at 11:11 AM, Matthew Tamayo <[email protected]> > wrote: > >> Sorry Brion, one more question! >> >> What's the easiest way to go in the reverse direction? That is if we >> receive a UInt8Array from the server, what's the easiest way to pass that >> into a C++ call? The conversion back is ideally just a reinterpret_cast of >> an unsigned char * to the correct C++ type. >> >> -mtr >> >> On Sat, Aug 22, 2015 at 4:41 PM, Brion Vibber <[email protected]> wrote: >> >>> Ok yeah, that actually make sense. :) >>> >>> >>> Two warnings about using memory views here: >>> >>> 1) As with sending a raw pointer, lifetime management is left up to you. >>> Make sure the underlying object doesn't change or get deallocated before >>> you've used the array contents, or your data may be corrupted. >>> >>> 2) Beware of the difference between *typed arrays* and *array buffers* >>> -- a typed array is always a view into an ArrayBuffer object, and when >>> returning data as a memory_view that means you've got a Uint8Array (or >>> other typed array subtype) that is associated with an ArrayBuffer that is >>> the entire, live emscripten heap. >>> >>> So if the API you're passing data to accepts a Uint8Array directly, >>> memory_view is perfect. This is good for, say, uploading WebGL textures. >>> However if your target API actually takes an ArrayBuffer as a single data >>> chunk, then you may need to manually slice out a copy of that portion of >>> the buffer before you can pass it in... >>> >>> Documentation on MDN seems to indicate that XHR.send() should accept a >>> typed array view directly < >>> https://developer.mozilla.org/en-US/docs/Web/API/XMLHttpRequest#send%28%29> >>> but the compatibility table is a bit spotty. Double-check that it works in >>> your target browsers. :) >>> >>> >>> If it does turns out that you need to copy into a separate ArrayBuffer, >>> you can do that on the JS side something like this: >>> >>> var arr = myobj.myfunc(); >>> var buf = arr.buffer.slice(arr.byteOffset, arr.byteOffset + >>> arr.byteLength); >>> xhr.send(buf); >>> >>> Should be able to do the equivalent via emscripten::val directly in your >>> binding function as well, though it's probably a bit verbose... >>> >>> -- brion >>> >>> On Sat, Aug 22, 2015 at 3:16 PM, Matthew Tamayo <[email protected] >>> > wrote: >>> >>>> Thanks, Brion! >>>> >>>> To add a little more context we have POD types that are contiguous >>>> blocks of memory representing multivariate polynomial functions. On the >>>> server side we are martially the raw byte buffers through Java into JNI >>>> back into native C++ data type. >>>> >>>> We're thinking the general flow should be: >>>> >>>> 1. Grab binary buffer view of POD data type through a memory view >>>> or some kind of object binding. >>>> 2. Post it as an arraybuffer type using XMLHttpRequest2 to a Java >>>> web service >>>> 3. Rehydrate on the server by taking byte[] -> jbyteArray (JNI) -> >>>> jByte * -> reinterpret_cast -> POD * >>>> >>>> For security reasons we have validations on the size of the byte arrays >>>> server side. It's a self-describing data structure and is generated by >>>> filling with random data in the first place so as long size is correct we >>>> are safe. >>>> >>>> These objects can get pretty big (100K -> 7 MB) so having zero cost >>>> serialization of the binary form would be ideal. >>>> >>>> Given this additional context is memory_view the correct approach? >>>> >>>> -mtr >>>> >>>> On Fri, Aug 21, 2015 at 5:44 PM, Brion Vibber <[email protected]> wrote: >>>> >>>>> Bindings for a class instance should typically use a class or value >>>>> object binding, not a raw memory view -- that mainly makes sense for >>>>> binary >>>>> buffers. >>>>> >>>>> See >>>>> https://kripken.github.io/emscripten-site/docs/porting/connecting_cpp_and_javascript/embind.html#classes >>>>> and >>>>> https://kripken.github.io/emscripten-site/docs/porting/connecting_cpp_and_javascript/embind.html#value-types >>>>> for the more typical bindings. >>>>> >>>>> >>>>> If you really need to pass raw object instance variable contents as >>>>> typed array views into JavaScript and then send them back... well it >>>>> probably works to receive the array in your binding as a std::string >>>>> (there's a standard incoming mapping from Uint8Array to std::string), then >>>>> get the string's data pointer and reinterpret_cast<> it. >>>>> >>>>> -- brion >>>>> >>>>> >>>>> On Fri, Aug 21, 2015 at 4:21 PM, Peng Hui How <[email protected]> >>>>> wrote: >>>>> >>>>>> Hi Brion, >>>>>> >>>>>> Please ignore the previous message as it was full of typos. >>>>>> >>>>>> The question is: >>>>>> Suppose we have an instance of class X in C++, converted it into >>>>>> emscripten val using the method you described previously, >>>>>> how do we retrieve it as an instance of class X in C++? >>>>>> >>>>>> It would be super helpful if you can look into this. Thanks a lot! >>>>>> >>>>>> Best, >>>>>> Peng Hui >>>>>> >>>>>> On Tuesday, August 18, 2015 at 7:17:40 PM UTC-7, Matthew Tamayo wrote: >>>>>>> >>>>>>> Thanks Brion! This is super useful. >>>>>>> >>>>>>> -mtr >>>>>>> ------------------------------ >>>>>>> From: Brion Vibber >>>>>>> Sent: 8/15/2015 5:53 AM >>>>>>> To: emscripten Mailing List >>>>>>> Subject: Re: Marshalling Data From C++ to JS Efficiently >>>>>>> >>>>>>> If you declare your C++ function to return an emscripten::val, a >>>>>>> memory view should correctly marshal out and return a Uint8Array (or >>>>>>> whichever type you selected). >>>>>>> >>>>>>> Here's an example in a getter on a value object, but it should work >>>>>>> fine on regular object methods too: >>>>>>> >>>>>>> >>>>>>> #include <emscripten/bind.h> >>>>>>> using namespace emscripten; >>>>>>> >>>>>>> ... >>>>>>> >>>>>>> struct H264Plane { >>>>>>> unsigned char *data; >>>>>>> int stride; >>>>>>> int height; >>>>>>> ... >>>>>>> val data_getter() const; >>>>>>> void data_setter(val v); >>>>>>> }; >>>>>>> >>>>>>> ... >>>>>>> >>>>>>> val H264Plane::data_getter() const >>>>>>> { >>>>>>> return val(memory_view<unsigned char>(stride * height, >>>>>>> (unsigned char *)data)); >>>>>>> } >>>>>>> >>>>>>> ... >>>>>>> >>>>>>> EMSCRIPTEN_BINDINGS(h264_decoder) { >>>>>>> >>>>>>> value_object<H264Plane>("H264Plane") >>>>>>> .field("data", &H264Plane::data_getter, >>>>>>> &H264Plane::data_setter) >>>>>>> ... >>>>>>> ; >>>>>>> ... >>>>>>> } >>>>>>> >>>>>>> -- brion >>>>>>> >>>>>>> >>>>>>> On Fri, Aug 14, 2015 at 11:22 PM, Matthew Tamayo < >>>>>>> [email protected]> wrote: >>>>>>> >>>>>>>> I've been looking around Emscripten to try and figure out how to >>>>>>>> marshall large binary objects from C++. >>>>>>>> >>>>>>>> We want to: >>>>>>>> >>>>>>>> 1. pass binaryArray = (unsigned *)&bigObject as a Uint8Array >>>>>>>> 2. Use XMLHttpRequest2 to send Uint8Array to server >>>>>>>> 3. Read in binary data on server (BigObject*) binaryArray in C++ >>>>>>>> >>>>>>>> The problem we're having is figuring out the most efficient way to >>>>>>>> get a Uint8Array of emscripten using embind. I've seen some threads >>>>>>>> that >>>>>>>> recommend emscripten::memory_view(output_size, output_ptr), but I don't >>>>>>>> know if it requires passing in a JS function via emscripten::val or if >>>>>>>> returning a memory view will get correctly marshalled. >>>>>>>> >>>>>>>> Does this seem like sane approach? Any easier ways in the most >>>>>>>> recent version of emscripten? >>>>>>>> >>>>>>>> -mtr >>>>>>>> >>>>>>>> -- >>>>>>>> You received this message because you are subscribed to the Google >>>>>>>> Groups "emscripten-discuss" group. >>>>>>>> To unsubscribe from this group and stop receiving emails from it, >>>>>>>> send an email to [email protected]. >>>>>>>> For more options, visit https://groups.google.com/d/optout. >>>>>>>> >>>>>>> >>>>>>> -- >>>>>>> You received this message because you are subscribed to a topic in >>>>>>> the Google Groups "emscripten-discuss" group. >>>>>>> To unsubscribe from this topic, visit >>>>>>> https://groups.google.com/d/topic/emscripten-discuss/l8GkOxZ79Ks/unsubscribe >>>>>>> . >>>>>>> To unsubscribe from this group and all its topics, send an email to >>>>>>> [email protected]. >>>>>>> For more options, visit https://groups.google.com/d/optout. >>>>>>> >>>>>> -- >>>>>> You received this message because you are subscribed to the Google >>>>>> Groups "emscripten-discuss" group. >>>>>> To unsubscribe from this group and stop receiving emails from it, >>>>>> send an email to [email protected]. >>>>>> For more options, visit https://groups.google.com/d/optout. >>>>>> >>>>> >>>>> -- >>>>> You received this message because you are subscribed to a topic in the >>>>> Google Groups "emscripten-discuss" group. >>>>> To unsubscribe from this topic, visit >>>>> https://groups.google.com/d/topic/emscripten-discuss/l8GkOxZ79Ks/unsubscribe >>>>> . >>>>> To unsubscribe from this group and all its topics, send an email to >>>>> [email protected]. >>>>> For more options, visit https://groups.google.com/d/optout. >>>>> >>>> >>>> -- >>>> You received this message because you are subscribed to the Google >>>> Groups "emscripten-discuss" group. >>>> To unsubscribe from this group and stop receiving emails from it, send >>>> an email to [email protected]. >>>> For more options, visit https://groups.google.com/d/optout. >>>> >>> >>> -- >>> You received this message because you are subscribed to a topic in the >>> Google Groups "emscripten-discuss" group. >>> To unsubscribe from this topic, visit >>> https://groups.google.com/d/topic/emscripten-discuss/l8GkOxZ79Ks/unsubscribe >>> . >>> To unsubscribe from this group and all its topics, send an email to >>> [email protected]. >>> For more options, visit https://groups.google.com/d/optout. >>> >> >> -- >> You received this message because you are subscribed to the Google Groups >> "emscripten-discuss" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to [email protected]. >> For more options, visit https://groups.google.com/d/optout. >> > > -- > You received this message because you are subscribed to a topic in the > Google Groups "emscripten-discuss" group. > To unsubscribe from this topic, visit > https://groups.google.com/d/topic/emscripten-discuss/l8GkOxZ79Ks/unsubscribe > . > To unsubscribe from this group and all its topics, send an email to > [email protected]. > For more options, visit https://groups.google.com/d/optout. > -- You received this message because you are subscribed to the Google Groups "emscripten-discuss" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/d/optout.
