I just opened https://github.com/kripken/emscripten/issues/5602 for general
wasm dynamic linking stuff. We should open issues on the various things,
including the ones you just mentioned, and link them to that general issue.
Or if they are minor I guess we can just mention them in the general issue.
(There is also a github "projects" option to create hierarchies of issues,
which might be useful for stuff like this, but I've not had time to look
into it.)
On Fri, Sep 15, 2017 at 1:41 PM, <dtippet...@gmail.com> wrote:
> Hmm, I also noticed a few additional details about the MAIN_MODULE=2 build
> that are not the case for the default option build.
>
> 1. Loading the JavaScript file will cause a warning in browsers
>
> AnimalLibraryMainModule2.js:formatted:159 on the web, we need the wasm
> binary to be preloaded and set on Module['wasmBinary']. emcc.py will do
> that for you when generating HTML (but not JS)
> shell_printErr @ AnimalLibraryMainModule2.js:formatted:159
> abort @ AnimalLibraryMainModule2.js:formatted:7257
> getBinary @ AnimalLibraryMainModule2.js:formatted:1604
> doNativeWasm @ AnimalLibraryMainModule2.js:formatted:1642
> Module.asm @ AnimalLibraryMainModule2.js:formatted:1719
> (anonymous) @ AnimalLibraryMainModule2.js:formatted:6874
> AnimalLibraryMainModule2.js:formatted:159 failed to compile wasm module:
> abort("on the web, we need the wasm binary to be preloaded and set on
> Module['wasmBinary']. emcc.py will do that for you when generating HTML
> (but not JS)") at Error
> at jsStackTrace (http://localhost/www/AnimalTest/
> AnimalLibraryMainModule2.js:1:21990)
> at stackTrace (http://localhost/www/AnimalTest/
> AnimalLibraryMainModule2.js:1:22161)
> at abort (http://localhost/www/AnimalTest/
> AnimalLibraryMainModule2.js:1:175924)
> at getBinary (http://localhost/www/AnimalTest/
> AnimalLibraryMainModule2.js:1:33806)
> at doNativeWasm (http://localhost/www/AnimalTest/
> AnimalLibraryMainModule2.js:1:34711)
> at Object.Module.asm (http://localhost/www/AnimalTest/
> AnimalLibraryMainModule2.js:1:36685)
> at http://localhost/www/AnimalTest/AnimalLibraryMainModule2.js:1:
> 145379
> If this abort() is unexpected, build with -s ASSERTIONS=1 which can give
> more information.
> shell_printErr @ AnimalLibraryMainModule2.js:formatted:159
> doNativeWasm @ AnimalLibraryMainModule2.js:formatted:1644
> Module.asm @ AnimalLibraryMainModule2.js:formatted:1719
> (anonymous) @ AnimalLibraryMainModule2.js:formatted:6874
>
> 2. All of the exports are set to undefined since
> Module["asm"](Module.asmGlobalArg,
> Module.asmLibraryArg, buffer); returns false. (which means var asm = false)
> For example even expected exports like the following
> var _ANM_CatCreateScottishFold = Module["_ANM_CatCreateScottishFold"] =
> asm["_ANM_CatCreateScottishFold"];
> will end up setting _ANM_CatCreateScottishFold and
> Module["_ANM_CatCreateScottishFold"]
> to undefined which is pretty clearly not the intended behaviour.
>
> I guess these will probably all be sorted out when shared modules are
> updated to work better with WASM. Still let me know if I should file
> another issue for this behaviour.
>
> Thanks,
>
> David
>
>
> All right. I filed the following issues:
>>
>> ALLOW_MEMORY_GROWTH not supported with -s MAIN_MODULE and -s WASM=1:
>> https://github.com/kripken/emscripten/issues/5585
>>
>> WASM shared modules export unnecessary symbols even with -s MAIN_MODULE
>> 2: https://github.com/kripken/emscripten/issues/5586
>>
>> Thanks,
>>
>> David
>>
>> On Wednesday, September 13, 2017 at 10:32:58 AM UTC-7, Alon Zakai wrote:
>>>
>>> Oh, the JS size increase is a separate issue. That's not specific to
>>> wasm, it's just that whatever symbols remain in the file, we export through
>>> JS so that others can access them. So we separate the issue of what is kept
>>> alive in the file - everything in mode 1, just what is exported in mode 2 -
>>> but what is kept alive is accessible from other modules, in both modes.
>>>
>>> This might not be the best thing, perhaps we should only do that for
>>> things on the list to be exported, but I'm not sure. Let's file an issue
>>> for discussion.
>>>
>>> On Tue, Sep 12, 2017 at 6:00 PM, <dtipp...@gmail.com> wrote:
>>>
>>>> > Thanks. Looking at this, the main module 2 is 23% larger. Looks like
>>>> much of that is due to us exporting all the functions in the table (which
>>>> limits dce). I think we don't need that for wasm shared modules, this is
>>>> just something that wasn't optimized yet.
>>>>
>>>> I guess in this example the thing that stands out to me more is that
>>>> the JS file is almost 3x the size. This seems to be mostly because of a
>>>> number of internal symbols including the following being exposed:
>>>>
>>>> var __ZN6animal8KittyCat16GetWhiskerLengthEv =
>>>> Module["__ZN6animal8KittyCat16GetWhiskerLengthEv"] =
>>>> asm["__ZN6animal8KittyCat16GetWhiskerLengthEv"];
>>>>
>>>> Not only does this increase the file size, but also since those symbols
>>>> are being exposed unnecessary parts of the internal structure are visible.
>>>>
>>>> Is this what you are referring to with "exporting all the functions in
>>>> the table"?
>>>>
>>>> > I'd suggest opening issues on this and on memory growth not allowed
>>>> in shared wasm modules (mentioned earlier in this thread), so that they are
>>>> not missed, and hopefully someone can get to them soon.
>>>>
>>>> Okay, I will submit a couple of bug reports for these two cases.
>>>>
>>>> Thanks,
>>>>
>>>> David
>>>>
>>>> On Tuesday, September 12, 2017 at 5:31:21 PM UTC-7, Alon Zakai wrote:
>>>>>
>>>>> Thanks. Looking at this, the main module 2 is 23% larger. Looks like
>>>>> much of that is due to us exporting all the functions in the table (which
>>>>> limits dce). I think we don't need that for wasm shared modules, this is
>>>>> just something that wasn't optimized yet.
>>>>>
>>>>> I'd suggest opening issues on this and on memory growth not allowed in
>>>>> shared wasm modules (mentioned earlier in this thread), so that they are
>>>>> not missed, and hopefully someone can get to them soon.
>>>>>
>>>>> On Mon, Sep 11, 2017 at 3:18 PM, <dtipp...@gmail.com> wrote:
>>>>>
>>>>>> All right. I was able to reproduce the problem with the sample at
>>>>>> https://drive.google.com/open?id=0B462aSLeSsPMUFhnNW1iYzhjeHM .
>>>>>> (created binaries also included in output folder) The issue doesn't
>>>>>> really
>>>>>> appear to be reproducible until some standard library logic is used. (in
>>>>>> this case std::vector)
>>>>>>
>>>>>> Please let me know if anything is unclear in the sample. In my case I
>>>>>> ran Emscripten on Windows using CMake to generate mingw32 makefiles.
>>>>>>
>>>>>> Thanks,
>>>>>>
>>>>>> David
>>>>>>
>>>>>> On Friday, September 8, 2017 at 10:05:39 AM UTC-7, Alon Zakai wrote:
>>>>>>>
>>>>>>> That's odd, can you make a small testcase showing the issue you are
>>>>>>> seeing? That would help understand what's going on.
>>>>>>>
>>>>>>> Mode 2 in main and side modules should only keep alive things you
>>>>>>> explicitly keep alive, so you may be hitting a bug in how we do that,
>>>>>>> or in
>>>>>>> LLVM's dce.
>>>>>>>
>>>>>>>
>>>>>>> On Thu, Sep 7, 2017 at 12:26 PM, <dtipp...@gmail.com> wrote:
>>>>>>>
>>>>>>>> Hmm, even with -s MAIN_MODULE=2 internal symbols are still being
>>>>>>>> exposed. I see a large number of internal symbols in the .js file
>>>>>>>> exposed
>>>>>>>> in the following manner:
>>>>>>>> var _internal_function_name = Module["_internal_function_name"] =
>>>>>>>> asm["_internal_function_name"];
>>>>>>>>
>>>>>>>> Most likely as a result the output files are still significantly
>>>>>>>> larger than without -s MAIN_MODULE
>>>>>>>>
>>>>>>>> Default: 264 KB .js file. 6.6 MB .wasm file.
>>>>>>>> MAIN_MODULE = 1: 7MB .js file 15MB .wasm file
>>>>>>>> MAIN_MODULE = 2: 3MB .js file 8MB .wasm file
>>>>>>>>
>>>>>>>> Is this expected?
>>>>>>>>
>>>>>>>> Thanks,
>>>>>>>>
>>>>>>>> David
>>>>>>>>
>>>>>>>> On Thursday, September 7, 2017 at 11:13:12 AM UTC-7,
>>>>>>>> dtipp...@gmail.com wrote:
>>>>>>>>>
>>>>>>>>> Yeah, sorry, I must have missed that comment in the documentation.
>>>>>>>>> Is dead code elimination also disabled for side modules and if so is
>>>>>>>>> SIDE_MODULE=2 expected to work?
>>>>>>>>>
>>>>>>>>> Thanks,
>>>>>>>>>
>>>>>>>>> David
>>>>>>>>>
>>>>>>>>> On Thursday, September 7, 2017 at 11:06:27 AM UTC-7, Alon Zakai
>>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>> Yes, linkable modules are larger by default because we disable
>>>>>>>>>> dead code elimination (since something might be used from something
>>>>>>>>>> it is
>>>>>>>>>> linked to). See MAIN_MODULE mode 2, that can help (see docs in
>>>>>>>>>> settings.js).
>>>>>>>>>>
>>>>>>>>>> On Thu, Sep 7, 2017 at 10:33 AM, <dtipp...@gmail.com> wrote:
>>>>>>>>>>
>>>>>>>>>>> Thanks, that allowed me to build.
>>>>>>>>>>>
>>>>>>>>>>> I noticed that by adding -s MAIN_MODULE=1 my wasm file size
>>>>>>>>>>> increased from 6.5 MB to 14.6 MB. Is this kind of increase expected
>>>>>>>>>>> or am I
>>>>>>>>>>> doing something incorrect? For reference my build settings look
>>>>>>>>>>> like the
>>>>>>>>>>> following:
>>>>>>>>>>>
>>>>>>>>>>> emcc -s DISABLE_EXCEPTION_CATCHING=0 --llvm-lto 3 -s WASM=1 -s
>>>>>>>>>>> EXTRA_EXPORTED_RUNTIME_METHODS=['UTF16ToString','stringToUTF16']
>>>>>>>>>>> -Oz libpdf_worker.so libPDFNetC.so -o ${WORKER_NAME}Wasm.js --bind
>>>>>>>>>>> -s
>>>>>>>>>>> MAIN_MODULE=1 --js-library NextAfter.js -s ALLOW_MEMORY_GROWTH=1 -s
>>>>>>>>>>> NO_EXIT_RUNTIME=1 -s TOTAL_MEMORY=50331648 -s
>>>>>>>>>>> RESERVED_FUNCTION_POINTERS=10 -s
>>>>>>>>>>> EXPORTED_FUNCTIONS=<big list of functions>
>>>>>>>>>>>
>>>>>>>>>>> Thanks,
>>>>>>>>>>>
>>>>>>>>>>> David
>>>>>>>>>>>
>>>>>>>>>>> On Thursday, September 7, 2017 at 1:00:14 AM UTC-7, jj wrote:
>>>>>>>>>>>>
>>>>>>>>>>>> Hmm, in WebAssembly, I believe that message should not be
>>>>>>>>>>>> relevant
>>>>>>>>>>>> anymore, and memory growth should be supported with dynamic
>>>>>>>>>>>> linking.
>>>>>>>>>>>> Or perhaps there was some reason that I can't remember now. You
>>>>>>>>>>>> can
>>>>>>>>>>>> try finding where that message comes out, and comment it out,
>>>>>>>>>>>> it might
>>>>>>>>>>>> apply only to asm.js case.
>>>>>>>>>>>>
>>>>>>>>>>>> 2017-09-07 9:58 GMT+03:00 <dtipp...@gmail.com>:
>>>>>>>>>>>> > Hmm, I encountered an error: "memory growth is not supported
>>>>>>>>>>>> with shared
>>>>>>>>>>>> > modules yet". Unfortunately I don't really have the option to
>>>>>>>>>>>> disable memory
>>>>>>>>>>>> > growth. (since the modules memory usage is dependent on user
>>>>>>>>>>>> input) Is there
>>>>>>>>>>>> > any workaround for this error? If not is adding memory growth
>>>>>>>>>>>> for shared
>>>>>>>>>>>> > modules something that will be supported soon?
>>>>>>>>>>>> >
>>>>>>>>>>>> > Thanks,
>>>>>>>>>>>> >
>>>>>>>>>>>> > David
>>>>>>>>>>>> >
>>>>>>>>>>>> >
>>>>>>>>>>>> > On Tuesday, September 5, 2017 at 5:49:28 PM UTC-7,
>>>>>>>>>>>> dtipp...@gmail.com wrote:
>>>>>>>>>>>> >>
>>>>>>>>>>>> >> Thanks, I will give it a try!
>>>>>>>>>>>> >>
>>>>>>>>>>>> >> David
>>>>>>>>>>>> >>
>>>>>>>>>>>> >> On Tuesday, September 5, 2017 at 4:57:43 PM UTC-7, Alon
>>>>>>>>>>>> Zakai wrote:
>>>>>>>>>>>> >>>
>>>>>>>>>>>> >>> To some extent yes. We have working dlopen() support, see
>>>>>>>>>>>> >>>
>>>>>>>>>>>> >>> https://github.com/kripken/emscripten/wiki/Linking
>>>>>>>>>>>> >>>
>>>>>>>>>>>> >>> However, it is not optimized yet, it uses a bunch of hacks.
>>>>>>>>>>>> As a result
>>>>>>>>>>>> >>> getting this working takes a bit more effort (see details
>>>>>>>>>>>> on that page), and
>>>>>>>>>>>> >>> it has more size and speed overhead than it should. This
>>>>>>>>>>>> will be fixed
>>>>>>>>>>>> >>> eventually with proper dynamic linking of wasm.
>>>>>>>>>>>> >>>
>>>>>>>>>>>> >>> On Tue, Sep 5, 2017 at 2:54 PM, <dtipp...@gmail.com>
>>>>>>>>>>>> wrote:
>>>>>>>>>>>> >>>>
>>>>>>>>>>>> >>>> Hi,
>>>>>>>>>>>> >>>>
>>>>>>>>>>>> >>>> My situation is this:
>>>>>>>>>>>> >>>>
>>>>>>>>>>>> >>>> I am dealing with a fairly large codebase and trying to
>>>>>>>>>>>> make WebAssembly
>>>>>>>>>>>> >>>> builds as small as possible to speed up loading times.
>>>>>>>>>>>> (especially a problem
>>>>>>>>>>>> >>>> on mobile) There are certain libraries that are rarely
>>>>>>>>>>>> used and contribute a
>>>>>>>>>>>> >>>> significant portion of the total size so my thought was to
>>>>>>>>>>>> load these
>>>>>>>>>>>> >>>> libraries only when their functions are first used.
>>>>>>>>>>>> >>>>
>>>>>>>>>>>> >>>> Given the above I was wondering if it is currently
>>>>>>>>>>>> possible to
>>>>>>>>>>>> >>>> dynamically and synchronously download wasm and load it
>>>>>>>>>>>> using dlopen or
>>>>>>>>>>>> >>>> equivalent on current browsers? This would be running in a
>>>>>>>>>>>> worker so any
>>>>>>>>>>>> >>>> synchronous wait would not cause the UI to hang.
>>>>>>>>>>>> >>>>
>>>>>>>>>>>> >>>> Thanks,
>>>>>>>>>>>> >>>>
>>>>>>>>>>>> >>>> David
>>>>>>>>>>>> >>>
>>>>>>>>>>>> >>>
>>>>>>>>>>>> > --
>>>>>>>>>>>>
>>>>>>>>>>> --
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>> --
>>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>
>>>>>
>>>>> --
>>>>
>>>
>>> --
> You received this message because you are subscribed to the Google Groups
> "emscripten-discuss" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to emscripten-discuss+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>
--
You received this message because you are subscribed to the Google Groups
"emscripten-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to emscripten-discuss+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
