On Wednesday, February 12, 2014 5:13:32 PM UTC-5, Dave Nicponski wrote:
>
> Hello, and thanks for the reply Alon.
>
> In the process of making a simple repo case, i better understand the
> problem(s) involved here.
> Executive summary: When a worker is run under nodejs with typed arrays
> option set for the worker, an emscripten assertion is triggered:
>
> "Assertion failed: Cannot fallback to non-typed array case: Code is too
> specialized"
>
> However, compiling the worker with USE_TYPED_ARRAYS=0 causes "no memory
> initializer" to be inserted into the generated code, instead of the
> traditional memory initializer block.
> This causes the emlink.py script to fail when it cannot find the memory
> initializer section.
>
> Attached find a pretty minimal repro. Note that where nodejs triggers the
> above listed emscripten assertion failure, the same code appears to work
> fine in a browser (at least under latest firefox). It's not entirely clear
> how i should continue development. Using the browser is incredibly
> inefficient for development for me compared to nodejs. I suspect a 'quick
> fix' here would be to fix the emlink.py script to handle this case
> correctly. Meanwhile, i'll try giving fastcomp a try if i can get it to
> work for me.
>
> Thanks,
> -dave-
>
> -------8<-------8<-------8<-------8<-------8<-------8<-------
> Building sequence follows
> -------8<-------8<-------8<-------8<-------8<-------8<-------
> # This shows the simple clang setup I use. It mostly just sets up the
> various
> # include paths and forces C++ mode and c++11
> $ echo $(./clang_compile_flags.py)
> -Qunused-arguments -Wall -Wextra -Werror -Wno-unused-parameter
> -fexceptions -std=c++11 -x c++ -iquote ./src -iquote
> /home/daven/src/emscripen/emscripten/system/include/ -I . -isystem
> ./include/protorpc/ -isystem ./include/openssl/ -isystem
> ./include/tinyxml/tinyxml/
>
> # Build the MAIN (shared, static) library for main()
> $ emcc $(./clang_compile_flags.py) -O1 -s MAIN_MODULE=1 -o
> src/trash/poc/split_worker_with_common_lib/build/lib.js
> src/trash/poc/split_worker_with_common_lib/lib.cc
>
> # Build the MAIN (shared, static) library for worker
> $ emcc $(./clang_compile_flags.py) -O1 -s MAIN_MODULE=1 -s
> BUILD_AS_WORKER=1 -o
> src/trash/poc/split_worker_with_common_lib/build/lib.js
> src/trash/poc/split_worker_with_common_lib/lib.cc
>
>
Oops, this should obviously have read:
$ emcc $(./clang_compile_flags.py) -O1 -s MAIN_MODULE=1 -s
BUILD_AS_WORKER=1 -o
src/trash/poc/split_worker_with_common_lib/build/lib_worker.js
src/trash/poc/split_worker_with_common_lib/lib.cc
> # Build the SIDE (main module) library
> $ emcc $(./clang_compile_flags.py) -O1 -s SIDE_MODULE=1 -o
> src/trash/poc/split_worker_with_common_lib/build/main.js
> src/trash/poc/split_worker_with_common_lib/main.cc
>
> # Link the main() module (SUCCESS)
> $ emlink.py
> src/trash/poc/split_worker_with_common_lib/build/{lib,main,main_module}.js
> #Main module: src/trash/poc/split_worker_with_common_lib/build/lib.js
> #Side module: src/trash/poc/split_worker_with_common_lib/build/main.js
> #Output: src/trash/poc/split_worker_with_common_lib/build/main_module.js
>
> # Build the SIDE (worker module) library
> $ emcc $(./clang_compile_flags.py) -O1 -s BUILD_AS_WORKER=1 -s
> SIDE_MODULE=1 -s EXPORTED_FUNCTIONS="['_do_work']" -o
> src/trash/poc/split_worker_with_common_lib/build/worker.js
> src/trash/poc/split_worker_with_common_lib/worker.cc
>
> # Link the worker do_work() module
> $ emlink.py
> src/trash/poc/split_worker_with_common_lib/build/{lib_worker,worker,worker_module}.js
> #Main module: src/trash/poc/split_worker_with_common_lib/build/lib.js
> #Side module: src/trash/poc/split_worker_with_common_lib/build/worker.js
> #Output: src/trash/poc/split_worker_with_common_lib/build/worker_module.js
>
> # Try running this in nodejs. It fails in emscripten worker setup code,
> # apparently. It works fine from a browser (firefox, at least) though.
> $ nodejs main_module.js
> Lib function returns This is some string
> Assertion failed: Cannot fallback to non-typed array case: Code is too
> specialized
>
> # Try again building SIDE (worker module) library without typed arrays, so
> # this can be run in nodejs
> $ emcc $(./clang_compile_flags.py) -O1 -s USE_TYPED_ARRAYS=0 -s
> BUILD_AS_WORKER=1 -s SIDE_MODULE=1 -s EXPORTED_FUNCTIONS="['_do_work']" -o
> src/trash/poc/split_worker_with_common_lib/build/worker.js
> src/trash/poc/split_worker_with_common_lib/worker.cc
>
> # Try again relinking the worker do_work module
> $ emlink.py
> src/trash/poc/split_worker_with_common_lib/build/{lib_worker,worker,worker_module}.js
> #Main module: src/trash/poc/split_worker_with_common_lib/build/lib.js
> #Side module: src/trash/poc/split_worker_with_common_lib/build/worker.js
> #Output: src/trash/poc/split_worker_with_common_lib/build/worker_module.js
> #Traceback (most recent call last):
> # File "/home/daven/symlinks/override/emlink.py", line 33, in <module>
> # run()
> # File "/home/daven/symlinks/override/emlink.py", line 27, in run
> # side = AsmModule(side)
> # File "/home/daven/src/emscripen/emscripten/tools/asm_module.py", line
> 24, in __init__
> # self.mem_init_js = re.search(shared.JS.memory_initializer_pattern,
> self.pre_js).group(0)
> #AttributeError: 'NoneType' object has no attribute 'group'
>
> # Sure enough... The same is true with USE_TYPED_ARRAYS=1 as well.
> $ grep 'no memory initializer'
> src/trash/poc/split_worker_with_common_lib/build/worker.js
> #/* no memory initializer */
>
>
>
>
> On Tuesday, February 11, 2014 1:35:43 PM UTC-5, Alon Zakai wrote:
>>
>> This is an intended use case, that sounds like a bug. Any chance you can
>> make a testcase showing it?
>>
>> Fastcomp is likely a better way to improve your iteration time though, as
>> it is much more heavily tested than the static linking feature (at least
>> currently). But please make a testcase if you can, that's how we can make
>> static linking more stable.
>>
>> - Alon
>>
>>
>>
>> On Mon, Feb 10, 2014 at 11:30 PM, Dave Nicponski <[email protected]>wrote:
>>
>>> Hello all,
>>>
>>> Sorry for the terse email, but it's 2:30am here and i'm about to give up
>>> for the night.
>>> I have a project roughly as follows:
>>>
>>> The project contains currently a main method and a set of worker code,
>>> intended to be run in web workers in browsers, or via the webworker-threads
>>> module in nodejs.
>>>
>>> Each of these requires a rather large support library (statically
>>> compiled, rarely changing) to work. It contains things like the protocol
>>> buffer libraries, for instance.
>>>
>>> The support libraries have been precompiled into llvm bitcode. Most of
>>> the development iteration is done on the main and worker modules. As such,
>>> compilation time after modifying either of those is very important to me.
>>>
>>> To minimize this, here's what I had in mind.
>>> 1) compile the static library bitcode into a javascript MAIN library.
>>>
>>> iterate one of:
>>>
>>> 2a) Modify and compile main() module into a SIDE library
>>> 2b) run "emlink.py static.js, main.js main_out.js"
>>>
>>> or
>>>
>>> 3a) Modify and compile worker module into a SIDE library with
>>> BUILD_AS_WORKER=1 option
>>> 3b) run "emlink.py static.js worker.js worker_out.js"
>>>
>>> Sadly, step 3a reliably fails since worker.js doesn't have a memory
>>> initialization section! This causes tools/asm_module.py to fail trying to
>>> spot memory_initializer_patter, when in fact only
>>> no_memory_initilizer_pattern is present in that file (worker.js).
>>>
>>> Is this know / meant to be an unsupported use case? Is there an easy
>>> workaround?
>>> Tomorrow i'll try using fastcomp (which i suspect would eventually
>>> work), but for tonight i don't have the energy.
>>>
>>> I'll supply additional info tomorrow if requested.
>>>
>>> Thanks,
>>> -dave-
>>>
>>> --
>>> You received this message because you are subscribed to the Google
>>> Groups "emscripten-discuss" group.
>>> To unsubscribe from this group and stop receiving emails from it, send
>>> an email to [email protected].
>>> For more options, visit https://groups.google.com/groups/opt_out.
>>>
>>
>>
--
You received this message because you are subscribed to the Google Groups
"emscripten-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
For more options, visit https://groups.google.com/groups/opt_out.
