The "code is too specialized" error is confusing, I'll fix that now. The issue is you are running the code in a JS engine which lacks full typed array support. Look in the generated code for what is tested, and you can check specifically for what is missing.
The non-typed arrays case is an old codegen option, and will not work with modules. - Alon On Wed, Feb 12, 2014 at 2:21 PM, Dave Nicponski <[email protected]>wrote: > > > On Wednesday, February 12, 2014 5:15:50 PM UTC-5, Dave Nicponski wrote: >> >> >> >> On Wednesday, February 12, 2014 5:13:32 PM UTC-5, Dave Nicponski wrote: >>> >>> Hello, and thanks for the reply Alon. >>> >>> In the process of making a simple repo case, i better understand the >>> problem(s) involved here. >>> Executive summary: When a worker is run under nodejs with typed arrays >>> option set for the worker, an emscripten assertion is triggered: >>> >>> "Assertion failed: Cannot fallback to non-typed array case: Code is too >>> specialized" >>> >>> However, compiling the worker with USE_TYPED_ARRAYS=0 causes "no memory >>> initializer" to be inserted into the generated code, instead of the >>> traditional memory initializer block. >>> This causes the emlink.py script to fail when it cannot find the memory >>> initializer section. >>> >>> Attached find a pretty minimal repro. Note that where nodejs triggers >>> the above listed emscripten assertion failure, the same code appears to >>> work fine in a browser (at least under latest firefox). It's not entirely >>> clear how i should continue development. Using the browser is incredibly >>> inefficient for development for me compared to nodejs. I suspect a 'quick >>> fix' here would be to fix the emlink.py script to handle this case >>> correctly. Meanwhile, i'll try giving fastcomp a try if i can get it to >>> work for me. >>> >>> Thanks, >>> -dave- >>> >>> -------8<-------8<-------8<-------8<-------8<-------8<------- >>> Building sequence follows >>> -------8<-------8<-------8<-------8<-------8<-------8<------- >>> # This shows the simple clang setup I use. It mostly just sets up the >>> various >>> # include paths and forces C++ mode and c++11 >>> $ echo $(./clang_compile_flags.py) >>> -Qunused-arguments -Wall -Wextra -Werror -Wno-unused-parameter >>> -fexceptions -std=c++11 -x c++ -iquote ./src -iquote >>> /home/daven/src/emscripen/emscripten/system/include/ -I . -isystem >>> ./include/protorpc/ -isystem ./include/openssl/ -isystem >>> ./include/tinyxml/tinyxml/ >>> >>> # Build the MAIN (shared, static) library for main() >>> $ emcc $(./clang_compile_flags.py) -O1 -s MAIN_MODULE=1 -o >>> src/trash/poc/split_worker_with_common_lib/build/lib.js >>> src/trash/poc/split_worker_with_common_lib/lib.cc >>> >>> # Build the MAIN (shared, static) library for worker >>> $ emcc $(./clang_compile_flags.py) -O1 -s MAIN_MODULE=1 -s >>> BUILD_AS_WORKER=1 -o src/trash/poc/split_worker_with_common_lib/build/lib.js >>> src/trash/poc/split_worker_with_common_lib/lib.cc >>> >>> >> Oops, this should obviously have read: >> $ emcc $(./clang_compile_flags.py) -O1 -s MAIN_MODULE=1 -s >> BUILD_AS_WORKER=1 -o src/trash/poc/split_worker_ >> with_common_lib/build/lib_worker.js src/trash/poc/split_worker_ >> with_common_lib/lib.cc >> > > Ahh, remembered one more important detail. > I slightly modified emscripten/src/shell.js to enable webworker emulated > support in nodejs. > > shell.js modifications: > > src/shell.js-67- }; > src/shell.js-68- > src/shell.js-69- Module['arguments'] = process['argv'].slice(2); > src/shell.js-70-+ > src/shell.js-71-+ // XXX(daven): Hack in webworker emulation. > src/shell.js:72:+ Module['Worker'] = Worker = > require('webworker-threads').Worker; > src/shell.js-73-+ // XXX > src/shell.js-74- > src/shell.js-75- module['exports'] = Module; > src/shell.js-76-} > src/shell.js-77-else if (ENVIRONMENT_IS_SHELL) { > > This uses the nodejs webworker-threads module (npm install > webworker-threads), which works fine in other tests i've done. > > Cheers, > -dave- > > > >> >> >>> # Build the SIDE (main module) library >>> $ emcc $(./clang_compile_flags.py) -O1 -s SIDE_MODULE=1 -o >>> src/trash/poc/split_worker_with_common_lib/build/main.js >>> src/trash/poc/split_worker_with_common_lib/main.cc >>> >>> # Link the main() module (SUCCESS) >>> $ emlink.py src/trash/poc/split_worker_with_common_lib/build/{lib, >>> main,main_module}.js >>> #Main module: src/trash/poc/split_worker_with_common_lib/build/lib.js >>> #Side module: src/trash/poc/split_worker_with_common_lib/build/main.js >>> #Output: src/trash/poc/split_worker_with_common_lib/build/main_module.js >>> >>> # Build the SIDE (worker module) library >>> $ emcc $(./clang_compile_flags.py) -O1 -s BUILD_AS_WORKER=1 -s >>> SIDE_MODULE=1 -s EXPORTED_FUNCTIONS="['_do_work']" -o >>> src/trash/poc/split_worker_with_common_lib/build/worker.js >>> src/trash/poc/split_worker_with_common_lib/worker.cc >>> >>> # Link the worker do_work() module >>> $ emlink.py src/trash/poc/split_worker_with_common_lib/build/{lib_ >>> worker,worker,worker_module}.js >>> #Main module: src/trash/poc/split_worker_with_common_lib/build/lib.js >>> #Side module: src/trash/poc/split_worker_with_common_lib/build/worker.js >>> #Output: src/trash/poc/split_worker_with_common_lib/build/worker_ >>> module.js >>> >>> # Try running this in nodejs. It fails in emscripten worker setup code, >>> # apparently. It works fine from a browser (firefox, at least) though. >>> $ nodejs main_module.js >>> Lib function returns This is some string >>> Assertion failed: Cannot fallback to non-typed array case: Code is too >>> specialized >>> >>> # Try again building SIDE (worker module) library without typed arrays, >>> so >>> # this can be run in nodejs >>> $ emcc $(./clang_compile_flags.py) -O1 -s USE_TYPED_ARRAYS=0 -s >>> BUILD_AS_WORKER=1 -s SIDE_MODULE=1 -s EXPORTED_FUNCTIONS="['_do_work']" >>> -o src/trash/poc/split_worker_with_common_lib/build/worker.js >>> src/trash/poc/split_worker_with_common_lib/worker.cc >>> >>> # Try again relinking the worker do_work module >>> $ emlink.py src/trash/poc/split_worker_with_common_lib/build/{lib_ >>> worker,worker,worker_module}.js >>> #Main module: src/trash/poc/split_worker_with_common_lib/build/lib.js >>> #Side module: src/trash/poc/split_worker_with_common_lib/build/worker.js >>> #Output: src/trash/poc/split_worker_with_common_lib/build/worker_ >>> module.js >>> #Traceback (most recent call last): >>> # File "/home/daven/symlinks/override/emlink.py", line 33, in <module> >>> # run() >>> # File "/home/daven/symlinks/override/emlink.py", line 27, in run >>> # side = AsmModule(side) >>> # File "/home/daven/src/emscripen/emscripten/tools/asm_module.py", >>> line 24, in __init__ >>> # self.mem_init_js = re.search(shared.JS.memory_initializer_pattern, >>> self.pre_js).group(0) >>> #AttributeError: 'NoneType' object has no attribute 'group' >>> >>> # Sure enough... The same is true with USE_TYPED_ARRAYS=1 as well. >>> $ grep 'no memory initializer' src/trash/poc/split_worker_ >>> with_common_lib/build/worker.js >>> #/* no memory initializer */ >>> >>> >>> >>> >>> On Tuesday, February 11, 2014 1:35:43 PM UTC-5, Alon Zakai wrote: >>>> >>>> This is an intended use case, that sounds like a bug. Any chance you >>>> can make a testcase showing it? >>>> >>>> Fastcomp is likely a better way to improve your iteration time though, >>>> as it is much more heavily tested than the static linking feature (at least >>>> currently). But please make a testcase if you can, that's how we can make >>>> static linking more stable. >>>> >>>> - Alon >>>> >>>> >>>> >>>> On Mon, Feb 10, 2014 at 11:30 PM, Dave Nicponski >>>> <[email protected]>wrote: >>>> >>>>> Hello all, >>>>> >>>>> Sorry for the terse email, but it's 2:30am here and i'm about to give >>>>> up for the night. >>>>> I have a project roughly as follows: >>>>> >>>>> The project contains currently a main method and a set of worker code, >>>>> intended to be run in web workers in browsers, or via the >>>>> webworker-threads >>>>> module in nodejs. >>>>> >>>>> Each of these requires a rather large support library (statically >>>>> compiled, rarely changing) to work. It contains things like the protocol >>>>> buffer libraries, for instance. >>>>> >>>>> The support libraries have been precompiled into llvm bitcode. Most >>>>> of the development iteration is done on the main and worker modules. As >>>>> such, compilation time after modifying either of those is very important >>>>> to >>>>> me. >>>>> >>>>> To minimize this, here's what I had in mind. >>>>> 1) compile the static library bitcode into a javascript MAIN library. >>>>> >>>>> iterate one of: >>>>> >>>>> 2a) Modify and compile main() module into a SIDE library >>>>> 2b) run "emlink.py static.js, main.js main_out.js" >>>>> >>>>> or >>>>> >>>>> 3a) Modify and compile worker module into a SIDE library with >>>>> BUILD_AS_WORKER=1 option >>>>> 3b) run "emlink.py static.js worker.js worker_out.js" >>>>> >>>>> Sadly, step 3a reliably fails since worker.js doesn't have a memory >>>>> initialization section! This causes tools/asm_module.py to fail trying to >>>>> spot memory_initializer_patter, when in fact only >>>>> no_memory_initilizer_pattern is present in that file (worker.js). >>>>> >>>>> Is this know / meant to be an unsupported use case? Is there an easy >>>>> workaround? >>>>> Tomorrow i'll try using fastcomp (which i suspect would eventually >>>>> work), but for tonight i don't have the energy. >>>>> >>>>> I'll supply additional info tomorrow if requested. >>>>> >>>>> Thanks, >>>>> -dave- >>>>> >>>>> -- >>>>> You received this message because you are subscribed to the Google >>>>> Groups "emscripten-discuss" group. >>>>> To unsubscribe from this group and stop receiving emails from it, send >>>>> an email to [email protected]. >>>>> For more options, visit https://groups.google.com/groups/opt_out. >>>>> >>>> >>>> -- > You received this message because you are subscribed to the Google Groups > "emscripten-discuss" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > For more options, visit https://groups.google.com/groups/opt_out. > -- You received this message because you are subscribed to the Google Groups "emscripten-discuss" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/groups/opt_out.
