I'm unconvinced about the the focus on mem_init - unless the backend output
is very confused, mem_init is a literal string like
/* memory initializer */ allocate([70,83,[...SNIP...],0,0,0,0], "i8",
ALLOC_NONE, Runtime.GLOBAL_BASE);
and so shouldn't have strange bytes in it. I don't think decoding the js
array in the python string tells us much either because regardless of what
it comes to, it's not going to have any impact on Python.
global_initializers is from json deserialisation so should be reasonably
safe (and I doubt clang would choose unicode names for initialisers).
Which leaves us with pre, which can be described as "the bit before
'EMSCRIPTEN_END_FUNCS'", per pre, post = glue.split('//
EMSCRIPTEN_END_FUNCS')
Which seems to imply the bad byte is in an emscripten library somewhere.
It just so happens I got rid of a '\xc2\xa0' byte sequence from an
emscripten library recently -
https://github.com/kripken/emscripten/pull/2904
This fix got into 1.26.0 - can you try that?
Thanks
Aidan
On 30 October 2014 17:30, Robert Goulet <[email protected]> wrote:
> Unfortunately I won't be able to dig into this for the next two-three
> weeks, but I will try to find the offending commit once I'm back. Thanks!
>
> Sent from mobile, please excuse typos.
>
> On Oct 29, 2014, at 4:59 PM, Alon Zakai <[email protected]> wrote:
>
> Not sure what you mean exactly, but in the python script, it is handling
> output from the JS backend, which handled both static initialization and
> executable code, basically everything.
>
> If it works on 1.22.0 but not on 1.25.0, one option is to bisect to find
> the breaking version (or better, commit).
>
> Another option is to reduce it manually, link less and less things in the
> final link (might need -s LINKABLE=1 to avoid dce killing everything once
> you remove the main() method), then continue narrowing it down to a single
> method, which hopefully you can provide a public testcase based on that.
>
> - Alon
>
>
> On Wed, Oct 29, 2014 at 12:55 PM, Robert Goulet <
> [email protected]> wrote:
>
>> That's very cleaver, I didn't think to convert those to ascii.
>>
>> Btw I completely uninstalled Emscripten 1.25.0 and re-installed it and
>> the issue was still present. I uninstalled it again and re-installed 1.22.0
>> and everything works fine. So we can rule that one out. Now back to
>> 1.25.0...
>>
>> Definitively a string in our code, so I am guessing this must be a change
>> in Clang since I wouldn't know how Emscripten could possibly introduce
>> issues like that. I will have to dig into Clang's compiler changes/options
>> to see if I can do anything about it. Its just annoying that the whole
>> engine was running perfectly in 1.22.0 and now it won't even finish linking.
>>
>> That file being processed by Emscripten in this case, is it only the .mem
>> content (static variables initialization?) or does it also process the
>> whole byte code?
>>
>>
>> On Wednesday, October 29, 2014 2:44:34 PM UTC-4, Alon Zakai wrote:
>>>
>>> Converting those last bytes to a string gives
>>> "ght_direction\x00:\x00\x00\x00\x00\x00\x00\x00slices\x00\x00d". There
>>> isn't anything in emscripten, LLVM or clang with the characters
>>> "ght_direction". Perhaps look for where that is in your codebase? Might be
>>> you have a function with non-unicode characters in its name or in a
>>> parameter name, and somehow clang and LLVM preserve that oddness when
>>> compiling.
>>>
>>> Another odd possibility is somehow emscripten.py (the file that crashes)
>>> got corrupted somehow, and contains a non-ascii character in the text
>>> processing right before that crash? Using a new copy of emscripten could
>>> rule that out.
>>>
>>> - Alon
>>>
>>>
>>> On Wed, Oct 29, 2014 at 11:02 AM, Robert Goulet <[email protected]>
>>> wrote:
>>>
>>>> Cool that worked.
>>>>
>>>> It prints 16,1 for 234335:234339
>>>>
>>>> It seems the offending character is the comma?
>>>>
>>>> There I printed 234300:234400 and I get:
>>>>
>>>> 103,104,116,95,100,105,114,101,99,116,105,111,110,0,58,0,
>>>> 0,0,0,0,0,0,115,108,105,99,101,115,0,0,100,
>>>>
>>>> I really don't see what is wrong with these characters... encoding with
>>>> utf-8 doesn't change the print output.
>>>>
>>>>
>>>>
>>>> On Wednesday, October 29, 2014 1:12:09 PM UTC-4, Bruce Mitchener wrote:
>>>>>
>>>>> I am half asleep but maybe:
>>>>>
>>>>> print >> sys.stderr, mem_init...
>>>>>
>>>>> http://stackoverflow.com/questions/5574702/how-to-print-to-
>>>>> stderr-in-python
>>>>>
>>>>> - Bruce
>>>>>
>>>>> Sent from my iPhone
>>>>>
>>>>> On Oct 30, 2014, at 12:05 AM, Robert Goulet <[email protected]>
>>>>> wrote:
>>>>>
>>>>> Ok I added the try/except block and I think it went into the except
>>>>> part but I can't see it; the print function does not write into the Visual
>>>>> Studio output window. So I tried to run the link command (emcc.bat with
>>>>> all
>>>>> the same parameters) and it still doesn't print the error in the console.
>>>>> How do I get this print function to write in the console (or VS output
>>>>> window) ?
>>>>>
>>>>> On Wednesday, October 29, 2014 11:10:30 AM UTC-4, Bruce Mitchener
>>>>> wrote:
>>>>>>
>>>>>> Sure.
>>>>>>
>>>>>> Around that block of code, wrap it in a try / except block:
>>>>>>
>>>>>> try:
>>>>>> ...
>>>>>> except:
>>>>>> print mem_init[234335:234339]
>>>>>>
>>>>>> And I think that'll work. If that print errors, you may need to try
>>>>>> printing mem_init[...].encode('utf-8') instead ...
>>>>>>
>>>>>> - Bruce
>>>>>>
>>>>>>
>>>>>> On Wed, Oct 29, 2014 at 9:44 PM, Robert Goulet <[email protected]
>>>>>> > wrote:
>>>>>>
>>>>>>> Sure I would like to debug it. I have never debugged Emscripten
>>>>>>> before so I will need advices about how to do what you just said. Can
>>>>>>> you
>>>>>>> provide tips?
>>>>>>>
>>>>>>> On Wednesday, October 29, 2014 10:34:37 AM UTC-4, Bruce Mitchener
>>>>>>> wrote:
>>>>>>>>
>>>>>>>> Someone else reported this same bug a week or two ago on this list.
>>>>>>>>
>>>>>>>> My suggestion to them is the same that I will make to you:
>>>>>>>>
>>>>>>>> Modify the code to dump some of that value around the area that it
>>>>>>>> is reporting the problem. What's in position 234337 of mem_init?
>>>>>>>> What's
>>>>>>>> around there?
>>>>>>>>
>>>>>>>> That should tell you something useful (and it is how I tracked down
>>>>>>>> my problem when I ran into this in the past).
>>>>>>>>
>>>>>>>> - Bruce
>>>>>>>>
>>>>>>>>
>>>>>>>> On Wed, Oct 29, 2014 at 9:31 PM, Robert Goulet <
>>>>>>>> [email protected]> wrote:
>>>>>>>>
>>>>>>>>> Yeah unfortunately the code is private, can't share it. I wasn't
>>>>>>>>> able to reproduce it with a simple new project. The project that
>>>>>>>>> reproduce
>>>>>>>>> it links with about 20 static libraries (.bc), and it seems that if I
>>>>>>>>> remove about 3/4 of them, the link works. It doesn't seems to be tied
>>>>>>>>> to a
>>>>>>>>> specific lib, but more about the total size of the lib payload.
>>>>>>>>> That's very
>>>>>>>>> odd indeed.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Wednesday, October 29, 2014 10:20:18 AM UTC-4, Robert Goulet
>>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>> Nope, that didn't work...
>>>>>>>>>>
>>>>>>>>>> Link:
>>>>>>>>>> Traceback (most recent call last):
>>>>>>>>>> File "C:\Program
>>>>>>>>>> Files\Emscripten\emscripten\1.25.0\emscripten.py",
>>>>>>>>>> line 1578, in <module>
>>>>>>>>>> _main(environ=os.environ)
>>>>>>>>>> File "C:\Program
>>>>>>>>>> Files\Emscripten\emscripten\1.25.0\emscripten.py",
>>>>>>>>>> line 1566, in _main
>>>>>>>>>> temp_files.run_and_clean(lambda: main(
>>>>>>>>>> File "C:\Program
>>>>>>>>>> Files\Emscripten\emscripten\1.25.0\tools\tempfiles.py",
>>>>>>>>>> line 39, in run_and_clean
>>>>>>>>>> return func()
>>>>>>>>>> File "C:\Program
>>>>>>>>>> Files\Emscripten\emscripten\1.25.0\emscripten.py",
>>>>>>>>>> line 1574, in <lambda>
>>>>>>>>>> DEBUG_CACHE=DEBUG_CACHE,
>>>>>>>>>> File "C:\Program
>>>>>>>>>> Files\Emscripten\emscripten\1.25.0\emscripten.py",
>>>>>>>>>> line 1461, in main
>>>>>>>>>> jcache=jcache, temp_files=temp_files, DEBUG=DEBUG,
>>>>>>>>>> DEBUG_CACHE=DEBUG_CACHE)
>>>>>>>>>> File "C:\Program
>>>>>>>>>> Files\Emscripten\emscripten\1.25.0\emscripten.py",
>>>>>>>>>> line 911, in emscript_fast
>>>>>>>>>> %s''' % (str(mem_init).count(',')+1, global_initializers,
>>>>>>>>>> str(mem_init))) # XXX wrong size calculation!
>>>>>>>>>> UnicodeDecodeError: 'ascii' codec can't decode byte 0xc2 in
>>>>>>>>>> position 234337: ordinal not in range(128)
>>>>>>>>>> Traceback (most recent call last):
>>>>>>>>>> File "C:\Program Files\Emscripten\emscripten\1.25.0\\emcc",
>>>>>>>>>> line 1259, in <module>
>>>>>>>>>> final = shared.Building.emscripten(final,
>>>>>>>>>> append_ext=False, extra_args=extra_args)
>>>>>>>>>> File "C:\Program
>>>>>>>>>> Files\Emscripten\emscripten\1.25.0\tools\shared.py",
>>>>>>>>>> line 1464, in emscripten
>>>>>>>>>> assert os.path.exists(filename + '.o.js'), 'Emscripten
>>>>>>>>>> failed to generate .js'
>>>>>>>>>> AssertionError: Emscripten failed to generate .js
>>>>>>>>>> C:\Program Files (x86)\MSBuild\Microsoft.Cpp\v4
>>>>>>>>>> .0\Platforms\Emscripten\Microsoft.Cpp.Emscripten.Targets(117,5):
>>>>>>>>>> error MSB6006: "C:\Program
>>>>>>>>>> Files\Emscripten\emscripten\1.25.0\emcc.bat"
>>>>>>>>>> exited with code 1.
>>>>>>>>>>
>>>>>>>>>> Build FAILED.
>>>>>>>>>>
>>>>>>>>>> On Tuesday, October 28, 2014 4:24:45 PM UTC-4, Alon Zakai wrote:
>>>>>>>>>>>
>>>>>>>>>>> Hmm, never seen that before. Odd.
>>>>>>>>>>>
>>>>>>>>>>> If this code is open, can you provide the bitcode for testing?
>>>>>>>>>>> If not, one random guess - if you change that line (911 in
>>>>>>>>>>> emscripten.py)
>>>>>>>>>>> to replace
>>>>>>>>>>>
>>>>>>>>>>> mem_init
>>>>>>>>>>>
>>>>>>>>>>> (in both places) with
>>>>>>>>>>>
>>>>>>>>>>> str(mem_init)
>>>>>>>>>>>
>>>>>>>>>>> , does that help?
>>>>>>>>>>>
>>>>>>>>>>> - Alon
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> On Tue, Oct 28, 2014 at 11:04 AM, Robert Goulet <
>>>>>>>>>>> [email protected]> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> Hi all,
>>>>>>>>>>>>
>>>>>>>>>>>> I just upgraded my project to Emscripten 1.25.0 and I have a
>>>>>>>>>>>> new link error that wasn't there with Emscripten 1.22.0.
>>>>>>>>>>>>
>>>>>>>>>>>> Here is what the linker outputs:
>>>>>>>>>>>>
>>>>>>>>>>>> Link:
>>>>>>>>>>>> Traceback (most recent call last):
>>>>>>>>>>>> File "C:\Program
>>>>>>>>>>>> Files\Emscripten\emscripten\1.25.0\emscripten.py",
>>>>>>>>>>>> line 1578, in <module>
>>>>>>>>>>>> _main(environ=os.environ)
>>>>>>>>>>>> File "C:\Program
>>>>>>>>>>>> Files\Emscripten\emscripten\1.25.0\emscripten.py",
>>>>>>>>>>>> line 1566, in _main
>>>>>>>>>>>> temp_files.run_and_clean(lambda: main(
>>>>>>>>>>>> File "C:\Program
>>>>>>>>>>>> Files\Emscripten\emscripten\1.25.0\tools\tempfiles.py",
>>>>>>>>>>>> line 39, in run_and_clean
>>>>>>>>>>>> return func()
>>>>>>>>>>>> File "C:\Program
>>>>>>>>>>>> Files\Emscripten\emscripten\1.25.0\emscripten.py",
>>>>>>>>>>>> line 1574, in <lambda>
>>>>>>>>>>>> DEBUG_CACHE=DEBUG_CACHE,
>>>>>>>>>>>> File "C:\Program
>>>>>>>>>>>> Files\Emscripten\emscripten\1.25.0\emscripten.py",
>>>>>>>>>>>> line 1461, in main
>>>>>>>>>>>> jcache=jcache, temp_files=temp_files, DEBUG=DEBUG,
>>>>>>>>>>>> DEBUG_CACHE=DEBUG_CACHE)
>>>>>>>>>>>> File "C:\Program
>>>>>>>>>>>> Files\Emscripten\emscripten\1.25.0\emscripten.py",
>>>>>>>>>>>> line 911, in emscript_fast
>>>>>>>>>>>> %s''' % (mem_init.count(',')+1, global_initializers,
>>>>>>>>>>>> mem_init)) # XXX wrong size calculation!
>>>>>>>>>>>> UnicodeDecodeError: 'ascii' codec can't decode byte 0xc2 in
>>>>>>>>>>>> position 234337: ordinal not in range(128)
>>>>>>>>>>>> Traceback (most recent call last):
>>>>>>>>>>>> File "C:\Program Files\Emscripten\emscripten\1.25.0\\emcc",
>>>>>>>>>>>> line 1259, in <module>
>>>>>>>>>>>> final = shared.Building.emscripten(final,
>>>>>>>>>>>> append_ext=False, extra_args=extra_args)
>>>>>>>>>>>> File "C:\Program
>>>>>>>>>>>> Files\Emscripten\emscripten\1.25.0\tools\shared.py",
>>>>>>>>>>>> line 1464, in emscripten
>>>>>>>>>>>> assert os.path.exists(filename + '.o.js'), 'Emscripten
>>>>>>>>>>>> failed to generate .js'
>>>>>>>>>>>> AssertionError: Emscripten failed to generate .js
>>>>>>>>>>>> C:\Program Files (x86)\MSBuild\Microsoft.Cpp\v4
>>>>>>>>>>>> .0\Platforms\Emscripten\Microsoft.Cpp.Emscripten.Targets(117,5):
>>>>>>>>>>>> error MSB6006: "C:\Program
>>>>>>>>>>>> Files\Emscripten\emscripten\1.25.0\emcc.bat"
>>>>>>>>>>>> exited with code 1.
>>>>>>>>>>>>
>>>>>>>>>>>> Here is the link command-line use:
>>>>>>>>>>>>
>>>>>>>>>>>> emcc -o f:/dev/build/main.html -O2 f:/dev/build/game/game.bc
>>>>>>>>>>>> -Wno-warn-absolute-paths -s NO_EXIT_RUNTIME=1 -s
>>>>>>>>>>>> TOTAL_MEMORY=64*1024*1024
>>>>>>>>>>>> --preload-file f:/dev/data/gamedata@/
>>>>>>>>>>>>
>>>>>>>>>>>> I did a clean/full rebuild to be sure there wasn't any
>>>>>>>>>>>> previously built files around and I always get the same results.
>>>>>>>>>>>> Did anyone
>>>>>>>>>>>> ever had this problem before? How do I debug this?
>>>>>>>>>>>> Thanks!
>>>>>>>>>>>>
>>>>>>>>>>>> --
>>>>>>>>>>>> You received this message because you are subscribed to the
>>>>>>>>>>>> Google Groups "emscripten-discuss" group.
>>>>>>>>>>>> To unsubscribe from this group and stop receiving emails from
>>>>>>>>>>>> it, send an email to emscripten-discuss+unsubscribe
>>>>>>>>>>>> @googlegroups.com.
>>>>>>>>>>>> For more options, visit https://groups.google.com/d/optout.
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> --
>>>>>>>>> You received this message because you are subscribed to the Google
>>>>>>>>> Groups "emscripten-discuss" group.
>>>>>>>>> To unsubscribe from this group and stop receiving emails from it,
>>>>>>>>> send an email to [email protected].
>>>>>>>>> For more options, visit https://groups.google.com/d/optout.
>>>>>>>>>
>>>>>>>>
>>>>>>>> --
>>>>>>> You received this message because you are subscribed to the Google
>>>>>>> Groups "emscripten-discuss" group.
>>>>>>> To unsubscribe from this group and stop receiving emails from it,
>>>>>>> send an email to [email protected].
>>>>>>> For more options, visit https://groups.google.com/d/optout.
>>>>>>>
>>>>>>
>>>>>> --
>>>>> You received this message because you are subscribed to the Google
>>>>> Groups "emscripten-discuss" group.
>>>>> To unsubscribe from this group and stop receiving emails from it, send
>>>>> an email to [email protected].
>>>>> For more options, visit https://groups.google.com/d/optout.
>>>>>
>>>>> --
>>>> You received this message because you are subscribed to the Google
>>>> Groups "emscripten-discuss" group.
>>>> To unsubscribe from this group and stop receiving emails from it, send
>>>> an email to [email protected].
>>>> For more options, visit https://groups.google.com/d/optout.
>>>>
>>>
>>> --
>> You received this message because you are subscribed to the Google Groups
>> "emscripten-discuss" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to [email protected].
>> For more options, visit https://groups.google.com/d/optout.
>>
>
> --
> You received this message because you are subscribed to a topic in the
> Google Groups "emscripten-discuss" group.
> To unsubscribe from this topic, visit
> https://groups.google.com/d/topic/emscripten-discuss/Kfn59Uv8MBs/unsubscribe
> .
> To unsubscribe from this group and all its topics, send an email to
> [email protected].
> For more options, visit https://groups.google.com/d/optout.
>
> --
> You received this message because you are subscribed to the Google Groups
> "emscripten-discuss" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to [email protected].
> For more options, visit https://groups.google.com/d/optout.
>
--
You received this message because you are subscribed to the Google Groups
"emscripten-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
For more options, visit https://groups.google.com/d/optout.
