Re: Moving large data between JS & wasm with IDL binder

[Beuc]

Hi,

On 14/07/2020 18:49, キャロウ マーク wrote:
>
>
>> On Jul 14, 2020, at 8:37, 'Sam Clegg' via emscripten-discuss
>> > <mailto:emscripten-discuss@googlegroups.com>> wrote:
>>
>>
>>
>> On Mon, Jul 13, 2020 at 6:43 PM キャロウ　マーク > <mailto:git...@callow.im>> wrote:
>>
>> Is is possible to use a typed array created on HEAP8 with
>> XMLHttpRequest as the destination for the incoming data?
>>
>>
>> I don't think so no.   IIRC  XMLHttpRequest API pre-dates the typed
>> array APIs.
>
> XMLHttpRequest most definitely supports receiving responses into
> ArrayBuffers. You set responseType=‘arrayBuffer’. However now I look
> more closely at the API there doesn’t seem to be any way to provide it
> with the destination ArrayBuffer. It looks like XMLHttpRequest always
> creates it itself which makes my question moot.
One optimization I did for RenPyWeb was importing XMLHttpRequest's
buffer into the filesystem (FS):
https://github.com/renpy/renpy/blob/8b0d3dd4986a5a8656b3a5ad634c62bd52b1afbd/renpy/webloader.py#L64
FS.writeFile(path, new Uint8Array(xhr.response), {canOwn:true});

I'm not sure what's your use case is, it might help.

Cheers!
Beuc

-- 
You received this message because you are subscribed to the Google Groups 
"emscripten-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to emscripten-discuss+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/emscripten-discuss/cb9bca95-877c-3a6d-921a-e9b093f57b33%40beuc.net.

Emscripten mobile and keyboards

[Beuc]

Hi,

Mobile browsers are making progress with WebAssembly support and I see
more interest in handling on-screen keyboards in web games.
(typically so your game's port doesn't get the mobile player stuck with
a mere "whose your name" prompt).

I explored 2 main ideas to make the virtual keyboard pop up in SDL2 web
applications in a generic way, typically on SDL_StartTextInput
<https://wiki.libsdl.org/SDL_StartTextInput> (as Android SDL2 games do)
but browsers don't make it easy:
https://github.com/emscripten-ports/SDL2/issues/80

At this point I'm not even sure at what layer it would be best to
implement virtual keyboard support (besides re-implementing it in each
application).

What's your experience with mobile webapps and keyboard support?

Cheers!
Beuc

-- 
You received this message because you are subscribed to the Google Groups 
"emscripten-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to emscripten-discuss+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/emscripten-discuss/1cf0d838-9340-e8bb-8977-0c38cff60c9a%40beuc.net.

Re: failed to call xz, please confirm that xz is installed in your system

[Beuc]

Hi Fernando,

You may want to check

https://github.com/kripken/zee.js

which exposes high-level functions of zlib through a secondary WASM module.

Cheers!
Beuc

On 12/06/2020 19:50, 'Thomas Lively' via emscripten-discuss wrote:
> WebAssembly in the browser has no notion of processes (or shells), so
> `system` will not work. Whether you compile xz to wasm or use the npm
> package, you will probably still have to modify the original code to
> call out to some JS that invokes xz rather than using `system`.
>
> On Fri, Jun 12, 2020 at 7:48 AM Fernando Bitti Loureiro
> mailto:fernando.bi...@gmail.com>> wrote:
>
>
> Let me resort again to the support of this list.
>
> I successfully compiled a tool to wasm just to find out it uses
> the C system command:
> https://linux.die.net/man/3/system
> ... to call another program, https://github.com/xz-mirror/xz.
>
> The wasm program still works when I run the program through node
> on Linux CLI, but it obviously fails when I run it on a browser,
> with the error on the subject of this message:
> "failed to call xz, please confirm that xz is installed in your
> system"
> (this is an error message created by the tool itself, not Emscripten)
>
> Researching this specific topic is being really difficult. I tried
> to find a single example of a C program with a system call being
> compiled to wasm and running on a browser.
> But, because the words "system" and "shell" mean different things
> and are used in a variety of contexts, I get a lot of unrelated
> search results, hence I'm resorting to this discussion list.
>
> I thought about the following possible solutions:
> 1. touch the original C code to remove the system shell calls to
> the xz tool and handle the compression on JS
> via https://www.npmjs.com/package/xz
> 2. compile the xz tool to wasm and (somehow) make calls between
> the two wasm functions.
> (is this possible)?
>
> Any guidance is welcome.
>
> Thank you so much,
>
> Fernando
> -- 
> You received this message because you are subscribed to the Google
> Groups "emscripten-discuss" group.
> To unsubscribe from this group and stop receiving emails from it,
> send an email to emscripten-discuss+unsubscr...@googlegroups.com
> <mailto:emscripten-discuss+unsubscr...@googlegroups.com>.
> To view this discussion on the web visit
> 
> https://groups.google.com/d/msgid/emscripten-discuss/3e2c0c37-1f99-4df1-88c0-0040a2c14438o%40googlegroups.com
> 
> <https://groups.google.com/d/msgid/emscripten-discuss/3e2c0c37-1f99-4df1-88c0-0040a2c14438o%40googlegroups.com?utm_medium=email_source=footer>.
>
> -- 
> You received this message because you are subscribed to the Google
> Groups "emscripten-discuss" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to emscripten-discuss+unsubscr...@googlegroups.com
> <mailto:emscripten-discuss+unsubscr...@googlegroups.com>.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/emscripten-discuss/CAJZD_EWOU8qj6KNEVORGvnaktCujrUXVRJkW_fcaY%3DnHMOnAYw%40mail.gmail.com
> <https://groups.google.com/d/msgid/emscripten-discuss/CAJZD_EWOU8qj6KNEVORGvnaktCujrUXVRJkW_fcaY%3DnHMOnAYw%40mail.gmail.com?utm_medium=email_source=footer>.

-- 
You received this message because you are subscribed to the Google Groups 
"emscripten-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to emscripten-discuss+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/emscripten-discuss/71d48c16-fc8b-5ed3-654b-e2c77b2ebaa2%40beuc.net.

Re: Real-time chat replacement

[Beuc]

Independently of the topic /per se/, I wish I didn't see herd mentality
as a decision basis in a R project - hurts my feelings.

Cheers!
Beuc

On 18/01/2020 01:53, Alon Zakai wrote:
> I definitely get that there are a lot of reasonable opinions on this.
> I honestly agree with most of the opinions (I wish discord were more
> open, for example), and I don't have a strong preference myself.
>
> But I think this is bigger than this project - as Sam said, the
> WebAssembly community has been using discord, and I think it would
> take an extremely strong reason for us to not continue with them
> there. This is sort of like github - I agree with the arguments
> against using it (not fully open, too centralized, policy issues,
> etc.) but in the end that's where the community is, so for most
> projects it's the place that is easiest and takes the least effort -
> in this case, almost no effort, as the wasm community already uses
> that discord and even has an #emscripten room.
>
> (I didn't know LLVM also uses discord - that's even more reason, then.)
>
> If people want to experiment with a bridge from discord to IRC or
> something like that, I'd be interested to see that happen!
>
> - Alon
>
>
>
> On Fri, Jan 17, 2020 at 10:53 AM Sam Clegg  <mailto:s...@google.com>> wrote:
>
> FWIW I think mozilla switched to https://matrix.org/.
>
> But LLVM and WebAssembly are both on discord now.     Since a lot
> of us also work on those projects I (personally) think discord
> makes sense for now.
>
> This might be a case of Alon having to make an DBFL decision
> because it unlikey people we agree here.
>
>
> On Fri, Jan 17, 2020 at 10:26 AM Shlomi Fish
> mailto:shlo...@shlomifish.org>> wrote:
>
> Hi Alon,
>
> On Wed, 15 Jan 2020 13:37:28 -0800
> Alon Zakai mailto:alonza...@gmail.com>>
> wrote:
>
> > Hi everyone,
> >
> > In around a month Mozilla will remove the irc.mozilla.org
> <http://irc.mozilla.org> IRC server we've
> > been using for #emscripten. So we need another option for
> real-time
> > discussion.
> >
> > One option is discord, where the wasm community has been
> gathering, and
> > there's already an #emscripten room there that someone set up,
> >
> >
> https://discordapp.com/channels/453584038356058112/590215642444202044
> (docs
> > PR: https://github.com/emscripten-core/emscripten/pull/10165)
> >
> > I don't particularly have an opinion between the options in
> this space.
> > Discord seems reasonable to go with for now since as I said
> it's already
> > used by wasm in general which obviously overlaps with
> emscripten users
> > quite a lot. But if there's interest, perhaps we could also
> set up a bridge
> > to other places people want, or consider other options.
> >
> > Thoughts?
> >
>
> I dislike discord's web UI because it keeps emitting beeps and the
> "servers"/chatrooms multiplicity is confusing. I also don't
> like slack too much.
> I like freenode for IRC and also use the telegram desktop client.
>
> > - Alon
> >
>
>
>
> -- 
>
> Shlomi Fish       https://www.shlomifish.org/
> Optimising Code for Speed - https://shlom.in/optimise
>
> The worst way to waste your time is to never waste it.
>
> Please reply to list if it's a mailing list post -
> http://shlom.in/reply .
>
> -- 
> You received this message because you are subscribed to the
> Google Groups "emscripten-discuss" group.
> To unsubscribe from this group and stop receiving emails from
> it, send an email to
> emscripten-discuss+unsubscr...@googlegroups.com
> <mailto:emscripten-discuss%2bunsubscr...@googlegroups.com>.
> To view this discussion on the web visit
> 
> https://groups.google.com/d/msgid/emscripten-discuss/20200117202608.1d671515%40telaviv1.shlomifish.org.
>
> -- 
> You received this message because you are subscribed to the Google
> Groups "emscripten-discuss" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to emscripten-discuss+unsubscr...@googlegroups.com
> <mailto:emscripten-discuss+unsubscr...@googlegroups.com>.
> To view this discussion on the web visit
> h

Re: Real-time chat replacement

[Beuc]

Hi,

On 15/01/2020 22:37, Alon Zakai wrote:
> Hi everyone,
>
> In around a month Mozilla will remove the irc.mozilla.org
> <http://irc.mozilla.org> IRC server we've been using for #emscripten.
> So we need another option for real-time discussion.
>
> One option is discord, where the wasm community has been gathering,
> and there's already an #emscripten room there that someone set up,
>
> https://discordapp.com/channels/453584038356058112/590215642444202044
> (docs PR: https://github.com/emscripten-core/emscripten/pull/10165)
>
> I don't particularly have an opinion between the options in this
> space. Discord seems reasonable to go with for now since as I said
> it's already used by wasm in general which obviously overlaps with
> emscripten users quite a lot. But if there's interest, perhaps we
> could also set up a bridge to other places people want, or consider
> other options.
>
> Thoughts?

AFAIK there is no usable free/open source client for Discord, offline
notifications are days-delayed, and users have to accept their ToS with
the usual abusive clauses.

- Beuc

-- 
You received this message because you are subscribed to the Google Groups 
"emscripten-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to emscripten-discuss+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/emscripten-discuss/9583386a-93c2-a152-8dcf-e31f7c6e7766%40beuc.net.

Re: Why is the memory ArrayBuffer called HEAP?

[Beuc]

While I regularly complain about unexpected/undocumented change, I think
Alon's plan is pretty careful and reasonable.

Badly named variables are a perpetual mental hassle, and triggering a
clear error message means the change won't be obscure.

Cheers!
Beuc

On 17/10/2019 17:02, Gabriel Cuvillier wrote:
>
> 100% agree with Floh there.     Changing such a thing is an open door
> to a very obscure, hard to find, and unwanted potential "project
> breaker", for a very minor added value...
>
> Having to live with some minor technical debt due to badly named
> things is acceptable I suppose.
>
>
> Le 17/10/2019 à 16:51, Floh a écrit :
>
>> Isn't there at lot code out there in the wild that would be
>> broken by
>> such as change?
>>
>> Yeah. TBH I'm not a fan of a name change even if the new name is
>> slightly more fitting. That's one of those obscure things that
>> suddenly breaks projects, usually at the worst possible time (even
>> with a deprecation note and period, this stuff is usually ignored).
>> If such a change would come with a real benefit (such as better
>> performance, or less code maintenance in the future) I'd be all for
>> it, but not when it's just one of the many "badly named things" in
>> computing :)

-- 
You received this message because you are subscribed to the Google Groups 
"emscripten-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to emscripten-discuss+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/emscripten-discuss/10c8a820-5a3a-d1a6-dec2-6694dd9631ea%40beuc.net.

Safari mobile support?

[Beuc]

Hi,

It seems that Safari/Safari mobile is supporting WebAssembly but with
limitations, which make it harder to tell the user whether they can play
the game or not, so they get cryptic errors instead.

I eventually got a few proper traces from a Safari Mobile user (I don't
own apple hardware):

For https://renpy.beuc.net/play/?game=the_question.zip:
  wasm instantiation failed!
  Error: Out of executable memory in function at index 15855
(even with freeing memory / restarting the browser)

For the lighter https://www.beuc.net/python-emscripten/demo/
  exception thrown: RangeError: Maximum call stack size exceeded.,
  .wasm-function[705]@[wasm code]
  ... [76 other wasm-function's]
  .wasm-function[5481]@[wasm code]
  wasm-stub@[wasm code]
  Gb@[native code]
  https://www.beuc.net/python-emscripten/demo/index.js:1:182840
  callMain@https://www.beuc.net/python-emscripten/demo/index.js:1:187051
  doRun@https://www.beuc.net/python-emscripten/demo/index.js:1:187638
  https://www.beuc.net/python-emscripten/demo/index.js:1:187790

This is the only browser where this happens.

Do you have similar issues with Safari?
Is there a way to properly detect such limited WebAssembly environment?
(I wouldn't want to blindly blacklist Safari)

Cheers!
Beuc

-- 
You received this message because you are subscribed to the Google Groups 
"emscripten-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to emscripten-discuss+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/emscripten-discuss/615b9d0a-ccfa-ae67-5902-1e0b68c4c46c%40beuc.net.

Re: automated testing of 3rd-party projects

[Beuc]

Hi,

In case you'd like to inspect some build recipes for inspiration :)
https://github.com/renpy/renpyweb/blob/master/scripts/ffmpeg.sh (broke in the 
past)
https://github.com/renpy/renpyweb/blob/master/scripts/libjpeg-turbo.sh (broke 
in the past)
https://github.com/renpy/renpyweb/blob/master/scripts/fribidi.sh (simple 
autotools)
https://github.com/renpy/renpyweb/blob/master/scripts/libpng.sh (less simple 
autotools)
https://github.com/renpy/renpyweb/blob/master/scripts/libzip.sh (cmake)
https://github.com/renpy/renpyweb/blob/master/scripts/SDL2_image.sh 
(cross-compile with webp support)
https://github.com/renpy/renpyweb/tree/master/scripts (for more)

All are currently built static-only, remove --disable-shared for testing 
dynamic builds.
Tarballs URLs are in https://github.com/renpy/renpyweb/blob/master/Makefile

- Beuc

On Fri, Oct 11, 2019 at 04:05:20PM -0700, Alon Zakai wrote:
> Sounds good to me.
> 
> Separate "contrib" directory sounds good too. We may want to move some
> existing ports to there, like say cocos2d.
> 
> On Fri, Oct 11, 2019 at 3:29 PM 'Sam Clegg' via emscripten-discuss <
> emscripten-discuss@googlegroups.com> wrote:
> 
> > On Fri, Oct 11, 2019 at 2:28 PM Alon Zakai  wrote:
> > >
> > > Yeah, I think we can add repos to emscripten-ports for things like that.
> > We don't need to add support in the emscripten repo itself for them, which
> > does make it more like nacl-ports. So there would be two tiers in
> > emscripten-ports support; they'd all be in emscripten-ports, but only some
> > would have code in emscripten itself.
> > >
> > > Concretely here, we could add ffmpeg to emscripten-ports but not to
> > emscripten, and add a test in emscripten, and then we'd get regression
> > tests to avoid issues there, which given the history could be useful. The
> > only objection I might have to that is if the test is very slow. Otherwise,
> > if someone's interested to do it, it sounds good to me.
> > >
> >
> > I broadly agree with the approach.  One thing to note is that more
> > ports should not need to be forked to exist in this system.   It
> > should be enough to commit a simple build recipe (e.g. emconfigure &&
> > emake) along with an upstream URL.The emscripten-ports
> > organization is mostly used/useful for hosting forked repos.   I think
> > I'm suggesting something more like the "emscripten/tools/ports" tree,
> > where each project is represented by some metadata and they all live
> > in the same tree.
> >
> > For now I propose the following:
> > 1. Add ffmpeg to "emscripten/tools/ports".  Use emmake and emconfgiure
> > here to mimic the build process used by upstream/Beuc.
> > 2. Add an optional flag to mark ports as "contrib" or "second tier".
> > This would signal that normal CI doesn't build them, and they don't
> > get built by "embuilder build ALL" or get listed by default by
> > "embuilder list".
> >
> > Perhaps we can use a subdirectory called "contrib" under
> > "emscripten/tools/ports".  Any ports in that tree would be less
> > frequently tested and have a lower bar for entry.  Then we can have
> > people add whatever ports they want there.
> >
> > >
> > > On Wed, Oct 9, 2019 at 10:59 PM Gabriel Cuvillier <
> > gabriel.cuvill...@gmail.com> wrote:
> > >>
> > >> Regarding the Emscripten "Ports", why not having something like NaCL
> > >> "Web Ports" system ?
> > >>
> > >> https://chromium.googlesource.com/webports
> > >>
> > >> For example, the list of ports for Pepper 47 version:
> > >>
> > >>
> > https://chromium.googlesource.com/webports/+/pepper_47/docs/port_list.md
> > >>
> > >>
> > >> This is a 3rd party repository (that is, not in NaCL main tree) for
> > >> projects buildable on the NaCl platform. It was a very handy by the
> > time.
> > >>
> > >> I understand the reluctance to have every possible ports in the
> > >> Emscripten tree (apart maybe some very common stuff, such as SDL2, zlib
> > >> or Freetype), though having a "ports ecosystem" is nevertheless
> > >> something needed
> > >>
> > >>
> > >> Le 09/10/2019 à 22:18, 'Sam Clegg' via emscripten-discuss a écrit :
> > >> > On Wed, Oct 9, 2019 at 3:05 AM Sylvain Beucler  wrote:
> > >> >> Hi,
> > >> >>
> > >> >> On 08/10/2019 21:12, 'Sam Clegg' via emscripte

Re: Patching Emscripten: preserving good practices

[Beuc]

I just upgraded 1.38.45->1.38.46 and I got a new build failure in an
unmodified ffmpeg build.
Which is also what happened last time I upgraded Emscripten 1 month ago
(different error).
I don't see wasm-ld-related notes in the ChangeLog, but apparently it
got stricter again.
Compiling a few external libs may actually be a good release test case
for the Emscripten project ¯\_(ツ)_/¯

- Beuc

On 20/09/2019 20:06, Beuc wrote:
>
> Hi,
>
> On 17/09/2019 00:18, Alon Zakai wrote:
>> On Mon, Sep 16, 2019 at 6:50 AM Beuc > <mailto:b...@beuc.net>> wrote:
>>
>> > * Are there tests in your codebase that we could run in
>> upstream emscripten?
>>
>> Come to think of it, there's one thorough automated test that we
>> have to run at each upgrade:
>> building dependencies!
>> [...]
>>
>> It might be useful to set up CI that runs the emscripten tip-of-tree
>> builds on that (emsdk install tot-fastcomp or tot-upstream). Those
>> are literally the very newest code, that passed chromium CI but is
>> not as heavily tested as the actual release tags. You may sometimes
>> see a temporary breakage you can ignore, but it would also catch
>> regressions.
>>
> There seem to be a misunderstanding, you asked for test you could run,
> not the other way around ;)
>
> It isn't worth it for us to constantly rebuild with ToT: most of the
> time we're working on Ren'Py/RenPyWeb itself and can't afford varying
> dependencies.
> Investigating all temporary and non-temporary breakages sounds like
> time I should reserve for properly upgrading and adapting to a newer,
> stable Emscripten once every other month.
>
>>> - Tagging: TBH I didn't figure out yet e.g. "what LLVM commit is
>>> used for emsdk 1.38.42" from
>>> https://chromium.googlesource.com/emscripten-releases/
>>>
>>
>> Is that still an issue? Are the docs at
>>
>> https://github.com/emscripten-core/emscripten/blob/incoming/docs/process.md#packaging-emscripten
>
> (I hadn't seen this question on first read.)
>
> My first attempts with binaryen seems to work, and for those
> interested Sam Clegg posted a recap:
>
> https://github.com/emscripten-core/emscripten/issues/8995#issuecomment-532984238
>

-- 
You received this message because you are subscribed to the Google Groups 
"emscripten-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to emscripten-discuss+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/emscripten-discuss/3f1f8c14-8d50-d66d-f6fb-31c588f42acf%40beuc.net.

Re: Patching Emscripten: preserving good practices

[Beuc]

HHi,

On 17/09/2019 00:18, Alon Zakai wrote:
> On Mon, Sep 16, 2019 at 6:50 AM Beuc  <mailto:b...@beuc.net>> wrote:
>
> > * Are there tests in your codebase that we could run in upstream
> emscripten?
>
> Come to think of it, there's one thorough automated test that we
> have to run at each upgrade:
> building dependencies!
> [...]
>
> It might be useful to set up CI that runs the emscripten tip-of-tree
> builds on that (emsdk install tot-fastcomp or tot-upstream). Those are
> literally the very newest code, that passed chromium CI but is not as
> heavily tested as the actual release tags. You may sometimes see a
> temporary breakage you can ignore, but it would also catch regressions.
>
There seem to be a misunderstanding, you asked for test you could run,
not the other way around ;)

It isn't worth it for us to constantly rebuild with ToT: most of the
time we're working on Ren'Py/RenPyWeb itself and can't afford varying
dependencies.
Investigating all temporary and non-temporary breakages sounds like time
I should reserve for properly upgrading and adapting to a newer, stable
Emscripten once every other month.

>> - Tagging: TBH I didn't figure out yet e.g. "what LLVM commit is used
>> for emsdk 1.38.42" from
>> https://chromium.googlesource.com/emscripten-releases/
>>
>
> Is that still an issue? Are the docs at
>
> https://github.com/emscripten-core/emscripten/blob/incoming/docs/process.md#packaging-emscripten

(I hadn't seen this question on first read.)

My first attempts with binaryen seems to work, and for those interested
Sam Clegg posted a recap:

https://github.com/emscripten-core/emscripten/issues/8995#issuecomment-532984238

Cheers!
Beuc

-- 
You received this message because you are subscribed to the Google Groups 
"emscripten-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to emscripten-discuss+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/emscripten-discuss/82726f26-e8eb-5b4d-4938-c82d79fa02cf%40beuc.net.

Asyncify feedback

[Beuc]

Hi,

I got feedback from a few gamedevs who tried to new RenPyWeb build
(with Emterpreter->Asyncify, and fastcomp->upstream):
so far it's been positive :)

I thought I'd share - great work :)

- Beuc

-- 
You received this message because you are subscribed to the Google Groups 
"emscripten-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to emscripten-discuss+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/emscripten-discuss/040f2d28-2839-eeb3-0760-f58620b59996%40beuc.net.

Re: Patching Emscripten: preserving good practices

[Beuc]

> * Are there tests in your codebase that we could run in upstream
emscripten?

Come to think of it, there's one thorough automated test that we have to
run at each upgrade:
building dependencies!

https://github.com/renpy/renpyweb
https://github.com/renpy/renpyweb/tree/master/scripts
https://www.beuc.net/python-emscripten/python/dir?ci=tip

Typing a simple:

$ make

will download and build all dependencies, and it's not uncommon that
something breaks on Emscripten upgrade.

Cheers!
Sylvain

On 10/09/2019 12:41, Sylvain Beucler wrote:
>
> Hi,
>
> (I was hoping other people would give their PoV...)
>
> Generally-speaking, we may need more patches because of:
>
> - the combination of Emscripten features (Python + uninterruptible
> main loop -- aka fpcast+async; thankfully I could get rid of dynamic
> linking...)
>
> - centrally improving Emscripten portability rather than adapting the
> codebases (e.g. spending weeks on the 1-line #8751 / LANG support,
> rather than adding a quick EM_ASM in all my projects)
>
> - working directly with Emscripten rather than using a framework that
> provides a layer of work-arounds (e.g. Unity)
>
> Again, at a given point of time we always have a buffer of pending
> patches (I forgot to mention #6511 and #7631 that were added to our
> shell.html template, btw), the ones we can upstream are replaced by
> newer ones: old bugs we eventually tackle like fullscreen support or
> autosave-on-quit; backports for regressions in a given Emscripten release.
>
> Hopefully one day the web ecosystem will be so mature that we'll be
> able to take our distro's years-old compiler, `./configure
> --host=browser` without any change, and publish. Meanwhile... we patch :)
>
> To answer your specific questions:
>
> - This is a game engine, much of the tests are manual, sadly (but I
> contribute tests when I submit Emscripten patches); we often get
> integration errors, or errors due to combining multiple Emscripten
> options.
>
> - I upgrade Emscripten to a tagged version ~every other month; since
> this is likely to introduce breaking changes, I only do it when I have
> the time to deal with the consequences (e.g. when I'm not in the
> middle of developing RenPyWeb itself). I avoid ToT for stability and
> reproducibility reasons.
>
> - EMT_STACK_MAX is the exception, not the rule ;) That's about the
> only thing I could have upstream but didn't due to lack of time -- and
> it's now deprecated.
>
> - Tagging: TBH I didn't figure out yet e.g. "what LLVM commit is used
> for emsdk 1.38.42" from
> https://chromium.googlesource.com/emscripten-releases/
>
> Cheers!
> Sylvain
>
> On 07/09/2019 00:58, Alon Zakai wrote:
>> I think those are all reasonable requests, thanks for opening those
>> issues and highlighting them here! I hope we can make progress on each.
>>
>> And hopefully other users can provide useful advice from their
>> perspective and experience.
>>
>> About "no matching tags for llvm and bynarien, commit IDs quite hard
>> to track (discussion in #9317/#9383)" - I think we have resolved
>> that? Please let me know if not.
>>
>> And a general comment: Your overall experience does surprise me
>> (which shows why it's good you wrote it up!) - that's a lot more
>> local changes than I've seen elsewhere, and that includes some very
>> large production user setups. So I hope we can work together to make
>> your experience closer to theirs.
>>
>>  * Are there tests in your codebase that we could run in upstream
>> emscripten?
>>  * Do you test latest emscripten releases on your codebase as they
>> are tagged? (Another option is to test the much more frequent
>> tip-of-tree builds, but you may see more noise there.)
>>  * It would be good to discuss upstreaming long-term changes you
>> mention like that EMT_STACK_MAX one - we may not be able to in all
>> cases, but let's try at least (I don't think I remember a discussion
>> on that one).
>>
>> - Alon
>>
>>
>>
>> On Fri, Sep 6, 2019 at 12:32 AM Beuc > <mailto:b...@beuc.net>> wrote:
>>
>> Hi,
>>
>> We almost never could use Emscripten without applying a few patches.
>> It's become increasingly difficult, not only to patch, but to
>> convince the developers that it's legitimate, so I thought I'd
>> detail our use case for good here.
>>
>> So we've been using Emscripten with patches in emscripten, llvm
>> and SDL2 port [1].
>> By the time a patch is officially included in a release, we
>> usually need more patches as we face new issues -- or regressions.
>> Rins

Patching Emscripten: preserving good practices

[Beuc]

Hi,

We almost never could use Emscripten without applying a few patches.
It's become increasingly difficult, not only to patch, but to convince the 
developers that it's legitimate, so I thought I'd detail our use case for good 
here.

So we've been using Emscripten with patches in emscripten, llvm and SDL2 port 
[1].
By the time a patch is officially included in a release, we usually need more 
patches as we face new issues -- or regressions.
Rinse and repeat.

Consequently for our production use (the RenPyWeb build that we publish to our 
users), we build an identified, versioned release (e.g. 1.38.42) with specific 
fixes.


New difficulties:
- scarse documentation for recompiling "upstream" (patch in progress #9383)
- no matching tags for llvm and bynarien, commit IDs quite hard to track 
(discussion in #9317/#9383)
- misleading versioning in tagged commits (#9282)
- grab-from-git port system; EMCC_LOCAL_PORTS considered for "local debugging", 
not cached, and causing issues (e.g. #9342 #9402)


Rebutals:

- "99% users use emsdk binaries"
Well, the remaining 1% will provide 99% of external contributions, take care of 
them :)
Also being able to test fixes before they finish the full review process 
allows spotting issues in advance.

- "users will either use the unmodified binaries, or work on master/tip-of-tree"
No, we use an identified, tagged Emscripten version, because it's more stable 
than a random commit.
Also when we publish a hot fix, sometimes a couple months after release, we 
need to reproduce the same build environment

- "emsdk can build from source"
This build process doesn't allow patching, so it's only when one doesn't trust 
the binaries and wants to rebuild them as-is AFAIU.
(To be honest, I'd like to be able to use the binaries, but Emscripten being 
bleeding-edge, I figure that'll have to wait :))


I hope this makes clear why we need to checkout emscripten/llvm/bynarien for an 
existing release, patch it, and build it.
Preferably without reverse-engineering the build process and fighting all the 
stack.
That's the first time I have to justify this while working on free/open source 
software.

How do you people deal with this?


[1] For instance, our current release needs these patches, some of them still 
needing work & official integration:

- https://github.com/emscripten-core/emscripten/issues/9257
Work-around ./configure misdetection

- https://github.com/emscripten-core/emscripten/issues/9097
Fix fullscreen exit causing wrong screen size

- hard-coded change to EMT_STACK_MAX which isn't configurable

- https://github.com/emscripten-core/emscripten-fastcomp/pull/195
Compiler limitation when combining several Emscripten features

- https://github.com/emscripten-ports/SDL2/issues/88
Send SDL_APP_TERMINATING on page exit

- https://github.com/emscripten-ports/SDL2/issues/70
Support Emterpreter/Asincify in SDL2

- Search "Emscripten" in https://renpy.beuc.net/#status_reports for all the 
other patches that were eventually integrated over the year

-- 
You received this message because you are subscribed to the Google Groups 
"emscripten-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to emscripten-discuss+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/emscripten-discuss/20190906073253.yh5wv677v3l4xn4k%40mail.beuc.net.

Re: Help investigating asyncify failure

[Beuc]

Hi,

I eventually tracked down a few missing functions using a simpler project.
If I understand correctly, those were tail calls.

The application feels a lot smoother, thanks :)

Note: I couldn't whitelist additional functions caused by
EMULATE_FUNCTION_POINTER_CASTS, such as
byn$fpcast_emu$__pyx_pw_11pygame_sdl2_7display_6Window_13flip (present
in the .wasm but not recognized as part of the whitelist). Is this expected?

Cheers!
Beuc

On 05/09/2019 02:07, Alon Zakai wrote:
>
>
> On Wed, Sep 4, 2019 at 4:45 PM Beuc  <mailto:b...@beuc.net>> wrote:
>
> Hi and thanks for the input,
>
> A few quick points:
>
> - the wildcard doesn't seem to work with '*' either:
>   warning: Asyncify whitelist contained a non-existing function
> name: __pyx_pf_5renpy_2gl_6gldraw_6GLDraw_*
> ($__pyx_pf_5renpy_2gl_6gldraw_6GLDraw_*)
>
>
> Ah, you're right, I was confused, sorry.
>
> ASYNCIFY_IMPORTS supports such a wildcard, but the lists can't, since
> "*" is a valid character in human-readable function names... we'd need
> to add some kind of escaping support if we want it.
>
> We do have ongoing work on adding a response file option to those, so
> that should make handling long lists easier.
>
> - I was using tot-upstream, '-s ASSERTIONS=1 -g', and
>   '-s ASYNCIFY_STACK_SIZE=65535' (just in case)
>
> - I'm using my patch from
>   https://github.com/emscripten-ports/SDL2/issues/70
>   hence emscripten_sleep()
>
>
> Hmm, no idea then. I'd suggest investigating this using -s WASM=0,
> that tends to be much easier to debug (can add console.logs, etc.).
> Sometimes just seeing the unreachable/abort in context helps
> understand what's going on.
>
>
> On Wed, Sep 04, 2019 at 04:22:17PM -0700, Alon Zakai wrote:
> > On Wed, Sep 4, 2019 at 2:10 PM Beuc  <mailto:b...@beuc.net>> wrote:
> >
> > > Hi,
> > >
> > > Trying to test my Emscripten project with Asyncify, I
> recompiled it with
> > > tot-upstream, with the same whitelist, but this is failing:
> > >
> > > Part 1) Handling function names that change (work-around)
> > >
> > > I read that using function prefixes worked as limited wildcard
> but that
> > > doesn't seem to be the case - did I do something wrong?
> > >
> >
> > Wildcards like "__py*" should work (with "*"). But the "*" can
> only be at
> > the end currently, I believe.
> >
> >   -s ASYNCIFY_WHITELIST='["__pyx_pf_5renpy_2gl_6gldraw_6GLDraw_"...'
> > >   warning: Asyncify whitelist contained a non-existing
> function name:
> > > __pyx_pf_5renpy_2gl_6gldraw_6GLDraw_
> ($__pyx_pf_5renpy_2gl_6gldraw_6GLDraw_)
> > >
> > > Also llvm-nm doesn't seem to support listing WASM.
> > >
> >
> > Correct, llvm-nm is only aware of LLVM IR. Wasm object files
> contain wasm.
> > To see the names in them, build with something like -g or
> --profiling (so
> > names are kept) and use binaryen's wasm-dis. You can also
> inspect the
> > binary directly.
> >
> > I managed to find the function through emcc ... -o fat.llvm &&
> llvm-nm
> > > fat.llvm :
> > >   003832c5 t __pyx_pf_5renpy_2gl_6gldraw_6GLDraw_28draw_screen
> > >
> > >
> > LLVM IR may encode the name differently than in wasm, as LLVM IR
> will use
> > standard name mangling while wasm names will be "human-readable"
> (contain
> > things like parentheses, types, ::, etc. - the output of c++filt).
> >
> >
> > > Part 2) Run-time failure
> > >
> > > With the updated WHITELIST, I got the dreaded:
> > >
> > > RuntimeError: unreachable executed index.wasm:386701:1
> > >
> >
> > An unreachable can be several things. If it in the binaryen API,
> it could
> > be the asyncify stack is too small. But here it looks like it's
> in a user
> > function - it could be that an import was called that started a
> sleep, but
> > asyncify was not aware of it, and control flow ends up in the
> wrong place.
> > Adding some debug logging in your sleep function, and making sure
> > ASYNCIFY_IMPORTS is correct, might help.
> >
> > Note that in ASSERTIONS mode we improved debug errors for that
> pretty
> > recently, not sure if in the last version o

Re: Help investigating asyncify failure

[Beuc]

Hi and thanks for the input,

A few quick points:

- the wildcard doesn't seem to work with '*' either:
  warning: Asyncify whitelist contained a non-existing function name: 
__pyx_pf_5renpy_2gl_6gldraw_6GLDraw_* ($__pyx_pf_5renpy_2gl_6gldraw_6GLDraw_*)

- I was using tot-upstream, '-s ASSERTIONS=1 -g', and
  '-s ASYNCIFY_STACK_SIZE=65535' (just in case)

- I'm using my patch from
  https://github.com/emscripten-ports/SDL2/issues/70
  hence emscripten_sleep()


On Wed, Sep 04, 2019 at 04:22:17PM -0700, Alon Zakai wrote:
> On Wed, Sep 4, 2019 at 2:10 PM Beuc  wrote:
> 
> > Hi,
> >
> > Trying to test my Emscripten project with Asyncify, I recompiled it with
> > tot-upstream, with the same whitelist, but this is failing:
> >
> > Part 1) Handling function names that change (work-around)
> >
> > I read that using function prefixes worked as limited wildcard but that
> > doesn't seem to be the case - did I do something wrong?
> >
> 
> Wildcards like "__py*" should work (with "*"). But the "*" can only be at
> the end currently, I believe.
> 
>   -s ASYNCIFY_WHITELIST='["__pyx_pf_5renpy_2gl_6gldraw_6GLDraw_"...'
> >   warning: Asyncify whitelist contained a non-existing function name:
> > __pyx_pf_5renpy_2gl_6gldraw_6GLDraw_ ($__pyx_pf_5renpy_2gl_6gldraw_6GLDraw_)
> >
> > Also llvm-nm doesn't seem to support listing WASM.
> >
> 
> Correct, llvm-nm is only aware of LLVM IR. Wasm object files contain wasm.
> To see the names in them, build with something like -g or --profiling (so
> names are kept) and use binaryen's wasm-dis. You can also inspect the
> binary directly.
> 
> I managed to find the function through emcc ... -o fat.llvm && llvm-nm
> > fat.llvm :
> >   003832c5 t __pyx_pf_5renpy_2gl_6gldraw_6GLDraw_28draw_screen
> >
> >
> LLVM IR may encode the name differently than in wasm, as LLVM IR will use
> standard name mangling while wasm names will be "human-readable" (contain
> things like parentheses, types, ::, etc. - the output of c++filt).
> 
> 
> > Part 2) Run-time failure
> >
> > With the updated WHITELIST, I got the dreaded:
> >
> > RuntimeError: unreachable executed index.wasm:386701:1
> >
> 
> An unreachable can be several things. If it in the binaryen API, it could
> be the asyncify stack is too small. But here it looks like it's in a user
> function - it could be that an import was called that started a sleep, but
> asyncify was not aware of it, and control flow ends up in the wrong place.
> Adding some debug logging in your sleep function, and making sure
> ASYNCIFY_IMPORTS is correct, might help.
> 
> Note that in ASSERTIONS mode we improved debug errors for that pretty
> recently, not sure if in the last version or since then.
> 
> 
> > __pyx_pw_11pygame_sdl2_7display_21flip
> > http://localhost:8000/index.wasm:386701
> > emu$__pyx_pw_11pygame_sdl2_7display_21flip
> > http://localhost:8000/index.wasm:8749855
> > 15 http://localhost:8000/index.wasm:2587748
> > __pyx_pf_5renpy_2gl_6gldraw_6GLDraw_28draw_screen
> > http://localhost:8000/index.wasm:2845914
> > __pyx_pw_5renpy_2gl_6gldraw_6GLDraw_29draw_screen
> > http://localhost:8000/index.wasm:2813529
> > emu$__pyx_pw_5renpy_2gl_6gldraw_6GLDraw_29draw_screen
> > http://localhost:8000/index.wasm:8760622
> > PyCFunction_Call http://localhost:8000/index.wasm:5272869
> > PyEval_EvalFrameEx http://localhost:8000/index.wasm:5750951
> > PyEval_EvalCodeEx http://localhost:8000/index.wasm:5714071
> > fast_function http://localhost:8000/index.wasm:5767904
> > PyEval_EvalFrameEx http://localhost:8000/index.wasm:5751409
> > PyEval_EvalCodeEx http://localhost:8000/index.wasm:5714071
> > function_call http://localhost:8000/index.wasm:5167287
> > emu$function_call http://localhost:8000/index.wasm:8793327
> > PyObject_Call http://localhost:8000/index.wasm:5030529
> > PyEval_EvalFrameEx http://localhost:8000/index.wasm:5754056
> > PyEval_EvalCodeEx http://localhost:8000/index.wasm:5714071
> > function_call http://localhost:8000/index.wasm:5167287
> > emu$function_call http://localhost:8000/index.wasm:8793327
> > PyObject_Call http://localhost:8000/index.wasm:5030529
> > PyEval_EvalFrameEx http://localhost:8000/index.wasm:5754056
> > PyEval_EvalCodeEx http://localhost:8000/index.wasm:5714071
> > fast_function http://localhost:8000/index.wasm:5767904
> > PyEval_EvalFrameEx http://localhost:8000/index.wasm:5751409
> > PyEval_EvalCodeEx http://localhost:8000/index.wasm:5714071
> >

Help investigating asyncify failure

[Beuc]

index.wasm:5751409
PyEval_EvalCodeEx http://localhost:8000/index.wasm:5714071
fast_function http://localhost:8000/index.wasm:5767904
PyEval_EvalFrameEx http://localhost:8000/index.wasm:5751409
PyEval_EvalCodeEx http://localhost:8000/index.wasm:5714071
fast_function http://localhost:8000/index.wasm:5767904
PyEval_EvalFrameEx http://localhost:8000/index.wasm:5751409
PyEval_EvalCodeEx http://localhost:8000/index.wasm:5714071
PyEval_EvalCode http://localhost:8000/index.wasm:5710418
PyRun_FileExFlags http://localhost:8000/index.wasm:5955369
PyRun_SimpleFileExFlags http://localhost:8000/index.wasm:5954716
pyapp_runmain http://localhost:8000/index.wasm:58353
x http://localhost:8000/index.js:17649
handleSleep http://localhost:8000/index.js:17712
safeSetTimeout http://localhost:8000/index.js:9068

My whitelist:
-s ASYNCIFY_WHITELIST='["main", "pyapp_runmain", "SDL_WaitEvent", 
"SDL_WaitEventTimeout", "SDL_Delay", "SDL_RenderPresent", 
"GLES2_RenderPresent", "SDL_GL_SwapWindow", "Emscripten_GLES_SwapWindow", 
"PyRun_SimpleFileExFlags", "PyRun_FileExFlags", "PyEval_EvalCode", 
"PyEval_EvalCodeEx", "PyEval_EvalFrameEx", "PyCFunction_Call", "PyObject_Call", 
"fast_function", "function_call", "instancemethod_call", "slot_tp_call", 
"__pyx_pw_11pygame_sdl2_5event_7wait", 
"__pyx_pw_11pygame_sdl2_7display_21flip", 
"__pyx_pw_11pygame_sdl2_7display_6Window_13flip", 
"__pyx_pf_5renpy_2gl_6gldraw_6GLDraw_28draw_screen", 
"__pyx_pw_5renpy_2gl_6gldraw_6GLDraw_29draw_screen", 
"__Pyx_PyObject_CallNoArg", "__pyx_pf_10emscripten_6sleep", 
"__pyx_pw_10emscripten_7sleep", "__pyx_pf_10emscripten_8sleep_with_yield", 
"__pyx_pw_10emscripten_9sleep_with_yield", "gen_send", "gen_send_ex", 
"gen_iternext", "type_call", "slot_tp_init", "builtin_eval"]'

Not sure if all the "emu$" and "fastfunction" are properly handled :/

Any clue?

Cheers!
Beuc

-- 
You received this message because you are subscribed to the Google Groups 
"emscripten-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to emscripten-discuss+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/emscripten-discuss/20190904211035.7a3v7rgwmiwser4n%40mail.beuc.net.

Re: Future of (a)synchronous

[Beuc]

Hi,

On 26/08/2019 22:09, Alon Zakai wrote:
> On Sun, Aug 25, 2019 at 2:21 PM Beuc  <mailto:b...@beuc.net>> wrote:
>
> > Is there some use case you have that asyncify doesn't handle?
>
> I had answered you in the link above :)
>
> - debug capacity: Emterpreter provides a detailed stack trace whenever
>   trying to pause in the wrong place, which is a must-have to populate
>   the WHITELIST in complex code with indirect calls (e.g. Python);
>   here AFAICT the ASYNCIFY'd application will just skip the
>   emscripten_sleep() and crash mysteriously later.
>
> - pattern-matching in WHITELIST
>   (which I had implemented for Emterpreter)
>
>
> Would be good to get issues filed for those things if there aren't
> already. (I can try to find time to get to them myself, but not sure
> when I will be able to.)

I will when I can test asyncify on my production code. I hit several
issues with -upstream that are currently being resolved.

Cheers!
Beuc

-- 
You received this message because you are subscribed to the Google Groups 
"emscripten-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to emscripten-discuss+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/emscripten-discuss/966f0c50-e3c4-e6f0-10f1-fec35a055f68%40beuc.net.

Re: Future of asynchronous

[Beuc]

Hi,

I know about new-asyncify.
However I understood Emterpreter wasn't ported to LLVM "yet", but now it
seems it won't be at all.
(see also
https://github.com/emscripten-core/emscripten/issues/9257#issuecomment-523670023
)
I'd like to know what the official plan is :)

Good point about MS Chromium Edge.
I need to test https://www.microsoftedgeinsider.com/

TBH I'd welcome a working SDL2 proxy-to-pthread (even if it worked only
for chrome-likes) more than another stack-unwind emulation.

Cheers!
Sylvain

On 22/08/2019 00:39, 'Derek Schuff' via emscripten-discuss wrote:
> The future of asynchronous is the new version of Asyncify
> (https://emscripten.org/docs/porting/asyncify.html). It serves the
> same use cases as emterpreter (it operates similarly to fastcomp's old
> "asyncify" pass, but should be faster than that and emterpreter).
>
> Unfortunately I don't know anything specific about Microsoft or
> Mozilla's threading plans. MS's Chromium-based version of Edge will
> inherit Chrome's site-isolation feature, so they can safely ship
> threads; presumably they will leave it enabled in their builds?
>
> On Tue, Aug 20, 2019 at 12:37 AM Beuc  <mailto:b...@beuc.net>> wrote:
>
> Hi,
>
> Currently I use Emterpreter for my Python-based project web port (
> https://renpy.beuc.net/ ).
> I don't see it referenced in the "upstream" roadmap though:
> https://github.com/emscripten-core/emscripten/projects/1
> Is EMTERPRETIFY planned for "upstream"?
>
> Also, what I'd really need is pthreads, but I can't find ETA for when
> Mozilla or Microsoft plan to bring
> SharedArrayBuffer/WebAssembly-threads/etc. back.
> Any info? :)
>
> Cheers!
> Beuc
>
> -- 
> You received this message because you are subscribed to the Google
> Groups "emscripten-discuss" group.
> To unsubscribe from this group and stop receiving emails from it,
> send an email to emscripten-discuss+unsubscr...@googlegroups.com
> <mailto:emscripten-discuss%2bunsubscr...@googlegroups.com>.
> To view this discussion on the web visit
> 
> https://groups.google.com/d/msgid/emscripten-discuss/406ad461-4b8e-cffb-62b3-8acd797cefcc%40beuc.net.
>
> -- 
> You received this message because you are subscribed to the Google
> Groups "emscripten-discuss" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to emscripten-discuss+unsubscr...@googlegroups.com
> <mailto:emscripten-discuss+unsubscr...@googlegroups.com>.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/emscripten-discuss/CAAEAhvdgBaJZpA%2BfpjFPHUeJFCk8d-FwCeXeuULKrOTtKdd-Tg%40mail.gmail.com
> <https://groups.google.com/d/msgid/emscripten-discuss/CAAEAhvdgBaJZpA%2BfpjFPHUeJFCk8d-FwCeXeuULKrOTtKdd-Tg%40mail.gmail.com?utm_medium=email_source=footer>.

-- 
You received this message because you are subscribed to the Google Groups 
"emscripten-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to emscripten-discuss+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/emscripten-discuss/524047cf-6173-9397-28ea-ee57125c3029%40beuc.net.

Future of asynchronous

[Beuc]

Hi,

Currently I use Emterpreter for my Python-based project web port (
https://renpy.beuc.net/ ).
I don't see it referenced in the "upstream" roadmap though:
https://github.com/emscripten-core/emscripten/projects/1
Is EMTERPRETIFY planned for "upstream"?

Also, what I'd really need is pthreads, but I can't find ETA for when
Mozilla or Microsoft plan to bring
SharedArrayBuffer/WebAssembly-threads/etc. back.
Any info? :)

Cheers!
Beuc

-- 
You received this message because you are subscribed to the Google Groups 
"emscripten-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to emscripten-discuss+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/emscripten-discuss/406ad461-4b8e-cffb-62b3-8acd797cefcc%40beuc.net.

Re: 32-bit SDK of Emscripten 1.38.x

[Beuc]

You can't use your system (i386) headers for an Emscripten (asmjs/wasm)
application :)

If JModelica can't be compiled without i386 headers (AFAIU), it probably
can't compiled for the asmjs/wasm target.
You'll need to improve JModelica so it supports more targets/architectures.

Cheers!
Beuc

On 09/07/2019 07:47, Manuel Häusler wrote:
>
> Without those, emcc will not find needed header files and stop with
> errors...
> What do you suggest then? Do I have to build a 32-Bit Emscripten on a
> 32-Bit system to overcome that problem?
>
>
> Am Montag, 8. Juli 2019 18:52:57 UTC+2 schrieb Alon Zakai:
>
> The use of local system includes can cause errors,
>
> -I /usr/include/ -I /usr/include/i386-linux-gnu/ -I
> /usr/include/i386-linux-gnu/bits/
>
> Without those emcc will properly use the emcc system headers. With
> those, it will use whatever is on the local system, which may be
> wrong (since you aren't compiling for that system natively, but
> cross-compiling for another target).
>

-- 
You received this message because you are subscribed to the Google Groups 
"emscripten-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to emscripten-discuss+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/emscripten-discuss/899cfbac-e9ee-13e2-7fd2-2e47557686c8%40beuc.net.
For more options, visit https://groups.google.com/d/optout.

Re: Request for comments: set LANG from HTTP Accept-Language

[Beuc]

Hi,

FYI the PR was accepted :)

Cheers!
Beuc

On 12/06/2019 12:06, Beuc wrote:
>
> Hi,
>
> This PR's point is improving the "standard-ness" of the Emscripten
> environment :)
> (One could easily make yet another emscripten-specific change and grab
> 'navigator.languages' using EM_JS.)
>
> With this PR, for instance, my RenPyWeb port can use Python's 'locale'
> package without any changes.
> Programs using gettext(3) would also work as-is.
>
> LANG doesn't affect the C runtime until the program calls setlocale(3)
> (cf. PR discussion).
> This is about language preference detection; changing the language
> in-app (sadly) cannot be solved at this level ;)
>
> Cheers!
> Beuc
>
> On 12/06/2019 11:26, Floh wrote:
>> What about a html5.h function to poll the language:
>>
>> const char* emscripten_get_navigator_language(void);
>>
>> Would this provide enough functionality while being less "intrusive"?
>>
>> I don't know how the ENV-variable approach affects C runtime
>> functions though,
>> but for showing a UI in the user's preferred language this would probably
>> suffice (as a user I guess I'd still like to have an additional
>> "in-app-option" to 
>> change the language though).
>>
>> On Tuesday, 11 June 2019 21:57:23 UTC+2, Beuc wrote:
>>
>> Hi,
>>
>> I issued a PR to expose the user's preferred language as
>> ENV['LANG']:
>> https://github.com/emscripten-core/emscripten/pull/8751
>> <https://github.com/emscripten-core/emscripten/pull/8751>
>>
>> It's based on navigator.languages which reflects HTTP
>> Accept-Language, e.g.:
>> - HTTP Accept-Language: fr,fr-FR;q=0.8,en-US;q=0.5,en;q=0.3
>> - navigator.languages = Array(4) [ "fr", "fr-FR", "en-US", "en" ]
>> - ENV['LANG'] = fr.UTF-8
>>
>> (Currently Emscripten uses a hard-coded 'C.UTF-8'.)
>>
>> This PR allows starting your WebAssembly program directly using the
>> user's preferred language (rather than shoving a language choice in
>> their face on start-up ;)).
>> Provided your program is i18n'd / multi-language, of course.
>> For nodejs no changes.
>>
>> While this environment variable is pretty standard on desktop, Alon
>> expressed concerned that this may be risky to start doing this now.
>>
>> Hence we moved the discussion to this list :)
>> What do you think?
>>

-- 
You received this message because you are subscribed to the Google Groups 
"emscripten-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to emscripten-discuss+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/emscripten-discuss/3a30dd72-dbe1-a455-25f1-b7cc008c6bec%40beuc.net.
For more options, visit https://groups.google.com/d/optout.

Re: Chromium, IDBFS and iframes

[Beuc]

Hi,

With some more testing:
- Chromium, Debian version, blocks third-party cookies by default:
https://sources.debian.org/patches/chromium/73.0.3683.75-1/disable/third-party-cookies.patch/
- Chrome and default Chromium builds do not
- Firefox, including Debian version, doesn't; with blocking even the
3rd-party WASM can't load (SecurityException)

So this is not a browser issue but a default settings issue.

AFAIU the reason why the iframe cannot use IndexDB is because otherwise
the top-level page could control the frame and have it set a cookie,
bypassing the "no 3rd-party cookie" setting.

There doesn't seem to be a way around this but I can be mistaken.

Cheers!
Beuc

On 17/06/2019 14:53, Beuc wrote:
>
> AFAICS this has been around for years :/
> See for instance
> https://forum.unity.com/threads/how-to-check-playerprefs-has-actually-written-data.389376/
>
> I really don't understand the logic of attempting to use the caller's
> / top-level IDBFS context instead of the iframe one?
>
> One work-around is unblocking "third-party cookies" in
> chrome://settings/content/cookies
>
> Another one is passing a sessionid to the iframe to it can save the
> data via the hoster's API, provided the player has an account there
> already.
> (I believe newgrounds does it, but not itch.io.)
>
> Do you all have some more feedback on this?
>
> Cheers!
> Beuc
>
> On 17/06/2019 12:45, Jukka Jylänki wrote:
>> If there's differing behavior between browsers, filing a bug is good
>> idea. Any chance you'd be able to handwrite an IndexedDB example JS
>> code that highlights the discrepancy? In general I find that filing
>> Emscripten compiled pages as bug test cases tend to go untriaged as
>> too complex. :)
>>
>> su 16. kesäk. 2019 klo 13.53 Beuc (b...@beuc.net) kirjoitti:
>>> Hi,
>>>
>>> I have savegame issues with Chromium and typical game hosting websites
>>> (itch.io, newgrounds...), where they present the game in their domain,
>>> and run the game within a iframe from a CDN (and a different domain name).
>>>
>>> While running the game from the iframe, Chromium still attempts to use
>>> IndexedDB storage from the top-level domain context.
>>> This results in FS.syncfs() failing with "DOMException: The user denied
>>> permission to access the database" (visible if you set an async error
>>> handler).
>>> If the user manually hits the iframe page directly, storage works.
>>>
>>> With Firefox, no issues, since the browser uses IDB with the iframe
>>> domain context
>>> (which makes much more sense to me).
>>>
>>> Are you aware of this?
>>> Are there workarounds?
>>> Should I file a bug in Chromium?
>>>
>>> Cheers!
>>> Beuc
>>>
>>> --
>>> You received this message because you are subscribed to the Google Groups 
>>> "emscripten-discuss" group.
>>> To unsubscribe from this group and stop receiving emails from it, send an 
>>> email to emscripten-discuss+unsubscr...@googlegroups.com.
>>> To view this discussion on the web visit 
>>> https://groups.google.com/d/msgid/emscripten-discuss/bf34857b-e7ed-0f97-a7f2-9402c0aad45b%40beuc.net.
>>> For more options, visit https://groups.google.com/d/optout.
> -- 
> You received this message because you are subscribed to the Google
> Groups "emscripten-discuss" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to emscripten-discuss+unsubscr...@googlegroups.com
> <mailto:emscripten-discuss+unsubscr...@googlegroups.com>.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/emscripten-discuss/9057fc5d-80fe-9b11-bf56-4ff506cf9759%40beuc.net
> <https://groups.google.com/d/msgid/emscripten-discuss/9057fc5d-80fe-9b11-bf56-4ff506cf9759%40beuc.net?utm_medium=email_source=footer>.
> For more options, visit https://groups.google.com/d/optout.

-- 
You received this message because you are subscribed to the Google Groups 
"emscripten-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to emscripten-discuss+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/emscripten-discuss/06c61392-c41b-8a66-068d-79229eb2bb7a%40beuc.net.
For more options, visit https://groups.google.com/d/optout.

Re: Chromium, IDBFS and iframes

[Beuc]

AFAICS this has been around for years :/
See for instance
https://forum.unity.com/threads/how-to-check-playerprefs-has-actually-written-data.389376/

I really don't understand the logic of attempting to use the caller's /
top-level IDBFS context instead of the iframe one?

One work-around is unblocking "third-party cookies" in
chrome://settings/content/cookies

Another one is passing a sessionid to the iframe to it can save the data
via the hoster's API, provided the player has an account there already.
(I believe newgrounds does it, but not itch.io.)

Do you all have some more feedback on this?

Cheers!
Beuc

On 17/06/2019 12:45, Jukka Jylänki wrote:
> If there's differing behavior between browsers, filing a bug is good
> idea. Any chance you'd be able to handwrite an IndexedDB example JS
> code that highlights the discrepancy? In general I find that filing
> Emscripten compiled pages as bug test cases tend to go untriaged as
> too complex. :)
>
> su 16. kesäk. 2019 klo 13.53 Beuc (b...@beuc.net) kirjoitti:
>> Hi,
>>
>> I have savegame issues with Chromium and typical game hosting websites
>> (itch.io, newgrounds...), where they present the game in their domain,
>> and run the game within a iframe from a CDN (and a different domain name).
>>
>> While running the game from the iframe, Chromium still attempts to use
>> IndexedDB storage from the top-level domain context.
>> This results in FS.syncfs() failing with "DOMException: The user denied
>> permission to access the database" (visible if you set an async error
>> handler).
>> If the user manually hits the iframe page directly, storage works.
>>
>> With Firefox, no issues, since the browser uses IDB with the iframe
>> domain context
>> (which makes much more sense to me).
>>
>> Are you aware of this?
>> Are there workarounds?
>> Should I file a bug in Chromium?
>>
>> Cheers!
>> Beuc
>>
>> --
>> You received this message because you are subscribed to the Google Groups 
>> "emscripten-discuss" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to emscripten-discuss+unsubscr...@googlegroups.com.
>> To view this discussion on the web visit 
>> https://groups.google.com/d/msgid/emscripten-discuss/bf34857b-e7ed-0f97-a7f2-9402c0aad45b%40beuc.net.
>> For more options, visit https://groups.google.com/d/optout.

-- 
You received this message because you are subscribed to the Google Groups 
"emscripten-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to emscripten-discuss+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/emscripten-discuss/9057fc5d-80fe-9b11-bf56-4ff506cf9759%40beuc.net.
For more options, visit https://groups.google.com/d/optout.

Chromium, IDBFS and iframes

[Beuc]

Hi,

I have savegame issues with Chromium and typical game hosting websites
(itch.io, newgrounds...), where they present the game in their domain,
and run the game within a iframe from a CDN (and a different domain name).

While running the game from the iframe, Chromium still attempts to use
IndexedDB storage from the top-level domain context.
This results in FS.syncfs() failing with "DOMException: The user denied
permission to access the database" (visible if you set an async error
handler).
If the user manually hits the iframe page directly, storage works.

With Firefox, no issues, since the browser uses IDB with the iframe
domain context
(which makes much more sense to me).

Are you aware of this?
Are there workarounds?
Should I file a bug in Chromium?

Cheers!
Beuc

-- 
You received this message because you are subscribed to the Google Groups 
"emscripten-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to emscripten-discuss+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/emscripten-discuss/bf34857b-e7ed-0f97-a7f2-9402c0aad45b%40beuc.net.
For more options, visit https://groups.google.com/d/optout.

Re: Request for comments: set LANG from HTTP Accept-Language

[Beuc]

Hi,

This PR's point is improving the "standard-ness" of the Emscripten
environment :)
(One could easily make yet another emscripten-specific change and grab
'navigator.languages' using EM_JS.)

With this PR, for instance, my RenPyWeb port can use Python's 'locale'
package without any changes.
Programs using gettext(3) would also work as-is.

LANG doesn't affect the C runtime until the program calls setlocale(3)
(cf. PR discussion).
This is about language preference detection; changing the language
in-app (sadly) cannot be solved at this level ;)

Cheers!
Beuc

On 12/06/2019 11:26, Floh wrote:
> What about a html5.h function to poll the language:
>
> const char* emscripten_get_navigator_language(void);
>
> Would this provide enough functionality while being less "intrusive"?
>
> I don't know how the ENV-variable approach affects C runtime functions
> though,
> but for showing a UI in the user's preferred language this would probably
> suffice (as a user I guess I'd still like to have an additional
> "in-app-option" to 
> change the language though).
>
> On Tuesday, 11 June 2019 21:57:23 UTC+2, Beuc wrote:
>
> Hi,
>
> I issued a PR to expose the user's preferred language as ENV['LANG']:
> https://github.com/emscripten-core/emscripten/pull/8751
> <https://github.com/emscripten-core/emscripten/pull/8751>
>
> It's based on navigator.languages which reflects HTTP
> Accept-Language, e.g.:
> - HTTP Accept-Language: fr,fr-FR;q=0.8,en-US;q=0.5,en;q=0.3
> - navigator.languages = Array(4) [ "fr", "fr-FR", "en-US", "en" ]
> - ENV['LANG'] = fr.UTF-8
>
> (Currently Emscripten uses a hard-coded 'C.UTF-8'.)
>
> This PR allows starting your WebAssembly program directly using the
> user's preferred language (rather than shoving a language choice in
> their face on start-up ;)).
> Provided your program is i18n'd / multi-language, of course.
> For nodejs no changes.
>
> While this environment variable is pretty standard on desktop, Alon
> expressed concerned that this may be risky to start doing this now.
>
> Hence we moved the discussion to this list :)
> What do you think?
>
> Cheers!
> Beuc
>
> -- 
> You received this message because you are subscribed to the Google
> Groups "emscripten-discuss" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to emscripten-discuss+unsubscr...@googlegroups.com
> <mailto:emscripten-discuss+unsubscr...@googlegroups.com>.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/emscripten-discuss/acc93acd-b423-41f9-995b-b870d02c58dc%40googlegroups.com
> <https://groups.google.com/d/msgid/emscripten-discuss/acc93acd-b423-41f9-995b-b870d02c58dc%40googlegroups.com?utm_medium=email_source=footer>.
> For more options, visit https://groups.google.com/d/optout.

-- 
You received this message because you are subscribed to the Google Groups 
"emscripten-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to emscripten-discuss+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/emscripten-discuss/961c284f-f35c-e1e6-29fb-882579ac820e%40beuc.net.
For more options, visit https://groups.google.com/d/optout.

Re: Debugging WASM with C++ source

[Beuc]

I'm interested :)

- Beuc

On 11/04/2019 20:15, Charles Vaughn wrote:
> At Tableau, what we've done is very similar to what Floh mentioned,
> but baked into our tooling. It's probably not for everyone, since we
> require calls into WASM be asynchronous, and Javascript access WASM's
> heap in a serializable way. From those restrictions, we're able to use
> a python server that loads a native version of our WASM, and exposes
> our methods via a REST interface. We even support native executing
> Javascript through Chrome's remote debugging protocol.
>
> If there's interest keeping in mind those restrictions, I can look
> into sharing source.
>
> On Friday, April 5, 2019 at 1:28:58 PM UTC-7, Alon Zakai wrote:
>
> Yeah, debugging is definitely one of the bigger missing pieces
> here. This will hopefully improve soon in toolchains and browsers,
> there is ongoing work on it. Meanwhile debugging a parallel native
> build is often best, as mentioned.
>
> On Thu, Apr 4, 2019 at 8:56 PM Александр Гурьянов
> > wrote:
>
> Agree, I never debug emscripten output. Source map never works
> for big
> code bases. So all bug fixing done by playing with C++ native
> build,
> in hope that it will be ported 1:1, actually it's almost true.
>
> пт, 5 апр. 2019 г. в 03:28, Brian Gavin  >:
> >
> > My experience with source maps is similar to Floh.   Kind of
> works for very simple hello world apps but even then there are
> some lines I can not set breakpoints.  No variable
> inspection.   Stepping into and out of functions does not
> always work.
> >
> > Unfortunately my debugging technique is lots of prints and
> dump callstack.   You need to plan for development time to
> take longer then a regular C++ application.
> >
> > Better debugging is my number 1 feature wish.
> >
> > Brian Gavin
> >
> > --
> > You received this message because you are subscribed to the
> Google Groups "emscripten-discuss" group.
> > To unsubscribe from this group and stop receiving emails
> from it, send an email to
> emscripten-discuss+unsubscr...@googlegroups.com .
> > For more options, visit https://groups.google.com/d/optout
> <https://groups.google.com/d/optout>.
>
> -- 
> You received this message because you are subscribed to the
> Google Groups "emscripten-discuss" group.
> To unsubscribe from this group and stop receiving emails from
> it, send an email to
> emscripten-discuss+unsubscr...@googlegroups.com .
> For more options, visit https://groups.google.com/d/optout
> <https://groups.google.com/d/optout>.
>
> -- 
> You received this message because you are subscribed to the Google
> Groups "emscripten-discuss" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to emscripten-discuss+unsubscr...@googlegroups.com
> <mailto:emscripten-discuss+unsubscr...@googlegroups.com>.
> For more options, visit https://groups.google.com/d/optout.

-- 
You received this message because you are subscribed to the Google Groups 
"emscripten-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to emscripten-discuss+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: managing views after heap realloc

[Beuc]

Hi,

Would you like to propose a pull request? :)

https://github.com/emscripten-core/emscripten/blob/incoming/site/source/docs/porting/connecting_cpp_and_javascript/Interacting-with-code.rst
https://github.com/emscripten-core/emscripten/edit/incoming/site/source/docs/porting/connecting_cpp_and_javascript/Interacting-with-code.rst

- Sylvain

On 10/04/2019 14:53, Eric Mandel wrote:
> I might be really useful to add a sentence to the "Access Memory from
> Javascript" section of
> https://emscripten.org/docs/porting/connecting_cpp_and_javascript/Interacting-with-code.html,
> referencing your illuminating memory growth note
> https://gist.github.com/kripken/949eab99b7bc34f67c12140814d2b595. It's
> a good example of the requirement that "you know what you are doing",
> especially for C programmers who can't wait to get a buffer pointer
> back into Javascript.

-- 
You received this message because you are subscribed to the Google Groups 
"emscripten-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to emscripten-discuss+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Trying to diagnose a crash

[Beuc]

Hi,

Also, infinite loops normally produce a pop-up from Chromium first,
asking the user whether to "Exit page | Wait".

In case this helps, last time I got an immediate "Aw snap!" was due to
using too much memory in the virtual filesystem (>1 GB).

Cheers!
Beuc

On 05/04/2019 11:15, Floh wrote:
> Hmm, hard to say what's going on, an "Aw snap!" can be something as
> trivial as your program accidentally entering an infinite loop. If the
> browser runtime doesn't hear back from your program for some time
> (15..30 seconds or so), it will kill the tab and print this "Aw Snap"
> message.
>
> Those [violation] messages is probably just a little reminder that
> your frame callback function shouldn't take longer than the interval
> requestAnimationFrame() is running with (e.g. 16 or 33 ms).
>
> On Thursday, 4 April 2019 22:51:28 UTC+2, Zajo wrote:
>
> Thank you for your help, we had forgotten to pass -g, but still no
> luck.
>
> We do see a stack trace when an assert fires but not in this case.
> Here is an updated log, it can be seen that we print on every
> frame, but there are some suspicious "Violations", they don't seem
> normal. In the end, after drawing ~100 frames, we get "Aw, Snap!
> Something went wrong while displaying this webpage" from the browser.
>
> c5t_new.html:1246 wasm streaming compile failed: TypeError: Failed
> to execute 'compile' on 'WebAssembly': Incorrect response MIME
> type. Expected 'application/wasm'.
> printErr @ c5t_new.html:1246
> (anonymous) @ c5t_new.js:1648
> Promise.then (async)
> createWasm @ c5t_new.js:1645
> Module.asm @ c5t_new.js:1675
> (anonymous) @ c5t_new.js:8678
> c5t_new.html:1246 falling back to ArrayBuffer instantiation
> printErr @ c5t_new.html:1246
> (anonymous) @ c5t_new.js:1649
> Promise.then (async)
> createWasm @ c5t_new.js:1645
> Module.asm @ c5t_new.js:1675
> (anonymous) @ c5t_new.js:8678
> c5t_new.html:1237 main function started
> c5t_new.html:1237 SM initialized.
> c5t_new.html:1237 You should see a circle in a canvas.
> c5t_new.js:9210 [Violation] 'setTimeout' handler took 710ms
> c5t_new.html:1237 Draw 0
> c5t_new.js:5947 [Violation] 'requestAnimationFrame' handler took 60ms
> c5t_new.html:1237 Draw 1
> c5t_new.html:1237 Draw 2
> c5t_new.html:1237 Draw 3
> c5t_new.html:1237 Draw 4
> c5t_new.html:1237 Draw 5
> c5t_new.html:1237 Draw 6
> c5t_new.html:1237 Draw 7
> c5t_new.html:1237 Draw 8
> c5t_new.html:1237 Draw 9
> c5t_new.html:1237 Draw 10
> c5t_new.html:1237 Draw 11
> c5t_new.html:1237 Draw 12
> c5t_new.html:1237 Draw 13
> c5t_new.html:1237 Draw 14
> c5t_new.html:1237 Draw 15
> c5t_new.html:1237 Draw 16
> c5t_new.html:1237 Draw 17
> c5t_new.html:1237 Draw 18
> c5t_new.html:1237 Draw 19
> c5t_new.html:1237 Draw 20
> c5t_new.html:1237 Draw 21
> c5t_new.html:1237 Draw 22
> c5t_new.html:1237 Draw 23
> c5t_new.html:1237 Draw 24
> c5t_new.html:1237 Draw 25
> c5t_new.html:1237 Draw 26
> c5t_new.html:1237 Draw 27
> c5t_new.html:1237 Draw 28
> c5t_new.html:1237 Draw 29
> c5t_new.html:1237 Draw 30
> c5t_new.html:1237 Draw 31
> c5t_new.html:1237 Draw 32
> c5t_new.html:1237 Draw 33
> c5t_new.html:1237 Draw 34
> c5t_new.html:1237 Draw 35
> c5t_new.html:1237 Draw 36
> c5t_new.html:1237 Draw 37
> c5t_new.html:1237 Draw 38
> c5t_new.html:1237 Draw 39
> c5t_new.html:1237 Draw 40
> c5t_new.html:1237 Draw 41
> c5t_new.html:1237 Draw 42
> c5t_new.html:1237 Draw 43
> c5t_new.html:1237 Draw 44
> c5t_new.html:1237 Draw 45
> c5t_new.html:1237 Draw 46
> c5t_new.html:1237 Draw 47
> c5t_new.html:1237 Draw 48
> c5t_new.html:1237 Draw 49
> c5t_new.html:1237 Draw 50
> c5t_new.html:1237 Draw 51
> c5t_new.html:1237 Draw 52
> c5t_new.html:1237 Draw 53
> c5t_new.html:1237 Draw 54
> c5t_new.html:1237 Draw 55
> c5t_new.html:1237 Draw 56
> c5t_new.html:1237 Draw 57
> c5t_new.html:1237 Draw 58
> c5t_new.html:1237 Draw 59
> c5t_new.html:1237 Draw 60
> c5t_new.html:1237 Draw 61
> c5t_new.html:1237 Draw 62
> c5t_new.html:1237 Draw 63
> c5t_new.html:1237 Draw 64
> c5t_new.html:1237 Draw 65
> c5t_new.html:1237 Draw 66
> c5t_new.html:1237 Draw 67
> c5t_new.html:1237 Draw 68
> c5t_new.html:1237 Draw 69
> c5t_new.html:1237 Draw 70
> c5t_new.html:1237 Draw 71
> c5t_new.html:1237 Draw 72
>   

Emterpreter and changing function names

[Beuc]

Hi,

When compiling my RenPyWeb project, which use Emterpreter, I often get
errors about new, non-whitelisted functions such as:

___Pyx_PyObject_CallNoArg_387
___Pyx_PyObject_CallNoArg_6345

This may happen when I make some changes in the code or in the
compilation flags.

AFAIU these are generated by Emscripten ('grep' only sees references in
Emscripten-generated files) and the numbers are some kind of offset.

This is annoying because I usually have to compile (5-10mn depending on
target), run in a browser and wait for complete startup (couple
minutes), get the new stack strace, edit my Makefile and
EMTERPRETIFY_WHITELIST, and repeat.
And I need to tell my users/contributors to do the same ;)

Is there anything I can do to avoid these dynamic function names?

Cheers!
Beuc

-- 
You received this message because you are subscribed to the Google Groups 
"emscripten-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to emscripten-discuss+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Emterpreter and changing function names

[Beuc]

Hi,

When compiling my RenPyWeb project, which use Emterpreter, I often get
errors about new, non-whitelisted functions such as:

  ___Pyx_PyObject_CallNoArg_387
  ___Pyx_PyObject_CallNoArg_6345

This may happen when I make some changes in the code or in the
compilation flags.

AFAIU these are generated by Emscripten ('grep' only sees references in
Emscripten-generated files) and the numbers are some kind of offset.

This is annoying because I usually have to compile (5-10mn depending on
target), run in a browser and wait for complete startup (couple
minutes), get the new stack strace, edit my Makefile and
EMTERPRETIFY_WHITELIST, and repeat.
And I need to tell my users/contributors to do the same ;)

Is there anything I can do to avoid these dynamic function names?

Cheers!
Beuc


-- 
You received this message because you are subscribed to the Google Groups 
"emscripten-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to emscripten-discuss+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Dynamic linking and Chromium

[Beuc]

What am I saying, the same .so files are not compatible between asmjs
and wasm.

- Beuc

On 02/02/2019 12:43, Beuc wrote:
>
> Thanks, I linked the doc in the wiki.
>
> One issue though: if one loads the same the same data file from an
> asmjs alternate build, this makes dlopen fail.
> Perhaps preloading .so files should be inhibited in that case.
>
> - Sylvain
>
> On 02/02/2019 04:01, Alon Zakai wrote:
>> There are some workarounds here, which might help in some cases. One
>> is to preload the library files, then the loader code will compile
>> them asynchronously before the main program runs. Basically just
>> preloading the files normally will work, if they have the suffix
>> ".so", see
>> https://github.com/emscripten-core/emscripten/blob/incoming/src/library_browser.js#L233
>>
>> Another option is to call loadWebAssemblyModule manually, and pass it
>> loadAsync: true, like the plugin does in the above link.
>>
>> Thanks for helping document this!
>>
>> On Fri, Feb 1, 2019 at 3:22 PM Beuc > <mailto:b...@beuc.net>> wrote:
>>
>> Hi,
>>
>> When experimenting with dlopen() I remembered that Chromium doesn't
>> support compiling non-trivial WASM on the main thread, and
>> consequently
>> dlopen() is unusable:
>>
>> RangeError: WebAssembly.Compile is disallowed on the main thread,
>> if the
>> buffer size is larger than 4KB. Use WebAssembly.compile, or
>> compile on a
>> worker thread.
>>
>>     I added a note at
>> https://github.com/emscripten-core/emscripten/wiki/Linking
>> Is there a work-around to document (aside from moving everything to a
>> worker, which is not practical)?
>>
>> - Beuc
>>
>> -- 
>> You received this message because you are subscribed to the
>> Google Groups "emscripten-discuss" group.
>> To unsubscribe from this group and stop receiving emails from it,
>> send an email to emscripten-discuss+unsubscr...@googlegroups.com
>> <mailto:emscripten-discuss%2bunsubscr...@googlegroups.com>.
>> For more options, visit https://groups.google.com/d/optout.
>>
>> -- 
>> You received this message because you are subscribed to the Google
>> Groups "emscripten-discuss" group.
>> To unsubscribe from this group and stop receiving emails from it,
>> send an email to emscripten-discuss+unsubscr...@googlegroups.com
>> <mailto:emscripten-discuss+unsubscr...@googlegroups.com>.
>> For more options, visit https://groups.google.com/d/optout.
> -- 
> You received this message because you are subscribed to the Google
> Groups "emscripten-discuss" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to emscripten-discuss+unsubscr...@googlegroups.com
> <mailto:emscripten-discuss+unsubscr...@googlegroups.com>.
> For more options, visit https://groups.google.com/d/optout.

-- 
You received this message because you are subscribed to the Google Groups 
"emscripten-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to emscripten-discuss+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Dynamic linking and Chromium

[Beuc]

Thanks, I linked the doc in the wiki.

One issue though: if one loads the same the same data file from an asmjs
alternate build, this makes dlopen fail.
Perhaps preloading .so files should be inhibited in that case.

- Sylvain

On 02/02/2019 04:01, Alon Zakai wrote:
> There are some workarounds here, which might help in some cases. One
> is to preload the library files, then the loader code will compile
> them asynchronously before the main program runs. Basically just
> preloading the files normally will work, if they have the suffix
> ".so", see
> https://github.com/emscripten-core/emscripten/blob/incoming/src/library_browser.js#L233
>
> Another option is to call loadWebAssemblyModule manually, and pass it
> loadAsync: true, like the plugin does in the above link.
>
> Thanks for helping document this!
>
> On Fri, Feb 1, 2019 at 3:22 PM Beuc  <mailto:b...@beuc.net>> wrote:
>
> Hi,
>
> When experimenting with dlopen() I remembered that Chromium doesn't
> support compiling non-trivial WASM on the main thread, and
> consequently
> dlopen() is unusable:
>
> RangeError: WebAssembly.Compile is disallowed on the main thread,
> if the
> buffer size is larger than 4KB. Use WebAssembly.compile, or
> compile on a
> worker thread.
>
> I added a note at
> https://github.com/emscripten-core/emscripten/wiki/Linking
> Is there a work-around to document (aside from moving everything to a
> worker, which is not practical)?
>
> - Beuc
>
> -- 
> You received this message because you are subscribed to the Google
> Groups "emscripten-discuss" group.
> To unsubscribe from this group and stop receiving emails from it,
> send an email to emscripten-discuss+unsubscr...@googlegroups.com
> <mailto:emscripten-discuss%2bunsubscr...@googlegroups.com>.
> For more options, visit https://groups.google.com/d/optout.
>
> -- 
> You received this message because you are subscribed to the Google
> Groups "emscripten-discuss" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to emscripten-discuss+unsubscr...@googlegroups.com
> <mailto:emscripten-discuss+unsubscr...@googlegroups.com>.
> For more options, visit https://groups.google.com/d/optout.

-- 
You received this message because you are subscribed to the Google Groups 
"emscripten-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to emscripten-discuss+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Dynamic linking and Chromium

[Beuc]

Hi,

When experimenting with dlopen() I remembered that Chromium doesn't
support compiling non-trivial WASM on the main thread, and consequently
dlopen() is unusable:

RangeError: WebAssembly.Compile is disallowed on the main thread, if the
buffer size is larger than 4KB. Use WebAssembly.compile, or compile on a
worker thread.

I added a note at https://github.com/emscripten-core/emscripten/wiki/Linking
Is there a work-around to document (aside from moving everything to a
worker, which is not practical)?

- Beuc

-- 
You received this message because you are subscribed to the Google Groups 
"emscripten-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to emscripten-discuss+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Stricter dlopen?

[Beuc]

Hi,

I won't have time to follow-up on this bug (I already have quite a few
and I need to make progress on my actual port, plus I'm not directly
impacted anymore).
If anybody wants to step in, please do :)

- Sylvain

On 29/01/2019 01:19, 'Sam Clegg' via emscripten-discuss wrote:
> I agree that this seems to be a regression we should fix.  Would you
> mind opening a bug?
>
> On Thu, Jan 24, 2019 at 10:51 AM Beuc  wrote:
>> I'm not sure how this should work; AFAICS dlopen works even with static 
>> binaries under GNU/Linux.
>> To the least, printing out the name of the failed symbol would help 
>> understand the context.
>>
>> - Beuc
>>
>> On 24/01/2019 18:54, Alon Zakai wrote:
>>
>> I see, thanks. So should we make dlopen/dlsym just return "library/symbol 
>> not found" if dynamic linking is not present? Would that have avoided the 
>> confusing error for you?
>>
>>
>> On Thu, Jan 24, 2019 at 8:48 AM Beuc  wrote:
>>> The issue turns out a bit more twisted, I used to query "glReadBuffer", 
>>> which doesn't exist in GLES2, and wasn't used it my code anyway.
>>>
>>> The SDL2 port actually already calls SDL_GL_GetProcAddress => 
>>> device->GL_GetProcAddress => Emscripten_GLES_GetProcAddress => 
>>> SDL_EGL_GetProcAddress => _this->egl_data->eglGetProcAddress => 
>>> _emscripten_GetProcAddress (no less ;)).
>>>
>>> When this fails, however, SDL then tries SDL_LoadFunction => dlopen.
>>> Which I assume used to return NULL, and now raises an exception.
>>>
>>> Dropping the non-existing function call while still using 
>>> SDL_GL_GetProcAddress for the rest also solves the issue in my case.
>>>
>>> This makes SDL_GL_GetProcAddress still work but less forgiving (with a 
>>> confusing error message).
>>>
>>> - Beuc
>>>
>>> On 24/01/2019 16:20, Alon Zakai wrote:
>>>
>>> Oh, thanks, I missed that this was for SDL_GL_GetProcAddress getting GL 
>>> functions specifically. That should definitely still work. Maybe what's 
>>> going on is that the SDL code calls dlopen? Instead, it should call 
>>> emscripten_GetProcAddress (impl is in system/lib/gl/gl.c). Looks like we 
>>> need to add it to emscripten.h too. Beuc, is that change to SDL practical?
>>>
>>> On Thu, Jan 24, 2019 at 3:09 AM キャロウ　マーク  wrote:
>>>>
>>>>
>>>>> On Jan 23, 2019, at 8:35, Alon Zakai  wrote:
>>>>>
>>>>> Well, what I think happened is we used to have a very simple dlopen, and 
>>>>> when we rewrote it to be more powerful, we simplified it to only work on 
>>>>> linkable code. So the case of a module loading symbols from itself is a 
>>>>> case that is not well handled, you're right, it adds overhead there. 
>>>>> However, perhaps you can work around it in the code, avoid using dlopen 
>>>>> and get the function pointers directly?
>>>>>
>>>> This change apparently broke SDL_GL_GetProcAddress, the complaint at the 
>>>> start of this thread. Suggesting workarounds is not acceptable, except as 
>>>> a strictly temporary measure. The SDL port needs to be fixed. Given that 
>>>> most Open GL {,ES} applications dynamically obtain their function 
>>>> pointers, you need to consider other fixes too. Such as EGLGetProcAddress 
>>>> in the EGL emulation.
>>>>
>>>> Regards
>> --
>> You received this message because you are subscribed to the Google Groups 
>> "emscripten-discuss" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to emscripten-discuss+unsubscr...@googlegroups.com.
>> For more options, visit https://groups.google.com/d/optout.

-- 
You received this message because you are subscribed to the Google Groups 
"emscripten-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to emscripten-discuss+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Stricter dlopen?

[Beuc]

I'm not sure how this should work; AFAICS dlopen works even with static
binaries under GNU/Linux.
To the least, printing out the name of the failed symbol would help
understand the context.

- Beuc

On 24/01/2019 18:54, Alon Zakai wrote:
> I see, thanks. So should we make dlopen/dlsym just return
> "library/symbol not found" if dynamic linking is not present? Would
> that have avoided the confusing error for you?
>
>
> On Thu, Jan 24, 2019 at 8:48 AM Beuc  <mailto:b...@beuc.net>> wrote:
>
> The issue turns out a bit more twisted, I used to query
> "glReadBuffer", which doesn't exist in GLES2, and wasn't used it
> my code anyway.
>
> The SDL2 port actually already calls SDL_GL_GetProcAddress =>
> device->GL_GetProcAddress => Emscripten_GLES_GetProcAddress =>
> SDL_EGL_GetProcAddress => _this->egl_data->eglGetProcAddress =>
> _emscripten_GetProcAddress (no less ;)).
>
> When this fails, however, SDL then tries SDL_LoadFunction => dlopen.
> Which I assume used to return NULL, and now raises an exception.
>
> Dropping the non-existing function call while still using
> SDL_GL_GetProcAddress for the rest also solves the issue in my case.
>
> This makes SDL_GL_GetProcAddress still work but less forgiving
> (with a confusing error message).
>
> - Beuc
>
> On 24/01/2019 16:20, Alon Zakai wrote:
>> Oh, thanks, I missed that this was for SDL_GL_GetProcAddress
>> getting GL functions specifically. That should definitely still
>> work. Maybe what's going on is that the SDL code calls dlopen?
>> Instead, it should call emscripten_GetProcAddress (impl is in
>> system/lib/gl/gl.c). Looks like we need to add it to emscripten.h
>> too. Beuc, is that change to SDL practical?
>>
>> On Thu, Jan 24, 2019 at 3:09 AM キャロウ　マーク > <mailto:git...@callow.im>> wrote:
>>
>>
>>
>> > On Jan 23, 2019, at 8:35, Alon Zakai > <mailto:alonza...@gmail.com>> wrote:
>> >
>> > Well, what I think happened is we used to have a very
>> simple dlopen, and when we rewrote it to be more powerful, we
>> simplified it to only work on linkable code. So the case of a
>> module loading symbols from itself is a case that is not well
>> handled, you're right, it adds overhead there. However,
>> perhaps you can work around it in the code, avoid using
>> dlopen and get the function pointers directly?
>> >
>>
>> This change apparently broke SDL_GL_GetProcAddress, the
>> complaint at the start of this thread. Suggesting workarounds
>> is not acceptable, except as a strictly temporary measure.
>> The SDL port needs to be fixed. Given that most Open GL {,ES}
>> applications dynamically obtain their function pointers, you
>> need to consider other fixes too. Such as EGLGetProcAddress
>> in the EGL emulation.
>>
>> Regards
>>

-- 
You received this message because you are subscribed to the Google Groups 
"emscripten-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to emscripten-discuss+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Stricter dlopen?

[Beuc]

The issue turns out a bit more twisted, I used to query "glReadBuffer",
which doesn't exist in GLES2, and wasn't used it my code anyway.

The SDL2 port actually already calls SDL_GL_GetProcAddress =>
device->GL_GetProcAddress => Emscripten_GLES_GetProcAddress =>
SDL_EGL_GetProcAddress => _this->egl_data->eglGetProcAddress =>
_emscripten_GetProcAddress (no less ;)).

When this fails, however, SDL then tries SDL_LoadFunction => dlopen.
Which I assume used to return NULL, and now raises an exception.

Dropping the non-existing function call while still using
SDL_GL_GetProcAddress for the rest also solves the issue in my case.

This makes SDL_GL_GetProcAddress still work but less forgiving (with a
confusing error message).

- Beuc

On 24/01/2019 16:20, Alon Zakai wrote:
> Oh, thanks, I missed that this was for SDL_GL_GetProcAddress getting
> GL functions specifically. That should definitely still work. Maybe
> what's going on is that the SDL code calls dlopen? Instead, it should
> call emscripten_GetProcAddress (impl is in system/lib/gl/gl.c). Looks
> like we need to add it to emscripten.h too. Beuc, is that change to
> SDL practical?
>
> On Thu, Jan 24, 2019 at 3:09 AM キャロウ　マーク  <mailto:git...@callow.im>> wrote:
>
>
>
> > On Jan 23, 2019, at 8:35, Alon Zakai  <mailto:alonza...@gmail.com>> wrote:
> >
> > Well, what I think happened is we used to have a very simple
> dlopen, and when we rewrote it to be more powerful, we simplified
> it to only work on linkable code. So the case of a module loading
> symbols from itself is a case that is not well handled, you're
> right, it adds overhead there. However, perhaps you can work
> around it in the code, avoid using dlopen and get the function
> pointers directly?
> >
>
> This change apparently broke SDL_GL_GetProcAddress, the complaint
> at the start of this thread. Suggesting workarounds is not
> acceptable, except as a strictly temporary measure. The SDL port
> needs to be fixed. Given that most Open GL {,ES} applications
> dynamically obtain their function pointers, you need to consider
> other fixes too. Such as EGLGetProcAddress in the EGL emulation.
>
> Regards
>
>     -Mark
>
> -- 
> You received this message because you are subscribed to the Google
> Groups "emscripten-discuss" group.
> To unsubscribe from this group and stop receiving emails from it,
> send an email to emscripten-discuss+unsubscr...@googlegroups.com
> <mailto:emscripten-discuss%2bunsubscr...@googlegroups.com>.
> For more options, visit https://groups.google.com/d/optout.
>
> -- 
> You received this message because you are subscribed to the Google
> Groups "emscripten-discuss" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to emscripten-discuss+unsubscr...@googlegroups.com
> <mailto:emscripten-discuss+unsubscr...@googlegroups.com>.
> For more options, visit https://groups.google.com/d/optout.

-- 
You received this message because you are subscribed to the Google Groups 
"emscripten-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to emscripten-discuss+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Managing SDL2 + emterpreter

[Beuc]

Hi,

Thinking back on this issue, is there some risk to call
emscripten_sleep/emscripten_sleep_with_yield from JavaScript, as you
suggested?
Emscripten can only restore C stack frames AFAIU.

Also I've opened https://github.com/emscripten-ports/SDL2/issues/70

- Beuc

On 04/01/2019 03:37, Alon Zakai wrote:
> Interesting, this does sound like an area we should improve in.
>
> Maybe ideally you wouldn't need a patched SDL2. Perhaps SDL2 in those
> two methods could call something - maybe
> "emscripten_frame_pause/delay()" or some better name - which would be
> defined to be a no-op normally. And your app could define it in JS to
> call emscripten_sleep().
>
> Separately, maybe there's a simple way to make the ports system
> extensible, basically to make EMCC_LOCAL_PORTS less error-prone, or
> replace it with something simpler. Perhaps a -s option to override the
> URL and version tag of a port or something like that.
>
>
>
>
> On Sun, Dec 30, 2018 at 10:54 AM Beuc  <mailto:b...@beuc.net>> wrote:
>
> Hi,
>
> I'm running my recursion-heavy game app with emterpreter so I can
> pause/resume execution.
>
> To achieve this I patched SDL2 so it calls emscripten_sleep() on
> SDL_Delay and Emscripten_GLES_SwapWindow().
>
> I get quite annoyed by the port system which gets SDL2 straight from
> github and caches the result: if I ever forget to set EMCC_LOCAL_PORTS
> appropriately, emcc will happily fetch the non-patched, inappropriate
> version and my app will freeze. I also have to clean the cache
> whenever
> I need to change to a non-emterpreter build.
> I eventually patched tools/ports/sdl.py to handle a new "-s
> USE_SDL=2emterpreter" variant.
> I hesitate to drop -s USE_SDL and recompile SDL manually as several .a
> libs (with/without emterpreter) but I'd rather stay close to the
> current
> SDL2 port (which I contribute to).
>
> How do you people cope with SDL2+emterpreter?
> (ideally there's a way to support it without patching around both SDL2
> and emscripten?)
>
> Cheers!
> Beuc
>
> -- 
> You received this message because you are subscribed to the Google
> Groups "emscripten-discuss" group.
> To unsubscribe from this group and stop receiving emails from it,
> send an email to emscripten-discuss+unsubscr...@googlegroups.com
> <mailto:emscripten-discuss%2bunsubscr...@googlegroups.com>.
> For more options, visit https://groups.google.com/d/optout.
>
> -- 
> You received this message because you are subscribed to the Google
> Groups "emscripten-discuss" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to emscripten-discuss+unsubscr...@googlegroups.com
> <mailto:emscripten-discuss+unsubscr...@googlegroups.com>.
> For more options, visit https://groups.google.com/d/optout.

-- 
You received this message because you are subscribed to the Google Groups 
"emscripten-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to emscripten-discuss+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Stricter dlopen?

[Beuc]

Yeah that's how I had fixed it:
https://git.savannah.gnu.org/cgit/freedink.git/commit/?id=b879f418619a8e87e0bb614e4b84ca054cb8cfe9
Glad to see we came to the same conclusion :)

- Beuc

On 23/01/2019 00:35, Alon Zakai wrote:
> Well, what I think happened is we used to have a very simple dlopen,
> and when we rewrote it to be more powerful, we simplified it to only
> work on linkable code. So the case of a module loading symbols from
> itself is a case that is not well handled, you're right, it adds
> overhead there. However, perhaps you can work around it in the code,
> avoid using dlopen and get the function pointers directly?
>
> On Tue, Jan 22, 2019 at 2:53 PM Beuc  <mailto:b...@beuc.net>> wrote:
>
> Hi,
>
> To be honest I didn't try MAIN_MODULE=1, because this is
> documented as being slower, and incompatible with
> Emterpreter/pthreads/etc.
> I'm somewhat puzzled it worked before really, I'm asking to make
> sure I didn't miss something obvious.
>
> - Beuc
>
> On 22/01/2019 23:27, Alon Zakai wrote:
>> Maybe the error is not clear enough - I think all you need to do
>> is build with -s MAIN_MODULE=1. Does that fix things for you?
>>
>> On Tue, Jan 22, 2019 at 6:29 AM Beuc > <mailto:b...@beuc.net>> wrote:
>>
>> Hi,
>>
>> For the GNU FreeDink project, which has a custom OpenGL
>> loader, I use
>> code like:
>>
>>     BlendFunc = (void (APIENTRY*)(GLenum sfactor, GLenum
>> dfactor))SDL_GL_GetProcAddress("glBlendFunc");
>>
>> This stopped working recently, I believe this still worked in
>> 1.38.21
>> which I used for my last build, though the error message I
>> now get was
>> introduced in 538f958a900 (right before that version):
>>
>>     +    abort("To use dlopen, you need to use Emscripten's
>> linking
>> support, see
>> https://github.com/kripken/emscripten/wiki/Linking;);
>>
>>
>> I guess Emscripten previously supported loading symbols from
>> the current
>> binary, and now rejects it?
>>
>> While it makes sense to modify FreeDink to locate GL functions
>> statically under Emscripten, I don't see anything related in
>> ChangeLog.md, so I'd like to make sure.
>>
>> -- 
>> Beuc
>>
>> -- 
>> You received this message because you are subscribed to the
>> Google Groups "emscripten-discuss" group.
>> To unsubscribe from this group and stop receiving emails from
>> it, send an email to
>> emscripten-discuss+unsubscr...@googlegroups.com
>> <mailto:emscripten-discuss%2bunsubscr...@googlegroups.com>.
>> For more options, visit https://groups.google.com/d/optout.
>>
>> -- 
>> You received this message because you are subscribed to the
>> Google Groups "emscripten-discuss" group.
>> To unsubscribe from this group and stop receiving emails from it,
>> send an email to emscripten-discuss+unsubscr...@googlegroups.com
>> <mailto:emscripten-discuss+unsubscr...@googlegroups.com>.
>> For more options, visit https://groups.google.com/d/optout.
> -- 
> You received this message because you are subscribed to the Google
> Groups "emscripten-discuss" group.
> To unsubscribe from this group and stop receiving emails from it,
> send an email to emscripten-discuss+unsubscr...@googlegroups.com
> <mailto:emscripten-discuss+unsubscr...@googlegroups.com>.
> For more options, visit https://groups.google.com/d/optout.
>
> -- 
> You received this message because you are subscribed to the Google
> Groups "emscripten-discuss" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to emscripten-discuss+unsubscr...@googlegroups.com
> <mailto:emscripten-discuss+unsubscr...@googlegroups.com>.
> For more options, visit https://groups.google.com/d/optout.

-- 
You received this message because you are subscribed to the Google Groups 
"emscripten-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to emscripten-discuss+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Stricter dlopen?

[Beuc]

Hi,

To be honest I didn't try MAIN_MODULE=1, because this is documented as
being slower, and incompatible with Emterpreter/pthreads/etc.
I'm somewhat puzzled it worked before really, I'm asking to make sure I
didn't miss something obvious.

- Beuc

On 22/01/2019 23:27, Alon Zakai wrote:
> Maybe the error is not clear enough - I think all you need to do is
> build with -s MAIN_MODULE=1. Does that fix things for you?
>
> On Tue, Jan 22, 2019 at 6:29 AM Beuc  <mailto:b...@beuc.net>> wrote:
>
> Hi,
>
> For the GNU FreeDink project, which has a custom OpenGL loader, I use
> code like:
>
>     BlendFunc = (void (APIENTRY*)(GLenum sfactor, GLenum
> dfactor))SDL_GL_GetProcAddress("glBlendFunc");
>
> This stopped working recently, I believe this still worked in 1.38.21
> which I used for my last build, though the error message I now get was
> introduced in 538f958a900 (right before that version):
>
>     +    abort("To use dlopen, you need to use Emscripten's linking
> support, see https://github.com/kripken/emscripten/wiki/Linking;);
>
>
> I guess Emscripten previously supported loading symbols from the
> current
> binary, and now rejects it?
>
> While it makes sense to modify FreeDink to locate GL functions
> statically under Emscripten, I don't see anything related in
> ChangeLog.md, so I'd like to make sure.
>
> -- 
> Beuc
>
> -- 
> You received this message because you are subscribed to the Google
> Groups "emscripten-discuss" group.
> To unsubscribe from this group and stop receiving emails from it,
> send an email to emscripten-discuss+unsubscr...@googlegroups.com
> <mailto:emscripten-discuss%2bunsubscr...@googlegroups.com>.
> For more options, visit https://groups.google.com/d/optout.
>
> -- 
> You received this message because you are subscribed to the Google
> Groups "emscripten-discuss" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to emscripten-discuss+unsubscr...@googlegroups.com
> <mailto:emscripten-discuss+unsubscr...@googlegroups.com>.
> For more options, visit https://groups.google.com/d/optout.

-- 
You received this message because you are subscribed to the Google Groups 
"emscripten-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to emscripten-discuss+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Stricter dlopen?

[Beuc]

Hi,

For the GNU FreeDink project, which has a custom OpenGL loader, I use
code like:

    BlendFunc = (void (APIENTRY*)(GLenum sfactor, GLenum
dfactor))SDL_GL_GetProcAddress("glBlendFunc");

This stopped working recently, I believe this still worked in 1.38.21
which I used for my last build, though the error message I now get was
introduced in 538f958a900 (right before that version):

    +    abort("To use dlopen, you need to use Emscripten's linking
support, see https://github.com/kripken/emscripten/wiki/Linking;);


I guess Emscripten previously supported loading symbols from the current
binary, and now rejects it?

While it makes sense to modify FreeDink to locate GL functions
statically under Emscripten, I don't see anything related in
ChangeLog.md, so I'd like to make sure.

-- 
Beuc

-- 
You received this message because you are subscribed to the Google Groups 
"emscripten-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to emscripten-discuss+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: GitHub org?

[Beuc]

> One (non-GitHub) guy chimed in to say he thought the only way
> GitHub should act was if there was an official trademark claim.
>
How about we do just that?

- Beuc

-- 
You received this message because you are subscribed to the Google Groups 
"emscripten-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to emscripten-discuss+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Faking a thread (with emterpreter)

[Beuc]

Hi,

pthread's requirements are currently enabled in chromium but not in
firefox. I wonder if people could be told to enable the right option in
about:config...
If you target a browser that's behind firefox esr though, you can forget
about any recent emscripten feature.

AFAIU emterpreter doesn't support running several asynchronous calls at
once, so currently you cannot use it for both your main loop and your
long function.
I'm not sure why that is, maybe you could modify emterpreter to support
several call stacks.

Cheers!
Beuc

On 08/01/2019 02:03, Raptisoft wrote:
> Thanks for the quick reply!
>
> IDEALLY I would use pthreads, but it is my understanding that
> currently most browsers are not offering pthread support...?  The
> Waterfox browser that I'm testing in has them turned off by default
> (that would really kill my user-base because my main loop is of the
> SDL pump & sleep variety).
>
> But isn't the emterpreter bytecode being processed by a custom js
> virtual machine?  I had hoped that this meant it could and would be
> able to run pseudo-threads in an emulated (albeit slow) way.

-- 
You received this message because you are subscribed to the Google Groups 
"emscripten-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to emscripten-discuss+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Using WASM to add new file format support to browsers

[Beuc]

Hi,

Another extension with a new image format if you're interested:
https://bellard.org/bpg/
https://bellard.org/bpg/gallery2.html
https://bellard.org/bpg/animation.html

Nice target choice for the emulator ;)

- Beuc

On 05/01/2019 22:20, Floh wrote:
> Not a question, just something interesting I've been doing in the past
> few days...
>
> I've been playing around with 8-bit computer emulation in WASM/asm.js
> for a while now, but
> only recently had the idea of "what if the emulator application would
> disappear completely"
> and would only serve as an "invisible" player/runner for emulator file
> formats...
>
> I experimented with this here in this blog post:
>
> https://floooh.github.io/2019/01/05/wasm-embedding.html
>
> There's an embedded emulator in there which acts like an "embedded
> video player",
> except it's "playing" an emulator snapshot file.
>
> The most interesting thing are the file sizes, the emulator is made
> from 3 files (sizes are compressed):
>
> cpc.html (2.1KB), cpc.wasm (72.9 KB), cpc.js (27.8 KB)... which adds
> up to about 100 KBytes before the
> emulator starts. Even on a slow connection this is nearly
> instantaneous, and the size of the 'player'
> is actually smaller than the size of the data it loads (129KB,
> although this is only because the .sna snapshot file isn't compressed
> by the github web servers).
>
> It's also interesting to note that the 73KB WASM size contains the
> embedded system ROMs for the
> Amstrad CPC6128: 48 KBytes Z80 machine code (uncompressed, which gzips
> to about 37 KBytes, so the actual compressed WASM bytecode is more
> like 35 KBytes!
>
> This also shows that the size of the Javascript runtime file starts to
> hurt :) In the beginning this was about
> 17 KBytes, but then I had to enable the ccall/cwrap stuff again, which
> I think was mainly responsible for the 10 KByte jump. The other big
> chunk of stuff is the WebGL shim. I hope the new WASM/JS binding 
> stuff will help here.
>
> But anyway... I think "extending" browsers with small WASM modules
> like this is a very interesting direction!
> -- 
> You received this message because you are subscribed to the Google
> Groups "emscripten-discuss" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to emscripten-discuss+unsubscr...@googlegroups.com
> <mailto:emscripten-discuss+unsubscr...@googlegroups.com>.
> For more options, visit https://groups.google.com/d/optout.

-- 
You received this message because you are subscribed to the Google Groups 
"emscripten-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to emscripten-discuss+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Managing SDL2 + emterpreter

[Beuc]

Hi,

I'm running my recursion-heavy game app with emterpreter so I can
pause/resume execution.

To achieve this I patched SDL2 so it calls emscripten_sleep() on
SDL_Delay and Emscripten_GLES_SwapWindow().

I get quite annoyed by the port system which gets SDL2 straight from
github and caches the result: if I ever forget to set EMCC_LOCAL_PORTS
appropriately, emcc will happily fetch the non-patched, inappropriate
version and my app will freeze. I also have to clean the cache whenever
I need to change to a non-emterpreter build.
I eventually patched tools/ports/sdl.py to handle a new "-s
USE_SDL=2emterpreter" variant.
I hesitate to drop -s USE_SDL and recompile SDL manually as several .a
libs (with/without emterpreter) but I'd rather stay close to the current
SDL2 port (which I contribute to).

How do you people cope with SDL2+emterpreter?
(ideally there's a way to support it without patching around both SDL2
and emscripten?)

Cheers!
Beuc

-- 
You received this message because you are subscribed to the Google Groups 
"emscripten-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to emscripten-discuss+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: GitHub org?

[Beuc]

The GitHub->GitLab migration process was well tested last June due to
the github management change ;)
https://about.gitlab.com/2018/06/03/movingtogitlab/

Doc:
https://docs.gitlab.com/ee/user/project/import/github.html
https://www.youtube.com/watch?v=VYOXuOg9tQI

An org would change a lot of URLs so this makes it is a good time to
think about such move.

I don't think there's a way to make github forward the git remotes to
gitlab; they don't recommend using this for long anyway:
https://help.github.com/articles/about-repository-transfers/

(not particularly recommending one (self)host or another, but the
current github concentration always had a bad sourceforge-era taste to me)

++
Beuc

On 21/12/2018 10:59, Jukka Jylänki wrote:
> Sounds good to me to migrate to an org. Btw, would that invalidate
> existing URL links to Emscripten repository, issue tracker, tagged zip
> downloads etc? Tagged zip downloads are particularly tricky, current
> emsdk downloads from kripken/emscripten tagged zip releases, so if
> that location changes, we'll need to update emsdk accordingly.
>
> A highly subjective opinion, but definitely would not prefer even
> thinking about using somethingwentwronglab. (apologies for such a
> biased comment)
>
> pe 21. jouluk. 2018 klo 0.41 Alon Zakai (alonza...@gmail.com) kirjoitti:
>> Gitlab is an interesting option, but is there a way to transfer our repos to 
>> there, that is, with automatic forwarding of people's remotes, like github 
>> does? There's also the matter of the issue tracker, etc.
>>
>> On Thu, Dec 20, 2018 at 1:12 PM Lorn Potter  wrote:
>>>
>>>
>>> On 20/12/18 10:49 am, Alon Zakai wrote:
>>>> Maybe there are better ideas?
>>>>
>>>> (Plain "emscripten" was already taken, and when I contacted github they
>>>> said they can privately ask the owner to transfer it over, but they
>>>> never got back to me - maybe someone reading this knows more?)
>>>>
>>> How about using gitlab instead?
>>>
>>> https://gitlab.com/emscripten
>>>
>>> emscripten group is available.
>>>
>>> --
>>> You received this message because you are subscribed to the Google Groups 
>>> "emscripten-discuss" group.
>>> To unsubscribe from this group and stop receiving emails from it, send an 
>>> email to emscripten-discuss+unsubscr...@googlegroups.com.
>>> For more options, visit https://groups.google.com/d/optout.
>> --
>> You received this message because you are subscribed to the Google Groups 
>> "emscripten-discuss" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to emscripten-discuss+unsubscr...@googlegroups.com.
>> For more options, visit https://groups.google.com/d/optout.

-- 
You received this message because you are subscribed to the Google Groups 
"emscripten-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to emscripten-discuss+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Emterpreter and fpcast

[Beuc]

Thanks for the feedback!

I was worried fpcast would somehow switch to non-emterpreter execution
and cause issues on stop/resume.

Beuc

On 09/12/2018 17:22, Alon Zakai wrote:
> I believe that's normal, yes. For function pointer cast emulation we
> run binaryens fpcast-emu pass. It turns each indirect call into a call
> like that, so you will see them on the stack. This happens after
> emterpretify, and there should be no interactions between them that I
> can think of atm.
>
> On Sun, Dec 9, 2018 at 6:03 AM Beuc  <mailto:b...@beuc.net>> wrote:
>
> Hi,
>
> My current Python+Emterpreter app (https://renpy.beuc.net/) sometimes
> crashes unexpectedly, e.g. when going back to the browser tab after a
> few minutes of inactivity.
>
> I'm still trying to figure out where that comes from, but something in
> the JS stack trace intrigues me:
>
> ...
> emterpret
> emterpret
> _function_call
> byn$fpcast-emu$_function_call
> emterpret
> emterpret
> ...
>
> while I do specify:
> -s EMTERPRETIFY_WHITELIST='["_main", ... "_function_call"...
>
> (adding "byn$fpcast-emu$_function_call" doesn't change the behavior)
>
> The application generally works correctly when calling
> emscripten_sleep(), so it doesn't look like this is the culprit, but
> I'd rather make sure.
>
> Is it normal that fpcast'd functions appear that way in the
> emterpreter stack?
>
> Cheers!
> Sylvain
>
> -- 
> You received this message because you are subscribed to the Google
> Groups "emscripten-discuss" group.
> To unsubscribe from this group and stop receiving emails from it,
> send an email to emscripten-discuss+unsubscr...@googlegroups.com
> <mailto:emscripten-discuss%2bunsubscr...@googlegroups.com>.
> For more options, visit https://groups.google.com/d/optout.
>
> -- 
> You received this message because you are subscribed to the Google
> Groups "emscripten-discuss" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to emscripten-discuss+unsubscr...@googlegroups.com
> <mailto:emscripten-discuss+unsubscr...@googlegroups.com>.
> For more options, visit https://groups.google.com/d/optout.

-- 
You received this message because you are subscribed to the Google Groups 
"emscripten-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to emscripten-discuss+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Emterpreter and fpcast

[Beuc]

Hi,

My current Python+Emterpreter app (https://renpy.beuc.net/) sometimes
crashes unexpectedly, e.g. when going back to the browser tab after a
few minutes of inactivity.

I'm still trying to figure out where that comes from, but something in
the JS stack trace intrigues me:

...
emterpret
emterpret
_function_call
byn$fpcast-emu$_function_call
emterpret
emterpret
...

while I do specify:
-s EMTERPRETIFY_WHITELIST='["_main", ... "_function_call"...

(adding "byn$fpcast-emu$_function_call" doesn't change the behavior)

The application generally works correctly when calling
emscripten_sleep(), so it doesn't look like this is the culprit, but
I'd rather make sure.

Is it normal that fpcast'd functions appear that way in the emterpreter stack?

Cheers!
Sylvain

-- 
You received this message because you are subscribed to the Google Groups 
"emscripten-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to emscripten-discuss+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Using npm install rather than check-in node_modules

[beuc]

Hi,

On Tue, Nov 20, 2018 at 02:18:02PM -0800, 'Sam Clegg' via emscripten-discuss 
wrote:
> On Tue, Nov 20, 2018 at 1:01 PM Sam Clegg  wrote:
> > On Tue, Nov 20, 2018 at 12:24 PM  wrote:
> > > On Tue, Nov 20, 2018 at 12:05:28PM -0800, 'Sam Clegg' via 
> > > emscripten-discuss wrote:
> > > > As part of a cleanup operation and an attempt to use node modules in a
> > > > more conventional way it is being proposed that we remove the
> > > > checked-in node_modules directories that currently exist in emscripten
> > > > and replace them dependencies listed in package.json.
> > > >
> > > > This will hopefully be transparent to all users as we can run `npm
> > > > install` as need as part of the compiler driver and we already depend
> > > > on node, which ships with npm.
> > > >
> > > > Issue: https://github.com/kripken/emscripten/issues/7538
> > > > With in progress CL: https://github.com/kripken/emscripten/issues/7538
> > > >
> > > > Any concens with doing doing things this way?
> > > >
> > > > One assumption we would be making is that anyone who as a `node`
> > > > binary has an `npm` binary alongside it.  Is that reasonable?
> > >
> > > Those 2 are separate packages in Debian.
> > > I typically do not have npm installed (without doing anything special).
> > > (also installing npm pulls 242 new packages)
> > >
> >
> > Interesting.  I didn't realize that.   Certainly anyone who downloads
> > the node binary package and npm for free.  As does anyone who uses
> > emsdk.   So we would be adding an extra dependency for those who work
> > directly on emscripten (i.e. don't use the emsdk) and also use a
> > packaging system that separates node and npm.
> 
> Sorry I meant to set ".. gets npm for free", because it comes with the
> upstream node binary package.
> 
> And to clarify, anyone who uses emsdk is using the version of node
> that emsdk downloads, which also has npm.
> 
> >
> > So the question becomes, is it reasonable to ask such developer to run
> > `apt-get install npm`?
> >
> > > Does that mean more stuff downloaded live from the Internet?
> > > I fear of builds that will suddently break without any change on the
> > > dev's config and/or won't compile offline.
> >
> > No, I don't think this can happen because we use a package-lock.json,
> > so dependencies can't change unless that file is changed in source
> > control.

Thanks a lot for the explanations.

I believe that npm installs packages in a project-specific location
(without cluttering the developer's setup) so that sounds reasonable
:)

(btw the original post links twice to #7538 - not sure what you meant
by "CL"?)

- Sylvain

-- 
You received this message because you are subscribed to the Google Groups 
"emscripten-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to emscripten-discuss+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Using npm install rather than check-in node_modules

[beuc]

Hi!

On Tue, Nov 20, 2018 at 12:05:28PM -0800, 'Sam Clegg' via emscripten-discuss 
wrote:
> As part of a cleanup operation and an attempt to use node modules in a
> more conventional way it is being proposed that we remove the
> checked-in node_modules directories that currently exist in emscripten
> and replace them dependencies listed in package.json.
> 
> This will hopefully be transparent to all users as we can run `npm
> install` as need as part of the compiler driver and we already depend
> on node, which ships with npm.
> 
> Issue: https://github.com/kripken/emscripten/issues/7538
> With in progress CL: https://github.com/kripken/emscripten/issues/7538
> 
> Any concens with doing doing things this way?
> 
> One assumption we would be making is that anyone who as a `node`
> binary has an `npm` binary alongside it.  Is that reasonable?

Those 2 are separate packages in Debian.
I typically do not have npm installed (without doing anything special).
(also installing npm pulls 242 new packages)

Does that mean more stuff downloaded live from the Internet?
I fear of builds that will suddently break without any change on the
dev's config and/or won't compile offline.

Cheers!
Sylvain

-- 
You received this message because you are subscribed to the Google Groups 
"emscripten-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to emscripten-discuss+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

PROXY_TO_PTHREAD + SDL2 status

[Beuc]

Hi,

I'm experimenting with a simple, non-async SDL2 loop in a background
thread/worker using PROXY_TO_PTHREAD=1.

I only get a:
  ReferenceError: screen is not defined
straight on SDL_Init(), with or without offscreen canvas, with both
Chromium 70 and FF Nightly.

I didn't see a matching PR nor a test case (only pure GL ones).

Is this working yet?
If not, how far are we? :)

Cheers!
Sylvain

-- 
You received this message because you are subscribed to the Google Groups 
"emscripten-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to emscripten-discuss+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: gzip fallback for annoying CDNs

[beuc]

On Tue, Nov 13, 2018 at 10:25:55AM +0100, b...@beuc.net wrote:
> Hi,
> 
> On Mon, Nov 12, 2018 at 03:40:32PM -0800, Alon Zakai wrote:
> > You can use zee.js to gunzip in JS on the client,
> > 
> > https://github.com/kripken/zee.js
> > 
> > (I can upload a new build there if people want). There are also some JS
> > libraries but I'm not very familiar with them, like pako and zlib.js. With
> > any of those, you just need to replace the loading of the main JS with code
> > that fetches, decompresses, and evals or adds a script tag, and for the
> > wasm you can preload it on Module.wasmBinary. So no shell.js modifications
> > should be necessary.
> 
> I'll try pre-setting Module.wasmBinary.
> The main.js' loading seems to come from emcc.py though.
> 
> > It would be best though if webservers did the compression so that the
> > browser can do it, which also allows wasm streaming compilation etc. - I
> > guess I don't fully understand the reasons why that isn't practical on some
> > hosting services.
> 
> AFAIU, HTML5 games have been downloading pre-compressed .gz resources
> and extracting them in JS for a while. Now if compression headers are
> enabled, this breaks those previous games (double-decompression). And
> CDN configuration is probably global for all hosted projects.
> More generally any change in configuration can break previous games so
> I guess nobody wants to do that.
> 
> Note the emscripten doc recommendation is not practical, as it
> compresses the .wasm/.data on the fly for each request. This makes the
> web server CPU-bound (for >10MB files, even for a single request).
> 
> Pre-compressing to .gz files, and adding content-negotation
> (MultiViews) is far more efficient.  Side-effect: this adds a
> 'Content-Encoding' header to both .js and .js.gz downloads, so direct
> .gz downloads are now pre-decompressed by the browser.
> 
> Unity and UnrealEngine seem to use a .jsgz extension but I'm not sure
> what semantics they're aiming at.  Anybody on this list familiar with
> those?
> 
> I was thinking of fetching the .gz resources, checking if they have a
> "Content-Encoding: gzip" header.
> If yes, proceed as usual.
> If not, decompress them from JS.
> If 404, download without .gz and proceed as usual.

Or rather:
- download .js/.wasm/.data/... (can stream-compile)
- if 404, download .js.gz/.wasm.gs/.data.gz/...
--- if "Content-Encoding: gzip", can stream-compile
--- if no "Content-Encoding: gzip", gunzip in JS

- Sylvain

-- 
You received this message because you are subscribed to the Google Groups 
"emscripten-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to emscripten-discuss+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: How To Use Freetype?

[beuc]

Hi,

What OpenGL context did you request?
Format 'GL_RED' requires GLES3/WebGL2 compatibility.

AFAICR SDL_ttf renders as RGBA and it's the programmer's responsibility
to upload that surface as a GL texture.

- Sylvain
(OpenGL Programming contributor ;))

On 11/11/2018 05:39, Bruce Davidson wrote:
> I am unable to get Freetype to display in enscriptem. I've found these
> tutorials:
>
> https://learnopengl.com/In-Practice/Text-Rendering
> https://en.wikibooks.org/wiki/OpenGL_Programming/Modern_OpenGL_Tutorial_Text_Rendering_01
>
> I've gotten both working on desktop with no issues, using shader
> verion 300 es.
> Then, when I compile with emcc I'm setting:
>
>     -s USE_FREETYPE=1 ^
>
> The assets are also included, and there are no compile errors, no
> runtime errors, but when I run it in Emscripten, the text is not
> displayed. When I check the browser console. it tells me that gles is
> unable to display because of invalid internal format, and it may be
> not-power-of-2. That actually makes sense, because the technique sets
>
>     glPixelStorei(GL_UNPACK_ALIGNMENT, 1); 
>
> which is not a power of 2 - fonts use a grayscale bitmap. So it seems
> like Freetype is not compatible, but then why does sdl2_ttf work, right? 
>
> How do I get Freetype to work in Emscripten - is there a working
> example anywhere? I checked the tests like
> https://github.com/kripken/emscripten/blob/master/tests/freetype/main.c,
> and the thing is that while the test passes, it doesn't really test if
> the texture is valid, it just prints the size.
>
>
>
> -- 
> You received this message because you are subscribed to the Google
> Groups "emscripten-discuss" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to emscripten-discuss+unsubscr...@googlegroups.com
> .
> For more options, visit https://groups.google.com/d/optout.

-- 
You received this message because you are subscribed to the Google Groups 
"emscripten-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to emscripten-discuss+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

gzip fallback for annoying CDNs

[beuc]

Hi,

As https://github.com/itchio/itch.io/issues/394 shows
some hosting services have tailored their HTTP server configuration to
Unity HTML5,
which decompresses .js/.wasm with a JS gzip.

This breaks
https://kripken.github.io/emscripten-site/docs/compiling/WebAssembly.html#web-server-setup
and fixing the CDN is clearly low priority.

So I'm wondering how one can do Unity-style JS decompression.
Does one has to patch e.g. shell.js? Other places?
Did anybody publish such alternative under a free license?

Cheers!
Sylvain

-- 
You received this message because you are subscribed to the Google Groups 
"emscripten-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to emscripten-discuss+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Removal of EMSCRIPTEN_ROOT from config file

[Beuc]

This makes sense, if we can configure EMSCRIPTEN_ROOT automagically, all
the better.
I mentioned LLVM_ROOT since Sam's use case was "parses the config file
and finds EMSCRIPTEN_ROOT pointing to different version" - I mean if
.emscripten is out-of-sync there will be other problems, but that's when
we'd have to use EM_CONFIG anyway.

- Sylvain

On 12/10/2018 00:37, Alon Zakai wrote:
> LLVM_ROOT is somewhat different - we have to be given the path to LLVM
> somehow. But emscripten code should know where emscripten is (since
> every python script can tell, etc.). So there is a risk of getting out
> of sync for EMSCRIPTEN_ROOT, and I think it makes sense to remove it,
> but not LLVM_ROOT, unless I'm missing something.
>
> On Wed, Oct 10, 2018 at 3:00 AM Beuc  <mailto:b...@beuc.net>> wrote:
>
> Hi,
>
> I believe we'd need to deal with LLVM_ROOT as well, since the fastcomp
> version needs to be in sync with emscripten's?
>
> - Sylvain
>
> On 10/10/2018 02:41, Sam Clegg wrote:
> > TLDR: There is a field in called EMSCRIPTEN_ROOT in the config file
> > which in theory can be used by external tools to find the "active"
> > emscripten.   I'm proposing to remove it.
> >
> > ---
> >
> > Maintaining this field has a cost and it can get out of sync
> with the
> > emscripten you are actually using.    Imagine you run `emcc` and if
> > parses the config file and finds EMSCRIPTEN_ROOT pointing to
> different
> > version of emscripten.
> >
> > The two current users of EMSCRIPTEN_ROOT that I know of are the
> scons support:
> >
> 
> https://github.com/kripken/emscripten/blob/incoming/tools/scons/site_scons/site_tools/emscripten/emscripten.py
> > And ammo.js:
> https://github.com/kripken/ammo.js/blob/master/make.py#L17
> >
> > In both of these cases a better solution would be either:
> > 1) looks for `emcc` in the $PATH
> > 2) check for EMSCRIPTEN_ROOT in the environment.
> >
> > Parsing the config file is also a rather brittle solution, and
> > prevents us from iterating on the config file format and how its
> > parses.  It also uses python's `eval` which is nasty.
> >
> > Any objections to following this path?
> >
>
> -- 
> You received this message because you are subscribed to the Google
> Groups "emscripten-discuss" group.
> To unsubscribe from this group and stop receiving emails from it,
> send an email to emscripten-discuss+unsubscr...@googlegroups.com
> <mailto:emscripten-discuss%2bunsubscr...@googlegroups.com>.
> For more options, visit https://groups.google.com/d/optout.
>
> -- 
> You received this message because you are subscribed to the Google
> Groups "emscripten-discuss" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to emscripten-discuss+unsubscr...@googlegroups.com
> <mailto:emscripten-discuss+unsubscr...@googlegroups.com>.
> For more options, visit https://groups.google.com/d/optout.

-- 
You received this message because you are subscribed to the Google Groups 
"emscripten-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to emscripten-discuss+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Removal of EMSCRIPTEN_ROOT from config file

[Beuc]

Hi,

I believe we'd need to deal with LLVM_ROOT as well, since the fastcomp
version needs to be in sync with emscripten's?

- Sylvain

On 10/10/2018 02:41, Sam Clegg wrote:
> TLDR: There is a field in called EMSCRIPTEN_ROOT in the config file
> which in theory can be used by external tools to find the "active"
> emscripten.   I'm proposing to remove it.
>
> ---
>
> Maintaining this field has a cost and it can get out of sync with the
> emscripten you are actually using.Imagine you run `emcc` and if
> parses the config file and finds EMSCRIPTEN_ROOT pointing to different
> version of emscripten.
>
> The two current users of EMSCRIPTEN_ROOT that I know of are the scons support:
> https://github.com/kripken/emscripten/blob/incoming/tools/scons/site_scons/site_tools/emscripten/emscripten.py
> And ammo.js: https://github.com/kripken/ammo.js/blob/master/make.py#L17
>
> In both of these cases a better solution would be either:
> 1) looks for `emcc` in the $PATH
> 2) check for EMSCRIPTEN_ROOT in the environment.
>
> Parsing the config file is also a rather brittle solution, and
> prevents us from iterating on the config file format and how its
> parses.  It also uses python's `eval` which is nasty.
>
> Any objections to following this path?
>

-- 
You received this message because you are subscribed to the Google Groups 
"emscripten-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to emscripten-discuss+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Catching "no binaryen method succeeded" under Chrome

[Beuc]

Thanks a lot, -incoming with this patch runs fine in my Chromium :)

- Sylvain

On 06/10/2018 15:56, Alon Zakai wrote:
> Looks like recent improvements make it easy to remove this specific
> limitation, I opened
>
> https://github.com/kripken/emscripten/pull/7235
>
> On Fri, Oct 5, 2018 at 4:32 PM Beuc  <mailto:b...@beuc.net>> wrote:
>
> Yeah that's my second point basically.
> Check tools/shared.py:is_wasm_only() for the flags.
> (Incidentally, any chance this issue could be fixed in chrom*? ;))
>
> On 06/10/2018 01:13, Sam Clegg wrote:
> > I think the thing we really want to fix here is the fact the async
> > compilation is disabled under certain circumstances.  For
> projects of
> > any reasonable size we want to always be doing async compiles, which
> > will fix the issue on chrome.  I'm not sure exactly which
> combination
> > of flags trigger "BINARYEN_ASYNC_COMPILATION disabled due to user
> > options" but we should work to remove this limitation.
> > On Fri, Oct 5, 2018 at 2:35 AM Sylvain  <mailto:b...@beuc.net>> wrote:
> >> Hi,
> >>
> >> Due to using EMULATE_FUNCTION_POINTER_CASTS (required for
> Python) I get when compiling:
> >> BINARYEN_ASYNC_COMPILATION disabled due to user options. This
> will reduce performance and compatibility (some browsers limit
> synchronous compilation)
> >>
> >> This indeed causes an error in Chrome:
> >> failed to compile wasm module: RangeError: WebAssembly.Compile
> is disallowed on the main thread, if the buffer size is larger
> than 4KB. Use WebAssembly.compile, or compile on a worker thread.
> >> Assertion failed: no binaryen method succeeded.
> >>
> >> Currently I have an alternate version compiled in asmjs (with
> decreased perf), and I blindly redirect to it when I detect Chrome.
> >>
> >> - Assuming Chrome fixes this later, is there a way to cleanly
> catch this "no binaryen method succeeded"?
> >>
> >> - Incidentally, what would be required to make
> EMULATE_FUNCTION_POINTER_CASTS compatible with Chrome wasm?
> >>
> >> Cheers!
> >>
>
> -- 
> You received this message because you are subscribed to the Google
> Groups "emscripten-discuss" group.
> To unsubscribe from this group and stop receiving emails from it,
> send an email to emscripten-discuss+unsubscr...@googlegroups.com
> <mailto:emscripten-discuss%2bunsubscr...@googlegroups.com>.
> For more options, visit https://groups.google.com/d/optout.
>
> -- 
> You received this message because you are subscribed to the Google
> Groups "emscripten-discuss" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to emscripten-discuss+unsubscr...@googlegroups.com
> <mailto:emscripten-discuss+unsubscr...@googlegroups.com>.
> For more options, visit https://groups.google.com/d/optout.

-- 
You received this message because you are subscribed to the Google Groups 
"emscripten-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to emscripten-discuss+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Catching "no binaryen method succeeded" under Chrome

[Beuc]

Yeah that's my second point basically.
Check tools/shared.py:is_wasm_only() for the flags.
(Incidentally, any chance this issue could be fixed in chrom*? ;))

On 06/10/2018 01:13, Sam Clegg wrote:
> I think the thing we really want to fix here is the fact the async
> compilation is disabled under certain circumstances.  For projects of
> any reasonable size we want to always be doing async compiles, which
> will fix the issue on chrome.  I'm not sure exactly which combination
> of flags trigger "BINARYEN_ASYNC_COMPILATION disabled due to user
> options" but we should work to remove this limitation.
> On Fri, Oct 5, 2018 at 2:35 AM Sylvain  wrote:
>> Hi,
>>
>> Due to using EMULATE_FUNCTION_POINTER_CASTS (required for Python) I get when 
>> compiling:
>> BINARYEN_ASYNC_COMPILATION disabled due to user options. This will reduce 
>> performance and compatibility (some browsers limit synchronous compilation)
>>
>> This indeed causes an error in Chrome:
>> failed to compile wasm module: RangeError: WebAssembly.Compile is disallowed 
>> on the main thread, if the buffer size is larger than 4KB. Use 
>> WebAssembly.compile, or compile on a worker thread.
>> Assertion failed: no binaryen method succeeded.
>>
>> Currently I have an alternate version compiled in asmjs (with decreased 
>> perf), and I blindly redirect to it when I detect Chrome.
>>
>> - Assuming Chrome fixes this later, is there a way to cleanly catch this "no 
>> binaryen method succeeded"?
>>
>> - Incidentally, what would be required to make 
>> EMULATE_FUNCTION_POINTER_CASTS compatible with Chrome wasm?
>>
>> Cheers!
>>

-- 
You received this message because you are subscribed to the Google Groups 
"emscripten-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to emscripten-discuss+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: SharedArrayBuffer status/roadmap

[Beuc]

Thanks a lot for the pointers!

- Beuc

On 05/10/2018 04:24, Alon Zakai wrote:
> SharedArrayBuffer has been re-enabled in Chrome meanwhile (after
> having implemented a good solution to the underlying security issues,
> Spectre etc.). You can use JS with threads there right now, including
> asm.js builds from emscripten.
>
> Other browsers are expected to also re-enable SAB in time, but I don't
> know of specific timetables for that.
>
> For wasm, in Chrome wasm-threads are not yet enabled by default but
> you can flip the flag in the settings to test with it right now.
>
> Regarding game engines and such using this, there are some PRs that
> are adding useful functionality for supporting rendering in a
> background thread etc., see this PR and others in the sequence,
>
> https://github.com/kripken/emscripten/pull/6201
>
>
>
>
> On Wed, Oct 3, 2018 at 2:53 AM Beuc  <mailto:b...@beuc.net>> wrote:
>
> Hi,
>
> I'm currently working on porting the Ren'Py game engine to the
> browser (
> see https://renpy.beuc.net/ ).
> Besides the challenge of handling Python + various Cython modules, the
> engine's main loop is deeply nested/recursive and pretty much
> impossible
> to rewrite asynchronously.
>
> I'm currently using EMTERPRETIFY_ASYNC, various patches in
> SDL2/Pygame_SDL2, and ad-hoc work-arounds for the game threads.
> Performances are far from optimal, even with emterpreter
> white-listing.
>
> I believe the long-term answer would be something like webgl-worker
> (
> 
> https://research.mozilla.org/2014/07/22/webgl-in-web-workers-today-and-faster-than-expected/
> )
> where the Ren'Py main loop would be in its own thread.
> AND, since the worker would need to grab events without returning from
> its main loop, we'd need a SharedArrayBuffer-based SDL2 event loop.
> This would allow a new class of apps to run unmodified in emscripten.
>
>
> Hence my post: is there a status of SharedArrayBuffer support?
> It was disabled in browser in January following Meltdown/Spectre (I
> didn't precisely got why), and still not re-enabled despite fixes and
> mitigation being available now.
> Also, is there interest/plans for a SharedArrayBuffer-based SDL2 port?
>
> (or maybe I missed an easier solution to my port? ;))
>
> Cheers!
> Beuc
>
> -- 
> You received this message because you are subscribed to the Google
> Groups "emscripten-discuss" group.
> To unsubscribe from this group and stop receiving emails from it,
> send an email to emscripten-discuss+unsubscr...@googlegroups.com
> <mailto:emscripten-discuss%2bunsubscr...@googlegroups.com>.
> For more options, visit https://groups.google.com/d/optout.
>
> -- 
> You received this message because you are subscribed to the Google
> Groups "emscripten-discuss" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to emscripten-discuss+unsubscr...@googlegroups.com
> <mailto:emscripten-discuss+unsubscr...@googlegroups.com>.
> For more options, visit https://groups.google.com/d/optout.

-- 
You received this message because you are subscribed to the Google Groups 
"emscripten-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to emscripten-discuss+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

SharedArrayBuffer status/roadmap

[Beuc]

Hi,

I'm currently working on porting the Ren'Py game engine to the browser (
see https://renpy.beuc.net/ ).
Besides the challenge of handling Python + various Cython modules, the
engine's main loop is deeply nested/recursive and pretty much impossible
to rewrite asynchronously.

I'm currently using EMTERPRETIFY_ASYNC, various patches in
SDL2/Pygame_SDL2, and ad-hoc work-arounds for the game threads.
Performances are far from optimal, even with emterpreter white-listing.

I believe the long-term answer would be something like webgl-worker
(
https://research.mozilla.org/2014/07/22/webgl-in-web-workers-today-and-faster-than-expected/
)
where the Ren'Py main loop would be in its own thread.
AND, since the worker would need to grab events without returning from
its main loop, we'd need a SharedArrayBuffer-based SDL2 event loop.
This would allow a new class of apps to run unmodified in emscripten.


Hence my post: is there a status of SharedArrayBuffer support?
It was disabled in browser in January following Meltdown/Spectre (I
didn't precisely got why), and still not re-enabled despite fixes and
mitigation being available now.
Also, is there interest/plans for a SharedArrayBuffer-based SDL2 port?

(or maybe I missed an easier solution to my port? ;))

Cheers!
Beuc

-- 
You received this message because you are subscribed to the Google Groups 
"emscripten-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to emscripten-discuss+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Linking many bitcode files into WASM

[beuc]

Hi!

On 02/10/2018 01:43, sha...@infograph.com wrote:
> Hello, I am new to WebAssembly and Emscriptem.
> I am converting a large C/C++ project (hundreds of source files) to WASM.
> I see how I can compile every source file to a bitcode file:
>
> emcc -O3 hello.c -o hello.bc
> emcc -O3 foo.c -o foo.bc
> emcc -O3 bar.c -o bar.bc
>
> When I link them into a wasm, js, html set of output files, it looks
> like this:
>
> emcc -O3 -s WASM=1 hello.bc foo.bc bar.bc -o hello.html
>
> But what if I have more than 3 bitcode files?  What if I have over 700?
> That won't fit on the commandline.  
> I tried using a wildcard:
>
> emcc -O3 -s WASM=1 *.bc -o hello.html
>
> but that gave an error.

And, what was the error?

> Any ideas?
> Could I write all the input files into a file and specify that
> filename instead?

Can you run xargs --show-limits and copy/paste the output?
Nowadays you get 2MB of command line size which should allow at least
8000+ files at once.
(if you're using GNU/Linux, that is)

Cheers!
Sylvain

-- 
You received this message because you are subscribed to the Google Groups 
"emscripten-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to emscripten-discuss+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.