Hi, (I was hoping other people would give their PoV...)
Generally-speaking, we may need more patches because of: - the combination of Emscripten features (Python + uninterruptible main loop -- aka fpcast+async; thankfully I could get rid of dynamic linking...) - centrally improving Emscripten portability rather than adapting the codebases (e.g. spending weeks on the 1-line #8751 / LANG support, rather than adding a quick EM_ASM in all my projects) - working directly with Emscripten rather than using a framework that provides a layer of work-arounds (e.g. Unity) Again, at a given point of time we always have a buffer of pending patches (I forgot to mention #6511 and #7631 that were added to our shell.html template, btw), the ones we can upstream are replaced by newer ones: old bugs we eventually tackle like fullscreen support or autosave-on-quit; backports for regressions in a given Emscripten release. Hopefully one day the web ecosystem will be so mature that we'll be able to take our distro's years-old compiler, `./configure --host=browser` without any change, and publish. Meanwhile... we patch :) To answer your specific questions: - This is a game engine, much of the tests are manual, sadly (but I contribute tests when I submit Emscripten patches); we often get integration errors, or errors due to combining multiple Emscripten options. - I upgrade Emscripten to a tagged version ~every other month; since this is likely to introduce breaking changes, I only do it when I have the time to deal with the consequences (e.g. when I'm not in the middle of developing RenPyWeb itself). I avoid ToT for stability and reproducibility reasons. - EMT_STACK_MAX is the exception, not the rule ;) That's about the only thing I could have upstream but didn't due to lack of time -- and it's now deprecated. - Tagging: TBH I didn't figure out yet e.g. "what LLVM commit is used for emsdk 1.38.42" from https://chromium.googlesource.com/emscripten-releases/ Cheers! Sylvain On 07/09/2019 00:58, Alon Zakai wrote: > I think those are all reasonable requests, thanks for opening those > issues and highlighting them here! I hope we can make progress on each. > > And hopefully other users can provide useful advice from their > perspective and experience. > > About "no matching tags for llvm and bynarien, commit IDs quite hard > to track (discussion in #9317/#9383)" - I think we have resolved that? > Please let me know if not. > > And a general comment: Your overall experience does surprise me (which > shows why it's good you wrote it up!) - that's a lot more local > changes than I've seen elsewhere, and that includes some very large > production user setups. So I hope we can work together to make your > experience closer to theirs. > > * Are there tests in your codebase that we could run in upstream > emscripten? > * Do you test latest emscripten releases on your codebase as they are > tagged? (Another option is to test the much more frequent tip-of-tree > builds, but you may see more noise there.) > * It would be good to discuss upstreaming long-term changes you > mention like that EMT_STACK_MAX one - we may not be able to in all > cases, but let's try at least (I don't think I remember a discussion > on that one). > > - Alon > > > > On Fri, Sep 6, 2019 at 12:32 AM Beuc <[email protected] > <mailto:[email protected]>> wrote: > > Hi, > > We almost never could use Emscripten without applying a few patches. > It's become increasingly difficult, not only to patch, but to > convince the developers that it's legitimate, so I thought I'd > detail our use case for good here. > > So we've been using Emscripten with patches in emscripten, llvm > and SDL2 port [1]. > By the time a patch is officially included in a release, we > usually need more patches as we face new issues -- or regressions. > Rinse and repeat. > > Consequently for our production use (the RenPyWeb build that we > publish to our users), we build an identified, versioned release > (e.g. 1.38.42) with specific fixes. > > > New difficulties: > - scarse documentation for recompiling "upstream" (patch in > progress #9383) > - no matching tags for llvm and bynarien, commit IDs quite hard to > track (discussion in #9317/#9383) > - misleading versioning in tagged commits (#9282) > - grab-from-git port system; EMCC_LOCAL_PORTS considered for > "local debugging", not cached, and causing issues (e.g. #9342 #9402) > > > Rebutals: > > - "99% users use emsdk binaries" > Well, the remaining 1% will provide 99% of external contributions, > take care of them :) > Also being able to test fixes before they finish the full > review&ci process allows spotting issues in advance. > > - "users will either use the unmodified binaries, or work on > master/tip-of-tree" > No, we use an identified, tagged Emscripten version, because it's > more stable than a random commit. > Also when we publish a hot fix, sometimes a couple months after > release, we need to reproduce the same build environment > > - "emsdk can build from source" > This build process doesn't allow patching, so it's only when one > doesn't trust the binaries and wants to rebuild them as-is AFAIU. > (To be honest, I'd like to be able to use the binaries, but > Emscripten being bleeding-edge, I figure that'll have to wait :)) > > > I hope this makes clear why we need to checkout > emscripten/llvm/bynarien for an existing release, patch it, and > build it. > Preferably without reverse-engineering the build process and > fighting all the stack. > That's the first time I have to justify this while working on > free/open source software. > > How do you people deal with this? > > > [1] For instance, our current release needs these patches, some of > them still needing work & official integration: > > - https://github.com/emscripten-core/emscripten/issues/9257 > Work-around ./configure misdetection > > - https://github.com/emscripten-core/emscripten/issues/9097 > Fix fullscreen exit causing wrong screen size > > - hard-coded change to EMT_STACK_MAX which isn't configurable > > - https://github.com/emscripten-core/emscripten-fastcomp/pull/195 > Compiler limitation when combining several Emscripten features > > - https://github.com/emscripten-ports/SDL2/issues/88 > Send SDL_APP_TERMINATING on page exit > > - https://github.com/emscripten-ports/SDL2/issues/70 > Support Emterpreter/Asincify in SDL2 > > - Search "Emscripten" in https://renpy.beuc.net/#status_reports > for all the other patches that were eventually integrated over the > year > -- You received this message because you are subscribed to the Google Groups "emscripten-discuss" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/emscripten-discuss/781c87ac-e4ac-b9bc-51ec-a46107470c0c%40beuc.net.
