That's an extremely neat idea, and dividing into smaller files should help the compilation to run to completion in limited memory scenarios like virtual machines and low-end laptops. I think this would also be required to make use of multiple CPU cores in the C compiler; running a 6-core machine on only one core is going to be much slower than it should be and will result in a lot of wasted time for the user.
It would also delay other projects' CI runs that install a fresh emsdk, wouldn't it? By delaying work that could be done once at build time to happen thousands of times, once at every installation? As a developer who uses a variety of computers to work, I personally would appreciate the performance improvements and the energy savings of making one build per release per platform and distributing the appropriate ones for direct use with no additional compilation stage. -- brion On Thu, Jul 9, 2020, 1:14 PM Alon Zakai <[email protected]> wrote: > Yeah, that's an issue with wasm2c. It takes a few minutes to build > (optimized) wasm-opt, and clang would be much worse, and maybe fail as you > said. > > I don't think anyone's looked into this, but ideas include: > > * Get wasm2c to emit separate files, each containing one or more > functions. Then you just need a Makefile or such and can even build in > parallel. > > * Write a little python script that splits up the file automatically. The > format is pretty simple so that's easy. And the python could also run > gcc/clang in a process pool for you. > > - Alon > > > > On Thu, Jul 9, 2020 at 1:01 PM 'Thomas Lively' via emscripten-discuss < > [email protected]> wrote: > >> How does the wasm2c build work with something as large as LLVM? I would >> assume that at some point the C file would get so large that the C compiler >> would fall over. >> >> On Thu, Jul 9, 2020 at 12:58 PM Alon Zakai <[email protected]> wrote: >> >>> At the risk of sounding like a broken record - I've been talking about >>> the next idea a lot recently ;) - I think this is something wasm2c can help >>> with. I just gave a talk about how: >>> >>> http://kripken.github.io/talks/2020/universal.html#/ >>> >>> The idea is that we add a single additional build target, "universal C", >>> which compiles LLVM and Binaryen to portable C code that can then be >>> compiled on practically any platform. So this new "build" would be C, and >>> people would download those C files and run a simple command to build them >>> locally. After the local build you end up with a normal executable that >>> just works. >>> >>> There is still that build step locally, but it's the simplest build >>> possible - no build system is needed, no special local setup, no cmake or >>> configure, just run gcc or clang on a self-contained C file. That should be >>> trivial on ARM64 MacOS or Linux as I believe they have a system C compiler >>> installed by default. >>> >>> I'm not opposed to adding "proper" ARM64 builds, though - ARM64 builds >>> would have some benefits over wasm2c. But wasm2c builds do cover the long >>> tail of less-common platforms, and we can probably set them up much quicker >>> too. I hope to do this for Binaryen soon anyhow. >>> >>> - Alon >>> >>> On Thu, Jul 9, 2020 at 11:30 AM Brion Vibber <[email protected]> wrote: >>> >>>> I've been a bit of an ARM64 enthusiast of late, trying out Linux, >>>> Windows 10, and iOS devices that run on the ARM64/Aarch64 family of >>>> processors. Emscripten works fine on these machines if one cares to do some >>>> light development work on them, but since there's no binaries built from >>>> CI, the standard emsdk can only install by building from source -- which >>>> can take hours on a middleweight portable machine. >>>> >>>> Now that Apple is switching their Mac product line to ARM64 processors >>>> over the next two years, it will likely become much more common next year >>>> for people to have ARM64-based laptop and desktop computers, and some of >>>> them will need to build something with emscripten in their workflows either >>>> on macOS or on a virtualized Linux in Docker etc. >>>> >>>> From what I've seen presented at WWDC, the ARM64 Macs will support >>>> emulated processes, so it may work to ship the x86_64 binaries with the >>>> caveat that they will run much slower than native builds. >>>> >>>> Virtualized Linux builds would also need native ARM64 binaries to run, >>>> or else they'd have to sit there for a couple hours compiling after every >>>> upgrade. >>>> >>>> And of course there are already Windows 10 and Linux computers >>>> available with ARM64 processes, on sale since a couple years ago and used >>>> in the wild in modest numbers. >>>> >>>> I get the impression that the biggest roadblock to explicit ARM64 >>>> support in emscripten is getting it into the CI infrastructure: >>>> * Linux/ARM64 builds and testing? >>>> * macOS/ARM64 builds and testing? >>>> * Windows/ARM64 builds and testing? >>>> >>>> It's too soon to start on macOS since it's in beta, dev kits aren't >>>> shipped yet, and there's no obvious way yet to figure out how to run tests >>>> on a macOS ARM system in CI. :) >>>> >>>> And I'm less sure how important Win/ARM64 is, given you can use the >>>> Linux/ARM64 version in WSL virtualization. But some folks prefer to develop >>>> on native Windows, too. >>>> >>>> If there's any way we can start talking about Linux/ARM64 builds and >>>> testing, I would be very happy about it! I would even kick in a few bucks >>>> for a VM or something if that would help any. ;) >>>> >>>> Thanks for your time and your consideration! >>>> >>>> -- brion vibber (brion @ pobox.com / brion @ wikimedia.org) >>>> Wikimedia Foundation >>>> >>>> -- >>>> You received this message because you are subscribed to the Google >>>> Groups "emscripten-discuss" group. >>>> To unsubscribe from this group and stop receiving emails from it, send >>>> an email to [email protected]. >>>> To view this discussion on the web visit >>>> https://groups.google.com/d/msgid/emscripten-discuss/CAFnWYT%3D3OLskq1KBtVrgKsq1PTS7eHDxyzpcBpmE8_Bfk7m6Kw%40mail.gmail.com >>>> <https://groups.google.com/d/msgid/emscripten-discuss/CAFnWYT%3D3OLskq1KBtVrgKsq1PTS7eHDxyzpcBpmE8_Bfk7m6Kw%40mail.gmail.com?utm_medium=email&utm_source=footer> >>>> . >>>> >>> -- >>> You received this message because you are subscribed to the Google >>> Groups "emscripten-discuss" group. >>> To unsubscribe from this group and stop receiving emails from it, send >>> an email to [email protected]. >>> To view this discussion on the web visit >>> https://groups.google.com/d/msgid/emscripten-discuss/CAEX4NpQoi1LZiw2dPjeMMy2zSf-Z8WFWjpP8K1C-gEx67O5jwg%40mail.gmail.com >>> <https://groups.google.com/d/msgid/emscripten-discuss/CAEX4NpQoi1LZiw2dPjeMMy2zSf-Z8WFWjpP8K1C-gEx67O5jwg%40mail.gmail.com?utm_medium=email&utm_source=footer> >>> . >>> >> -- >> You received this message because you are subscribed to the Google Groups >> "emscripten-discuss" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to [email protected]. >> To view this discussion on the web visit >> https://groups.google.com/d/msgid/emscripten-discuss/CAJZD_EVjH_yPoWO3YC3NdjQOPx9xJbBh-kfbiB6xiVXcK378Mw%40mail.gmail.com >> <https://groups.google.com/d/msgid/emscripten-discuss/CAJZD_EVjH_yPoWO3YC3NdjQOPx9xJbBh-kfbiB6xiVXcK378Mw%40mail.gmail.com?utm_medium=email&utm_source=footer> >> . >> > -- > You received this message because you are subscribed to the Google Groups > "emscripten-discuss" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > To view this discussion on the web visit > https://groups.google.com/d/msgid/emscripten-discuss/CAEX4NpQwE%2BVA682EdL0ozCeQc-S%2B5Ko%3DnQ0NVvMgfz1Y%2Bgf9aw%40mail.gmail.com > <https://groups.google.com/d/msgid/emscripten-discuss/CAEX4NpQwE%2BVA682EdL0ozCeQc-S%2B5Ko%3DnQ0NVvMgfz1Y%2Bgf9aw%40mail.gmail.com?utm_medium=email&utm_source=footer> > . > -- You received this message because you are subscribed to the Google Groups "emscripten-discuss" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/emscripten-discuss/CAFnWYTnh%2BqW8_fM5CBGa5oPrqtQS%3DyX9TaK7oWPTNUTwBcZmyA%40mail.gmail.com.
