I did a little more digging and emscripten is the same as musl, and wasi-libc is different. I will ask wasi-libc why they didn't use the original constants as there is nothing in their git history about it - it's always been different
On Tuesday, March 1, 2022 at 11:32:09 AM UTC-5 [email protected] wrote: > For clarity, here is the linke to `O_RDONLY` for wasi: > > https://github.com/WebAssembly/wasi-libc/blob/ad5133410f66b93a2381db5b542aad5e0964db96/libc-bottom-half/headers/public/__header_fcntl.h#L20 > > On Saturday, February 26, 2022 at 5:21:11 PM UTC-5 [email protected] > wrote: > >> Its the constants in fcntl.h / in wasi: __header_fcntl.h and api.h : >> >> ``` >> >> #define O_WASI_RDONLY (0x04000000) >> #define O_WASI_WRONLY (0x10000000) >> #define O_WASI_RDWR (O_WASI_RDONLY | O_WASI_WRONLY) >> >> >> typedef uint16_t __wasi_oflags_t; >> typedef uint16_t __wasi_fdflags_t; >> >> >> #define WASI_OFLAGS_CREAT ((__wasi_oflags_t)(1 << 0)) >> #define WASI_OFLAGS_EXCL ((__wasi_oflags_t)(1 << 2)) >> #define WASI_OFLAGS_TRUNC ((__wasi_oflags_t)(1 << 3)) >> #define WASI_FDFLAGS_SYNC ((__wasi_fdflags_t)(1 << 4)) >> >> >> #define O_WASI_CREAT (WASI_OFLAGS_CREAT << 12) >> #define O_WASI_EXCL (WASI_OFLAGS_EXCL << 12) >> #define O_WASI_TRUNC (WASI_OFLAGS_TRUNC << 12) >> #define O_WASI_SYNC WASI_FDFLAGS_SYNC #define O_WASI_CLOEXEC 0 >> ``` >> Remove the WASI_ from these and you see that in emscripten they are >> different. E.g. for O_RDONLY in fcntl.h under system/lib/libc/musl/include >> it is: >> ``` >> #define O_RDONLY 00 >> ``` >> But For WASI I "overwrite" it with 0x04000000. It's a hack ( >> https://github.com/dotnet/runtimelab/pull/1850/files#diff-5d86cce7b76e1bece0aec26ee2bc4830503046428aecbbdefbf845832573020f), >> >> and I do that switch with a "special" options flag. >> >> Interesting that these constants are different between emscripten and >> WASI as both are musl. There must be some history there I suppose. >> On Thursday, February 10, 2022 at 12:34:43 PM UTC-5 [email protected] >> wrote: >> >>> I see... yeah, mixing and matching object files from different >>> toolchains is not guaranteed to work. They need C ABI compatibility, I'm >>> afraid. >>> >>> It is a little sad to need separate runtimes, though, so I hope we can >>> find a way to avoid that for you. What is the specific code that would be >>> different between the two runtimes? >>> >>> On Thu, Feb 10, 2022 at 5:23 AM yowl yowlxx <[email protected]> wrote: >>> >>>> " (Of course, header constants *can* be an issue if you link wasm >>>> object files together that were built using different SDKs or even >>>> different versions of the same SDK.)" >>>> >>>> Yes, that is what I naively tried to do. The CoreCLR runtime is >>>> compiled using emscripten's SDK, but I wanted to link it with the WASI SDK >>>> libc++ to get file open support. What I'll probably have to do in the >>>> long >>>> run is have two CoreCLR runtimes, one for Emscripten and one for WASI SDK. >>>> >>>> On Mon, 7 Feb 2022 at 14:39, Alon Zakai <[email protected]> wrote: >>>> >>>>> Header constants like O_CREAT should not be a problem between wasm and >>>>> the host. WASI defines a very clear API between wasm and the runtime, and >>>>> those constants are not part of it. That is, "wasi-sdk host" is a >>>>> confusing >>>>> way to put it: there are VMs with WASI support, but they are not tied to >>>>> the wasi-sdk toolchain, which is just one toolchain that emits WASI. >>>>> >>>>> (Of course, header constants *can* be an issue if you link wasm object >>>>> files together that were built using different SDKs or even different >>>>> versions of the same SDK.) >>>>> >>>>> Emscripten could add more complete WASI support, if that were useful - >>>>> it's just a matter of implementing the APIs. PRs would be welcome! >>>>> >>>>> >>>>> >>>>> On Sat, Feb 5, 2022 at 9:59 AM [email protected] <[email protected]> >>>>> wrote: >>>>> >>>>>> My own observations mixing Emscripten and WASI-SDK are that some >>>>>> constants, e.g. `O_CREAT` are different in Emscripten (0x64) to Wasi-SDK >>>>>> (0x10000000). Allowing emscripten to build something that would run >>>>>> under >>>>>> a wasi-sdk host, like wasmtime, is tricky - I think. >>>>>> >>>>>> On Friday, December 17, 2021 at 8:01:14 AM UTC-5 Floh wrote: >>>>>> >>>>>>> Ah alright, that toolchain file looks a lot simpler (it's under >>>>>>> "share/cmake" in the SDK download). I'll use that as base for my own >>>>>>> cmake >>>>>>> toolchain file instead. Thanks! >>>>>>> >>>>>>> On Thursday, 16 December 2021 at 21:18:46 UTC+1 [email protected] >>>>>>> wrote: >>>>>>> >>>>>>>> On Thu, Dec 16, 2021 at 5:53 AM Floh <[email protected]> wrote: >>>>>>>> >>>>>>>>> > Unless you plan to run those tools on the web then I think >>>>>>>>> wasi-sdk is most likely the way to go. How do you plan to run the >>>>>>>>> binaries BTW? >>>>>>>>> >>>>>>>>> Currently I'm using wasmtime. >>>>>>>>> >>>>>>>>> I was actually successful to build my shader compiler through >>>>>>>>> wasi-sdk, and run it through wasmtime, the file system access worked >>>>>>>>> as >>>>>>>>> expected. >>>>>>>>> >>>>>>>>> Installing the wasi-sdk is just downloading and unpacking >>>>>>>>> somewhere (e.g. no env-variables or search path updating needed), and >>>>>>>>> then >>>>>>>>> I created a cmake toolchain file which looks a lot like my Emscripten >>>>>>>>> toolchain file, but with all the Emscripten-specific parts removed: >>>>>>>>> >>>>>>>>> >>>>>>>>> https://github.com/floooh/fips/blob/master/cmake-toolchains/wasisdk.toolchain.cmake >>>>>>>>> >>>>>>>> >>>>>>>> FYI wasi-sdk includes a cmake toolchain fail upstream: >>>>>>>> https://github.com/WebAssembly/wasi-sdk/blob/main/wasi-sdk.cmake >>>>>>>> >>>>>>>> Maybe we forgot to include it in the download? But it should be >>>>>>>> usable I think. >>>>>>>> >>>>>>>> >>>>>>>>> The only problem (vs the Emscripten SDK) was that the wasi-sdk >>>>>>>>> doesn't come with pthread headers/stubs, so I had to tinker a bit >>>>>>>>> with the >>>>>>>>> glslangValidator source code (which for some reason needs >>>>>>>>> thread-local data >>>>>>>>> and mutexes, but then doesn't actually spawn any threads). >>>>>>>>> >>>>>>>>> The popen() calls (to run native Metal or D3D shader compiler >>>>>>>>> executables) simply seem to fail at runtime (which I'm already >>>>>>>>> handling in >>>>>>>>> my code). >>>>>>>>> >>>>>>>>> All in all pretty smooth sailing, at the moment it's just an >>>>>>>>> experiment though, but a promising one :) >>>>>>>>> >>>>>>>>> >>>>>>>>> On Tuesday, 14 December 2021 at 23:41:10 UTC+1 [email protected] >>>>>>>>> wrote: >>>>>>>>> >>>>>>>>>> On Tue, Dec 14, 2021 at 9:28 AM Floh <[email protected]> wrote: >>>>>>>>>> >>>>>>>>>>> Oki doki, thanks for the clarification. I just wanted to check >>>>>>>>>>> if I'm missing something, using the wasi-sdk makes sense in that >>>>>>>>>>> case. >>>>>>>>>>> >>>>>>>>>>> Maybe my use case helps a bit to prioritize "proper" WASI >>>>>>>>>>> support a bit :) >>>>>>>>>>> >>>>>>>>>>> I basically want to replace native command line tools (in my >>>>>>>>>>> case: a shader compiler built out of the the Khronos GLSL compiler, >>>>>>>>>>> SPIRVTools and SPIRVCross) with a WASI version, because right now I >>>>>>>>>>> need to >>>>>>>>>>> build this tool in 4 variants (Windows x86_64, Linux x86_64, macOS >>>>>>>>>>> x86_64 >>>>>>>>>>> and macOS arm64) and then "distribute" the binaries through a git >>>>>>>>>>> repository. My plan is to replace this with a single WASI binary >>>>>>>>>>> (building >>>>>>>>>>> on the target machine is also not an option because these are >>>>>>>>>>> complex C++ >>>>>>>>>>> dependencies which can take up to 15 minutes to build). >>>>>>>>>>> >>>>>>>>>> >>>>>>>>>> Unless you plan to run those tools on the web then I think >>>>>>>>>> wasi-sdk is most likely the way to go. How do you plan to run the >>>>>>>>>> binaries BTW? >>>>>>>>>> >>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> One missing piece in the WASI API is popen() support though. The >>>>>>>>>>> shader compiler optionally needs to run the proprietaty D3D and >>>>>>>>>>> Metal >>>>>>>>>>> shader compilers to generate shader binary blobs. Not sure yet how >>>>>>>>>>> I'll >>>>>>>>>>> tackle that eventually, but a WASI executable which just generates >>>>>>>>>>> shader >>>>>>>>>>> source code (not binary blobs) would be a good start nonetheless. >>>>>>>>>>> >>>>>>>>>>> Cheers! >>>>>>>>>>> -Floh. >>>>>>>>>>> >>>>>>>>>>> On Tuesday, 14 December 2021 at 17:43:10 UTC+1 [email protected] >>>>>>>>>>> wrote: >>>>>>>>>>> >>>>>>>>>>>> The standalone/wasi support in emscripten is very basic and >>>>>>>>>>>> doesn't have full fileystem support yet. I would certainly >>>>>>>>>>>> recommend using wasi-sdk if you want to run something on wasmtime. >>>>>>>>>>>> >>>>>>>>>>>> If I ever get around to landing this PR then a lot more of the >>>>>>>>>>>> FS stuff might start working: >>>>>>>>>>>> https://github.com/emscripten-core/emscripten/pull/12704. >>>>>>>>>>>> But this has not been a priority recently. The interesting part >>>>>>>>>>>> for me >>>>>>>>>>>> would be that it might allow existing WASI applications to be run >>>>>>>>>>>> in the JS >>>>>>>>>>>> glue code. i.e. take a pre-built wasi module and run `emcc >>>>>>>>>>>> --post-link` to >>>>>>>>>>>> run on the web. >>>>>>>>>>>> >>>>>>>>>>>> On Tue, Dec 14, 2021 at 6:45 AM Floh <[email protected]> wrote: >>>>>>>>>>>> >>>>>>>>>>>>> I'm currently tinkering with Emscripten's WASI output and >>>>>>>>>>>>> can't get filesystem access to work. In short, everything >>>>>>>>>>>>> compiles, but >>>>>>>>>>>>> then when running via: >>>>>>>>>>>>> >>>>>>>>>>>>> wasmtime --dir . bla.wasm >>>>>>>>>>>>> >>>>>>>>>>>>> ...all filesystem operations fail. >>>>>>>>>>>>> >>>>>>>>>>>>> When compiling with the clang included in the wasi-sdk it >>>>>>>>>>>>> works as expected. Is this something that can be easily fixed or >>>>>>>>>>>>> worked >>>>>>>>>>>>> around on my side, or should I switch to the wasi-sdk instead? >>>>>>>>>>>>> >>>>>>>>>>>>> Thanks! >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> -- >>>>>>>>>>>>> You received this message because you are subscribed to the >>>>>>>>>>>>> Google Groups "emscripten-discuss" group. >>>>>>>>>>>>> To unsubscribe from this group and stop receiving emails from >>>>>>>>>>>>> it, send an email to [email protected]. >>>>>>>>>>>>> To view this discussion on the web visit >>>>>>>>>>>>> https://groups.google.com/d/msgid/emscripten-discuss/527adec2-ee4d-44f3-a783-ce901632c30cn%40googlegroups.com >>>>>>>>>>>>> >>>>>>>>>>>>> <https://groups.google.com/d/msgid/emscripten-discuss/527adec2-ee4d-44f3-a783-ce901632c30cn%40googlegroups.com?utm_medium=email&utm_source=footer> >>>>>>>>>>>>> . >>>>>>>>>>>>> >>>>>>>>>>>> -- >>>>>>>>>>> You received this message because you are subscribed to the >>>>>>>>>>> Google Groups "emscripten-discuss" group. >>>>>>>>>>> To unsubscribe from this group and stop receiving emails from >>>>>>>>>>> it, send an email to [email protected]. >>>>>>>>>>> >>>>>>>>>> To view this discussion on the web visit >>>>>>>>>>> https://groups.google.com/d/msgid/emscripten-discuss/17b87056-ad33-404a-850b-141028b20f43n%40googlegroups.com >>>>>>>>>>> >>>>>>>>>>> <https://groups.google.com/d/msgid/emscripten-discuss/17b87056-ad33-404a-850b-141028b20f43n%40googlegroups.com?utm_medium=email&utm_source=footer> >>>>>>>>>>> . >>>>>>>>>>> >>>>>>>>>> -- >>>>>>>>> You received this message because you are subscribed to the Google >>>>>>>>> Groups "emscripten-discuss" group. >>>>>>>>> To unsubscribe from this group and stop receiving emails from it, >>>>>>>>> send an email to [email protected]. >>>>>>>>> >>>>>>>> To view this discussion on the web visit >>>>>>>>> https://groups.google.com/d/msgid/emscripten-discuss/484d2d4a-fd79-42a5-b9f7-90cc39b5cce8n%40googlegroups.com >>>>>>>>> >>>>>>>>> <https://groups.google.com/d/msgid/emscripten-discuss/484d2d4a-fd79-42a5-b9f7-90cc39b5cce8n%40googlegroups.com?utm_medium=email&utm_source=footer> >>>>>>>>> . >>>>>>>>> >>>>>>>> -- >>>>>> You received this message because you are subscribed to the Google >>>>>> Groups "emscripten-discuss" group. >>>>>> To unsubscribe from this group and stop receiving emails from it, >>>>>> send an email to [email protected]. >>>>>> To view this discussion on the web visit >>>>>> https://groups.google.com/d/msgid/emscripten-discuss/67a4ca70-36e3-47a5-879c-f4ca6df0d282n%40googlegroups.com >>>>>> >>>>>> <https://groups.google.com/d/msgid/emscripten-discuss/67a4ca70-36e3-47a5-879c-f4ca6df0d282n%40googlegroups.com?utm_medium=email&utm_source=footer> >>>>>> . >>>>>> >>>>> -- >>>>> You received this message because you are subscribed to a topic in the >>>>> Google Groups "emscripten-discuss" group. >>>>> To unsubscribe from this topic, visit >>>>> https://groups.google.com/d/topic/emscripten-discuss/a97HaictQLo/unsubscribe >>>>> . >>>>> To unsubscribe from this group and all its topics, send an email to >>>>> [email protected]. >>>>> To view this discussion on the web visit >>>>> https://groups.google.com/d/msgid/emscripten-discuss/CAEX4NpQ7uW5SZhsrqfi0HFptrsv87QnTzaOixr3L0J26kxJ0GA%40mail.gmail.com >>>>> >>>>> <https://groups.google.com/d/msgid/emscripten-discuss/CAEX4NpQ7uW5SZhsrqfi0HFptrsv87QnTzaOixr3L0J26kxJ0GA%40mail.gmail.com?utm_medium=email&utm_source=footer> >>>>> . >>>>> >>>> -- >>>> You received this message because you are subscribed to the Google >>>> Groups "emscripten-discuss" group. >>>> To unsubscribe from this group and stop receiving emails from it, send >>>> an email to [email protected]. >>>> >>> To view this discussion on the web visit >>>> https://groups.google.com/d/msgid/emscripten-discuss/CADJ3TLdKqmSEsWZ6_TY4Ef9semHv11ousfda_FbX9Jmiow5XJg%40mail.gmail.com >>>> >>>> <https://groups.google.com/d/msgid/emscripten-discuss/CADJ3TLdKqmSEsWZ6_TY4Ef9semHv11ousfda_FbX9Jmiow5XJg%40mail.gmail.com?utm_medium=email&utm_source=footer> >>>> . >>>> >>> -- You received this message because you are subscribed to the Google Groups "emscripten-discuss" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/emscripten-discuss/9c8bb35a-d8a7-4473-bc54-6b458a49b1bfn%40googlegroups.com.
