>From their zulipchat: >
Dan Gohman: The original reason is that cloudlibc did it that way, and the code comes from there. Dan Gohman: From a libc maintainer perspective, these values are implementation details. musl only has a single value because musl only supports Linux, where O_RDONLY always has the same value. But across other platforms, it can have different values. Dan Gohman: wasi-libc is not ABI-compatible with Emscripten in numerous ways, and no one has ever volunteered to do the work to make it be so. Dan Gohman: That said, from a practical perspective, some C / POSIX constants are sufficiently "well known" that we do make efforts to align their values with the well-known values. This is somewhat subjective, so whether O_RDONLY should qualify as having a well-known value would come down to how it's used in practice, how you're using it. In summary, while it would be easier from my perspective if the libcs where ABI compatible, its perhaps not a reasonable expectation, and the "right" thing would not to use libc for things that WASI supplies, but go straight to the WASI calls. On Tuesday, March 8, 2022 at 4:54:29 PM UTC-5 [email protected] wrote: > I did a little more digging and emscripten is the same as musl, and > wasi-libc is different. I will ask wasi-libc why they didn't use the > original constants as there is nothing in their git history about it - it's > always been different > > On Tuesday, March 1, 2022 at 11:32:09 AM UTC-5 [email protected] wrote: > >> For clarity, here is the linke to `O_RDONLY` for wasi: >> >> https://github.com/WebAssembly/wasi-libc/blob/ad5133410f66b93a2381db5b542aad5e0964db96/libc-bottom-half/headers/public/__header_fcntl.h#L20 >> >> On Saturday, February 26, 2022 at 5:21:11 PM UTC-5 [email protected] >> wrote: >> >>> Its the constants in fcntl.h / in wasi: __header_fcntl.h and api.h : >>> >>> ``` >>> >>> #define O_WASI_RDONLY (0x04000000) >>> #define O_WASI_WRONLY (0x10000000) >>> #define O_WASI_RDWR (O_WASI_RDONLY | O_WASI_WRONLY) >>> >>> >>> typedef uint16_t __wasi_oflags_t; >>> typedef uint16_t __wasi_fdflags_t; >>> >>> >>> #define WASI_OFLAGS_CREAT ((__wasi_oflags_t)(1 << 0)) >>> #define WASI_OFLAGS_EXCL ((__wasi_oflags_t)(1 << 2)) >>> #define WASI_OFLAGS_TRUNC ((__wasi_oflags_t)(1 << 3)) >>> #define WASI_FDFLAGS_SYNC ((__wasi_fdflags_t)(1 << 4)) >>> >>> >>> #define O_WASI_CREAT (WASI_OFLAGS_CREAT << 12) >>> #define O_WASI_EXCL (WASI_OFLAGS_EXCL << 12) >>> #define O_WASI_TRUNC (WASI_OFLAGS_TRUNC << 12) >>> #define O_WASI_SYNC WASI_FDFLAGS_SYNC #define O_WASI_CLOEXEC 0 >>> ``` >>> Remove the WASI_ from these and you see that in emscripten they are >>> different. E.g. for O_RDONLY in fcntl.h under system/lib/libc/musl/include >>> it is: >>> ``` >>> #define O_RDONLY 00 >>> ``` >>> But For WASI I "overwrite" it with 0x04000000. It's a hack ( >>> https://github.com/dotnet/runtimelab/pull/1850/files#diff-5d86cce7b76e1bece0aec26ee2bc4830503046428aecbbdefbf845832573020f), >>> >>> and I do that switch with a "special" options flag. >>> >>> Interesting that these constants are different between emscripten and >>> WASI as both are musl. There must be some history there I suppose. >>> On Thursday, February 10, 2022 at 12:34:43 PM UTC-5 [email protected] >>> wrote: >>> >>>> I see... yeah, mixing and matching object files from different >>>> toolchains is not guaranteed to work. They need C ABI compatibility, I'm >>>> afraid. >>>> >>>> It is a little sad to need separate runtimes, though, so I hope we can >>>> find a way to avoid that for you. What is the specific code that would be >>>> different between the two runtimes? >>>> >>>> On Thu, Feb 10, 2022 at 5:23 AM yowl yowlxx <[email protected]> wrote: >>>> >>>>> " (Of course, header constants *can* be an issue if you link wasm >>>>> object files together that were built using different SDKs or even >>>>> different versions of the same SDK.)" >>>>> >>>>> Yes, that is what I naively tried to do. The CoreCLR runtime is >>>>> compiled using emscripten's SDK, but I wanted to link it with the WASI >>>>> SDK >>>>> libc++ to get file open support. What I'll probably have to do in the >>>>> long >>>>> run is have two CoreCLR runtimes, one for Emscripten and one for WASI SDK. >>>>> >>>>> On Mon, 7 Feb 2022 at 14:39, Alon Zakai <[email protected]> wrote: >>>>> >>>>>> Header constants like O_CREAT should not be a problem between wasm >>>>>> and the host. WASI defines a very clear API between wasm and the >>>>>> runtime, >>>>>> and those constants are not part of it. That is, "wasi-sdk host" is a >>>>>> confusing way to put it: there are VMs with WASI support, but they are >>>>>> not >>>>>> tied to the wasi-sdk toolchain, which is just one toolchain that emits >>>>>> WASI. >>>>>> >>>>>> (Of course, header constants *can* be an issue if you link wasm >>>>>> object files together that were built using different SDKs or even >>>>>> different versions of the same SDK.) >>>>>> >>>>>> Emscripten could add more complete WASI support, if that were useful >>>>>> - it's just a matter of implementing the APIs. PRs would be welcome! >>>>>> >>>>>> >>>>>> >>>>>> On Sat, Feb 5, 2022 at 9:59 AM [email protected] <[email protected]> >>>>>> wrote: >>>>>> >>>>>>> My own observations mixing Emscripten and WASI-SDK are that some >>>>>>> constants, e.g. `O_CREAT` are different in Emscripten (0x64) to >>>>>>> Wasi-SDK >>>>>>> (0x10000000). Allowing emscripten to build something that would run >>>>>>> under >>>>>>> a wasi-sdk host, like wasmtime, is tricky - I think. >>>>>>> >>>>>>> On Friday, December 17, 2021 at 8:01:14 AM UTC-5 Floh wrote: >>>>>>> >>>>>>>> Ah alright, that toolchain file looks a lot simpler (it's under >>>>>>>> "share/cmake" in the SDK download). I'll use that as base for my own >>>>>>>> cmake >>>>>>>> toolchain file instead. Thanks! >>>>>>>> >>>>>>>> On Thursday, 16 December 2021 at 21:18:46 UTC+1 [email protected] >>>>>>>> wrote: >>>>>>>> >>>>>>>>> On Thu, Dec 16, 2021 at 5:53 AM Floh <[email protected]> wrote: >>>>>>>>> >>>>>>>>>> > Unless you plan to run those tools on the web then I think >>>>>>>>>> wasi-sdk is most likely the way to go. How do you plan to run the >>>>>>>>>> binaries BTW? >>>>>>>>>> >>>>>>>>>> Currently I'm using wasmtime. >>>>>>>>>> >>>>>>>>>> I was actually successful to build my shader compiler through >>>>>>>>>> wasi-sdk, and run it through wasmtime, the file system access worked >>>>>>>>>> as >>>>>>>>>> expected. >>>>>>>>>> >>>>>>>>>> Installing the wasi-sdk is just downloading and unpacking >>>>>>>>>> somewhere (e.g. no env-variables or search path updating needed), >>>>>>>>>> and then >>>>>>>>>> I created a cmake toolchain file which looks a lot like my >>>>>>>>>> Emscripten >>>>>>>>>> toolchain file, but with all the Emscripten-specific parts removed: >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> https://github.com/floooh/fips/blob/master/cmake-toolchains/wasisdk.toolchain.cmake >>>>>>>>>> >>>>>>>>> >>>>>>>>> FYI wasi-sdk includes a cmake toolchain fail upstream: >>>>>>>>> https://github.com/WebAssembly/wasi-sdk/blob/main/wasi-sdk.cmake >>>>>>>>> >>>>>>>>> Maybe we forgot to include it in the download? But it should be >>>>>>>>> usable I think. >>>>>>>>> >>>>>>>>> >>>>>>>>>> The only problem (vs the Emscripten SDK) was that the wasi-sdk >>>>>>>>>> doesn't come with pthread headers/stubs, so I had to tinker a bit >>>>>>>>>> with the >>>>>>>>>> glslangValidator source code (which for some reason needs >>>>>>>>>> thread-local data >>>>>>>>>> and mutexes, but then doesn't actually spawn any threads). >>>>>>>>>> >>>>>>>>>> The popen() calls (to run native Metal or D3D shader compiler >>>>>>>>>> executables) simply seem to fail at runtime (which I'm already >>>>>>>>>> handling in >>>>>>>>>> my code). >>>>>>>>>> >>>>>>>>>> All in all pretty smooth sailing, at the moment it's just an >>>>>>>>>> experiment though, but a promising one :) >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> On Tuesday, 14 December 2021 at 23:41:10 UTC+1 [email protected] >>>>>>>>>> wrote: >>>>>>>>>> >>>>>>>>>>> On Tue, Dec 14, 2021 at 9:28 AM Floh <[email protected]> wrote: >>>>>>>>>>> >>>>>>>>>>>> Oki doki, thanks for the clarification. I just wanted to check >>>>>>>>>>>> if I'm missing something, using the wasi-sdk makes sense in that >>>>>>>>>>>> case. >>>>>>>>>>>> >>>>>>>>>>>> Maybe my use case helps a bit to prioritize "proper" WASI >>>>>>>>>>>> support a bit :) >>>>>>>>>>>> >>>>>>>>>>>> I basically want to replace native command line tools (in my >>>>>>>>>>>> case: a shader compiler built out of the the Khronos GLSL >>>>>>>>>>>> compiler, >>>>>>>>>>>> SPIRVTools and SPIRVCross) with a WASI version, because right now >>>>>>>>>>>> I need to >>>>>>>>>>>> build this tool in 4 variants (Windows x86_64, Linux x86_64, macOS >>>>>>>>>>>> x86_64 >>>>>>>>>>>> and macOS arm64) and then "distribute" the binaries through a git >>>>>>>>>>>> repository. My plan is to replace this with a single WASI binary >>>>>>>>>>>> (building >>>>>>>>>>>> on the target machine is also not an option because these are >>>>>>>>>>>> complex C++ >>>>>>>>>>>> dependencies which can take up to 15 minutes to build). >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> Unless you plan to run those tools on the web then I think >>>>>>>>>>> wasi-sdk is most likely the way to go. How do you plan to run the >>>>>>>>>>> binaries BTW? >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> One missing piece in the WASI API is popen() support though. >>>>>>>>>>>> The shader compiler optionally needs to run the proprietaty D3D >>>>>>>>>>>> and Metal >>>>>>>>>>>> shader compilers to generate shader binary blobs. Not sure yet how >>>>>>>>>>>> I'll >>>>>>>>>>>> tackle that eventually, but a WASI executable which just generates >>>>>>>>>>>> shader >>>>>>>>>>>> source code (not binary blobs) would be a good start nonetheless. >>>>>>>>>>>> >>>>>>>>>>>> Cheers! >>>>>>>>>>>> -Floh. >>>>>>>>>>>> >>>>>>>>>>>> On Tuesday, 14 December 2021 at 17:43:10 UTC+1 [email protected] >>>>>>>>>>>> wrote: >>>>>>>>>>>> >>>>>>>>>>>>> The standalone/wasi support in emscripten is very basic and >>>>>>>>>>>>> doesn't have full fileystem support yet. I would certainly >>>>>>>>>>>>> recommend using wasi-sdk if you want to run something on wasmtime. >>>>>>>>>>>>> >>>>>>>>>>>>> If I ever get around to landing this PR then a lot more of the >>>>>>>>>>>>> FS stuff might start working: >>>>>>>>>>>>> https://github.com/emscripten-core/emscripten/pull/12704. >>>>>>>>>>>>> But this has not been a priority recently. The interesting >>>>>>>>>>>>> part for me >>>>>>>>>>>>> would be that it might allow existing WASI applications to be run >>>>>>>>>>>>> in the JS >>>>>>>>>>>>> glue code. i.e. take a pre-built wasi module and run `emcc >>>>>>>>>>>>> --post-link` to >>>>>>>>>>>>> run on the web. >>>>>>>>>>>>> >>>>>>>>>>>>> On Tue, Dec 14, 2021 at 6:45 AM Floh <[email protected]> wrote: >>>>>>>>>>>>> >>>>>>>>>>>>>> I'm currently tinkering with Emscripten's WASI output and >>>>>>>>>>>>>> can't get filesystem access to work. In short, everything >>>>>>>>>>>>>> compiles, but >>>>>>>>>>>>>> then when running via: >>>>>>>>>>>>>> >>>>>>>>>>>>>> wasmtime --dir . bla.wasm >>>>>>>>>>>>>> >>>>>>>>>>>>>> ...all filesystem operations fail. >>>>>>>>>>>>>> >>>>>>>>>>>>>> When compiling with the clang included in the wasi-sdk it >>>>>>>>>>>>>> works as expected. Is this something that can be easily fixed or >>>>>>>>>>>>>> worked >>>>>>>>>>>>>> around on my side, or should I switch to the wasi-sdk instead? >>>>>>>>>>>>>> >>>>>>>>>>>>>> Thanks! >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> -- >>>>>>>>>>>>>> You received this message because you are subscribed to the >>>>>>>>>>>>>> Google Groups "emscripten-discuss" group. >>>>>>>>>>>>>> To unsubscribe from this group and stop receiving emails from >>>>>>>>>>>>>> it, send an email to [email protected]. >>>>>>>>>>>>>> To view this discussion on the web visit >>>>>>>>>>>>>> https://groups.google.com/d/msgid/emscripten-discuss/527adec2-ee4d-44f3-a783-ce901632c30cn%40googlegroups.com >>>>>>>>>>>>>> >>>>>>>>>>>>>> <https://groups.google.com/d/msgid/emscripten-discuss/527adec2-ee4d-44f3-a783-ce901632c30cn%40googlegroups.com?utm_medium=email&utm_source=footer> >>>>>>>>>>>>>> . >>>>>>>>>>>>>> >>>>>>>>>>>>> -- >>>>>>>>>>>> You received this message because you are subscribed to the >>>>>>>>>>>> Google Groups "emscripten-discuss" group. >>>>>>>>>>>> To unsubscribe from this group and stop receiving emails from >>>>>>>>>>>> it, send an email to [email protected]. >>>>>>>>>>>> >>>>>>>>>>> To view this discussion on the web visit >>>>>>>>>>>> https://groups.google.com/d/msgid/emscripten-discuss/17b87056-ad33-404a-850b-141028b20f43n%40googlegroups.com >>>>>>>>>>>> >>>>>>>>>>>> <https://groups.google.com/d/msgid/emscripten-discuss/17b87056-ad33-404a-850b-141028b20f43n%40googlegroups.com?utm_medium=email&utm_source=footer> >>>>>>>>>>>> . >>>>>>>>>>>> >>>>>>>>>>> -- >>>>>>>>>> You received this message because you are subscribed to the >>>>>>>>>> Google Groups "emscripten-discuss" group. >>>>>>>>>> To unsubscribe from this group and stop receiving emails from it, >>>>>>>>>> send an email to [email protected]. >>>>>>>>>> >>>>>>>>> To view this discussion on the web visit >>>>>>>>>> https://groups.google.com/d/msgid/emscripten-discuss/484d2d4a-fd79-42a5-b9f7-90cc39b5cce8n%40googlegroups.com >>>>>>>>>> >>>>>>>>>> <https://groups.google.com/d/msgid/emscripten-discuss/484d2d4a-fd79-42a5-b9f7-90cc39b5cce8n%40googlegroups.com?utm_medium=email&utm_source=footer> >>>>>>>>>> . >>>>>>>>>> >>>>>>>>> -- >>>>>>> You received this message because you are subscribed to the Google >>>>>>> Groups "emscripten-discuss" group. >>>>>>> To unsubscribe from this group and stop receiving emails from it, >>>>>>> send an email to [email protected]. >>>>>>> To view this discussion on the web visit >>>>>>> https://groups.google.com/d/msgid/emscripten-discuss/67a4ca70-36e3-47a5-879c-f4ca6df0d282n%40googlegroups.com >>>>>>> >>>>>>> <https://groups.google.com/d/msgid/emscripten-discuss/67a4ca70-36e3-47a5-879c-f4ca6df0d282n%40googlegroups.com?utm_medium=email&utm_source=footer> >>>>>>> . >>>>>>> >>>>>> -- >>>>>> You received this message because you are subscribed to a topic in >>>>>> the Google Groups "emscripten-discuss" group. >>>>>> To unsubscribe from this topic, visit >>>>>> https://groups.google.com/d/topic/emscripten-discuss/a97HaictQLo/unsubscribe >>>>>> . >>>>>> To unsubscribe from this group and all its topics, send an email to >>>>>> [email protected]. >>>>>> To view this discussion on the web visit >>>>>> https://groups.google.com/d/msgid/emscripten-discuss/CAEX4NpQ7uW5SZhsrqfi0HFptrsv87QnTzaOixr3L0J26kxJ0GA%40mail.gmail.com >>>>>> >>>>>> <https://groups.google.com/d/msgid/emscripten-discuss/CAEX4NpQ7uW5SZhsrqfi0HFptrsv87QnTzaOixr3L0J26kxJ0GA%40mail.gmail.com?utm_medium=email&utm_source=footer> >>>>>> . >>>>>> >>>>> -- >>>>> You received this message because you are subscribed to the Google >>>>> Groups "emscripten-discuss" group. >>>>> To unsubscribe from this group and stop receiving emails from it, send >>>>> an email to [email protected]. >>>>> >>>> To view this discussion on the web visit >>>>> https://groups.google.com/d/msgid/emscripten-discuss/CADJ3TLdKqmSEsWZ6_TY4Ef9semHv11ousfda_FbX9Jmiow5XJg%40mail.gmail.com >>>>> >>>>> <https://groups.google.com/d/msgid/emscripten-discuss/CADJ3TLdKqmSEsWZ6_TY4Ef9semHv11ousfda_FbX9Jmiow5XJg%40mail.gmail.com?utm_medium=email&utm_source=footer> >>>>> . >>>>> >>>> -- You received this message because you are subscribed to the Google Groups "emscripten-discuss" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/emscripten-discuss/fa678432-0dc2-4495-8aa0-604a26d184a4n%40googlegroups.com.
