Its the constants in fcntl.h / in wasi: __header_fcntl.h and api.h : ```
#define O_WASI_RDONLY (0x04000000) #define O_WASI_WRONLY (0x10000000) #define O_WASI_RDWR (O_WASI_RDONLY | O_WASI_WRONLY) typedef uint16_t __wasi_oflags_t; typedef uint16_t __wasi_fdflags_t; #define WASI_OFLAGS_CREAT ((__wasi_oflags_t)(1 << 0)) #define WASI_OFLAGS_EXCL ((__wasi_oflags_t)(1 << 2)) #define WASI_OFLAGS_TRUNC ((__wasi_oflags_t)(1 << 3)) #define WASI_FDFLAGS_SYNC ((__wasi_fdflags_t)(1 << 4)) #define O_WASI_CREAT (WASI_OFLAGS_CREAT << 12) #define O_WASI_EXCL (WASI_OFLAGS_EXCL << 12) #define O_WASI_TRUNC (WASI_OFLAGS_TRUNC << 12) #define O_WASI_SYNC WASI_FDFLAGS_SYNC #define O_WASI_CLOEXEC 0 ``` Remove the WASI_ from these and you see that in emscripten they are different. E.g. for O_RDONLY in fcntl.h under system/lib/libc/musl/include it is: ``` #define O_RDONLY 00 ``` But For WASI I "overwrite" it with 0x04000000. It's a hack (https://github.com/dotnet/runtimelab/pull/1850/files#diff-5d86cce7b76e1bece0aec26ee2bc4830503046428aecbbdefbf845832573020f), and I do that switch with a "special" options flag. Interesting that these constants are different between emscripten and WASI as both are musl. There must be some history there I suppose. On Thursday, February 10, 2022 at 12:34:43 PM UTC-5 alon...@gmail.com wrote: > I see... yeah, mixing and matching object files from different toolchains > is not guaranteed to work. They need C ABI compatibility, I'm afraid. > > It is a little sad to need separate runtimes, though, so I hope we can > find a way to avoid that for you. What is the specific code that would be > different between the two runtimes? > > On Thu, Feb 10, 2022 at 5:23 AM yowl yowlxx <scot...@gmail.com> wrote: > >> " (Of course, header constants *can* be an issue if you link wasm object >> files together that were built using different SDKs or even different >> versions of the same SDK.)" >> >> Yes, that is what I naively tried to do. The CoreCLR runtime is compiled >> using emscripten's SDK, but I wanted to link it with the WASI SDK libc++ to >> get file open support. What I'll probably have to do in the long run is >> have two CoreCLR runtimes, one for Emscripten and one for WASI SDK. >> >> On Mon, 7 Feb 2022 at 14:39, Alon Zakai <alon...@gmail.com> wrote: >> >>> Header constants like O_CREAT should not be a problem between wasm and >>> the host. WASI defines a very clear API between wasm and the runtime, and >>> those constants are not part of it. That is, "wasi-sdk host" is a confusing >>> way to put it: there are VMs with WASI support, but they are not tied to >>> the wasi-sdk toolchain, which is just one toolchain that emits WASI. >>> >>> (Of course, header constants *can* be an issue if you link wasm object >>> files together that were built using different SDKs or even different >>> versions of the same SDK.) >>> >>> Emscripten could add more complete WASI support, if that were useful - >>> it's just a matter of implementing the APIs. PRs would be welcome! >>> >>> >>> >>> On Sat, Feb 5, 2022 at 9:59 AM scot...@gmail.com <scot...@gmail.com> >>> wrote: >>> >>>> My own observations mixing Emscripten and WASI-SDK are that some >>>> constants, e.g. `O_CREAT` are different in Emscripten (0x64) to Wasi-SDK >>>> (0x10000000). Allowing emscripten to build something that would run under >>>> a wasi-sdk host, like wasmtime, is tricky - I think. >>>> >>>> On Friday, December 17, 2021 at 8:01:14 AM UTC-5 Floh wrote: >>>> >>>>> Ah alright, that toolchain file looks a lot simpler (it's under >>>>> "share/cmake" in the SDK download). I'll use that as base for my own >>>>> cmake >>>>> toolchain file instead. Thanks! >>>>> >>>>> On Thursday, 16 December 2021 at 21:18:46 UTC+1 s...@google.com wrote: >>>>> >>>>>> On Thu, Dec 16, 2021 at 5:53 AM Floh <flo...@gmail.com> wrote: >>>>>> >>>>>>> > Unless you plan to run those tools on the web then I think >>>>>>> wasi-sdk is most likely the way to go. How do you plan to run the >>>>>>> binaries BTW? >>>>>>> >>>>>>> Currently I'm using wasmtime. >>>>>>> >>>>>>> I was actually successful to build my shader compiler through >>>>>>> wasi-sdk, and run it through wasmtime, the file system access worked as >>>>>>> expected. >>>>>>> >>>>>>> Installing the wasi-sdk is just downloading and unpacking somewhere >>>>>>> (e.g. no env-variables or search path updating needed), and then I >>>>>>> created >>>>>>> a cmake toolchain file which looks a lot like my Emscripten toolchain >>>>>>> file, >>>>>>> but with all the Emscripten-specific parts removed: >>>>>>> >>>>>>> >>>>>>> https://github.com/floooh/fips/blob/master/cmake-toolchains/wasisdk.toolchain.cmake >>>>>>> >>>>>> >>>>>> FYI wasi-sdk includes a cmake toolchain fail upstream: >>>>>> https://github.com/WebAssembly/wasi-sdk/blob/main/wasi-sdk.cmake >>>>>> >>>>>> Maybe we forgot to include it in the download? But it should be >>>>>> usable I think. >>>>>> >>>>>> >>>>>>> The only problem (vs the Emscripten SDK) was that the wasi-sdk >>>>>>> doesn't come with pthread headers/stubs, so I had to tinker a bit with >>>>>>> the >>>>>>> glslangValidator source code (which for some reason needs thread-local >>>>>>> data >>>>>>> and mutexes, but then doesn't actually spawn any threads). >>>>>>> >>>>>>> The popen() calls (to run native Metal or D3D shader compiler >>>>>>> executables) simply seem to fail at runtime (which I'm already handling >>>>>>> in >>>>>>> my code). >>>>>>> >>>>>>> All in all pretty smooth sailing, at the moment it's just an >>>>>>> experiment though, but a promising one :) >>>>>>> >>>>>>> >>>>>>> On Tuesday, 14 December 2021 at 23:41:10 UTC+1 s...@google.com >>>>>>> wrote: >>>>>>> >>>>>>>> On Tue, Dec 14, 2021 at 9:28 AM Floh <flo...@gmail.com> wrote: >>>>>>>> >>>>>>>>> Oki doki, thanks for the clarification. I just wanted to check if >>>>>>>>> I'm missing something, using the wasi-sdk makes sense in that case. >>>>>>>>> >>>>>>>>> Maybe my use case helps a bit to prioritize "proper" WASI support >>>>>>>>> a bit :) >>>>>>>>> >>>>>>>>> I basically want to replace native command line tools (in my case: >>>>>>>>> a shader compiler built out of the the Khronos GLSL compiler, >>>>>>>>> SPIRVTools >>>>>>>>> and SPIRVCross) with a WASI version, because right now I need to >>>>>>>>> build this >>>>>>>>> tool in 4 variants (Windows x86_64, Linux x86_64, macOS x86_64 and >>>>>>>>> macOS >>>>>>>>> arm64) and then "distribute" the binaries through a git repository. >>>>>>>>> My plan >>>>>>>>> is to replace this with a single WASI binary (building on the target >>>>>>>>> machine is also not an option because these are complex C++ >>>>>>>>> dependencies >>>>>>>>> which can take up to 15 minutes to build). >>>>>>>>> >>>>>>>> >>>>>>>> Unless you plan to run those tools on the web then I think wasi-sdk >>>>>>>> is most likely the way to go. How do you plan to run the binaries >>>>>>>> BTW? >>>>>>>> >>>>>>>> >>>>>>>>> >>>>>>>>> One missing piece in the WASI API is popen() support though. The >>>>>>>>> shader compiler optionally needs to run the proprietaty D3D and Metal >>>>>>>>> shader compilers to generate shader binary blobs. Not sure yet how >>>>>>>>> I'll >>>>>>>>> tackle that eventually, but a WASI executable which just generates >>>>>>>>> shader >>>>>>>>> source code (not binary blobs) would be a good start nonetheless. >>>>>>>>> >>>>>>>>> Cheers! >>>>>>>>> -Floh. >>>>>>>>> >>>>>>>>> On Tuesday, 14 December 2021 at 17:43:10 UTC+1 s...@google.com >>>>>>>>> wrote: >>>>>>>>> >>>>>>>>>> The standalone/wasi support in emscripten is very basic and >>>>>>>>>> doesn't have full fileystem support yet. I would certainly >>>>>>>>>> recommend using wasi-sdk if you want to run something on wasmtime. >>>>>>>>>> >>>>>>>>>> If I ever get around to landing this PR then a lot more of the FS >>>>>>>>>> stuff might start working: >>>>>>>>>> https://github.com/emscripten-core/emscripten/pull/12704. But >>>>>>>>>> this has not been a priority recently. The interesting part for me >>>>>>>>>> would >>>>>>>>>> be that it might allow existing WASI applications to be run in the >>>>>>>>>> JS glue >>>>>>>>>> code. i.e. take a pre-built wasi module and run `emcc --post-link` >>>>>>>>>> to run >>>>>>>>>> on the web. >>>>>>>>>> >>>>>>>>>> On Tue, Dec 14, 2021 at 6:45 AM Floh <flo...@gmail.com> wrote: >>>>>>>>>> >>>>>>>>>>> I'm currently tinkering with Emscripten's WASI output and can't >>>>>>>>>>> get filesystem access to work. In short, everything compiles, but >>>>>>>>>>> then when >>>>>>>>>>> running via: >>>>>>>>>>> >>>>>>>>>>> wasmtime --dir . bla.wasm >>>>>>>>>>> >>>>>>>>>>> ...all filesystem operations fail. >>>>>>>>>>> >>>>>>>>>>> When compiling with the clang included in the wasi-sdk it works >>>>>>>>>>> as expected. Is this something that can be easily fixed or worked >>>>>>>>>>> around on >>>>>>>>>>> my side, or should I switch to the wasi-sdk instead? >>>>>>>>>>> >>>>>>>>>>> Thanks! >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> -- >>>>>>>>>>> You received this message because you are subscribed to the >>>>>>>>>>> Google Groups "emscripten-discuss" group. >>>>>>>>>>> To unsubscribe from this group and stop receiving emails from >>>>>>>>>>> it, send an email to emscripten-disc...@googlegroups.com. >>>>>>>>>>> To view this discussion on the web visit >>>>>>>>>>> https://groups.google.com/d/msgid/emscripten-discuss/527adec2-ee4d-44f3-a783-ce901632c30cn%40googlegroups.com >>>>>>>>>>> >>>>>>>>>>> <https://groups.google.com/d/msgid/emscripten-discuss/527adec2-ee4d-44f3-a783-ce901632c30cn%40googlegroups.com?utm_medium=email&utm_source=footer> >>>>>>>>>>> . >>>>>>>>>>> >>>>>>>>>> -- >>>>>>>>> You received this message because you are subscribed to the Google >>>>>>>>> Groups "emscripten-discuss" group. >>>>>>>>> To unsubscribe from this group and stop receiving emails from it, >>>>>>>>> send an email to emscripten-disc...@googlegroups.com. >>>>>>>>> >>>>>>>> To view this discussion on the web visit >>>>>>>>> https://groups.google.com/d/msgid/emscripten-discuss/17b87056-ad33-404a-850b-141028b20f43n%40googlegroups.com >>>>>>>>> >>>>>>>>> <https://groups.google.com/d/msgid/emscripten-discuss/17b87056-ad33-404a-850b-141028b20f43n%40googlegroups.com?utm_medium=email&utm_source=footer> >>>>>>>>> . >>>>>>>>> >>>>>>>> -- >>>>>>> You received this message because you are subscribed to the Google >>>>>>> Groups "emscripten-discuss" group. >>>>>>> To unsubscribe from this group and stop receiving emails from it, >>>>>>> send an email to emscripten-disc...@googlegroups.com. >>>>>>> >>>>>> To view this discussion on the web visit >>>>>>> https://groups.google.com/d/msgid/emscripten-discuss/484d2d4a-fd79-42a5-b9f7-90cc39b5cce8n%40googlegroups.com >>>>>>> >>>>>>> <https://groups.google.com/d/msgid/emscripten-discuss/484d2d4a-fd79-42a5-b9f7-90cc39b5cce8n%40googlegroups.com?utm_medium=email&utm_source=footer> >>>>>>> . >>>>>>> >>>>>> -- >>>> You received this message because you are subscribed to the Google >>>> Groups "emscripten-discuss" group. >>>> To unsubscribe from this group and stop receiving emails from it, send >>>> an email to emscripten-disc...@googlegroups.com. >>>> To view this discussion on the web visit >>>> https://groups.google.com/d/msgid/emscripten-discuss/67a4ca70-36e3-47a5-879c-f4ca6df0d282n%40googlegroups.com >>>> >>>> <https://groups.google.com/d/msgid/emscripten-discuss/67a4ca70-36e3-47a5-879c-f4ca6df0d282n%40googlegroups.com?utm_medium=email&utm_source=footer> >>>> . >>>> >>> -- >>> You received this message because you are subscribed to a topic in the >>> Google Groups "emscripten-discuss" group. >>> To unsubscribe from this topic, visit >>> https://groups.google.com/d/topic/emscripten-discuss/a97HaictQLo/unsubscribe >>> . >>> To unsubscribe from this group and all its topics, send an email to >>> emscripten-disc...@googlegroups.com. >>> To view this discussion on the web visit >>> https://groups.google.com/d/msgid/emscripten-discuss/CAEX4NpQ7uW5SZhsrqfi0HFptrsv87QnTzaOixr3L0J26kxJ0GA%40mail.gmail.com >>> >>> <https://groups.google.com/d/msgid/emscripten-discuss/CAEX4NpQ7uW5SZhsrqfi0HFptrsv87QnTzaOixr3L0J26kxJ0GA%40mail.gmail.com?utm_medium=email&utm_source=footer> >>> . >>> >> -- >> You received this message because you are subscribed to the Google Groups >> "emscripten-discuss" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to emscripten-disc...@googlegroups.com. >> > To view this discussion on the web visit >> https://groups.google.com/d/msgid/emscripten-discuss/CADJ3TLdKqmSEsWZ6_TY4Ef9semHv11ousfda_FbX9Jmiow5XJg%40mail.gmail.com >> >> <https://groups.google.com/d/msgid/emscripten-discuss/CADJ3TLdKqmSEsWZ6_TY4Ef9semHv11ousfda_FbX9Jmiow5XJg%40mail.gmail.com?utm_medium=email&utm_source=footer> >> . >> > -- You received this message because you are subscribed to the Google Groups "emscripten-discuss" group. To unsubscribe from this group and stop receiving emails from it, send an email to emscripten-discuss+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/emscripten-discuss/f5e559d7-4cca-41a7-b9d0-c6dd34ad7812n%40googlegroups.com.
