For clarity, here is the linke to `O_RDONLY` for wasi: https://github.com/WebAssembly/wasi-libc/blob/ad5133410f66b93a2381db5b542aad5e0964db96/libc-bottom-half/headers/public/__header_fcntl.h#L20
On Saturday, February 26, 2022 at 5:21:11 PM UTC-5 [email protected] wrote: > Its the constants in fcntl.h / in wasi: __header_fcntl.h and api.h : > > ``` > > #define O_WASI_RDONLY (0x04000000) > #define O_WASI_WRONLY (0x10000000) > #define O_WASI_RDWR (O_WASI_RDONLY | O_WASI_WRONLY) > > > typedef uint16_t __wasi_oflags_t; > typedef uint16_t __wasi_fdflags_t; > > > #define WASI_OFLAGS_CREAT ((__wasi_oflags_t)(1 << 0)) > #define WASI_OFLAGS_EXCL ((__wasi_oflags_t)(1 << 2)) > #define WASI_OFLAGS_TRUNC ((__wasi_oflags_t)(1 << 3)) > #define WASI_FDFLAGS_SYNC ((__wasi_fdflags_t)(1 << 4)) > > > #define O_WASI_CREAT (WASI_OFLAGS_CREAT << 12) > #define O_WASI_EXCL (WASI_OFLAGS_EXCL << 12) > #define O_WASI_TRUNC (WASI_OFLAGS_TRUNC << 12) > #define O_WASI_SYNC WASI_FDFLAGS_SYNC #define O_WASI_CLOEXEC 0 > ``` > Remove the WASI_ from these and you see that in emscripten they are > different. E.g. for O_RDONLY in fcntl.h under system/lib/libc/musl/include > it is: > ``` > #define O_RDONLY 00 > ``` > But For WASI I "overwrite" it with 0x04000000. It's a hack ( > https://github.com/dotnet/runtimelab/pull/1850/files#diff-5d86cce7b76e1bece0aec26ee2bc4830503046428aecbbdefbf845832573020f), > > and I do that switch with a "special" options flag. > > Interesting that these constants are different between emscripten and WASI > as both are musl. There must be some history there I suppose. > On Thursday, February 10, 2022 at 12:34:43 PM UTC-5 [email protected] > wrote: > >> I see... yeah, mixing and matching object files from different toolchains >> is not guaranteed to work. They need C ABI compatibility, I'm afraid. >> >> It is a little sad to need separate runtimes, though, so I hope we can >> find a way to avoid that for you. What is the specific code that would be >> different between the two runtimes? >> >> On Thu, Feb 10, 2022 at 5:23 AM yowl yowlxx <[email protected]> wrote: >> >>> " (Of course, header constants *can* be an issue if you link wasm object >>> files together that were built using different SDKs or even different >>> versions of the same SDK.)" >>> >>> Yes, that is what I naively tried to do. The CoreCLR runtime is >>> compiled using emscripten's SDK, but I wanted to link it with the WASI SDK >>> libc++ to get file open support. What I'll probably have to do in the long >>> run is have two CoreCLR runtimes, one for Emscripten and one for WASI SDK. >>> >>> On Mon, 7 Feb 2022 at 14:39, Alon Zakai <[email protected]> wrote: >>> >>>> Header constants like O_CREAT should not be a problem between wasm and >>>> the host. WASI defines a very clear API between wasm and the runtime, and >>>> those constants are not part of it. That is, "wasi-sdk host" is a >>>> confusing >>>> way to put it: there are VMs with WASI support, but they are not tied to >>>> the wasi-sdk toolchain, which is just one toolchain that emits WASI. >>>> >>>> (Of course, header constants *can* be an issue if you link wasm object >>>> files together that were built using different SDKs or even different >>>> versions of the same SDK.) >>>> >>>> Emscripten could add more complete WASI support, if that were useful - >>>> it's just a matter of implementing the APIs. PRs would be welcome! >>>> >>>> >>>> >>>> On Sat, Feb 5, 2022 at 9:59 AM [email protected] <[email protected]> >>>> wrote: >>>> >>>>> My own observations mixing Emscripten and WASI-SDK are that some >>>>> constants, e.g. `O_CREAT` are different in Emscripten (0x64) to Wasi-SDK >>>>> (0x10000000). Allowing emscripten to build something that would run >>>>> under >>>>> a wasi-sdk host, like wasmtime, is tricky - I think. >>>>> >>>>> On Friday, December 17, 2021 at 8:01:14 AM UTC-5 Floh wrote: >>>>> >>>>>> Ah alright, that toolchain file looks a lot simpler (it's under >>>>>> "share/cmake" in the SDK download). I'll use that as base for my own >>>>>> cmake >>>>>> toolchain file instead. Thanks! >>>>>> >>>>>> On Thursday, 16 December 2021 at 21:18:46 UTC+1 [email protected] >>>>>> wrote: >>>>>> >>>>>>> On Thu, Dec 16, 2021 at 5:53 AM Floh <[email protected]> wrote: >>>>>>> >>>>>>>> > Unless you plan to run those tools on the web then I think >>>>>>>> wasi-sdk is most likely the way to go. How do you plan to run the >>>>>>>> binaries BTW? >>>>>>>> >>>>>>>> Currently I'm using wasmtime. >>>>>>>> >>>>>>>> I was actually successful to build my shader compiler through >>>>>>>> wasi-sdk, and run it through wasmtime, the file system access worked >>>>>>>> as >>>>>>>> expected. >>>>>>>> >>>>>>>> Installing the wasi-sdk is just downloading and unpacking somewhere >>>>>>>> (e.g. no env-variables or search path updating needed), and then I >>>>>>>> created >>>>>>>> a cmake toolchain file which looks a lot like my Emscripten toolchain >>>>>>>> file, >>>>>>>> but with all the Emscripten-specific parts removed: >>>>>>>> >>>>>>>> >>>>>>>> https://github.com/floooh/fips/blob/master/cmake-toolchains/wasisdk.toolchain.cmake >>>>>>>> >>>>>>> >>>>>>> FYI wasi-sdk includes a cmake toolchain fail upstream: >>>>>>> https://github.com/WebAssembly/wasi-sdk/blob/main/wasi-sdk.cmake >>>>>>> >>>>>>> Maybe we forgot to include it in the download? But it should be >>>>>>> usable I think. >>>>>>> >>>>>>> >>>>>>>> The only problem (vs the Emscripten SDK) was that the wasi-sdk >>>>>>>> doesn't come with pthread headers/stubs, so I had to tinker a bit with >>>>>>>> the >>>>>>>> glslangValidator source code (which for some reason needs thread-local >>>>>>>> data >>>>>>>> and mutexes, but then doesn't actually spawn any threads). >>>>>>>> >>>>>>>> The popen() calls (to run native Metal or D3D shader compiler >>>>>>>> executables) simply seem to fail at runtime (which I'm already >>>>>>>> handling in >>>>>>>> my code). >>>>>>>> >>>>>>>> All in all pretty smooth sailing, at the moment it's just an >>>>>>>> experiment though, but a promising one :) >>>>>>>> >>>>>>>> >>>>>>>> On Tuesday, 14 December 2021 at 23:41:10 UTC+1 [email protected] >>>>>>>> wrote: >>>>>>>> >>>>>>>>> On Tue, Dec 14, 2021 at 9:28 AM Floh <[email protected]> wrote: >>>>>>>>> >>>>>>>>>> Oki doki, thanks for the clarification. I just wanted to check if >>>>>>>>>> I'm missing something, using the wasi-sdk makes sense in that case. >>>>>>>>>> >>>>>>>>>> Maybe my use case helps a bit to prioritize "proper" WASI support >>>>>>>>>> a bit :) >>>>>>>>>> >>>>>>>>>> I basically want to replace native command line tools (in my >>>>>>>>>> case: a shader compiler built out of the the Khronos GLSL compiler, >>>>>>>>>> SPIRVTools and SPIRVCross) with a WASI version, because right now I >>>>>>>>>> need to >>>>>>>>>> build this tool in 4 variants (Windows x86_64, Linux x86_64, macOS >>>>>>>>>> x86_64 >>>>>>>>>> and macOS arm64) and then "distribute" the binaries through a git >>>>>>>>>> repository. My plan is to replace this with a single WASI binary >>>>>>>>>> (building >>>>>>>>>> on the target machine is also not an option because these are >>>>>>>>>> complex C++ >>>>>>>>>> dependencies which can take up to 15 minutes to build). >>>>>>>>>> >>>>>>>>> >>>>>>>>> Unless you plan to run those tools on the web then I think >>>>>>>>> wasi-sdk is most likely the way to go. How do you plan to run the >>>>>>>>> binaries BTW? >>>>>>>>> >>>>>>>>> >>>>>>>>>> >>>>>>>>>> One missing piece in the WASI API is popen() support though. The >>>>>>>>>> shader compiler optionally needs to run the proprietaty D3D and >>>>>>>>>> Metal >>>>>>>>>> shader compilers to generate shader binary blobs. Not sure yet how >>>>>>>>>> I'll >>>>>>>>>> tackle that eventually, but a WASI executable which just generates >>>>>>>>>> shader >>>>>>>>>> source code (not binary blobs) would be a good start nonetheless. >>>>>>>>>> >>>>>>>>>> Cheers! >>>>>>>>>> -Floh. >>>>>>>>>> >>>>>>>>>> On Tuesday, 14 December 2021 at 17:43:10 UTC+1 [email protected] >>>>>>>>>> wrote: >>>>>>>>>> >>>>>>>>>>> The standalone/wasi support in emscripten is very basic and >>>>>>>>>>> doesn't have full fileystem support yet. I would certainly >>>>>>>>>>> recommend using wasi-sdk if you want to run something on wasmtime. >>>>>>>>>>> >>>>>>>>>>> If I ever get around to landing this PR then a lot more of the >>>>>>>>>>> FS stuff might start working: >>>>>>>>>>> https://github.com/emscripten-core/emscripten/pull/12704. But >>>>>>>>>>> this has not been a priority recently. The interesting part for >>>>>>>>>>> me would >>>>>>>>>>> be that it might allow existing WASI applications to be run in the >>>>>>>>>>> JS glue >>>>>>>>>>> code. i.e. take a pre-built wasi module and run `emcc --post-link` >>>>>>>>>>> to run >>>>>>>>>>> on the web. >>>>>>>>>>> >>>>>>>>>>> On Tue, Dec 14, 2021 at 6:45 AM Floh <[email protected]> wrote: >>>>>>>>>>> >>>>>>>>>>>> I'm currently tinkering with Emscripten's WASI output and can't >>>>>>>>>>>> get filesystem access to work. In short, everything compiles, but >>>>>>>>>>>> then when >>>>>>>>>>>> running via: >>>>>>>>>>>> >>>>>>>>>>>> wasmtime --dir . bla.wasm >>>>>>>>>>>> >>>>>>>>>>>> ...all filesystem operations fail. >>>>>>>>>>>> >>>>>>>>>>>> When compiling with the clang included in the wasi-sdk it works >>>>>>>>>>>> as expected. Is this something that can be easily fixed or worked >>>>>>>>>>>> around on >>>>>>>>>>>> my side, or should I switch to the wasi-sdk instead? >>>>>>>>>>>> >>>>>>>>>>>> Thanks! >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> -- >>>>>>>>>>>> You received this message because you are subscribed to the >>>>>>>>>>>> Google Groups "emscripten-discuss" group. >>>>>>>>>>>> To unsubscribe from this group and stop receiving emails from >>>>>>>>>>>> it, send an email to [email protected]. >>>>>>>>>>>> To view this discussion on the web visit >>>>>>>>>>>> https://groups.google.com/d/msgid/emscripten-discuss/527adec2-ee4d-44f3-a783-ce901632c30cn%40googlegroups.com >>>>>>>>>>>> >>>>>>>>>>>> <https://groups.google.com/d/msgid/emscripten-discuss/527adec2-ee4d-44f3-a783-ce901632c30cn%40googlegroups.com?utm_medium=email&utm_source=footer> >>>>>>>>>>>> . >>>>>>>>>>>> >>>>>>>>>>> -- >>>>>>>>>> You received this message because you are subscribed to the >>>>>>>>>> Google Groups "emscripten-discuss" group. >>>>>>>>>> To unsubscribe from this group and stop receiving emails from it, >>>>>>>>>> send an email to [email protected]. >>>>>>>>>> >>>>>>>>> To view this discussion on the web visit >>>>>>>>>> https://groups.google.com/d/msgid/emscripten-discuss/17b87056-ad33-404a-850b-141028b20f43n%40googlegroups.com >>>>>>>>>> >>>>>>>>>> <https://groups.google.com/d/msgid/emscripten-discuss/17b87056-ad33-404a-850b-141028b20f43n%40googlegroups.com?utm_medium=email&utm_source=footer> >>>>>>>>>> . >>>>>>>>>> >>>>>>>>> -- >>>>>>>> You received this message because you are subscribed to the Google >>>>>>>> Groups "emscripten-discuss" group. >>>>>>>> To unsubscribe from this group and stop receiving emails from it, >>>>>>>> send an email to [email protected]. >>>>>>>> >>>>>>> To view this discussion on the web visit >>>>>>>> https://groups.google.com/d/msgid/emscripten-discuss/484d2d4a-fd79-42a5-b9f7-90cc39b5cce8n%40googlegroups.com >>>>>>>> >>>>>>>> <https://groups.google.com/d/msgid/emscripten-discuss/484d2d4a-fd79-42a5-b9f7-90cc39b5cce8n%40googlegroups.com?utm_medium=email&utm_source=footer> >>>>>>>> . >>>>>>>> >>>>>>> -- >>>>> You received this message because you are subscribed to the Google >>>>> Groups "emscripten-discuss" group. >>>>> To unsubscribe from this group and stop receiving emails from it, send >>>>> an email to [email protected]. >>>>> To view this discussion on the web visit >>>>> https://groups.google.com/d/msgid/emscripten-discuss/67a4ca70-36e3-47a5-879c-f4ca6df0d282n%40googlegroups.com >>>>> >>>>> <https://groups.google.com/d/msgid/emscripten-discuss/67a4ca70-36e3-47a5-879c-f4ca6df0d282n%40googlegroups.com?utm_medium=email&utm_source=footer> >>>>> . >>>>> >>>> -- >>>> You received this message because you are subscribed to a topic in the >>>> Google Groups "emscripten-discuss" group. >>>> To unsubscribe from this topic, visit >>>> https://groups.google.com/d/topic/emscripten-discuss/a97HaictQLo/unsubscribe >>>> . >>>> To unsubscribe from this group and all its topics, send an email to >>>> [email protected]. >>>> To view this discussion on the web visit >>>> https://groups.google.com/d/msgid/emscripten-discuss/CAEX4NpQ7uW5SZhsrqfi0HFptrsv87QnTzaOixr3L0J26kxJ0GA%40mail.gmail.com >>>> >>>> <https://groups.google.com/d/msgid/emscripten-discuss/CAEX4NpQ7uW5SZhsrqfi0HFptrsv87QnTzaOixr3L0J26kxJ0GA%40mail.gmail.com?utm_medium=email&utm_source=footer> >>>> . >>>> >>> -- >>> You received this message because you are subscribed to the Google >>> Groups "emscripten-discuss" group. >>> To unsubscribe from this group and stop receiving emails from it, send >>> an email to [email protected]. >>> >> To view this discussion on the web visit >>> https://groups.google.com/d/msgid/emscripten-discuss/CADJ3TLdKqmSEsWZ6_TY4Ef9semHv11ousfda_FbX9Jmiow5XJg%40mail.gmail.com >>> >>> <https://groups.google.com/d/msgid/emscripten-discuss/CADJ3TLdKqmSEsWZ6_TY4Ef9semHv11ousfda_FbX9Jmiow5XJg%40mail.gmail.com?utm_medium=email&utm_source=footer> >>> . >>> >> -- You received this message because you are subscribed to the Google Groups "emscripten-discuss" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/emscripten-discuss/e5f691cf-5c54-42a3-9140-0e81f9f3a1bdn%40googlegroups.com.
