Sorry to be slow replying, things have been busy.
> Did you run this from the emsdk directory?
No. I don't read Python well and didn't realise it was necessary. Silly of
me.
Can you add `print(os.stat('upstream/emscripten/cache')` too?
Sure, and the missing ')'.
$ echo $EMSDK
/local/kernel_webasm/tools/emsdk
$ pushd $EMSDK
/local/kernel_webasm/tools/emsdk ~/regimes/tools_webasm/patssy/lx86
$ cat ~/regimes/tools_webasm/pytest.sh
python3 -c "import os; print(os.stat('upstream/emscripten/cache'))"
python3 -c "import os; print(os.access('upstream/emscripten/cache',
os.W_OK))"
$ source ~/regimes/tools_webasm/pytest.sh
os.stat_result(st_mode=16893, st_ino=101453189, st_dev=64774, st_nlink=4,
st_uid=5790, st_gid=5200, st_size=4096, st_atime=1730862972,
st_mtime=1730455667, st_ctime=1730455667)
True
Thanks very much,
John
On Fri, Nov 1, 2024 at 5:49 PM 'Sam Clegg' via emscripten-discuss <
emscripten-discuss@googlegroups.com> wrote:
>
>
> On Fri, Nov 1, 2024 at 3:18 AM John Dallman <jgdatsiem...@gmail.com>
> wrote:
>
>> > Are you able to touch a new file in `upstream/emscripten/cache`?
>>
>> Yes:
>>
>> $ echo $EMSDK
>> /local/kernel_webasm/tools/emsdk
>> $ echo $EMCC_CACHE
>> <blank line>
>> $ touch
>> /local/kernel_webasm/tools/emsdk/upstream/emscripten/cache/test.jgd
>> $ ls -l
>> /local/kernel_webasm/tools/emsdk/upstream/emscripten/cache/test.jgd
>> -rw-rw-r-- 1 jgd UK-Parasolid-GG 0 Nov 1 10:07
>> /local/kernel_webasm/tools/emsdk/upstream/emscripten/cache/test.jgd
>>
>> > What does python show when you run: python3 -c "import os;
>> print(os.access('upstream/emscripten/cache', os.W_OK))"
>>
>> $> cat ../../pytest.sh
>> python3 -c "import os; print(os.access('upstream/emscripten/cache',
>> os.W_OK))"
>> $ source ../../pytest.sh
>>
>
> Did you run this from the emsdk directory?
>
> Can you add `print(os.stat('upstream/emscripten/cache')` too?
>
>
>>
> False
>>
>> Is my Python too old? It's the one that came with Rocky Linux 8.10.
>>
>> $ python3 --version
>> Python 3.6.8
>>
>> Thanks very much,
>>
>> John
>>
>> On Thu, Oct 31, 2024 at 6:43 PM 'Sam Clegg' via emscripten-discuss <
>> emscripten-discuss@googlegroups.com> wrote:
>>
>>> Interesting. It seems like `os.access(path, os.W_OK)` must be returning
>>> `False` in our python code for `upstream/emscripten/cache`.
>>>
>>> Are you able to touch a new file in `upstream/emscripten/cache`?
>>>
>>> What does python show when you run: python3 -c "import os;
>>> print(os.access('upstream/emscripten/cache', os.W_OK))"
>>>
>>> On Thu, Oct 31, 2024 at 7:56 AM John Dallman <jgdatsiem...@gmail.com>
>>> wrote:
>>>
>>>> > Can you try running `emcc` with `EMCC_DEBUG=1` as well
>>>> as `EMCC_CACHE` set?
>>>>
>>>> Done.
>>>>
>>>> > Do you see the ` 'Using home-directory for emscripten cache due to
>>>> read-only root` message?
>>>>
>>>> Yes, I do. I have:
>>>>
>>>> $ touch $EMCC_CACHE/test.jgd
>>>> $ ls -l $EMCC_CACHE
>>>> total 0
>>>> -rw-rw-r-- 1 jgd UK-Parasolid-GG 0 Oct 31 14:42 test.jgd
>>>> $ ls -ld $EMCC_CACHE
>>>> drwxrwxr-x 2 kerman UK-Parasolid-GG 4096 Oct 31 14:42
>>>> /local/kernel_webasm/tools/emcache/
>>>> $ python3 --version
>>>> Python 3.6.8
>>>>
>>>> > Where is your emscripten config coming from? Are you using `source
>>>> emsdk_env.sh`?
>>>>
>>>> Yes, I'm sourcing that script. I have not altered any config files.
>>>>
>>>> > Making `emsdk/upstream/emscripten/cache` group writable is the
>>>> correct solution, there
>>>> > should be no need to serate of alternate cache directories in this
>>>> case I think.
>>>>
>>>> Checking that:
>>>>
>>>> $ whoami
>>>> kerman
>>>> $ pwd
>>>> /local/kernel_webasm/tools/emsdk
>>>> $ ls -ld upstream/emscripten/cache
>>>> drwxrwxr-x 4 kerman UK-Parasolid-GG 4096 Oct 24 23:29
>>>> upstream/emscripten/cache
>>>> $ ls -l upstream/emscripten/cache
>>>> total 12
>>>> drwxr-xr-x 2 kerman UK-Parasolid-GG 4096 Oct 24 23:29 build
>>>> -rwxr-xr-x 1 kerman UK-Parasolid-GG 0 Oct 24 23:29 cache.lock
>>>> drwxr-xr-x 5 kerman UK-Parasolid-GG 4096 Oct 24 23:18 sysroot
>>>> -rw-r--r-- 1 kerman UK-Parasolid-GG 1 Oct 24 23:18
>>>> sysroot_install.stamp
>>>>
>>>> Those files were extracted with those timestamps: Oct 24 is well before
>>>> I installed this Emscripten, and I don't work that late at night. Is the
>>>> presence of that cache.lock file the problem?
>>>>
>>>> Thanks,
>>>>
>>>> John
>>>>
>>>>
>>>> On Wed, Oct 30, 2024 at 6:33 PM 'Sam Clegg' via emscripten-discuss <
>>>> emscripten-discuss@googlegroups.com> wrote:
>>>>
>>>>>
>>>>>
>>>>> On Wed, Oct 30, 2024 at 5:46 AM John Dallman <jgdatsiem...@gmail.com>
>>>>> wrote:
>>>>>
>>>>>> This doesn't seem to be working. I'm using two accounts: "kerman" is
>>>>>> my system management account, which can sudo, and "jgd" is my personal
>>>>>> account. Both are members of the same primary group.
>>>>>>
>>>>>> I installed 3.1.70, the latest as of today, into
>>>>>> /local/kernel_webasm/tools, and made emsdk/upstream/emscripten/cache
>>>>>> group-writable. I then did a compile as jgd and found
>>>>>> ~jgd/.emscripten_cache was populated and used. I removed that and thought
>>>>>> again.
>>>>>>
>>>>>> I tried making the emsdk directory group-writable, and tried again.
>>>>>> ~jgd/.emscripten_cache was populated and used. I removed it again.
>>>>>>
>>>>>> I created a cache directory, /local/kernel_webasm/tools/emcache/,
>>>>>> made sure it was group-writable, and in my jgd session, did:
>>>>>>
>>>>>> EMCC_CACHE=/local/kernel_webasm/tools/emcache
>>>>>> export EMCC_CACHE
>>>>>>
>>>>>> I tried compiling again, and once again, ~jgd/.emscripten_cache was
>>>>>> populated and used.
>>>>>>
>>>>>
>>>>> Something strange must be going on here. Can you try running `emcc`
>>>>> with `EMCC_DEBUG=1` as well as `EMCC_CACHE` set? Do you see the ` 'Using
>>>>> home-directory for emscripten cache due to read-only root` message?
>>>>>
>>>>> The `CACHE = os.path.expanduser(os.path.join('~',
>>>>> '.emscripten_cache'))` should not even execute when you have an explicit
>>>>> cache configured.
>>>>>
>>>>> Where is your emscripten config coming from? Are you using `source
>>>>> emsdk_env.sh`?
>>>>>
>>>>> Making `emsdk/upstream/emscripten/cache` group writable is the correct
>>>>> solution, there should be no need to serate of alternate cache directories
>>>>> in this case I think.
>>>>>
>>>>> cheers,
>>>>> sam
>>>>>
>>>>> P.S. I am about to land a change that completely removes the fallback
>>>>> to `$HOME/.emscripten_cache`:
>>>>> https://github.com/emscripten-core/emscripten/pull/22801
>>>>>
>>>>>
>>>>>
>>>>>> I'm running Rocky Linux 8.10. I really do want to have many accounts
>>>>>> capable of using the same Emscripten installation, because putting it on
>>>>>> a
>>>>>> network drive makes standardizing development tools very much easier.
>>>>>>
>>>>>> Thanks,
>>>>>>
>>>>>> John
>>>>>>
>>>>>> On Tue, Oct 29, 2024 at 9:28 PM 'Sam Clegg' via emscripten-discuss <
>>>>>> emscripten-discuss@googlegroups.com> wrote:
>>>>>>
>>>>>>> The emscripten cache actually defaults to living inside of the
>>>>>>> emscripten directory. The line where this occurs is `CACHE =
>>>>>>> path_from_root('cache')`. Then `os.path.expanduser(os.path.join('~',
>>>>>>> '.emscripten_cache'))` location is only used when the emscripten
>>>>>>> directory
>>>>>>> is read only.
>>>>>>>
>>>>>>> When using emsdk the cache location should always be
>>>>>>> `emsdk/upstream/emscripten/cache`. emsdk also shipped with a
>>>>>>> pre-populated cache so most system libraries are already built there.
>>>>>>>
>>>>>>> For those who want to use a different cache location you can use the
>>>>>>> `EMCC_CACHE` environment variable or the `CACHE` key in your emscripten
>>>>>>> config file (both of these override the default). However, it doesn't
>>>>>>> sounds like you need to do either of those things and the default
>>>>>>> location
>>>>>>> inside the emsdk tree should work for you. Side note: emcc will also
>>>>>>> using the $HOME location if the in-tree location is not writable. You
>>>>>>> can
>>>>>>> set `EMCC_FROZEN_CACHE` if you want to accept this read-only location
>>>>>>> (obviously new libraries cannot be placed there in that case though).
>>>>>>> See
>>>>>>> https://github.com/emscripten-core/emscripten/blob/31f9fb3c71b1aacf5edf65e35515d41b59c10391/tools/config.py#L82-L92
>>>>>>> .
>>>>>>>
>>>>>>> cheers,
>>>>>>> sam
>>>>>>>
>>>>>>> On Mon, Oct 28, 2024 at 10:25 AM John Dallman <
>>>>>>> jgdatsiem...@gmail.com> wrote:
>>>>>>>
>>>>>>>> Greetings, all!
>>>>>>>>
>>>>>>>> I have a somewhat unusual first project with Emscripten. I need to
>>>>>>>> get a domain-specific C-like language generating WebAssembly. This is
>>>>>>>> not
>>>>>>>> too bad, because the DSL compiles into C code, which I then feed to
>>>>>>>> the C
>>>>>>>> compiler for the relevant platform. At this level, WebAssembly is "just
>>>>>>>> another platform" but as always, the details are more complicated.
>>>>>>>>
>>>>>>>> I need to provide declarations for the platform's C run-time
>>>>>>>> library functions, types, constants, and so on to the DSL. This is a
>>>>>>>> fairly
>>>>>>>> routine task, given the platform's C headers, but there are a few
>>>>>>>> things
>>>>>>>> about the Emscripten headers that are puzzling me.
>>>>>>>>
>>>>>>>> To forestall the obvious question, no, I can't just use the
>>>>>>>> provided headers. The DSL is C-like, but has some syntax differences.
>>>>>>>> Unlike C++, many C programs are not valid programs in the DSL, which
>>>>>>>> gives
>>>>>>>> much more freedom in the language design. It's a separate development
>>>>>>>> that
>>>>>>>> split from normal C in the mid-eighties and is still very much worth
>>>>>>>> using
>>>>>>>> for its specialised role. Yes, new hires have to learn it, which takes
>>>>>>>> about two days for someone who knows C or C++. Learning about the
>>>>>>>> specialised application area it is used for takes much longer.
>>>>>>>>
>>>>>>>> I'm running Emscripten on Linux. I started by looking at Emscripten
>>>>>>>> 3.1.41, since that's the version used by a couple of other product
>>>>>>>> teams
>>>>>>>> that work on my site. That is fairly simple when I run a few emcc
>>>>>>>> compiles
>>>>>>>> with -H to get a report of what files are referenced. The top-level
>>>>>>>> headers
>>>>>>>> come from emsdk/upstream/emscripten/cache/sysroot/include. A few
>>>>>>>> Clang-associated ones come from emsdk/upstream/lib/clang/17/include.
>>>>>>>> That's
>>>>>>>> all fine with me.
>>>>>>>>
>>>>>>>> Then I decided to check the latest emsdk, and found that with
>>>>>>>> 3.1.69, the headers come from a different place. It builds a cache of
>>>>>>>> the
>>>>>>>> headers and other files it uses under ~/.emscripten_cache. The problems
>>>>>>>> with that are:
>>>>>>>>
>>>>>>>> Storage: It will take 36MB in the user directory of everyone who
>>>>>>>> ever compiles with Emscripten. We keep all our user directories on a
>>>>>>>> server
>>>>>>>> disk, because that's enormously convenient in many ways. But we really
>>>>>>>> don't want to burn space with duplicates of that cache.
>>>>>>>>
>>>>>>>> Version lock: We need to be able to have several Emscripten
>>>>>>>> versions in use simultaneously, by the same account, without
>>>>>>>> conflicts. Our
>>>>>>>> reason for this is that we plan to release products on WebAssembly, and
>>>>>>>> from time to time, update the version of Emscripten we use, to get
>>>>>>>> access
>>>>>>>> to new C and C++ standards, compiler bug fixes, and so on. But we will
>>>>>>>> not
>>>>>>>> update the tools used to build a product version that's been released
>>>>>>>> and
>>>>>>>> is under maintenance, because we'd have to re-do a lot of the QA that
>>>>>>>> we do
>>>>>>>> at a release. So the service accounts that run our builds can't have
>>>>>>>> caches
>>>>>>>> of version-specific headers in their user directories.
>>>>>>>>
>>>>>>>> I need a way to tell Emscripten to put that cache somewhere else. I
>>>>>>>> did some grep'ing of the Emscripten scripts, and found this line, in
>>>>>>>> both
>>>>>>>> 3.1.41 and 3.1.69:
>>>>>>>>
>>>>>>>> upstream/emscripten/tools/config.py: CACHE =
>>>>>>>> os.path.expanduser(os.path.join('~', '.emscripten_cache'))
>>>>>>>>
>>>>>>>> I have not yet attempted to read the Python code and learn how it
>>>>>>>> all works, because that could take ages; I don't know Python well at
>>>>>>>> all
>>>>>>>> and am not keen on it. I'm a naturally low-level programmer, much
>>>>>>>> happier
>>>>>>>> with assembly code than object orientation
>>>>>>>>
>>>>>>>> Is there an environment variable I can use to relocate that cache?
>>>>>>>>
>>>>>>>> Thanks very much,
>>>>>>>>
>>>>>>>> John
>>>>>>>>
>>>>>>>> --
>>>>>>>> You received this message because you are subscribed to the Google
>>>>>>>> Groups "emscripten-discuss" group.
>>>>>>>> To unsubscribe from this group and stop receiving emails from it,
>>>>>>>> send an email to emscripten-discuss+unsubscr...@googlegroups.com.
>>>>>>>> To view this discussion visit
>>>>>>>> https://groups.google.com/d/msgid/emscripten-discuss/CAH1xqgm39n%2B-%3DAeT09zJ38Bp3SMkiWk_TxMK0cEMcinnaaQn4w%40mail.gmail.com
>>>>>>>> <https://groups.google.com/d/msgid/emscripten-discuss/CAH1xqgm39n%2B-%3DAeT09zJ38Bp3SMkiWk_TxMK0cEMcinnaaQn4w%40mail.gmail.com?utm_medium=email&utm_source=footer>
>>>>>>>> .
>>>>>>>>
>>>>>>> --
>>>>>>> You received this message because you are subscribed to the Google
>>>>>>> Groups "emscripten-discuss" group.
>>>>>>> To unsubscribe from this group and stop receiving emails from it,
>>>>>>> send an email to emscripten-discuss+unsubscr...@googlegroups.com.
>>>>>>> To view this discussion visit
>>>>>>> https://groups.google.com/d/msgid/emscripten-discuss/CAL_va2-imxQ2O0x3kKhpRvtanvYLRLEASCx2FrWUwy%3D2PxE94A%40mail.gmail.com
>>>>>>> <https://groups.google.com/d/msgid/emscripten-discuss/CAL_va2-imxQ2O0x3kKhpRvtanvYLRLEASCx2FrWUwy%3D2PxE94A%40mail.gmail.com?utm_medium=email&utm_source=footer>
>>>>>>> .
>>>>>>>
>>>>>> --
>>>>>> You received this message because you are subscribed to the Google
>>>>>> Groups "emscripten-discuss" group.
>>>>>> To unsubscribe from this group and stop receiving emails from it,
>>>>>> send an email to emscripten-discuss+unsubscr...@googlegroups.com.
>>>>>> To view this discussion visit
>>>>>> https://groups.google.com/d/msgid/emscripten-discuss/CAH1xqgkjhQF8QVh80-7eCQ2UGU3jqYScoKK44AzU-oAqMuXRXw%40mail.gmail.com
>>>>>> <https://groups.google.com/d/msgid/emscripten-discuss/CAH1xqgkjhQF8QVh80-7eCQ2UGU3jqYScoKK44AzU-oAqMuXRXw%40mail.gmail.com?utm_medium=email&utm_source=footer>
>>>>>> .
>>>>>>
>>>>> --
>>>>> You received this message because you are subscribed to the Google
>>>>> Groups "emscripten-discuss" group.
>>>>> To unsubscribe from this group and stop receiving emails from it, send
>>>>> an email to emscripten-discuss+unsubscr...@googlegroups.com.
>>>>> To view this discussion visit
>>>>> https://groups.google.com/d/msgid/emscripten-discuss/CAL_va2_BoKxF%3D40hQGihs4Ay8ht4eNJGyVJvzdR6-W5Q%3DddNhg%40mail.gmail.com
>>>>> <https://groups.google.com/d/msgid/emscripten-discuss/CAL_va2_BoKxF%3D40hQGihs4Ay8ht4eNJGyVJvzdR6-W5Q%3DddNhg%40mail.gmail.com?utm_medium=email&utm_source=footer>
>>>>> .
>>>>>
>>>> --
>>>> You received this message because you are subscribed to the Google
>>>> Groups "emscripten-discuss" group.
>>>> To unsubscribe from this group and stop receiving emails from it, send
>>>> an email to emscripten-discuss+unsubscr...@googlegroups.com.
>>>> To view this discussion visit
>>>> https://groups.google.com/d/msgid/emscripten-discuss/CAH1xqgnNnZBNPwqrT_vEd-6A%3DpuSKt3wE%3DrxaPqz%3DwYLgth0kQ%40mail.gmail.com
>>>> <https://groups.google.com/d/msgid/emscripten-discuss/CAH1xqgnNnZBNPwqrT_vEd-6A%3DpuSKt3wE%3DrxaPqz%3DwYLgth0kQ%40mail.gmail.com?utm_medium=email&utm_source=footer>
>>>> .
>>>>
>>> --
>>> You received this message because you are subscribed to the Google
>>> Groups "emscripten-discuss" group.
>>> To unsubscribe from this group and stop receiving emails from it, send
>>> an email to emscripten-discuss+unsubscr...@googlegroups.com.
>>> To view this discussion visit
>>> https://groups.google.com/d/msgid/emscripten-discuss/CAL_va2837g9UFhYbz9o6r_eF5RYp5--Xog4UbPmuPQX_65j7Pw%40mail.gmail.com
>>> <https://groups.google.com/d/msgid/emscripten-discuss/CAL_va2837g9UFhYbz9o6r_eF5RYp5--Xog4UbPmuPQX_65j7Pw%40mail.gmail.com?utm_medium=email&utm_source=footer>
>>> .
>>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "emscripten-discuss" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to emscripten-discuss+unsubscr...@googlegroups.com.
>> To view this discussion visit
>> https://groups.google.com/d/msgid/emscripten-discuss/CAH1xqgmpJbfFdmeGLtsTrvAwKAgd8x%2B1jaNGJtuobp8PYVcp7Q%40mail.gmail.com
>> <https://groups.google.com/d/msgid/emscripten-discuss/CAH1xqgmpJbfFdmeGLtsTrvAwKAgd8x%2B1jaNGJtuobp8PYVcp7Q%40mail.gmail.com?utm_medium=email&utm_source=footer>
>> .
>>
> --
> You received this message because you are subscribed to the Google Groups
> "emscripten-discuss" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to emscripten-discuss+unsubscr...@googlegroups.com.
> To view this discussion visit
> https://groups.google.com/d/msgid/emscripten-discuss/CAL_va288ZgNXryD78HOp-dTgC9CLJ%3DU5xNFhLOy_DY_HD-50hw%40mail.gmail.com
> <https://groups.google.com/d/msgid/emscripten-discuss/CAL_va288ZgNXryD78HOp-dTgC9CLJ%3DU5xNFhLOy_DY_HD-50hw%40mail.gmail.com?utm_medium=email&utm_source=footer>
> .
>
--
You received this message because you are subscribed to the Google Groups
"emscripten-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to emscripten-discuss+unsubscr...@googlegroups.com.
To view this discussion visit
https://groups.google.com/d/msgid/emscripten-discuss/CAH1xqg%3DgisAnDkhFmLz3EVuTVh5pFhkffp_umeCDKk6-CQN_NQ%40mail.gmail.com.
