Re: [Pharo-dev] Bug 18632 : Virtual Machine parameters need to be documented

[Nicolas Cellier]

Hi Benoit,
you are right to say that documentation is lacking.
Alas, it's not going to fall from the sky: there's no such thing like a VM
Pantheon with gods ruling the VM world, not even half-gods
(well maybe one super-powered hero, we have to acknowledge that a 10x
speedup was brought by his experience).
Changes must come from the ground up, and if some metaphor is to be used,
we'd rather want to cultivate the VM I would say.
Every initiative is welcome, and encouraged, the source code is open, there
is no will to protect by secret or obfuscation.
We have almost 50 contributors reported on github, which is already a good
starting point.
I'd say every little contribution counts. So if someone wants to take a
subject like VM parameters, I'm pretty sure that he/she will get all
necessary precision from those sharing parts of the knowledge. It's also a
very interesting way to get more insights in the VM.

What should be discussed IMO is the form of documentation. Blog posts are
nice.
But they are individual initiatives, rather than collectively maintained
and versioned documentation.


Le sam. 8 févr. 2020 à 04:17, Benoit St-Jean via Pharo-dev <
pharo-dev@lists.pharo.org> a écrit :

> Eliot,
>
> I was referring to VM parameters (aka #vmParameterPut:at:).  But now that
> you mention it, it adds another level of complexity. When you specify
> something in the command line arguments, how do you resolve the fact that
> the command line argument tells you something that is different from what
> is in the image? The mechanics of the VM stuff is like black magic to most
> of us! You see, that's the type of question that is kinda "oral tradition"
> : if no one ever told you, you don't know since it's not obvious and rarely
> documented properly but, eh, it's written "somewhere" in the code of the
> VM! It might seem obvious to a very few of you, especially yourself, since
> you've been working on the VM long before my Vassili-Bykov-Days but for
> most of us, it's a case of very-interesting-to-read-but-way-too-complex ! ;)
>
> To make a long story short, I was thinking about experimenting and trying
> to implement "stuff" with something similar to VisualWorks's MemoryPolicy
> (AbstractMemoryPolicy and subclasses to be more precise) for Pharo. It did
> save my/our/TheProjects' ass quite a few times. Yes, you can write scripts
> to customize a few things before you produce the "final" image but it is a
> pain in the as_ (the polite SQL way of describing it) and you have to keep
> track of artifacts (scripts, files, workspace code, whatever) to produce
> the final image.  With custom memory policies, everything resides in the
> image and it is sooo much simpler to deal with!
>
> There is no such thing as a one-configuration-fits-all.  That's why
> different types of applications have different needs for GC, memory,
> tenuring, memory-space sizes, etc. And quite frankly, VW' MemoryPolicy is
> so far and by far the best solution I've seen to cope with that. There
> ain't such thing as 2 identical applications that have the exact same
> VM/GC/memory needs!
>
> I was able to build/read a "collection" of articles/posts/PDFs about GC
> and the VM but, quite honestly, I kinda lost track of what is what and
> which is which.  Cog, Spur, stack or not, NewSpeak vs Squeak vs Pharo, 32
> vs 64 bits, jitted or not, and all permutations you can think of, for 99.9%
> of us it is like trying to learn Mandarin in 2 days! I do/did read your
> blog as well as the blog of Clément Béra as well as Andres Valloud
> posts/papers on the subject but in the VM world, things quickly get
> complicated unless, like Obélix, you were born in it! I've read a gazillion
> papers on the GC and various Smalltalk memory models written by John
> McIntosh but are those papers/posts/articles still relevant?
>
> In other words Eliot (and your VM wizard friends), the VM (aka GC and
> memory model) do need some "documentation love" for "the average
> Smalltalker" !  ;)
> On 2020-02-07 13:59, Eliot Miranda wrote:
>
> Hi Benoit,
>
> On Fri, Feb 7, 2020 at 8:20 AM Nicolas Cellier <
> nicolas.cellier.aka.n...@gmail.com> wrote:
>
>> Hi Benoit,
>> for some reason (?), your email was marked as SPAM in gmail...
>> There is a short description in primitiveVMParameter, which is found in
>> generated code, for example:
>>
>> https://raw.githubusercontent.com/OpenSmalltalk/opensmalltalk-vm/Cog/src/vm/gcc3x-cointerp.c
>> or VMMaker source code (you can also find VMMaker source code in pharo vm
>> github repo, no time to dig).
>>
>> Le jeu. 6 févr. 2020 à 12:45, Benoit St-Jean via Pharo-dev <
>> pharo-dev@lists.pharo.org> a écrit :
>>
>>> I was looking for some documentation on VM parameters (are they read,
>>> read-write or write only as well as the expected argument type) and came
>>> across that issue on the old FogBugz database.
>>>
>>
> Are you talking just about vmParameterAt:[put:]/vm parameterAt:[put:] or
> are you also talking about command line arguments 

Re: [Pharo-dev] Bug 18632 : Virtual Machine parameters need to be documented

[Benoit St-Jean via Pharo-dev]

--- Begin Message ---

Eliot,

I was referring to VM parameters (aka #vmParameterPut:at:).  But now 
that you mention it, it adds another level of complexity. When you 
specify something in the command line arguments, how do you resolve the 
fact that the command line argument tells you something that is 
different from what is in the image? The mechanics of the VM stuff is 
like black magic to most of us! You see, that's the type of question 
that is kinda "oral tradition" : if no one ever told you, you don't know 
since it's not obvious and rarely documented properly but, eh, it's 
written "somewhere" in the code of the VM! It might seem obvious to a 
very few of you, especially yourself, since you've been working on the 
VM long before my Vassili-Bykov-Days but for most of us, it's a case of 
very-interesting-to-read-but-way-too-complex ! ;)


To make a long story short, I was thinking about experimenting and 
trying to implement "stuff" with something similar to VisualWorks's 
MemoryPolicy (AbstractMemoryPolicy and subclasses to be more precise) 
for Pharo. It did save my/our/TheProjects' ass quite a few times. Yes, 
you can write scripts to customize a few things before you produce the 
"final" image but it is a pain in the as_ (the polite SQL way of 
describing it) and you have to keep track of artifacts (scripts, files, 
workspace code, whatever) to produce the final image.  With custom 
memory policies, everything resides in the image and it is sooo 
much simpler to deal with!


There is no such thing as a one-configuration-fits-all.  That's why 
different types of applications have different needs for GC, memory, 
tenuring, memory-space sizes, etc. And quite frankly, VW' MemoryPolicy 
is so far and by far the best solution I've seen to cope with that. 
There ain't such thing as 2 identical applications that have the exact 
same VM/GC/memory needs!


I was able to build/read a "collection" of articles/posts/PDFs about GC 
and the VM but, quite honestly, I kinda lost track of what is what and 
which is which.  Cog, Spur, stack or not, NewSpeak vs Squeak vs Pharo, 
32 vs 64 bits, jitted or not, and all permutations you can think of, for 
99.9% of us it is like trying to learn Mandarin in 2 days! I do/did read 
your blog as well as the blog of Clément Béra as well as Andres Valloud 
posts/papers on the subject but in the VM world, things quickly get 
complicated unless, like Obélix, you were born in it! I've read a 
gazillion papers on the GC and various Smalltalk memory models written 
by John McIntosh but are those papers/posts/articles still relevant?


In other words Eliot (and your VM wizard friends), the VM (aka GC and 
memory model) do need some "documentation love" for "the average 
Smalltalker" !  ;)


On 2020-02-07 13:59, Eliot Miranda wrote:

Hi Benoit,

On Fri, Feb 7, 2020 at 8:20 AM Nicolas Cellier 
> wrote:


Hi Benoit,
for some reason (?), your email was marked as SPAM in gmail...
There is a short description in primitiveVMParameter, which is
found in generated code, for example:

https://raw.githubusercontent.com/OpenSmalltalk/opensmalltalk-vm/Cog/src/vm/gcc3x-cointerp.c
or VMMaker source code (you can also find VMMaker source code in
pharo vm github repo, no time to dig).

Le jeu. 6 févr. 2020 à 12:45, Benoit St-Jean via Pharo-dev
mailto:pharo-dev@lists.pharo.org>> a
écrit :

I was looking for some documentation on VM parameters (are
they read,
read-write or write only as well as the expected argument
type) and came
across that issue on the old FogBugz database.


Are you talking just about vmParameterAt:[put:]/vm parameterAt:[put:] 
or are you also talking about command line arguments (pharos-vm 
--help/squeak -help)?
If the former, there are much more current comments in trunk Squeak 
SmalltalkImage>>vmParameterAt:[put:] than in Pharo; you could usefully 
copy that across. The comments in the two methods accurately specify 
which parameters are read/write, and which persist in the image header.


But I agree, more verbose documentation would be very helpful; the 
current documentation is very terse, and while it describes what the 
parameters are and whether they are writable, it really doesn't 
describe what the systems that they operate on actually do.  I'm happy 
to collaborate with you in developing better documentation.  If you 
will do the writing I will happily consult.  Does that work?




https://pharo.fogbugz.com/f/cases/18632/Virtual-Machine-parameters-need-to-be-documented

I tried looking it up on GitHub but couldn't find it. Am I
missing
something or I should open an issue?  I'm working on something
related
to various memory settings and not having any info on a lot of
these
parameters doesn't help.  While I could meticulously read tons
of C code
of the VM and 

Re: [Pharo-dev] Bug 18632 : Virtual Machine parameters need to be documented

[Eliot Miranda]

Hi Benoit,

On Fri, Feb 7, 2020 at 8:20 AM Nicolas Cellier <
nicolas.cellier.aka.n...@gmail.com> wrote:

> Hi Benoit,
> for some reason (?), your email was marked as SPAM in gmail...
> There is a short description in primitiveVMParameter, which is found in
> generated code, for example:
>
> https://raw.githubusercontent.com/OpenSmalltalk/opensmalltalk-vm/Cog/src/vm/gcc3x-cointerp.c
> or VMMaker source code (you can also find VMMaker source code in pharo vm
> github repo, no time to dig).
>
> Le jeu. 6 févr. 2020 à 12:45, Benoit St-Jean via Pharo-dev <
> pharo-dev@lists.pharo.org> a écrit :
>
>> I was looking for some documentation on VM parameters (are they read,
>> read-write or write only as well as the expected argument type) and came
>> across that issue on the old FogBugz database.
>>
>
Are you talking just about vmParameterAt:[put:]/vm parameterAt:[put:] or
are you also talking about command line arguments (pharos-vm --help/squeak
-help)?
If the former, there are much more current comments in trunk Squeak
SmalltalkImage>>vmParameterAt:[put:] than in Pharo; you could usefully copy
that across. The comments in the two methods accurately specify which
parameters are read/write, and which persist in the image header.

But I agree, more verbose documentation would be very helpful; the current
documentation is very terse, and while it describes what the parameters are
and whether they are writable, it really doesn't describe what the systems
that they operate on actually do.  I'm happy to collaborate with you in
developing better documentation.  If you will do the writing I will happily
consult.  Does that work?


>>
>> https://pharo.fogbugz.com/f/cases/18632/Virtual-Machine-parameters-need-to-be-documented
>>
>> I tried looking it up on GitHub but couldn't find it.  Am I missing
>> something or I should open an issue?  I'm working on something related
>> to various memory settings and not having any info on a lot of these
>> parameters doesn't help.  While I could meticulously read tons of C code
>> of the VM and pinpoint exactly what I need to know for each and everyone
>> of these undocumented parameters, I'd gladly do it myself if any of the
>> VM guys (Clément, Pablo, Alexandre or Eliot) have notes/documents/links
>> that could help me do it.
>>
>> tia
>>
>> --
>> -
>> Benoît St-Jean
>> Yahoo! Messenger: bstjean
>> Twitter: @BenLeChialeux
>> Pinterest: benoitstjean
>> Instagram: Chef_Benito
>> IRC: lamneth
>> GitHub: bstjean
>> Blogue: endormitoire.wordpress.com
>> "A standpoint is an intellectual horizon of radius zero".  (A. Einstein)
>>
>>
>>

-- 
_,,,^..^,,,_
best, Eliot

Re: [Pharo-dev] Bug 18632 : Virtual Machine parameters need to be documented

[Nicolas Cellier]

Hi Benoit,
for some reason (?), your email was marked as SPAM in gmail...
There is a short description in primitiveVMParameter, which is found in
generated code, for example:
https://raw.githubusercontent.com/OpenSmalltalk/opensmalltalk-vm/Cog/src/vm/gcc3x-cointerp.c
or VMMaker source code (you can also find VMMaker source code in pharo vm
github repo, no time to dig).

Le jeu. 6 févr. 2020 à 12:45, Benoit St-Jean via Pharo-dev <
pharo-dev@lists.pharo.org> a écrit :

> I was looking for some documentation on VM parameters (are they read,
> read-write or write only as well as the expected argument type) and came
> across that issue on the old FogBugz database.
>
>
> https://pharo.fogbugz.com/f/cases/18632/Virtual-Machine-parameters-need-to-be-documented
>
> I tried looking it up on GitHub but couldn't find it.  Am I missing
> something or I should open an issue?  I'm working on something related
> to various memory settings and not having any info on a lot of these
> parameters doesn't help.  While I could meticulously read tons of C code
> of the VM and pinpoint exactly what I need to know for each and everyone
> of these undocumented parameters, I'd gladly do it myself if any of the
> VM guys (Clément, Pablo, Alexandre or Eliot) have notes/documents/links
> that could help me do it.
>
> tia
>
> --
> -
> Benoît St-Jean
> Yahoo! Messenger: bstjean
> Twitter: @BenLeChialeux
> Pinterest: benoitstjean
> Instagram: Chef_Benito
> IRC: lamneth
> GitHub: bstjean
> Blogue: endormitoire.wordpress.com
> "A standpoint is an intellectual horizon of radius zero".  (A. Einstein)
>
>
>