Hi Eliot,
Pharo does AFAIK the same with the source files when you're navigating
source code.
Now, I remember that while adapting the finalization scheme that you
sent me to Pharo (because there are indeed subtle differences), I
noticed that there was a missing loop.
[looking for the code... found!]
finalizationProcess
"The finalization process arranges to send mourn to each element of
the VM's finalization queue,
which is accessed via primitiveFetchMourner. The VM signals
FinalizationSemaphore whenever
the queue is non-empty. This process loops, waiting on the
semaphore, fetches the first element
of the queue and then spawns a process at a higher priority to
acually send the mourn messages.
If an error occurs in the higher priority mourn loop process then
this process will simply spawn
another process, hence ensuring that errors in finalization
methods don't break finalization.
In addition this process also runs the old finalization scheme,
supporting clients of the older,
WeakRegistry based scheme. Hopefully this will go away when all
cleints have moved over."
| throttle firstMourner |
throttle := Semaphore new.
[FinalizationSemaphore wait; initSignals.
"Support the old scheme until things have changed over..."
self doOldFinalization.
[firstMourner := self primitiveFetchMourner.
firstMourner notNil] whileTrue:
[[throttle signal.
self mournLoopWith: firstMourner] forkAt: Processor
activePriority + 1.
throttle wait]]
At first I was using that code that you sent me and I noticed that the
finalization process in there is a loop that is never evaluated! So I
updated it to the following using a [true] whileTrue:
finalizationProcess
"The finalization process arranges to send mourn to each element of
the VM's finalization queue,
which is accessed via primitiveFetchMourner. The VM signals
FinalizationSemaphore whenever
the queue is non-empty. This process loops, waiting on the
semaphore, fetches the first element
of the queue and then spawns a process at a higher priority to
acually send the mourn messages.
If an error occurs in the higher priority mourn loop process then
this process will simply spawn
another process, hence ensuring that errors in finalization
methods don't break finalization.
In addition this process also runs the old finalization scheme,
supporting clients of the older,
WeakRegistry based scheme. Hopefully this will go away when all
cleints have moved over."
| throttle firstMourner |
throttle := Semaphore new.
[true] whileTrue: [FinalizationSemaphore wait; initSignals.
"Support the old scheme until things have changed over..."
self doOldFinalization.
[firstMourner := self primitiveFetchMourner.
firstMourner notNil] whileTrue:
[[throttle signal.
self mournLoopWith: firstMourner] forkAt: Processor
activePriority + 1.
throttle wait]]
Maybe that's the reason of weak arrays not being finalized in squeak?
Guille
-------- Original Message --------
Hi Guille,
good news! But I'm seeing something wrong with finalisation of
weak arrays using the new scheme. In Squeak method source access is
done by default by opening a new read-only file for each method's
source read (crazy, but that's not the issue). So by default
something that accesses lots of source ends up running out of file
descriptors, which causes primOpen:writable: to fail. The surrounding
code then uses retryWithGC:until:forFileNamed: to do a GC to try and
reclaim non-longer referenced files, close file descriptors and continue:
StandardFileStream retryWithGC:[self primOpen: f writable:
writeMode]
until:[:id| id notNil]
forFileNamed: fileName.
But in my tests I'm not seeing any files reclaimed. I wonder whether
the new finalisation code is failing to finalise weak arrays
properly. I wonder whether your weak tests work properly with the new
scheme or not.
Anyway, I think I can reproduce the pathology in the simulator if I
modify it to implement a small limit on the number of file
descriptors. I'm going to try that to shed light on the problem. I
can't easily debug in a running image because...I run out of file
descriptors ;-)
On Tue, Jun 7, 2016 at 2:32 AM, Guille Polito
<[email protected] <mailto:[email protected]>> wrote:
Hi All,
Since this morning, in Pharo #60065, Ephemeron support is in the
image. Most of the changes are infrastructural, so far transparent
for the users. It is important to notice that even while the support
is there, it is not enabled by default. Also, this required changes
in the virtual machine that are not yet distributed everywhere. For
the ones that would like more detail, I invite you to read the
following :)
* On the infrastructure side
- There is support to create Ephemeric classes and load them from
monticello
- There is a new finalization mechanism (by default disabled) that
will process Ephemerons using a finalization queue. This will avoid
scanning collections in look for weak objects to finalize ,as it
happens now with the WeakDependent mechanism in WeakArray.
- System-Finalization features two new classes *Ephemeron* and
*EphemeronRegistry*. For the ones that want more details on
Ephemerons, you can read the associated paper [1], or the class
comment of Ephemeron:
I represent ephemeric key-value objects. Ephemerons are
key-value objects (subclasses of Association) with special
semantics during garbage collection. My special behavior can
resumed as follows:
- The garbage collection will iterate my instances only if the key
is not referenced strongly by another object.
- Then, if no strong references to the key are found, then the
values of this ephemeron are hold weakly.
- Otherwise, the values are hold strongly.
In this implementation, an Ephemeron can hold more than one value,
which are all treated in the same manner. This ephemeron instance
knows its container, which allows the ephemeron to remove itself
from a container (such as a Dictionary) upon finalization.
!! Example usages
In general terms, do not use myself directly. Use instead an
Ephemeric container like EphemeronRegistry. An Ephemeron registry
will guarantee the collection of keys and values of the object
inside the Ephemeron.
Otherwise, if you want to use it, you can create an Ephemeron as
any association:
ephemeron := Ephemeron key: aKey value: aValue.
ephemeron container: aContainer.
!! Ephemeron Finalization
When an ephemeron's key is hold strongly just by the ephemeron
itself, the Ephemeron will be mourned (finalized). That means that
the VM will:
- put the Ephemeron in the mourning queue waiting for the image to
take care of mourning
- make the Ephemeron non ephemeric. That is, the ephemeron
instance cannot be reused.
On the image side, the finalization process will send the message
#mourn to an Ephemeron. #mourn will #finalize the Ephemeron's
key, and remove the Ephemeron from it's container to allow its
collection during a subsequent garbage collection.
!! More Documentation
You can read the associated paper to understand better the
semantics of ephemerons:
[1]Ephemerons: A New Finalization Mechanism. Barry Hayes. OOPSLA
'97
- WARNING: to be able to use ephemerons, you need to use the
*latestVm* that has several fixes for making ephemerons work, and
you need to enable ephemerons on the image side by evaluating:
Smalltalk supportsQueueingFinalization: true.
- With latest vm and ephemerons enabled, tests should be green,
otherwise they are skipped
* From the user point of view:
- The Weak registries were not yet migrated to the new
finalization mechanism.
- We expect nothing will change from the user point of view. Just
less memory leaks.
* Next steps (in order)
1) Bless the latest vm as stable
2) Enable queueing finalization by default
3) Replace Weak Registry by Ephemeron Registry.
Informed by: Guille
--
_,,,^..^,,,_
best, Eliot
