On Wed, Oct 14, 2015 at 12:09 PM, Robert Withers
<[email protected] <mailto:[email protected]>> wrote:
On 10/14/2015 11:01 AM, Mariano Martinez Peck wrote:
Robert,
As far as I can remember, the problem of substitutions at
materialziation time was because... as you may have read, Fuel first
serializes "objects" and then the references. At materialization, it
first creates the objects and then, on a second step, sets the
references betwen objects. So the problem was WHERE to place the
hook,
at which point in time. If we do it just after objects were created,
then the substitution code would NOT have access to any of the
instVars
of that object. If we do it AFTER objects creation and after objects
references, then there is no easy way to "substitute" which doesn't
involve #become: (because the graph is already constructed) And
you know
that #become: is still terrible slow because it scans full
memory. That
will change in Spur.
The trick I learned from Gemstone is to use forwarding proxies,
Well, Spur will/does something similar called lazy become. Basically it
lets a forwarding pointer object and then takes advantage of next GC
pass or whatever in order to resolve such proxy in lazy manner.
looking up into the FLobjectId dictionary (decoder>>objects?) when
stitching the references. When you copy the proxies on substitution,
it stitches normally at reference time.
Yes, that could work. The problem is if you need instVars of the object
you want to substitute. Imagine you have a class called Client and
instVar 'age'. And you want to substitute Client with instances of
OldClient if 'age' is > 10. At that point in time, the reference to the
instVar 10 has not yet been filled. Yet, you need to replace the object
at graph construction time.
As for Marea and Ghost,
Ghost proxies paper: https://hal.inria.fr/hal-01081236/document
Current Ghost repo: http://smalltalkhub.com/#!/~CAR/Ghost
Marea paper: http://www.jot.fm/issues/issue_2013_01/article2.pdf
Current repo: http://ss3.gemstone.com/ss/Marea.html
And finally, my PhD thesis:
https://tel.archives-ouvertes.fr/tel-00764991/document
Nicely done, sir. I'll check them out, thankyou.
Thanks, feel free to ask questions.
In Marea I needed custom clusters for my proxies because the
serializer
itself sends messages to the objects being serialized. My
proxies would
bring back graphs from a secondary memory. So if I was serializing a
graph that had proxies already, I didn't want that. So I hooked my
custom cluster for proxies that send only a few special messages
to the
proxy that these understand and answer rather than intercept those
messages.
As for how to extend Fuel for this, I recommend to check the code of
Marea. See categories 'Marea-Serializers' and 'Marea-Proxies'.
I have a Marea one click here:
https://www.dropbox.com/sh/xp8jzyypmz0898j/AACRdHno6V7UfhaJ1ofTPPXva?dl=0
Cheers,
thanks so much ^^
Robert
On Wed, Oct 14, 2015 at 11:40 AM, Robert Withers
<[email protected] <mailto:[email protected]>
<mailto:[email protected]
<mailto:[email protected]>>> wrote:
Good morning, Max. Thank you for the example. I got a little
confused, between migrations and substitutions. My issue
with no-arg
blocks, I believe, is the inability to access my scope
object to
maintain the object tables.
I'm attempting to write my own Materializer, Decoder and
Materialization. At the moment, I'm just going to walk the
graph,
testing and do #becomes:. See how well that works when I
get a test.
It's really good to know about that other list.
thanks so much ^^
Robert
On 10/14/2015 04:15 AM, Max Leske wrote:
BTW, there is a dedicated Fuel mailing list:
[email protected]
<mailto:[email protected]>
<mailto:[email protected]
<mailto:[email protected]>>
<mailto:[email protected]
<mailto:[email protected]>
<mailto:[email protected]
<mailto:[email protected]>>>
Max
On 14 Oct 2015, at 09:45, Max Leske
<[email protected] <mailto:[email protected]>
<mailto:[email protected] <mailto:[email protected]>>
<mailto:[email protected]
<mailto:[email protected]> <mailto:[email protected]
<mailto:[email protected]>>>> wrote:
On 14 Oct 2015, at 04:39, Robert Withers
<[email protected]
<mailto:[email protected]>
<mailto:[email protected]
<mailto:[email protected]>>
<mailto:[email protected]
<mailto:[email protected]>
<mailto:[email protected]
<mailto:[email protected]>>>> wrote:
On 10/13/2015 09:43 PM, Mariano Martinez Peck
wrote:
On Tue, Oct 13, 2015 at 10:33 PM, Robert
Withers
<[email protected]
<mailto:[email protected]>
<mailto:[email protected]
<mailto:[email protected]>>
<mailto:[email protected]
<mailto:[email protected]>
<mailto:[email protected]
<mailto:[email protected]>>>
<mailto:[email protected]
<mailto:[email protected]>
<mailto:[email protected]
<mailto:[email protected]>>>> wrote:
Hi Mariano,
This presents me with a big challenge,
then. I
read the docs and
explored the code and the only other
aspect not
mentioned, beyond
instance creation (#fuelNew #fuelNew:) and
postMaterialization
(#fuelAfterMaterialization), is migrations.
However, migration only
allows for instanceVar mappings, no code
blocks.
What do you mean that migrations only
allows instVar
mappings , and no
code blocks? I mean, what do you mean by
code blocks?
Sounds to me like this (see FuelOutStackDebuAction):
serializeTestFailureContext: aContext toFileNamed:
aFilename
| serializer |
serializer := FLSerializer newDefault.
self encodeDebugInformationOn: serializer.
serializer addPostMaterializationAction: [
:materialization |
Smalltalk tools debugger
openOn: Processor activeProcess
context: materialization root
label: 'External stack'
contents: nil
fullView: false ].
serializer
" use the sender context, generally the current
context is not
interesting"
serialize: aContext
toFileNamed: aFilename
This stores a block in the serialization which is
evaluated
after
materialization. The only requirement is that it’s
a clean
block (no
closure!).
We also support class renames. This is here:
http://rmod.inria.fr/web/software/Fuel/Version1.9/Documentation/Migration?_s=Pvs4DYqPRjfEy8sg&_k=X6ltJu7FDSxumm4X&_n&22
Which kind of migration example you have in
mind
that would not be
supported? An example would help.
Well, my pics will demonstrate. I am interested
in doing
more than
mappping ivars or a class rename. I want to do
a total
substitution,
then a further substitution on the receiving,
import side:
Vat1: anObject (Class A) ---> On wire: desc
(Descriptor)
---> Vat2:
aProxy (Class FarERef)
A desc is substituted for a PassByProxy object,
then a
FarERef is
substituted for the desc.
#fuelAccept: is a serialization side method.
If Fuel supports substitution on
serialization, I
don't understand
why no substitution support on
materialization.
There was a reason, which I cannot remember
completely. Maybe Martin or
Max can remember.
It seems your focus was pickling to disk then
back. My
focus is
distributed proxy graphs, which has different
use cases.
I am definitely going to use the
world-class Fuel
binary
serialization system. However, I find myself
needing to extend Fuel
to support substitution on materialization.
Perhaps the solution is
a custom decoder.
I have made custom clusters for example for
my Ghost
proxies of Marea
system. It was a perfect example of how I could
extent Fuel besides the
common hooks. Fuel provides many places for
extending , like clusters,
analyzer, etc
Right on, exactly! Could you tell me more about
your
Ghost proxies
and Marea, please? As well, could you mention
how you
select a custom
cluster on the serialization side?
thanks so much ^^
Robert
No, a bit more. It looks like I need a new
FLSubstitutePointerObjectCluster, write
them on
serialization with
the substitute, then do unsubstitution on
materialization, since the
cluster controls materialization and not
the decoder.
Does this approach seem sound to you,
from a you
know architecture
and design approach?
There was an issue. Hope other can
remember. If not,
I will try to
explan what I remember tomorrow.
thanks so much ^^
Robert
On 10/13/2015 04:49 PM, Mariano Martinez
Peck wrote:
No, unfortunately, as far as I can
remember,
we do not have
that. There
are another hooks you may use but
only in
certain scenarios
(#fuelNew,
#fuelAfterMaterialization, global sends,
etc). But everything is
listed
in
http://rmod.inria.fr/web/software/Fuel/Version1.9/Documentation/CustomizingGraph
so if you didn't find anything of
help in
there there are
chances
there isn't anything.
Cheers,
On Tue, Oct 13, 2015 at 5:30 PM,
Robert Withers
<[email protected]
<mailto:[email protected]>
<mailto:[email protected]
<mailto:[email protected]>>
<mailto:[email protected]
<mailto:[email protected]>
<mailto:[email protected]
<mailto:[email protected]>>>
<mailto:[email protected]
<mailto:[email protected]>
<mailto:[email protected]
<mailto:[email protected]>>>
<mailto:[email protected]
<mailto:[email protected]>
<mailto:[email protected]
<mailto:[email protected]>>
<mailto:[email protected]
<mailto:[email protected]>
<mailto:[email protected]
<mailto:[email protected]>>>>> wrote:
Yes, I meant dynamic
substitution on
materialization, to
use the
correct terminology.
thanks,
Robert
On 10/13/2015 11:40 AM, Max
Leske wrote:
On 13 Oct 2015, at
17:16, Robert
Withers
<[email protected] <mailto:[email protected]>
<mailto:[email protected]
<mailto:[email protected]>>
<mailto:[email protected]
<mailto:[email protected]>
<mailto:[email protected]
<mailto:[email protected]>>>
<mailto:[email protected]
<mailto:[email protected]>
<mailto:[email protected]
<mailto:[email protected]>>>
<mailto:[email protected]
<mailto:[email protected]>
<mailto:[email protected]
<mailto:[email protected]>>
<mailto:[email protected]
<mailto:[email protected]>
<mailto:[email protected]
<mailto:[email protected]>>>>> wrote:
Every extra source
helps, thank
you. I see how to do
non-stream substitutions on
materializations, but the
documentation did not
indicate a
way to do non-stream
substitutions on
serialization.
Is it possible?
I don’t understand what you
mean by
“non-stream”. Could
you give
an example?
thanks,
Robert
On 10/13/2015 09:00 AM,
Mariano
Martinez Peck wrote:
Hi Robert,
As for the
documentation,
you have LOTS of
tests, you
have the chapter
Torsten pasted, you
have
this documentation:
http://rmod.inria.fr/web/software/Fuel
But also, as for
internals,
there is a journal
paper we
wrote:
http://rmod.lille.inria.fr/archives/papers/Dias12a-SPE-Fuel.pdf
Let us know how it
goes,
On Tue, Oct 13,
2015 at 6:00
AM, Torsten Bergmann
<[email protected]
<mailto:[email protected]>
<mailto:[email protected]
<mailto:[email protected]>> <mailto:[email protected]
<mailto:[email protected]>
<mailto:[email protected]
<mailto:[email protected]>>>
<mailto:[email protected]
<mailto:[email protected]> <mailto:[email protected]
<mailto:[email protected]>>>
<mailto:[email protected]
<mailto:[email protected]>
<mailto:[email protected]
<mailto:[email protected]>> <mailto:[email protected]
<mailto:[email protected]>
<mailto:[email protected]
<mailto:[email protected]>>>>
<mailto:[email protected] <mailto:[email protected]>
<mailto:[email protected]
<mailto:[email protected]>> <mailto:[email protected]
<mailto:[email protected]>
<mailto:[email protected]
<mailto:[email protected]>>>
<mailto:[email protected]
<mailto:[email protected]>
<mailto:[email protected]
<mailto:[email protected]>> <mailto:[email protected]
<mailto:[email protected]>
<mailto:[email protected]
<mailto:[email protected]>>>>>> wrote:
Hi Robert,
Also checkout the
chapter on Fuel in Pharo
Enterprise book:
https://ci.inria.fr/pharo-contribution/view/Books/job/EnterprisePharoBook/lastSuccessfulBuild/artifact/book-result/EnterprisePharo-A4.pdf
Bye
Torsten
Gesendet: Dienstag, 13. Oktober 2015 um
09:44 Uhr
Von: "Robert Withers"
<[email protected] <mailto:[email protected]>
<mailto:[email protected]
<mailto:[email protected]>>
<mailto:[email protected]
<mailto:[email protected]>
<mailto:[email protected]
<mailto:[email protected]>>>
<mailto:[email protected]
<mailto:[email protected]>
<mailto:[email protected]
<mailto:[email protected]>>>
<mailto:[email protected]
<mailto:[email protected]>
<mailto:[email protected]
<mailto:[email protected]>>
<mailto:[email protected]
<mailto:[email protected]>
<mailto:[email protected]
<mailto:[email protected]>>>>
<mailto:[email protected]
<mailto:[email protected]>
<mailto:[email protected]
<mailto:[email protected]>>
<mailto:[email protected]
<mailto:[email protected]>
<mailto:[email protected]
<mailto:[email protected]>>>
<mailto:[email protected]
<mailto:[email protected]>
<mailto:[email protected]
<mailto:[email protected]>>
<mailto:[email protected]
<mailto:[email protected]>
<mailto:[email protected]
<mailto:[email protected]>>>>>>
An: [email protected]
<mailto:[email protected]>
<mailto:[email protected]
<mailto:[email protected]>>
<mailto:[email protected]
<mailto:[email protected]>
<mailto:[email protected]
<mailto:[email protected]>>>
<mailto:[email protected]
<mailto:[email protected]>
<mailto:[email protected]
<mailto:[email protected]>>>
<mailto:[email protected]
<mailto:[email protected]>
<mailto:[email protected]
<mailto:[email protected]>>
<mailto:[email protected]
<mailto:[email protected]>
<mailto:[email protected]
<mailto:[email protected]>>>>
<mailto:[email protected]
<mailto:[email protected]>
<mailto:[email protected]
<mailto:[email protected]>>
<mailto:[email protected]
<mailto:[email protected]>
<mailto:[email protected]
<mailto:[email protected]>>>
<mailto:[email protected]
<mailto:[email protected]>
<mailto:[email protected]
<mailto:[email protected]>>
<mailto:[email protected]
<mailto:[email protected]>
<mailto:[email protected]
<mailto:[email protected]>>>>>
Betreff: Re: [Pharo-dev] binary
serialization
Yes, I have to do object substitutions.
Thanks
for the link!
thanks,
Robert
On 10/13/2015 03:43 AM, Max Leske wrote:
On 13 Oct 2015, at 09:40,
Robert Withers
<[email protected]
<mailto:[email protected]>
<mailto:[email protected]
<mailto:[email protected]>>
<mailto:[email protected]
<mailto:[email protected]>
<mailto:[email protected]
<mailto:[email protected]>>>
<mailto:[email protected]
<mailto:[email protected]>
<mailto:[email protected]
<mailto:[email protected]>>>
<mailto:[email protected]
<mailto:[email protected]>
<mailto:[email protected]
<mailto:[email protected]>>
<mailto:[email protected]
<mailto:[email protected]>
<mailto:[email protected]
<mailto:[email protected]>>>>
<mailto:[email protected]
<mailto:[email protected]>
<mailto:[email protected]
<mailto:[email protected]>>
<mailto:[email protected]
<mailto:[email protected]>
<mailto:[email protected]
<mailto:[email protected]>>>
<mailto:[email protected]
<mailto:[email protected]>
<mailto:[email protected]
<mailto:[email protected]>>
<mailto:[email protected]
<mailto:[email protected]>
<mailto:[email protected]
<mailto:[email protected]>>>>>> wrote:
Sven and Torsten, that's a binary
serialization
library! It
will take time
to learn
it and how to use
mappers.
What is the format; is it language
neutral?
For quick serialization you don’t
need to do
anything. It works
for (almost) all
objects. Only if you
want to
exclude things or
treat some
objects in a
special way, you
will need
to do some stuff.
Documentation:
http://rmod.inria.fr/web/software/Fuel.
thanks,
Robert
On 10/13/2015 01:21 AM, Sven Van
Caekenberghe
wrote:
Yes, it is called FUEL and
it is a
standard
part of the
image. See
FLSerializer
and FLMaterializer.
On 13 Oct 2015, at
06:59, Robert
Withers
<[email protected]
<mailto:[email protected]>
<mailto:[email protected]
<mailto:[email protected]>>
<mailto:[email protected]
<mailto:[email protected]>
<mailto:[email protected]
<mailto:[email protected]>>>
<mailto:[email protected]
<mailto:[email protected]>
<mailto:[email protected]
<mailto:[email protected]>>>
<mailto:[email protected]
<mailto:[email protected]>
<mailto:[email protected]
<mailto:[email protected]>>
<mailto:[email protected]
<mailto:[email protected]>
<mailto:[email protected]
<mailto:[email protected]>>>>
<mailto:[email protected]
<mailto:[email protected]>
<mailto:[email protected]
<mailto:[email protected]>>
<mailto:[email protected]
<mailto:[email protected]>
<mailto:[email protected]
<mailto:[email protected]>>>
<mailto:[email protected]
<mailto:[email protected]>
<mailto:[email protected]
<mailto:[email protected]>>
<mailto:[email protected]
<mailto:[email protected]>
<mailto:[email protected]
<mailto:[email protected]>>>>>> wrote:
Does Pharo have stream
classes to
binary
de/serialize an
object, such
that the
protocol accepts an
object as
an argument and
converts it to
a byteArray?
--
thanks,
Robert
--
Mariano
http://marianopeck.wordpress.com
--
Mariano
http://marianopeck.wordpress.com
--
Mariano
http://marianopeck.wordpress.com
<Exporting Vat.jpg><Importing Vat.jpg>
--
Mariano
http://marianopeck.wordpress.com
--
Mariano
http://marianopeck.wordpress.com
