Re: [Pharo-users] voyage mongo and transactionality

[sebastianconcept]

Hi guys,

I came to this post today due to having what can be described as an
unexpected voyage behavior and was greatly impressed by all the detailed
descriptions and explanations. Thanks everybody for that!

Also thanks Peter for mentioning that port I did back in the day (~2009) for
using OmniBase [1] in Aiflowing [2] where it achieved sub-second page
renderings with a full ACID, multi-image scalable setup. Is not actively
maintained but I think is a piece of software that is worth not having it
lost. The concept alone, filesystem based B-Tree object persistance, is neat
and can be the foundation of a lot of stuff.

Now I wonder what Esteban's fork has that couldn't be a merge request, I
would gladly take a look if open.

I feel bad about you loosing time with testBTreeKeyLocking because you won't
find the issue inside smalltalk. As it depends on the OS' file system to
detect concurrent changes in an object, you need the kernel of that system
to be supporting mandatory lockings [3] which is a feature that linux and
macOS does not have turned on by default. As mentioned in the readme of the
project you need to activate it in order to OmniBase raise an exception
preventing you of a race condition writing an object (coming it from a
transaction from another image or from another thread in that very image)
that would create db corruption or inconsistency. Note that, with mandatory
locking active in your OS kernel, the transaction that comes later will
raise an exception hence that test will pass. By the way, Windows has
mandatory locking by default.

*Back to the issue about voyage* here, I see the whole issue happening
simply because Voyage is caching by default hence forcing all Voyage users
to deal with one of the nastiest problems in computing: cache invalidation.

Yesterday I faced this issue: 
1. changed a mongo document from a mongo client and 
2. querying it from Voyage revealed different state (the state before 1).

This makes no sense at all. It makes Voyage not to be useful for
interoperation with other applications and languages, it makes it impossible
to scale an application in more than one Pharo image (all the main well
proven cloud platforms requires stateless built artifacts to horizontally
scale apps), on top of the lack of horizontal scaling issue there is the
issue of high availability that now you can't do because you are forced to
have a single point of failure (one vm behind the load balancer instead of
N).

But all those issues go away if Voyage simply does what a persistent client
is normally expected to do: save and query documents in the mongo
repository. By trying to do too much, like deciding that caching all inside
the session open in a given image is a valid design, it actually becomes
less powerful and surprises users in the worst way. Might be valid for a
prototype but certainly not for high-availability and high-load demanding
applications in production.

Caching is a great thing but is an application design decision, not a
persistency client decision. That concern is not for the client but for the
engineers who use it to make an application that might have /some/ of its
persisted models cached but certainly not all, except on very limited
scenarios. If you think about it, this is how things become niche.

Now in an application where I'm using Voyage expecting that queries return
what is actually in mongo I have to architect things in a way that
workaround this issue, like purging that cache somehow with some kind of
policy (time or a given event).

I like Voyage and I'm grateful for all the effort Esteban and the
contributors are putting into it. I might add a merge request myself. Is
documented, works well, has really nice features and is intuitive. It's just
this /cache all by default/ part that I don't think belongs to it and if
removed it would be a more powerful mongo client.



[1] https://github.com/sebastianconcept/OmniBase
[2] https://pharo.org/success/AirFlowing
[3] In *nix systems it requires mandatory file locking, so be sure you use
mand on your fstab.
https://www.hackinglinuxexposed.com/articles/20030623.html



--
Sent from: http://forum.world.st/Pharo-Smalltalk-Users-f1310670.html

Re: [Pharo-users] voyage mongo and transactionality

[jtuc...@objektfabrik.de]

Jonathan,

I was a bit destracted by work this week, so I am a little late to the 
show...


Am 10.10.19 um 12:26 schrieb Jonathan van Alteren:

Hi Joachim,

Thank you for your feedback.

It feels good to know we're not alone in this :-)


Well, I guess there are far more people in this boat than actually know 
they are ;-)



Unfortunately, the things you describe are familiar to me. My business 
partner Dave West has a lot of experience with applying a behavioral, 
'pure' object design approach. We're looking hard into simplifying 
these matters through the application of inversion of control and by 
making objects as autonomous as they can possible be.
sigh, This is really, really hard. You'd need a guard in the background 
that autonomously register all objects that get 
visited/accessed/modified during an operation and create a memento just 
before this actually happens. Ideally, this would only happen if the 
upcoming operation on that objects modifies its state once the operation 
happens. Or ot would have to clean up non-modified ones after the 
operation finished. I guess this second alternative is even feasible 
using slots and such, but there remain a few questions about 
synchronizing all this and the underlying database transactions...
At the same time, we haven't found the pot of gold at the end of the 
rainbow yet ;-)


You know, sometimes I fear it is better not to have anybody come back 
from that end of the rainbow. They might have found out there is no pot 
or anything similar to one ;-)



Nowadays, a fair amount of applications have a very direct way of 
handling user interactions and resulting state changes. For example, 
the settings of my Firefox browser doesn't have a save button anymore. 
Anything I change looks to be automatically persisted. Perhaps there 
is some value in that approach for 'enterprise' applications as well, 
although I think there will remain a lot of use cases where an 
explicit save will be needed for whatever reason.


Hmm. I am afraid there is a certain distance between changes that only 
affect a single setting value and more complex values that imply a lot 
of recomputation of values in a more complex model. Sometimes you also 
need to modify or at least check consequences of changes that don't fit 
onto a single screen, and there you need to find ways to allow a user to 
step back from what they just started to do...





Let's keep talking about this. I need to do some research on the 
memento pattern first :-) If you have any information sources you can 
point me to, I would greatly appreciate that.


I don't follow any interesting research on mementos. Too much work to do 
for that. The memento pattern itself is just "make a copy that the user 
can play with without any consequences to the original object". Works 
perfect for a single object or a well-defined object graph. But I am not 
aware of any working solutions that would solve our problems for more 
complex graphs, or even a general one... If you find something, let us know!



Joachim





Kind regards,

Jonathan van Alteren

Founding Member | Object Guild
jvalte...@objectguild.com
On 9 Oct 2019, 16:49 +0200, jtuc...@objektfabrik.de 
, wrote:


This is a tricky mine field. Sometimes you need a lot of business 
functionality in objects referenced in your objects that are 
currently in the editor. So I'm still to see a project in which the 
memento pattern really worked for more complex scenarios. How deep do 
you dive to have enough memento objects to provide the functionality 
needed. I guess you can do that with some sort of object-level 
transaction framework that automatically creates mementos of whatever 
object is being navigated to during some kind of processing-context. 
I guess slots could be of use here. But this is not trivial for 
general cases.


In my experience, this problem area makes for the other 70% of the 
time spent on developing GUI or Web applications, besides the 60% for 
GUI design and implementation and 25% business logic...


I'd be interested to learn about patterns to handle such more complex 
things. We constantly travel back and forth between implementing 
stuff in the GUI handlers (copying values to the GUI classes that 
access themselves during GUI operations and push values to the 
business objects when the users clicks on OK), using mementos (which 
most of the times are nets of mementos that are created manually - 
"we know what we'll touch in this Editor") and operating on business 
objects directly and relying on the persistence mechanism (Glorp in 
our case) and its rollback behaviour. All three have lots of 
weaknesses and seem to have their place nevertheless.


So this is a very interesting discussion and I think this is an area 
that has not been solved yet.



Joachim







Am 09.10.19 um 16:25 schrieb James Foster:
Thanks for the explanation. And, yes, this is an artifact of your 
design; if you put intermediate values into domain objects then they 

Re: [Pharo-users] voyage mongo and transactionality

[Jonathan van Alteren]

Hi Peter,

Thanks for elaborating. I found the thread on the Pharo users list and looked 
at Esteban's repository.

I'm hesitant to spend much time on it, since that is my scarcest resource at 
the moment.

You can try forking Esteban's repository and committing your modifications 
there. Then you can give them back to Esteban by doing a pull request.


Kind regards,

Jonathan van Alteren

Founding Member | Object Guild
jvalte...@objectguild.com
On 10 Oct 2019, 18:14 +0200, PBKResearch , wrote:
> Hi Jonathan
>
> If you are interested in OmniBase on Pharo, there was a port by Sebastian 
> Sastre in about 2010, which was forked and updated by Esteban Lorenzano in 
> June 2018 (https://github.com/estebanlm/OmniBase). There were a few bugs in 
> the port, which were sorted out by Matias Maretto, ma...@omeleon.de and 
> myself. I have all these corrections in my local repo, but they have never 
> been ported back to Esteban’s github repo, because I don’t know how to do it. 
> So do not use Esteban’s repo until updated.
> Esteban’s port was for Pharo 6, and in the case of marco and me was only 
> tried on 32 bits. If you decided to install on 64-bit Pharo 7, you would be 
> on new territory. The only problems are likely to be interactions with the 
> file system. As a quick test, I installed Omnibase and tests in 32-bit Pharo 
> 7. One test (#testBTreeKeyLocking) was red, all the rest green – I have not 
> yet traced the cause.
> The biggest problem is likely to be documentation. I have tried only the 
> simplest uses, because that is all I can work out from the slide-show notes. 
> Maybe someone else who has used it more extensively might still have fuller 
> documentation – e.g. Bruno, who also contributed to this thread.
>
> Peter Kenny
>
>
>
> From: Pharo-users  On Behalf Of Jonathan 
> van Alteren
> Sent: 10 October 2019 11:39
> To: Any question about pharo is welcome 
> Subject: Re: [Pharo-users] voyage mongo and transactionality
>
> Hi Peter,
>
> Thanks for your reply.
>
> That sounds very interesting. For similar reasons, I tried to check out 
> Magma. However, since I'm still a novice in Pharo/Smalltalk and it's not very 
> well documented (and mostly refers to Squeak), it's quite painful to figure 
> out how it works and how to get it going in Pharo. Not to mention actually 
> using it in production for an 'enterprise' application for an actual 
> customer... I haven't given that up yet, but currently don't have the time.
>
> I will look into OmniBase. Anything you're willing to share about getting it 
> up and running (in Pharo 7.0 64-bit) is greatly appreciated!
>
>
> Kind regards,
>
> Jonathan van Alteren
>
> Founding Member | Object Guild
> jvalte...@objectguild.com
> On 9 Oct 2019, 20:34 +0200, PBKResearch , wrote:
>
> > It may be irrelevant, but I have been playing recently with OmniBase, which 
> > is a fully object-oriented database system, now over 20 years old, but it 
> > still works very well for my uses. David Gorišek, the author, claims that 
> > it has ACID properties. From my reading, updates operate on a proxy object, 
> > which is not written to the database until an explicit commit is given. A 
> > second transaction accessing the same object will still see the original 
> > until the change is committed. What happens to a proxy which is never 
> > committed is not clear, but if Gorišek is right, the stored data can never 
> > be contaminated. I think a proxy in this sense is equivalent to a memento.
> >
> > Thanks to Esteban Lorenzano, OmniBase is now available on Pharo. The code 
> > is ancient, there is no documentation and obviously no support, but it 
> > might be worth while for someone to try some software archaeology and put 
> > it to use. I have found it possible to create and maintain a small database 
> > of natural language information, and access is fairly quick and easy – and 
> > it’s all Smalltalk.
> > It claims to store all kinds of Smalltalk objects, except block closures, 
> > and skimming through the code it seems to incorporate a serializer similar 
> > to Fuel.
> >
> > The only documentation I have found is a slideshow at 
> > https://www.slideshare.net/esug/omni-baseobjectdatabase. I have found out a 
> > few things about it, if anyone is interested.
> >
> > Peter Kenny
> >
> >
> > From: Pharo-users  On Behalf Of 
> > Norbert Hartl
> > Sent: 09 October 2019 18:08
> > To: Any question about pharo is welcome 
> > Subject: Re: [Pharo-users] voyage mongo and transactionality
> >
> >
> >
> > Am 09.10.2019 um 16:48 schrieb "jtuc...@objektfabrik.de" 
> > :
> > >
&

Re: [Pharo-users] voyage mongo and transactionality

[PBKResearch]

Hi Jonathan

 

If you are interested in OmniBase on Pharo, there was a port by Sebastian 
Sastre in about 2010, which was forked and updated by Esteban Lorenzano in June 
2018 (https://github.com/estebanlm/OmniBase). There were a few bugs in the 
port, which were sorted out by Matias Maretto, ma...@omeleon.de 
<mailto:ma...@omeleon.de>  and myself. I have all these corrections in my local 
repo, but they have never been ported back to Esteban’s github repo, because I 
don’t know how to do it. So do not use Esteban’s repo until updated.

Esteban’s port was for Pharo 6, and in the case of marco and me was only tried 
on 32 bits. If you decided to install on 64-bit Pharo 7, you would be on new 
territory. The only problems are likely to be interactions with the file 
system. As a quick test, I installed Omnibase and tests in 32-bit Pharo 7. One 
test (#testBTreeKeyLocking) was red, all the rest green – I have not yet traced 
the cause. 

The biggest problem is likely to be documentation. I have tried only the 
simplest uses, because that is all I can work out from the slide-show notes. 
Maybe someone else who has used it more extensively might still have fuller 
documentation – e.g. Bruno, who also contributed to this thread.

 

Peter Kenny

 

 

 

From: Pharo-users  On Behalf Of Jonathan 
van Alteren
Sent: 10 October 2019 11:39
To: Any question about pharo is welcome 
Subject: Re: [Pharo-users] voyage mongo and transactionality

 

Hi Peter, 

 

Thanks for your reply.

 

That sounds very interesting. For similar reasons, I tried to check out Magma. 
However, since I'm still a novice in Pharo/Smalltalk and it's not very well 
documented (and mostly refers to Squeak), it's quite painful to figure out how 
it works and how to get it going in Pharo. Not to mention actually using it in 
production for an 'enterprise' application for an actual customer... I haven't 
given that up yet, but currently don't have the time.

 

I will look into OmniBase. Anything you're willing to share about getting it up 
and running (in Pharo 7.0 64-bit) is greatly appreciated!

 

 

Kind regards,

Jonathan van Alteren

Founding Member | Object Guild
jvalte...@objectguild.com <mailto:jvalte...@objectguild.com> 

On 9 Oct 2019, 20:34 +0200, PBKResearch mailto:pe...@pbkresearch.co.uk> >, wrote:



It may be irrelevant, but I have been playing recently with OmniBase, which is 
a fully object-oriented database system, now over 20 years old, but it still 
works very well for my uses. David Gorišek, the author, claims that it has ACID 
properties. From my reading, updates operate on a proxy object, which is not 
written to the database until an explicit commit is given. A second transaction 
accessing the same object will still see the original until the change is 
committed. What happens to a proxy which is never committed is not clear, but 
if Gorišek is right, the stored data can never be contaminated. I think a proxy 
in this sense is equivalent to a memento.

 

Thanks to Esteban Lorenzano, OmniBase is now available on Pharo. The code is 
ancient, there is no documentation and obviously no support, but it might be 
worth while for someone to try some software archaeology and put it to use. I 
have found it possible to create and maintain a small database of natural 
language information, and access is fairly quick and easy – and it’s all 
Smalltalk.

It claims to store all kinds of Smalltalk objects, except block closures, and 
skimming through the code it seems to incorporate a serializer similar to Fuel.

 

The only documentation I have found is a slideshow at  
<https://www.slideshare.net/esug/omni-baseobjectdatabase> 
https://www.slideshare.net/esug/omni-baseobjectdatabase. I have found out a few 
things about it, if anyone is interested.

 

Peter Kenny

 

 

From: Pharo-users < <mailto:pharo-users-boun...@lists.pharo.org> 
pharo-users-boun...@lists.pharo.org> On Behalf Of Norbert Hartl
Sent: 09 October 2019 18:08
To: Any question about pharo is welcome < <mailto:pharo-users@lists.pharo.org> 
pharo-users@lists.pharo.org>
Subject: Re: [Pharo-users] voyage mongo and transactionality

 

 


Am 09.10.2019 um 16:48 schrieb " <mailto:jtuc...@objektfabrik.de> 
jtuc...@objektfabrik.de" < <mailto:jtuc...@objektfabrik.de> 
jtuc...@objektfabrik.de>:

 

This is a tricky mine field. Sometimes you need a lot of business functionality 
in objects referenced in your objects that are currently in the editor. So I'm 
still to see a project in which the memento pattern really worked for more 
complex scenarios. How deep do you dive to have enough memento objects to 
provide the functionality needed. I guess you can do that with some sort of 
object-level transaction framework that automatically creates mementos of 
whatever object is being navigated to during some kind of processing-context. I 
guess slots could be of use here. But this is not trivial for 

Re: [Pharo-users] voyage mongo and transactionality

[Jonathan van Alteren]

Hi,

That clears up some things, and I understand how the 'cache' works now, thanks!

I did a little bit of research on the lifecycle of a Seaside session, but 
wasn't very successful. Largely because there is a lot of different/outdated 
documentation around, including class comments in Pharo, which has caused me to 
get lost 'in the woods'.

I haven't figured out exactly how the session expiration actually operates. The 
class comment of WASession mentions #defaultTimeoutSeconds or #expire, but both 
of these seem to be absent from Seaside. The WAExpirySession class only keeps 
some counters.

Can you perhaps point me in the right direction?

For example, which object manages the Seaside sessions and where can I find 
(and influence) the expiration behavior? How can I be sure that our WASession 
is properly cleaned up so it won't prevent domain model objects from being 
'released' from the weak dictionary that is used by the Voyage 'cache'?



Kind regards,

Jonathan van Alteren

Founding Member | Object Guild
jvalte...@objectguild.com
On 10 Oct 2019, 12:55 +0200, Norbert Hartl , wrote:
> Hi,
>
>
> Am 10.10.2019 um 11:48 schrieb Jonathan van Alteren 
> :
>
> > Hi Norbert,
> >
> > Thank you very much for your extensive answer.
> >
> > For starters, we would be happy to have an option available for the course 
> > grained handling that you mention. I'd be interested to hear if there are 
> > any options besides GemStone available to be used with Voyage/MongoDB.
>
> I think Gematone has a model that serves these use cases really well. In 
> Pharo there is not much that makes your life easy. But most of this problems 
> can be solved sufficiently kust not generically. Every use case has its 
> constraints and you can model a solution for it. The main problem in voyage 
> stays that the cache is per repository. We need to think along that line if a 
> session or request cannot get its private copy of the database object.
>
> > Your mention of the Mongo cache being a weak dictionary is very 
> > interesting, I did not know that (it doesn't seem to be documented). Yes, 
> > the Seaside sessions seem to be the path through which these objects remain 
> > referenced. When I inspect all instances of our session class, I see old 
> > session objects hanging around that still reference our root application 
> > component with the rendering tree containing various references to our 
> > domain model objects.
>
> You asked Esteban about the cache. I answer that here. Everybody stumbles 
> over the name cache and Esteban is reluctant to give it another name 
> The cache is only for establishing identity in the image. The basic thing 
> with objects in an external system is that its identity (location in the 
> heap) becomes an identity parameter like an id. So when you have externalized 
> objects and you load the object with the same id twice in the image you will 
> make sure that there will only be one object for it and the object from the 
> first query and from the second query are identical meaning comparing both 
> with == returns true. So the cache keeps the database reference and the 
> business object in a weak dictionary. If the object with the same id is 
> queried from the database it returns the object it already has. If the object 
> is not referenced anymore the entry will be removed from the cache. It is 
> nothing more than this.
> >
> > Can you tell me why the session objects don't get garbage collected?
> > Should we manually clean up our session object somehow?
> >
> I‘m not sure what you mean. If the session is not referenced than it just 
> means the garbage collector did not remove it. But this won‘t keep the cache 
> from removing the entry. But seaside sessions have an expiry time. So there 
> is a duration while seaside keeps referencing the session and therefor all 
> objects that are kept in the session are still connecte keeping the objects 
> in the voyage cache.
>
> > I must admit I'm a bit out of my comfort zone here, after working in Java 
> > for close to 20 years ;-) We explicitly don't want to use any relational 
> > databases for our own application (perhaps only for integrating with 
> > customer data). I still haven't fully integrated the conceptual differences 
> > in my mental model, about how to work with objects in an image based/object 
> > persistence environment.
> >
> > I did look into using a Voyage repository filter, which is mentioned (or 
> > buried deep I should say ;-)) in this post: 
> > https://pharoweekly.wordpress.com/2017/02/20/consortium-action-13-17-feb/. 
> > What if we were to use a dynamic variable like this for each session 
> > instance? Then at least each user will have her/his own cache. But that 
> > doesn't answer the need for a kind of rollback mechanism within the same 
> > Voyage cache/session. And then there is your comment about potentially 
> > having multiple copies of an identical object...
>
> As I told in my last mail. The session or 

Re: [Pharo-users] voyage mongo and transactionality

[Smalltalk]

I have used OmniBase extensively long ago with Dolphin  Smalltalk (the 
Dolphin repository can be read & writen by Visual  Works too).


Even the Dolphin repository can be under Windows and Linux (using 
Samba). You can distribute instances ODBContainer through a LAN.


OmniBase use "multiversion concurrency control" to hold different 
version of the same object (in different transactions).


OmniBase is pretty cool but if need a lot of indexes (indexes on more 
than one inst var ) you have to solve the problem yourself using 
OmniBase b-trees.


regards

bruno

El 09/10/2019 a las 15:33, PBKResearch escribió:


It may be irrelevant, but I have been playing recently with OmniBase, 
which is a fully object-oriented database system, now over 20 years 
old, but it still works very well for my uses. David Gorišek, the 
author, claims that it has ACID properties. From my reading, updates 
operate on a proxy object, which is not written to the database until 
an explicit commit is given. A second transaction accessing the same 
object will still see the original until the change is committed. What 
happens to a proxy which is never committed is not clear, but if 
Gorišek is right, the stored data can never be contaminated. I think a 
proxy in this sense is equivalent to a memento.


Thanks to Esteban Lorenzano, OmniBase is now available on Pharo. The 
code is ancient, there is no documentation and obviously no support, 
but it might be worth while for someone to try some software 
archaeology and put it to use. I have found it possible to create and 
maintain a small database of natural language information, and access 
is fairly quick and easy – and it’s all Smalltalk.


It claims to store all kinds of Smalltalk objects, except block 
closures, and skimming through the code it seems to incorporate a 
serializer similar to Fuel.


The only documentation I have found is a slideshow at 
https://www.slideshare.net/esug/omni-baseobjectdatabase. I have found 
out a few things about it, if anyone is interested.


Peter Kenny

*From:*Pharo-users  *On Behalf Of 
*Norbert Hartl

*Sent:* 09 October 2019 18:08
*To:* Any question about pharo is welcome 
*Subject:* Re: [Pharo-users] voyage mongo and transactionality


Am 09.10.2019 um 16:48 schrieb "jtuc...@objektfabrik.de 
<mailto:jtuc...@objektfabrik.de>" <mailto:jtuc...@objektfabrik.de>>:


This is a tricky mine field. Sometimes you need a lot of business
functionality in objects referenced in your objects that are
currently in the editor. So I'm still to see a project in which
the memento pattern really worked for more complex scenarios. How
deep do you dive to have enough memento objects to provide the
functionality needed. I guess you can do that with some sort of
object-level transaction framework that automatically creates
mementos of whatever object is being navigated to during some kind
of processing-context. I guess slots could be of use here. But
this is not trivial for general cases.

Yes it is tricky. You can have copies of business objects but you have 
always references to the business objects not pointing to the copy.


And you need to know which objects should be tracked. In Gemstone IIRC 
it is easy as it is the time the object is copied from the stone to 
the gem it is registered in the current transaction. So you can check 
it and committing if it changed because you have to write it back. The 
important point here might be get noticed when a reference is 
acquired. In pharo it is not that easy but could be done if object 
would be reified and interceptable.


In my experience, this problem area makes for the other 70% of the
time spent on developing GUI or Web applications, besides the 60%
for GUI design and implementation and 25% business logic...

70% + 60% + 25% + 30% = 185%

sounds indeed very realistic if it comes to project planning. There 
is even a rule saying that for the first 90% of the project you need 
the first 90% of time and for the last 10% of the project you need the 
second 90% of time.


I'd be interested to learn about patterns to handle such more
complex things. We constantly travel back and forth between
implementing stuff in the GUI handlers (copying values to the GUI
classes that access themselves during GUI operations and push
values to the business objects when the users clicks on OK), using
mementos (which most of the times are nets of mementos that are
created manually - "we know what we'll touch in this Editor") and
operating on business objects directly and relying on the
persistence mechanism (Glorp in our case) and its rollback
behaviour. All three have lots of weaknesses and seem to have
their place nevertheless.

So this is a very interesting discussion and I think this is an
area that has not been solved yet.

I think it isn‘t solved and I find every piece of 

Re: [Pharo-users] voyage mongo and transactionality

[Norbert Hartl]

> Am 10.10.2019 um 12:31 schrieb Jonathan van Alteren 
> :
> 
> Hah, those percentages feel very real to me at the moment :-S
> 
> Can you explain what GemStone IIRC means? (Novice speaking here :-)) If 
> GemStone solves this at the moment an object goes from Stone to Gem, perhaps 
> we can take that architecture as an example and somehow apply it locally 
> within the same image? What would be required for something like that? Just 
> thinking out loud here...
> 
IIRC means „if I remember correctly“ ;)
Gemstone as the name says is a combination of gem and stone. Stone is the 
database part in the system that manages the global heap. A gem is more or less 
what a pharo vm is. When a gem opens a transaction all used objects get copied 
from the stone in the gem heap. On committing  the transaction the gem writes 
back the data to the stone. This is very brief and mostly not exactly like that 
but gives the picture

Norbert
> 
> Kind regards,
> 
> Jonathan van Alteren
> 
> Founding Member | Object Guild
> jvalte...@objectguild.com
>> On 9 Oct 2019, 19:09 +0200, Norbert Hartl , wrote:
>> 
>> 
>> Am 09.10.2019 um 16:48 schrieb "jtuc...@objektfabrik.de" 
>> :
>> 
>>> 
>>> This is a tricky mine field. Sometimes you need a lot of business 
>>> functionality in objects referenced in your objects that are currently in 
>>> the editor. So I'm still to see a project in which the memento pattern 
>>> really worked for more complex scenarios. How deep do you dive to have 
>>> enough memento objects to provide the functionality needed. I guess you can 
>>> do that with some sort of object-level transaction framework that 
>>> automatically creates mementos of whatever object is being navigated to 
>>> during some kind of processing-context. I guess slots could be of use here. 
>>> But this is not trivial for general cases.
>> 
>> Yes it is tricky. You can have copies of business objects but you have 
>> always references to the business objects not pointing to the copy. 
>> And you need to know which objects should be tracked. In Gemstone IIRC it is 
>> easy as it is the time the object is copied from the stone to the gem it is 
>> registered in the current transaction. So you can check it and committing if 
>> it changed because you have to write it back. The important point here might 
>> be get noticed when a reference is acquired. In pharo it is not that easy 
>> but could be done if object would be reified and interceptable. 
>>> In my experience, this problem area makes for the other 70% of the time 
>>> spent on developing GUI or Web applications, besides the 60% for GUI design 
>>> and implementation and 25% business logic...
>>> 
>> 70% + 60% + 25% + 30% = 185%
>> 
>> sounds indeed very realistic if it comes to project planning. There is even 
>> a rule saying that for the first 90% of the project you need the first 90% 
>> of time and for the last 10% of the project you need the second 90% of time. 
>>> I'd be interested to learn about patterns to handle such more complex 
>>> things. We constantly travel back and forth between implementing stuff in 
>>> the GUI handlers (copying values to the GUI classes that access themselves 
>>> during GUI operations and push values to the business objects when the 
>>> users clicks on OK), using mementos (which most of the times are nets of 
>>> mementos that are created manually - "we know what we'll touch in this 
>>> Editor") and operating on business objects directly and relying on the 
>>> persistence mechanism (Glorp in our case) and its rollback behaviour. All 
>>> three have lots of weaknesses and seem to have their place nevertheless.
>>> 
>>> So this is a very interesting discussion and I think this is an area that 
>>> has not been solved yet.
>>> 
>> I think it isn‘t solved and I find every piece of information about it very 
>> interesting.
>> 
>> Norbert
>>> Joachim
>>> 
>>> 
>>> 
>>> 
>>> 
>>> 
>>> 
>>> 
>>> 
>>> 
>>> 
>>> 
 Am 09.10.19 um 16:25 schrieb James Foster:
 Thanks for the explanation. And, yes, this is an artifact of your design; 
 if you put intermediate values into domain objects then they will remain 
 in your domain objects to be seen later. From what you’ve described, I 
 don’t see how it would be any different in a non-image environment (Java, 
 C#, etc.), unless you re-read the entire object graph from the database. 
 As someone else mentioned, this would be a good place for the Memento 
 Pattern.
 
 James
 
> On Oct 9, 2019, at 1:59 AM, Jonathan van Alteren 
>  wrote:
> 
> Hi James,
> 
> I see how my explanation might be unclear.
> 
> We have a main form for the agenda and a subform for an item, which is 
> shown using Seaside call/answer. The save button of the subform is 
> clicked, which adds the item to the underlying agenda model object, but 
> the save button of the main form is not clicked by the user. The callback 
> for the main save 

Re: [Pharo-users] voyage mongo and transactionality

[Norbert Hartl]

Hi,


> Am 10.10.2019 um 11:48 schrieb Jonathan van Alteren 
> :
> 
> Hi Norbert,
> 
> Thank you very much for your extensive answer.
> 
> For starters, we would be happy to have an option available for the course 
> grained handling that you mention. I'd be interested to hear if there are any 
> options besides GemStone available to be used with Voyage/MongoDB.

I think Gematone has a model that serves these use cases really well. In Pharo 
there is not much that makes your life easy. But most of this problems can be 
solved sufficiently kust not generically. Every use case has its constraints 
and you can model a solution for it. The main problem in voyage stays that the 
cache is per repository. We need to think along that line if a session or 
request cannot get its private copy of the database object.

> Your mention of the Mongo cache being a weak dictionary is very interesting, 
> I did not know that (it doesn't seem to be documented). Yes, the Seaside 
> sessions seem to be the path through which these objects remain referenced. 
> When I inspect all instances of our session class, I see old session objects 
> hanging around that still reference our root application component with the 
> rendering tree containing various references to our domain model objects.

You asked Esteban about the cache. I answer that here. Everybody stumbles over 
the name cache and Esteban is reluctant to give it another name 
The cache is only for establishing identity in the image. The basic thing with 
objects in an external system is that its identity (location in the heap) 
becomes an identity parameter like an id. So when you have externalized objects 
and you load the object with the same id twice in the image you will make sure 
that there will only be one object for it and the object from the first query 
and from the second query are identical meaning comparing both with == returns 
true. So the cache keeps the database reference and the business object in a 
weak dictionary. If the object with the same id is queried from the database it 
returns the object it already has. If the object is not referenced anymore the 
entry will be removed from the cache. It is nothing more than this.
> 
> Can you tell me why the session objects don't get garbage collected?
> Should we manually clean up our session object somehow?
> 
I‘m not sure what you mean. If the session is not referenced than it just means 
the garbage collector did not remove it. But this won‘t keep the cache from 
removing the entry. But seaside sessions have an expiry time. So there is a 
duration while seaside keeps referencing the session and therefor all objects 
that are kept in the session are still connecte keeping the objects in the 
voyage cache.

> I must admit I'm a bit out of my comfort zone here, after working in Java for 
> close to 20 years ;-) We explicitly don't want to use any relational 
> databases for our own application (perhaps only for integrating with customer 
> data). I still haven't fully integrated the conceptual differences in my 
> mental model, about how to work with objects in an image based/object 
> persistence environment.
> 
> I did look into using a Voyage repository filter, which is mentioned (or 
> buried deep I should say ;-)) in this post: 
> https://pharoweekly.wordpress.com/2017/02/20/consortium-action-13-17-feb/. 
> What if we were to use a dynamic variable like this for each session 
> instance? Then at least each user will have her/his own cache. But that 
> doesn't answer the need for a kind of rollback mechanism within the same 
> Voyage cache/session. And then there is your comment about potentially having 
> multiple copies of an identical object...

As I told in my last mail. The session or request based cache has also problems 
because if you attach an object you queried while in the session/request to 
somewhere more global you introduce subtle bugs. Maybe we need a two level 
cache, one that knows all objects referenced in the image and one that keeps 
the private copies based on the image based cache. This way we could know that 
multiple sessions/request are keeping a copy of the same object and we could 
determine merge conflicts. Need to think more about this
> 
> I wonder what patterns other Seaside 'enterprise' application developers are 
> using.
> 

> You are right, the term object transactionality doesn't make much sense ;-) 
> We are not using Magritte (and probably won't). I don't know much about 
> Magritte, but it feels like it might be incompatible with the behavioral, 
> 'pure' object approach that we want to use. However, I am interested to 
> investigate. Can you recommend any good documentation sources for learning 
> Magritte from an application architecture perspective?
> 
Magritte is a meta description of your objects. It is like having powerful 
annotation objects. These can be used to create different views on your model. 
There is the PhD paper of Lukas Renggli on the net and 

Re: [Pharo-users] voyage mongo and transactionality

[Jonathan van Alteren]

Hi Peter,

Thanks for your reply.

That sounds very interesting. For similar reasons, I tried to check out Magma. 
However, since I'm still a novice in Pharo/Smalltalk and it's not very well 
documented (and mostly refers to Squeak), it's quite painful to figure out how 
it works and how to get it going in Pharo. Not to mention actually using it in 
production for an 'enterprise' application for an actual customer... I haven't 
given that up yet, but currently don't have the time.

I will look into OmniBase. Anything you're willing to share about getting it up 
and running (in Pharo 7.0 64-bit) is greatly appreciated!


Kind regards,

Jonathan van Alteren

Founding Member | Object Guild
jvalte...@objectguild.com
On 9 Oct 2019, 20:34 +0200, PBKResearch , wrote:
> It may be irrelevant, but I have been playing recently with OmniBase, which 
> is a fully object-oriented database system, now over 20 years old, but it 
> still works very well for my uses. David Gorišek, the author, claims that it 
> has ACID properties. From my reading, updates operate on a proxy object, 
> which is not written to the database until an explicit commit is given. A 
> second transaction accessing the same object will still see the original 
> until the change is committed. What happens to a proxy which is never 
> committed is not clear, but if Gorišek is right, the stored data can never be 
> contaminated. I think a proxy in this sense is equivalent to a memento.
>
> Thanks to Esteban Lorenzano, OmniBase is now available on Pharo. The code is 
> ancient, there is no documentation and obviously no support, but it might be 
> worth while for someone to try some software archaeology and put it to use. I 
> have found it possible to create and maintain a small database of natural 
> language information, and access is fairly quick and easy – and it’s all 
> Smalltalk.
> It claims to store all kinds of Smalltalk objects, except block closures, and 
> skimming through the code it seems to incorporate a serializer similar to 
> Fuel.
>
> The only documentation I have found is a slideshow at 
> https://www.slideshare.net/esug/omni-baseobjectdatabase. I have found out a 
> few things about it, if anyone is interested.
>
> Peter Kenny
>
>
> From: Pharo-users  On Behalf Of Norbert 
> Hartl
> Sent: 09 October 2019 18:08
> To: Any question about pharo is welcome 
> Subject: Re: [Pharo-users] voyage mongo and transactionality
>
>
>
> Am 09.10.2019 um 16:48 schrieb "jtuc...@objektfabrik.de" 
> :
> >
> > This is a tricky mine field. Sometimes you need a lot of business 
> > functionality in objects referenced in your objects that are currently in 
> > the editor. So I'm still to see a project in which the memento pattern 
> > really worked for more complex scenarios. How deep do you dive to have 
> > enough memento objects to provide the functionality needed. I guess you can 
> > do that with some sort of object-level transaction framework that 
> > automatically creates mementos of whatever object is being navigated to 
> > during some kind of processing-context. I guess slots could be of use here. 
> > But this is not trivial for general cases.
>
> Yes it is tricky. You can have copies of business objects but you have always 
> references to the business objects not pointing to the copy.
> And you need to know which objects should be tracked. In Gemstone IIRC it is 
> easy as it is the time the object is copied from the stone to the gem it is 
> registered in the current transaction. So you can check it and committing if 
> it changed because you have to write it back. The important point here might 
> be get noticed when a reference is acquired. In pharo it is not that easy but 
> could be done if object would be reified and interceptable.
> > In my experience, this problem area makes for the other 70% of the time 
> > spent on developing GUI or Web applications, besides the 60% for GUI design 
> > and implementation and 25% business logic...
> 70% + 60% + 25% + 30% = 185%
>
> sounds indeed very realistic if it comes to project planning. There is even 
> a rule saying that for the first 90% of the project you need the first 90% of 
> time and for the last 10% of the project you need the second 90% of time.
>
> > I'd be interested to learn about patterns to handle such more complex 
> > things. We constantly travel back and forth between implementing stuff in 
> > the GUI handlers (copying values to the GUI classes that access themselves 
> > during GUI operations and push values to the business objects when the 
> > users clicks on OK), using mementos (which most of the times are nets of 
> > mementos that are created manually - "we know what we'll touch in thi

Re: [Pharo-users] voyage mongo and transactionality

[Jonathan van Alteren]

Hah, those percentages feel very real to me at the moment :-S

Can you explain what GemStone IIRC means? (Novice speaking here :-)) If 
GemStone solves this at the moment an object goes from Stone to Gem, perhaps we 
can take that architecture as an example and somehow apply it locally within 
the same image? What would be required for something like that? Just thinking 
out loud here...


Kind regards,

Jonathan van Alteren

Founding Member | Object Guild
jvalte...@objectguild.com
On 9 Oct 2019, 19:09 +0200, Norbert Hartl , wrote:
>
>
> Am 09.10.2019 um 16:48 schrieb "jtuc...@objektfabrik.de" 
> :
>
> >
> > This is a tricky mine field. Sometimes you need a lot of business 
> > functionality in objects referenced in your objects that are currently in 
> > the editor. So I'm still to see a project in which the memento pattern 
> > really worked for more complex scenarios. How deep do you dive to have 
> > enough memento objects to provide the functionality needed. I guess you can 
> > do that with some sort of object-level transaction framework that 
> > automatically creates mementos of whatever object is being navigated to 
> > during some kind of processing-context. I guess slots could be of use here. 
> > But this is not trivial for general cases.
>
> Yes it is tricky. You can have copies of business objects but you have always 
> references to the business objects not pointing to the copy.
> And you need to know which objects should be tracked. In Gemstone IIRC it is 
> easy as it is the time the object is copied from the stone to the gem it is 
> registered in the current transaction. So you can check it and committing if 
> it changed because you have to write it back. The important point here might 
> be get noticed when a reference is acquired. In pharo it is not that easy but 
> could be done if object would be reified and interceptable.
> > In my experience, this problem area makes for the other 70% of the time 
> > spent on developing GUI or Web applications, besides the 60% for GUI design 
> > and implementation and 25% business logic...
> 70% + 60% + 25% + 30% = 185%
>
> sounds indeed very realistic if it comes to project planning. There is even 
> a rule saying that for the first 90% of the project you need the first 90% of 
> time and for the last 10% of the project you need the second 90% of time.
> > I'd be interested to learn about patterns to handle such more complex 
> > things. We constantly travel back and forth between implementing stuff in 
> > the GUI handlers (copying values to the GUI classes that access themselves 
> > during GUI operations and push values to the business objects when the 
> > users clicks on OK), using mementos (which most of the times are nets of 
> > mementos that are created manually - "we know what we'll touch in this 
> > Editor") and operating on business objects directly and relying on the 
> > persistence mechanism (Glorp in our case) and its rollback behaviour. All 
> > three have lots of weaknesses and seem to have their place nevertheless.
> > So this is a very interesting discussion and I think this is an area that 
> > has not been solved yet.
> I think it isn‘t solved and I find every piece of information about it very 
> interesting.
>
> Norbert
> > Joachim
> >
> >
> >
> >
> >
> >
> > Am 09.10.19 um 16:25 schrieb James Foster:
> > > Thanks for the explanation. And, yes, this is an artifact of your design; 
> > > if you put intermediate values into domain objects then they will remain 
> > > in your domain objects to be seen later. From what you’ve described, I 
> > > don’t see how it would be any different in a non-image environment (Java, 
> > > C#, etc.), unless you re-read the entire object graph from the database. 
> > > As someone else mentioned, this would be a good place for the Memento 
> > > Pattern.
> > >
> > > James
> > >
> > > > On Oct 9, 2019, at 1:59 AM, Jonathan van Alteren 
> > > >  wrote:
> > > >
> > > > Hi James,
> > > >
> > > > I see how my explanation might be unclear.
> > > >
> > > > We have a main form for the agenda and a subform for an item, which is 
> > > > shown using Seaside call/answer. The save button of the subform is 
> > > > clicked, which adds the item to the underlying agenda model object, but 
> > > > the save button of the main form is not clicked by the user. The 
> > > > callback for the main save button sends the save message to the agenda 
> > > > object, causing the database to be updated.
> > > >
> > > > So yes, the browser does submit the data on the subform, it's the main 
> > > > form component that doesn't receive the save button callback. I realize 
> > > > that this is in large part an issue with our design. However, the way 
> > > > object persistence seems to work in the image environment plays a large 
> > > > role.
> > > >
> > > >
> > > > Kind regards,
> > > >
> > > > Jonathan van Alteren
> > > >
> > > > Founding Member | Object Guild
> > > > jvalte...@objectguild.com
> > > > On 8 Oct 2019, 

Re: [Pharo-users] voyage mongo and transactionality

[Jonathan van Alteren]

Hi Joachim,

Thank you for your feedback.

It feels good to know we're not alone in this :-) Unfortunately, the things you 
describe are familiar to me. My business partner Dave West has a lot of 
experience with applying a behavioral, 'pure' object design approach. We're 
looking hard into simplifying these matters through the application of 
inversion of control and by making objects as autonomous as they can possible 
be. At the same time, we haven't found the pot of gold at the end of the 
rainbow yet ;-)

Nowadays, a fair amount of applications have a very direct way of handling user 
interactions and resulting state changes. For example, the settings of my 
Firefox browser doesn't have a save button anymore. Anything I change looks to 
be automatically persisted. Perhaps there is some value in that approach for 
'enterprise' applications as well, although I think there will remain a lot of 
use cases where an explicit save will be needed for whatever reason.

Let's keep talking about this. I need to do some research on the memento 
pattern first :-) If you have any information sources you can point me to, I 
would greatly appreciate that.


Kind regards,

Jonathan van Alteren

Founding Member | Object Guild
jvalte...@objectguild.com
On 9 Oct 2019, 16:49 +0200, jtuc...@objektfabrik.de , 
wrote:
>
> This is a tricky mine field. Sometimes you need a lot of business 
> functionality in objects referenced in your objects that are currently in the 
> editor. So I'm still to see a project in which the memento pattern really 
> worked for more complex scenarios. How deep do you dive to have enough 
> memento objects to provide the functionality needed. I guess you can do that 
> with some sort of object-level transaction framework that automatically 
> creates mementos of whatever object is being navigated to during some kind of 
> processing-context. I guess slots could be of use here. But this is not 
> trivial for general cases.
> In my experience, this problem area makes for the other 70% of the time spent 
> on developing GUI or Web applications, besides the 60% for GUI design and 
> implementation and 25% business logic...
> I'd be interested to learn about patterns to handle such more complex things. 
> We constantly travel back and forth between implementing stuff in the GUI 
> handlers (copying values to the GUI classes that access themselves during GUI 
> operations and push values to the business objects when the users clicks on 
> OK), using mementos (which most of the times are nets of mementos that are 
> created manually - "we know what we'll touch in this Editor") and operating 
> on business objects directly and relying on the persistence mechanism (Glorp 
> in our case) and its rollback behaviour. All three have lots of weaknesses 
> and seem to have their place nevertheless.
> So this is a very interesting discussion and I think this is an area that has 
> not been solved yet.
>
> Joachim
>
>
>
>
>
>
> Am 09.10.19 um 16:25 schrieb James Foster:
> > Thanks for the explanation. And, yes, this is an artifact of your design; 
> > if you put intermediate values into domain objects then they will remain in 
> > your domain objects to be seen later. From what you’ve described, I don’t 
> > see how it would be any different in a non-image environment (Java, C#, 
> > etc.), unless you re-read the entire object graph from the database. As 
> > someone else mentioned, this would be a good place for the Memento Pattern.
> >
> > James
> >
> > > On Oct 9, 2019, at 1:59 AM, Jonathan van Alteren 
> > >  wrote:
> > >
> > > Hi James,
> > >
> > > I see how my explanation might be unclear.
> > >
> > > We have a main form for the agenda and a subform for an item, which is 
> > > shown using Seaside call/answer. The save button of the subform is 
> > > clicked, which adds the item to the underlying agenda model object, but 
> > > the save button of the main form is not clicked by the user. The callback 
> > > for the main save button sends the save message to the agenda object, 
> > > causing the database to be updated.
> > >
> > > So yes, the browser does submit the data on the subform, it's the main 
> > > form component that doesn't receive the save button callback. I realize 
> > > that this is in large part an issue with our design. However, the way 
> > > object persistence seems to work in the image environment plays a large 
> > > role.
> > >
> > >
> > > Kind regards,
> > >
> > > Jonathan van Alteren
> > >
> > > Founding Member | Object Guild
> > > jvalte...@objectguild.com
> > > On 8 Oct 2019, 15:41 +0200, James Foster , wrote:
> > > >
> > > > > On Oct 8, 2019, at 3:05 AM, Jonathan van Alteren 
> > > > >  wrote:
> > > > >
> > > > > We've encountered an issue where a user makes changes to an agenda, 
> > > > > but does not click the Save button. Instead, the user closes the 
> > > > > browser or uses the navigation to go to a different part of the 
> > > > > application. When navigating back to the 

Re: [Pharo-users] voyage mongo and transactionality

[Jonathan van Alteren]

Hi James,

Thank you for your feedback.

If I remember correctly, in Java a persistence framework like JPA/Hibernate 
would track 'dirty' objects somehow and handle caching together with 
transaction management. There might be some things we might apply from those 
frameworks in Pharo/Smalltalk. However, our challenge is that we don't have 
much time to look at this because we need it to develop customer applications 
;-)

I will look into the Memento Pattern, thanks!


Kind regards,

Jonathan van Alteren

Founding Member | Object Guild
jvalte...@objectguild.com
On 9 Oct 2019, 16:26 +0200, James Foster , wrote:
> Thanks for the explanation. And, yes, this is an artifact of your design; if 
> you put intermediate values into domain objects then they will remain in your 
> domain objects to be seen later. From what you’ve described, I don’t see how 
> it would be any different in a non-image environment (Java, C#, etc.), unless 
> you re-read the entire object graph from the database. As someone else 
> mentioned, this would be a good place for the Memento Pattern.
>
> James
>
> > On Oct 9, 2019, at 1:59 AM, Jonathan van Alteren 
> >  wrote:
> >
> > Hi James,
> >
> > I see how my explanation might be unclear.
> >
> > We have a main form for the agenda and a subform for an item, which is 
> > shown using Seaside call/answer. The save button of the subform is clicked, 
> > which adds the item to the underlying agenda model object, but the save 
> > button of the main form is not clicked by the user. The callback for the 
> > main save button sends the save message to the agenda object, causing the 
> > database to be updated.
> >
> > So yes, the browser does submit the data on the subform, it's the main form 
> > component that doesn't receive the save button callback. I realize that 
> > this is in large part an issue with our design. However, the way object 
> > persistence seems to work in the image environment plays a large role.
> >
> >
> > Kind regards,
> >
> > Jonathan van Alteren
> >
> > Founding Member | Object Guild
> > jvalte...@objectguild.com
> > On 8 Oct 2019, 15:41 +0200, James Foster , wrote:
> > >
> > > > On Oct 8, 2019, at 3:05 AM, Jonathan van Alteren 
> > > >  wrote:
> > > >
> > > > We've encountered an issue where a user makes changes to an agenda, but 
> > > > does not click the Save button. Instead, the user closes the browser or 
> > > > uses the navigation to go to a different part of the application. When 
> > > > navigating back to the original agenda, the changes made previously 
> > > > (e.g. items added) are still being displayed, even though they were 
> > > > never explicitly saved.
> > >
> > > Here is what I don’t understand: how did the change get from the user’s 
> > > client agent (browser) to the server? If you make a change to a field in 
> > > a form and then close the browser, who sent the change to the server? If 
> > > you show the save domain value in a different location, with a 
> > > dynamically-generated id and name (so it isn’t cached in the browser), or 
> > > written to the Pharo Transcript, does the value still change? That is, 
> > > are you sure that the change is in the reflected in the Smalltalk image 
> > > and not just somehow cached in the browser?
> > >
> > > James
> > >
> > >
>

Re: [Pharo-users] voyage mongo and transactionality

[Jonathan van Alteren]

Hi Estaban,

Thanks for your feedback.

I agree that there might not be any real problem with Voyage. And yes, I think 
it was some kind of in-memory transaction handling that is was looking for.

Can you explain a bit more about how the cache in Voyage works and why it is 
the way it is?

The application in question doesn't have a lot of users, so changes are we 
could do without caching and still have good performance. I was used to working 
with JPA/Hibernate in Java, which doesn't seem to have the issue that we run 
into. Any idea why it works differently with Voyage/MongoDB? Is that because 
MongoDB doesn't support database transactions, hence missing a good trigger to 
flush the cache?

Making copies of domain model objects somehow 'feels wrong'. When I have a 
person in my CRM domain, I feel strongly that it should always ever be 
represented by 1 object instance of class Person.

The search continues... :-)


Kind regards,

Jonathan van Alteren

Founding Member | Object Guild
jvalte...@objectguild.com
On 8 Oct 2019, 12:54 +0200, Esteban Lorenzano , wrote:
> Hi,
>
> That looks a lot more like a Seaside mishandling than a problem with Voyage 
> itself.
>
> > On 8 Oct 2019, at 12:05, Jonathan van Alteren  
> > wrote:
> >
> > Hello all,
> >
> > We are having some issues with using Voyage/Mongo for a customer project 
> > that I'd like to get your feedback on.
> >
> > The customer application is a form based business web application using 
> > Seaside with object persistence using Voyage with MongoDB on Pharo 7.0. The 
> > application is deployed on a dedicated Linux production server running 
> > MongoDB version 4.2.
> >
> > The application is used to manage meeting agendas and minutes. After 
> > opening the agenda view of a future meeting, the user can add an item to 
> > the agenda by clicking a button. This calls an item editor component which 
> > answers when a Save or Cancel button is clicked. The agenda view component 
> > itself also has a Save button, which performs a Voyage save of the object 
> > aggregate (agenda + items).
> >
> > We've encountered an issue where a user makes changes to an agenda, but 
> > does not click the Save button. Instead, the user closes the browser or 
> > uses the navigation to go to a different part of the application. When 
> > navigating back to the original agenda, the changes made previously (e.g. 
> > items added) are still being displayed, even though they were never 
> > explicitly saved.
> >
> > It does not matter if we select the agenda aggregate object instance using 
> > Voyage or access it in a different way. Changes to the state of the object 
> > are retained, even though a Voyage save was never sent to the agenda 
> > instance. The cause seems to be that the Voyage repository caches the 
> > object instance and thus on select, it returns an object that is in a 
> > different state than how it was persisted.
> >
> > This all seems to come down to object transactionality.
> >
> > We have a need to 'cancel' changes made to an object instance. Before 
> > working with Pharo/Smalltalk in a non-image based environment, I was used 
> > to do this by retrieving the original object from persistence again. This 
> > also allowed for a convenient way to detect changes to an object's state, 
> > which we are missing at the moment too.
>
> In Voyage, new objects are just persisted (hence they enter the cache) once 
> you do the first “save”.
> Retrieved objects are indeed in the cache, and since there is just one copy 
> for Pharo instance, if you do an update of a property (even if you do not 
> save), the cached instance will have the new attribute value, since there is 
> no in-memory transaction handling in Voyage.
>
> Which means: you need to handle it by your self. One solution is to work on a 
> copy and on “save” flush the changes before (it is easy to develop methods to 
> automatise this).
>
> Well, something like that :)
>
> >
> > We know that moving to GemStone can help us with these issues, but our 
> > current planning of customer projects does not allow us to do this within 
> > the next 3 months. And we really need to find a way to tackle these issues .
>
> Well, not really, you will still need to copy modified values, retrieve new 
> and apply your changes.
> Of course if you want to discard possible in-memory changes, yes, you can 
> cancel the transaction in gemstone… and you can also re-read the object in 
> Voyage (no special api for that, even if we could add it… you need to 
> re-execute a query for it).
>
> Esteban
>
> >
> >
> > Your feedback is greatly appreciated!
> >
> >
> >
> > Kind regards,
> >
> >
> > Jonathan van Alteren
> >
> > Founding Member | Object Guild
> > jvalte...@objectguild.com
> >
>
>

Re: [Pharo-users] voyage mongo and transactionality

[Jonathan van Alteren]

Hi Norbert,

Thank you very much for your extensive answer.

For starters, we would be happy to have an option available for the course 
grained handling that you mention. I'd be interested to hear if there are any 
options besides GemStone available to be used with Voyage/MongoDB.

Your mention of the Mongo cache being a weak dictionary is very interesting, I 
did not know that (it doesn't seem to be documented). Yes, the Seaside sessions 
seem to be the path through which these objects remain referenced. When I 
inspect all instances of our session class, I see old session objects hanging 
around that still reference our root application component with the rendering 
tree containing various references to our domain model objects.

Can you tell me why the session objects don't get garbage collected?
Should we manually clean up our session object somehow?

I must admit I'm a bit out of my comfort zone here, after working in Java for 
close to 20 years ;-) We explicitly don't want to use any relational databases 
for our own application (perhaps only for integrating with customer data). I 
still haven't fully integrated the conceptual differences in my mental model, 
about how to work with objects in an image based/object persistence environment.

I did look into using a Voyage repository filter, which is mentioned (or buried 
deep I should say ;-)) in this post: 
https://pharoweekly.wordpress.com/2017/02/20/consortium-action-13-17-feb/. What 
if we were to use a dynamic variable like this for each session instance? Then 
at least each user will have her/his own cache. But that doesn't answer the 
need for a kind of rollback mechanism within the same Voyage cache/session. And 
then there is your comment about potentially having multiple copies of an 
identical object...

I wonder what patterns other Seaside 'enterprise' application developers are 
using.

You are right, the term object transactionality doesn't make much sense ;-) We 
are not using Magritte (and probably won't). I don't know much about Magritte, 
but it feels like it might be incompatible with the behavioral, 'pure' object 
approach that we want to use. However, I am interested to investigate. Can you 
recommend any good documentation sources for learning Magritte from an 
application architecture perspective?

Also, I'm interested to hear more about the modification tracking approach you 
are working on. Please drop me a personal note if you are willing to 
collaborate on this.


Kind regards,

Jonathan van Alteren

Founding Member | Object Guild
jvalte...@objectguild.com
On 8 Oct 2019, 12:54 +0200, Norbert Hartl , wrote:
> Hi,
>
> > Am 08.10.2019 um 12:05 schrieb Jonathan van Alteren 
> > :
> >
> > Hello all,
> >
> > We are having some issues with using Voyage/Mongo for a customer project 
> > that I'd like to get your feedback on.
> >
> > The customer application is a form based business web application using 
> > Seaside with object persistence using Voyage with MongoDB on Pharo 7.0. The 
> > application is deployed on a dedicated Linux production server running 
> > MongoDB version 4.2.
> >
> > The application is used to manage meeting agendas and minutes. After 
> > opening the agenda view of a future meeting, the user can add an item to 
> > the agenda by clicking a button. This calls an item editor component which 
> > answers when a Save or Cancel button is clicked. The agenda view component 
> > itself also has a Save button, which performs a Voyage save of the object 
> > aggregate (agenda + items).
> >
> > We've encountered an issue where a user makes changes to an agenda, but 
> > does not click the Save button. Instead, the user closes the browser or 
> > uses the navigation to go to a different part of the application. When 
> > navigating back to the original agenda, the changes made previously (e.g. 
> > items added) are still being displayed, even though they were never 
> > explicitly saved.
> >
> > It does not matter if we select the agenda aggregate object instance using 
> > Voyage or access it in a different way. Changes to the state of the object 
> > are retained, even though a Voyage save was never sent to the agenda 
> > instance. The cause seems to be that the Voyage repository caches the 
> > object instance and thus on select, it returns an object that is in a 
> > different state than how it was persisted.
> >
> > This all seems to come down to object transactionality.
> >
> > We have a need to 'cancel' changes made to an object instance. Before 
> > working with Pharo/Smalltalk in a non-image based environment, I was used 
> > to do this by retrieving the original object from persistence again. This 
> > also allowed for a convenient way to detect changes to an object's state, 
> > which we are missing at the moment too.
> >
> > We know that moving to GemStone can help us with these issues, but our 
> > current planning of customer projects does not allow us to do this within 
> > the next 3 months. And we 

Re: [Pharo-users] voyage mongo and transactionality

[PBKResearch]

It may be irrelevant, but I have been playing recently with OmniBase, which is 
a fully object-oriented database system, now over 20 years old, but it still 
works very well for my uses. David Gorišek, the author, claims that it has ACID 
properties. From my reading, updates operate on a proxy object, which is not 
written to the database until an explicit commit is given. A second transaction 
accessing the same object will still see the original until the change is 
committed. What happens to a proxy which is never committed is not clear, but 
if Gorišek is right, the stored data can never be contaminated. I think a proxy 
in this sense is equivalent to a memento.

 

Thanks to Esteban Lorenzano, OmniBase is now available on Pharo. The code is 
ancient, there is no documentation and obviously no support, but it might be 
worth while for someone to try some software archaeology and put it to use. I 
have found it possible to create and maintain a small database of natural 
language information, and access is fairly quick and easy – and it’s all 
Smalltalk.

It claims to store all kinds of Smalltalk objects, except block closures, and 
skimming through the code it seems to incorporate a serializer similar to Fuel.

 

The only documentation I have found is a slideshow at 
https://www.slideshare.net/esug/omni-baseobjectdatabase. I have found out a few 
things about it, if anyone is interested.

 

Peter Kenny

 

 

From: Pharo-users  On Behalf Of Norbert 
Hartl
Sent: 09 October 2019 18:08
To: Any question about pharo is welcome 
Subject: Re: [Pharo-users] voyage mongo and transactionality

 

 


Am 09.10.2019 um 16:48 schrieb "jtuc...@objektfabrik.de 
<mailto:jtuc...@objektfabrik.de> " mailto:jtuc...@objektfabrik.de> >:

 

This is a tricky mine field. Sometimes you need a lot of business functionality 
in objects referenced in your objects that are currently in the editor. So I'm 
still to see a project in which the memento pattern really worked for more 
complex scenarios. How deep do you dive to have enough memento objects to 
provide the functionality needed. I guess you can do that with some sort of 
object-level transaction framework that automatically creates mementos of 
whatever object is being navigated to during some kind of processing-context. I 
guess slots could be of use here. But this is not trivial for general cases.

 

Yes it is tricky. You can have copies of business objects but you have always 
references to the business objects not pointing to the copy. 

And you need to know which objects should be tracked. In Gemstone IIRC it is 
easy as it is the time the object is copied from the stone to the gem it is 
registered in the current transaction. So you can check it and committing if it 
changed because you have to write it back. The important point here might be 
get noticed when a reference is acquired. In pharo it is not that easy but 
could be done if object would be reified and interceptable. 

In my experience, this problem area makes for the other 70% of the time spent 
on developing GUI or Web applications, besides the 60% for GUI design and 
implementation and 25% business logic...

70% + 60% + 25% + 30% = 185%

 

sounds indeed very realistic if it comes to project planning. There is even a 
rule saying that for the first 90% of the project you need the first 90% of 
time and for the last 10% of the project you need the second 90% of time. 



I'd be interested to learn about patterns to handle such more complex things. 
We constantly travel back and forth between implementing stuff in the GUI 
handlers (copying values to the GUI classes that access themselves during GUI 
operations and push values to the business objects when the users clicks on 
OK), using mementos (which most of the times are nets of mementos that are 
created manually - "we know what we'll touch in this Editor") and operating on 
business objects directly and relying on the persistence mechanism (Glorp in 
our case) and its rollback behaviour. All three have lots of weaknesses and 
seem to have their place nevertheless.

So this is a very interesting discussion and I think this is an area that has 
not been solved yet.

I think it isn‘t solved and I find every piece of information about it very 
interesting.

 

Norbert



Joachim

 

 

 

 

 

 

Am 09.10.19 um 16:25 schrieb James Foster:

Thanks for the explanation. And, yes, this is an artifact of your design; if 
you put intermediate values into domain objects then they will remain in your 
domain objects to be seen later. From what you’ve described, I don’t see how it 
would be any different in a non-image environment (Java, C#, etc.), unless you 
re-read the entire object graph from the database. As someone else mentioned, 
this would be a good place for the Memento Pattern. 

 

James





On Oct 9, 2019, at 1:59 AM, Jonathan van Alteren mailto:jvalte...@objectguild.com> > wrote:

 

Hi James, 

 

I s

Re: [Pharo-users] voyage mongo and transactionality

[Norbert Hartl]

> Am 09.10.2019 um 16:48 schrieb "jtuc...@objektfabrik.de" 
> :
> 
> 
> This is a tricky mine field. Sometimes you need a lot of business 
> functionality in objects referenced in your objects that are currently in the 
> editor. So I'm still to see a project in which the memento pattern really 
> worked for more complex scenarios. How deep do you dive to have enough 
> memento objects to provide the functionality needed. I guess you can do that 
> with some sort of object-level transaction framework that automatically 
> creates mementos of whatever object is being navigated to during some kind of 
> processing-context. I guess slots could be of use here. But this is not 
> trivial for general cases.

Yes it is tricky. You can have copies of business objects but you have always 
references to the business objects not pointing to the copy. 
And you need to know which objects should be tracked. In Gemstone IIRC it is 
easy as it is the time the object is copied from the stone to the gem it is 
registered in the current transaction. So you can check it and committing if it 
changed because you have to write it back. The important point here might be 
get noticed when a reference is acquired. In pharo it is not that easy but 
could be done if object would be reified and interceptable. 
> In my experience, this problem area makes for the other 70% of the time spent 
> on developing GUI or Web applications, besides the 60% for GUI design and 
> implementation and 25% business logic...
> 
70% + 60% + 25% + 30% = 185%

sounds indeed very realistic if it comes to project planning. There is even a 
rule saying that for the first 90% of the project you need the first 90% of 
time and for the last 10% of the project you need the second 90% of time. 
> I'd be interested to learn about patterns to handle such more complex things. 
> We constantly travel back and forth between implementing stuff in the GUI 
> handlers (copying values to the GUI classes that access themselves during GUI 
> operations and push values to the business objects when the users clicks on 
> OK), using mementos (which most of the times are nets of mementos that are 
> created manually - "we know what we'll touch in this Editor") and operating 
> on business objects directly and relying on the persistence mechanism (Glorp 
> in our case) and its rollback   behaviour. All three have lots of 
> weaknesses and seem to have their place nevertheless.
> 
> So this is a very interesting discussion and I think this is an area that has 
> not been solved yet.
> 
I think it isn‘t solved and I find every piece of information about it very 
interesting.

Norbert
> Joachim
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
>> Am 09.10.19 um 16:25 schrieb James Foster:
>> Thanks for the explanation. And, yes, this is an artifact of your design; if 
>> you put intermediate values into domain objects then they will remain in 
>> your domain objects to be seen later. From what you’ve described, I don’t 
>> see how it would be any different in a non-image environment (Java, C#, 
>> etc.), unless you re-read the entire object graph from the database. As 
>> someone else mentioned, this would be a good place for the Memento Pattern.
>> 
>> James
>> 
>>> On Oct 9, 2019, at 1:59 AM, Jonathan van Alteren 
>>>  wrote:
>>> 
>>> Hi James,
>>> 
>>> I see how my explanation might be unclear.
>>> 
>>> We have a main form for the agenda and a subform for an item, which is 
>>> shown using Seaside call/answer. The save button of the subform is clicked, 
>>> which adds the item to the underlying agenda model object, but the save 
>>> button of the main form is not clicked by the user. The callback for the 
>>> main save button sends the save message to the agenda object, causing the 
>>> database to be updated.
>>> 
>>> So yes, the browser does submit the data on the subform, it's the main form 
>>> component that doesn't receive the save button callback. I realize that 
>>> this is in large part an issue with our design. However, the way object 
>>> persistence seems to work in the image environment plays a large role. 
>>> 
>>> 
>>> Kind regards,
>>> 
>>> Jonathan van Alteren
>>> 
>>> Founding Member | Object Guild
>>> jvalte...@objectguild.com
 On 8 Oct 2019, 15:41 +0200, James Foster , wrote:
 
> On Oct 8, 2019, at 3:05 AM, Jonathan van Alteren 
>  wrote:
> 
> We've encountered an issue where a user makes changes to an agenda, but 
> does not click the Save button. Instead, the user closes the browser or 
> uses the navigation to go to a different part of the application. When 
> navigating back to the original agenda, the changes made previously (e.g. 
>   items added) are still being displayed, even
>though they were never explicitly saved.
 
 Here is what I don’t understand: how did the change get from the user’s 
 client agent (browser) to the server? If you make a change to a 

Re: [Pharo-users] voyage mongo and transactionality

[jtuc...@objektfabrik.de]

Am 09.10.19 um 16:48 schrieb jtuc...@objektfabrik.de:


In my experience, this problem area makes for the other 70% of the 
time spent on developing GUI or Web applications, besides the 60% for 
GUI design and implementation and 25% business logic...



I forgot the 30% for O/R Mapping when you use an ORM ;-)


Joachim


--
---
Objektfabrik Joachim Tuchel  mailto:jtuc...@objektfabrik.de
Fliederweg 1 http://www.objektfabrik.de
D-71640 Ludwigsburg  http://joachimtuchel.wordpress.com
Telefon: +49 7141 56 10 86 0 Fax: +49 7141 56 10 86 1

Re: [Pharo-users] voyage mongo and transactionality

[jtuc...@objektfabrik.de]

This is a tricky mine field. Sometimes you need a lot of business 
functionality in objects referenced in your objects that are currently 
in the editor. So I'm still to see a project in which the memento 
pattern really worked for more complex scenarios. How deep do you dive 
to have enough memento objects to provide the functionality needed. I 
guess you can do that with some sort of object-level transaction 
framework that automatically creates mementos of whatever object is 
being navigated to during some kind of processing-context. I guess slots 
could be of use here. But this is not trivial for general cases.


In my experience, this problem area makes for the other 70% of the time 
spent on developing GUI or Web applications, besides the 60% for GUI 
design and implementation and 25% business logic...


I'd be interested to learn about patterns to handle such more complex 
things. We constantly travel back and forth between implementing stuff 
in the GUI handlers (copying values to the GUI classes that access 
themselves during GUI operations and push values to the business objects 
when the users clicks on OK), using mementos (which most of the times 
are nets of mementos that are created manually - "we know what we'll 
touch in this Editor") and operating on business objects directly and 
relying on the persistence mechanism (Glorp in our case) and its 
rollback behaviour. All three have lots of weaknesses and seem to have 
their place nevertheless.


So this is a very interesting discussion and I think this is an area 
that has not been solved yet.



Joachim







Am 09.10.19 um 16:25 schrieb James Foster:
Thanks for the explanation. And, yes, this is an artifact of your 
design; if you put intermediate values into domain objects then they 
will remain in your domain objects to be seen later. From what you’ve 
described, I don’t see how it would be any different in a non-image 
environment (Java, C#, etc.), unless you re-read the entire object 
graph from the database. As someone else mentioned, this would be a 
good place for the Memento Pattern.


James

On Oct 9, 2019, at 1:59 AM, Jonathan van Alteren 
mailto:jvalte...@objectguild.com>> wrote:


Hi James,

I see how my explanation might be unclear.

We have a main form for the agenda and a subform for an item, which 
is shown using Seaside call/answer. The save button of the subform is 
clicked, which adds the item to the underlying agenda model object, 
but the save button of the main form _is not_ clicked by the user. 
The callback for the main save button sends the save message to the 
agenda object, causing the database to be updated.


So yes, the browser does submit the data on the subform, it's the 
main form component that doesn't receive the save button callback. I 
realize that this is in large part an issue with our design. However, 
the way object persistence seems to work in the image environment 
plays a large role.



Kind regards,

Jonathan van Alteren

Founding Member | Object Guild
jvalte...@objectguild.com 
On 8 Oct 2019, 15:41 +0200, James Foster >, wrote:


On Oct 8, 2019, at 3:05 AM, Jonathan van Alteren 
mailto:jvalte...@objectguild.com>> wrote:


We've encountered an issue where a user makes changes to an agenda, 
but does not click the Save button. Instead, the user closes the 
browser or uses the navigation to go to a different part of the 
application. When navigating back to the original agenda, the 
changes made previously (e.g. items added) are still being 
displayed, even though they were never explicitly saved.


Here is what I don’t understand: how did the change get from the 
user’s client agent (browser) to the server? If you make a change to 
a field in a form and then close the browser, who sent the change to 
the server? If you show the save domain value in a different 
location, with a dynamically-generated id and name (so it isn’t 
cached in the browser), or written to the Pharo Transcript, does the 
value still change? That is, are you sure that the change is in the 
reflected in the Smalltalk image and not just somehow cached in the 
browser?


James






--
---
Objektfabrik Joachim Tuchel  mailto:jtuc...@objektfabrik.de
Fliederweg 1 http://www.objektfabrik.de
D-71640 Ludwigsburg  http://joachimtuchel.wordpress.com
Telefon: +49 7141 56 10 86 0 Fax: +49 7141 56 10 86 1

Re: [Pharo-users] voyage mongo and transactionality

[James Foster]

Thanks for the explanation. And, yes, this is an artifact of your design; if 
you put intermediate values into domain objects then they will remain in your 
domain objects to be seen later. From what you’ve described, I don’t see how it 
would be any different in a non-image environment (Java, C#, etc.), unless you 
re-read the entire object graph from the database. As someone else mentioned, 
this would be a good place for the Memento Pattern.

James

> On Oct 9, 2019, at 1:59 AM, Jonathan van Alteren  
> wrote:
> 
> Hi James,
> 
> I see how my explanation might be unclear.
> 
> We have a main form for the agenda and a subform for an item, which is shown 
> using Seaside call/answer. The save button of the subform is clicked, which 
> adds the item to the underlying agenda model object, but the save button of 
> the main form is not clicked by the user. The callback for the main save 
> button sends the save message to the agenda object, causing the database to 
> be updated.
> 
> So yes, the browser does submit the data on the subform, it's the main form 
> component that doesn't receive the save button callback. I realize that this 
> is in large part an issue with our design. However, the way object 
> persistence seems to work in the image environment plays a large role. 
> 
> 
> Kind regards,
> 
> Jonathan van Alteren
> 
> Founding Member | Object Guild
> jvalte...@objectguild.com
> On 8 Oct 2019, 15:41 +0200, James Foster , wrote:
>> 
>>> On Oct 8, 2019, at 3:05 AM, Jonathan van Alteren 
>>>  wrote:
>>> 
>>> We've encountered an issue where a user makes changes to an agenda, but 
>>> does not click the Save button. Instead, the user closes the browser or 
>>> uses the navigation to go to a different part of the application. When 
>>> navigating back to the original agenda, the changes made previously (e.g. 
>>> items added) are still being displayed, even though they were never 
>>> explicitly saved.
>> 
>> Here is what I don’t understand: how did the change get from the user’s 
>> client agent (browser) to the server? If you make a change to a field in a 
>> form and then close the browser, who sent the change to the server? If you 
>> show the save domain value in a different location, with a 
>> dynamically-generated id and name (so it isn’t cached in the browser), or 
>> written to the Pharo Transcript, does the value still change? That is, are 
>> you sure that the change is in the reflected in the Smalltalk image and not 
>> just somehow cached in the browser?
>> 
>> James
>> 
>> 

Re: [Pharo-users] voyage mongo and transactionality

[Jonathan van Alteren]

Hi James,

I see how my explanation might be unclear.

We have a main form for the agenda and a subform for an item, which is shown 
using Seaside call/answer. The save button of the subform is clicked, which 
adds the item to the underlying agenda model object, but the save button of the 
main form is not clicked by the user. The callback for the main save button 
sends the save message to the agenda object, causing the database to be updated.

So yes, the browser does submit the data on the subform, it's the main form 
component that doesn't receive the save button callback. I realize that this is 
in large part an issue with our design. However, the way object persistence 
seems to work in the image environment plays a large role.


Kind regards,

Jonathan van Alteren

Founding Member | Object Guild
jvalte...@objectguild.com
On 8 Oct 2019, 15:41 +0200, James Foster , wrote:
>
> > On Oct 8, 2019, at 3:05 AM, Jonathan van Alteren 
> >  wrote:
> >
> > We've encountered an issue where a user makes changes to an agenda, but 
> > does not click the Save button. Instead, the user closes the browser or 
> > uses the navigation to go to a different part of the application. When 
> > navigating back to the original agenda, the changes made previously (e.g. 
> > items added) are still being displayed, even though they were never 
> > explicitly saved.
>
> Here is what I don’t understand: how did the change get from the user’s 
> client agent (browser) to the server? If you make a change to a field in a 
> form and then close the browser, who sent the change to the server? If you 
> show the save domain value in a different location, with a 
> dynamically-generated id and name (so it isn’t cached in the browser), or 
> written to the Pharo Transcript, does the value still change? That is, are 
> you sure that the change is in the reflected in the Smalltalk image and not 
> just somehow cached in the browser?
>
> James
>
>

Re: [Pharo-users] voyage mongo and transactionality

[James Foster]

> On Oct 8, 2019, at 3:05 AM, Jonathan van Alteren  
> wrote:
> 
> We've encountered an issue where a user makes changes to an agenda, but does 
> not click the Save button. Instead, the user closes the browser or uses the 
> navigation to go to a different part of the application. When navigating back 
> to the original agenda, the changes made previously (e.g. items added) are 
> still being displayed, even though they were never explicitly saved.

Here is what I don’t understand: how did the change get from the user’s client 
agent (browser) to the server? If you make a change to a field in a form and 
then close the browser, who sent the change to the server? If you show the save 
domain value in a different location, with a dynamically-generated id and name 
(so it isn’t cached in the browser), or written to the Pharo Transcript, does 
the value still change? That is, are you sure that the change is in the 
reflected in the Smalltalk image and not just somehow cached in the browser?

James

Re: [Pharo-users] voyage mongo and transactionality

[Norbert Hartl]

Hi,

> Am 08.10.2019 um 12:05 schrieb Jonathan van Alteren 
> :
> 
> Hello all,
> 
> We are having some issues with using Voyage/Mongo for a customer project that 
> I'd like to get your feedback on. 
> 
> The customer application is a form based business web application using 
> Seaside with object persistence using Voyage with MongoDB on Pharo 7.0. The 
> application is deployed on a dedicated Linux production server running 
> MongoDB version 4.2.
> 
> The application is used to manage meeting agendas and minutes. After opening 
> the agenda view of a future meeting, the user can add an item to the agenda 
> by clicking a button. This calls an item editor component which answers when 
> a Save or Cancel button is clicked. The agenda view component itself also has 
> a Save button, which performs a Voyage save of the object aggregate (agenda + 
> items).
> 
> We've encountered an issue where a user makes changes to an agenda, but does 
> not click the Save button. Instead, the user closes the browser or uses the 
> navigation to go to a different part of the application. When navigating back 
> to the original agenda, the changes made previously (e.g. items added) are 
> still being displayed, even though they were never explicitly saved.
> 
> It does not matter if we select the agenda aggregate object instance using 
> Voyage or access it in a different way. Changes to the state of the object 
> are retained, even though a Voyage save was never sent to the agenda 
> instance. The cause seems to be that the Voyage repository caches the object 
> instance and thus on select, it returns an object that is in a different 
> state than how it was persisted.
> 
> This all seems to come down to object transactionality. 
> 
> We have a need to 'cancel' changes made to an object instance. Before working 
> with Pharo/Smalltalk in a non-image based environment, I was used to do this 
> by retrieving the original object from persistence again. This also allowed 
> for a convenient way to detect changes to an object's state, which we are 
> missing at the moment too.
> 
> We know that moving to GemStone can help us with these issues, but our 
> current planning of customer projects does not allow us to do this within the 
> next 3 months. And we really need to find a way to tackle these issues .
> 
> 
> Your feedback is greatly appreciated!
> 
this is none to be confusing to a lot of people. If you map a memory object 
graph to a database there are no intrinsic points of transactions unless you 
put them into your application model. Gemstone or any transaction based 
handling won‘t help you if your use case is not as coarse grained as discarding 
all modified data and not just some. Or you need to hop into nested 
transactions which are supported by Gemstone IMHO but there is rules to care 
about, too, if you want to use them.

Regarding your application you modify data and this is kept. The cache in mongo 
is a weak dictionary. This means that if you get the object with the changes a 
second time it means these objects are referenced somewhere. Seaside sessions 
seems to be the obvious thing here. If you have a reachable object (e.g. server 
singleton -> handler -> seaside -> seaside session -> business object) then 
mongo keeps the object in its cache because it needs to be able to establish 
identity on further queries. Ein emoving all the changes from an object you 
keep in memory wouldn‘t be really good if the persistence layer would do it. 

The problem is obvious but the solution is not. We could make the cache session 
or request based which would circumvent the problem. But if an object would be 
attached to something outside the session/request which you cannot forbid it 
causes real problems like multiple copies to an identical object.

I don‘t know what you mean with object transactionality because it makes no 
sense in my head. But you have basically two approaches to solve the problem. 
If you would use magritte you would have it at hand. There each UI component 
gets just a memento of the business object for modification. You need to commit 
the UI component in order to modify the business object. So you can commit on 
user interaction and separately on persistence which allows a huge set of use 
cases.
The other one I‘m working on since a while is a proper modification tracking. 
In this you change the objects directly but have a registry with the dirty 
objects which you can act upon. I have a sample modification tracker that uses 
readOnly object write barrier for objects. On write attempt it registers the 
object and the write operation as command pattern. You can undo modifications 
of a single object where all commmands are undone on that object. Or you abort 
all modifications at once. It is basically working but has the problems on 
concurrent access. As the mongo cache is image wide two concurrent requests 
interfere with each other.
So you can see that the problem is not that easy to solve.

Re: [Pharo-users] voyage mongo and transactionality

[Esteban Lorenzano]

Hi, 

That looks a lot more like a Seaside mishandling than a problem with Voyage 
itself. 

> On 8 Oct 2019, at 12:05, Jonathan van Alteren  
> wrote:
> 
> Hello all,
> 
> We are having some issues with using Voyage/Mongo for a customer project that 
> I'd like to get your feedback on. 
> 
> The customer application is a form based business web application using 
> Seaside with object persistence using Voyage with MongoDB on Pharo 7.0. The 
> application is deployed on a dedicated Linux production server running 
> MongoDB version 4.2.
> 
> The application is used to manage meeting agendas and minutes. After opening 
> the agenda view of a future meeting, the user can add an item to the agenda 
> by clicking a button. This calls an item editor component which answers when 
> a Save or Cancel button is clicked. The agenda view component itself also has 
> a Save button, which performs a Voyage save of the object aggregate (agenda + 
> items).
> 
> We've encountered an issue where a user makes changes to an agenda, but does 
> not click the Save button. Instead, the user closes the browser or uses the 
> navigation to go to a different part of the application. When navigating back 
> to the original agenda, the changes made previously (e.g. items added) are 
> still being displayed, even though they were never explicitly saved.
> 
> It does not matter if we select the agenda aggregate object instance using 
> Voyage or access it in a different way. Changes to the state of the object 
> are retained, even though a Voyage save was never sent to the agenda 
> instance. The cause seems to be that the Voyage repository caches the object 
> instance and thus on select, it returns an object that is in a different 
> state than how it was persisted.
> 
> This all seems to come down to object transactionality. 
> 
> We have a need to 'cancel' changes made to an object instance. Before working 
> with Pharo/Smalltalk in a non-image based environment, I was used to do this 
> by retrieving the original object from persistence again. This also allowed 
> for a convenient way to detect changes to an object's state, which we are 
> missing at the moment too.

In Voyage, new objects are just persisted (hence they enter the cache) once you 
do the first “save”. 
Retrieved objects are indeed in the cache, and since there is just one copy for 
Pharo instance, if you do an update of a property (even if you do not save), 
the cached instance will have the new attribute value, since there is no 
in-memory transaction handling in Voyage.

Which means: you need to handle it by your self. One solution is to work on a 
copy and on “save” flush the changes before (it is easy to develop methods to 
automatise this). 

Well, something like that :)

> 
> We know that moving to GemStone can help us with these issues, but our 
> current planning of customer projects does not allow us to do this within the 
> next 3 months. And we really need to find a way to tackle these issues .

Well, not really, you will still need to copy modified values, retrieve new and 
apply your changes. 
Of course if you want to discard possible in-memory changes, yes, you can 
cancel the transaction in gemstone… and you can also re-read the object in 
Voyage (no special api for that, even if we could add it… you need to 
re-execute a query for it).

Esteban

> 
> 
> Your feedback is greatly appreciated!
> 
> 
> 
> Kind regards,
> 
> 
> Jonathan van Alteren
> 
> Founding Member | Object Guild
> jvalte...@objectguild.com
> 

[Pharo-users] voyage mongo and transactionality

[Jonathan van Alteren]

Hello all,

We are having some issues with using Voyage/Mongo for a customer project that 
I'd like to get your feedback on.

The customer application is a form based business web application using Seaside 
with object persistence using Voyage with MongoDB on Pharo 7.0. The application 
is deployed on a dedicated Linux production server running MongoDB version 4.2.

The application is used to manage meeting agendas and minutes. After opening 
the agenda view of a future meeting, the user can add an item to the agenda by 
clicking a button. This calls an item editor component which answers when a 
Save or Cancel button is clicked. The agenda view component itself also has a 
Save button, which performs a Voyage save of the object aggregate (agenda + 
items).

We've encountered an issue where a user makes changes to an agenda, but does 
not click the Save button. Instead, the user closes the browser or uses the 
navigation to go to a different part of the application. When navigating back 
to the original agenda, the changes made previously (e.g. items added) are 
still being displayed, even though they were never explicitly saved.

It does not matter if we select the agenda aggregate object instance using 
Voyage or access it in a different way. Changes to the state of the object are 
retained, even though a Voyage save was never sent to the agenda instance. The 
cause seems to be that the Voyage repository caches the object instance and 
thus on select, it returns an object that is in a different state than how it 
was persisted.

This all seems to come down to object transactionality.

We have a need to 'cancel' changes made to an object instance. Before working 
with Pharo/Smalltalk in a non-image based environment, I was used to do this by 
retrieving the original object from persistence again. This also allowed for a 
convenient way to detect changes to an object's state, which we are missing at 
the moment too.

We know that moving to GemStone can help us with these issues, but our current 
planning of customer projects does not allow us to do this within the next 3 
months. And we really need to find a way to tackle these issues .


Your feedback is greatly appreciated!



Kind regards,


Jonathan van Alteren

Founding Member | Object Guild
jvalte...@objectguild.com