Re: EOF inserts already existing M:N relationships/empty snapshot?!?

[o...@ocs.cz]

Chuck,

> On 21. 9. 2016, at 10:47 PM, Chuck Hill <ch...@gevityinc.com> wrote:
> 
> Ooh.  I had forgotten about that.  Does that qualify as a Heisenbug?

Pretty much, I guess.

Matter of fact, after I found the culprit in one of my test clones, I have 
fixed it in the main application... ... ...and the fix did not work. What?!? 
D'oh: of course, it could not work, for there still were all those snapshot 
logs here and there, through which I have tried to observe the snapshot 
creation progress; only after I have removed *all* committedSnapshotForObject's 
it started to work properly :)

Incidentally, finding the culprit was sort of at the funny side too.

First I have found in one clone that removing an audit call on user login, it 
helps. What?!?

Then, going through the audit code, I found the actual culprit was

===
switch (arg) { // which arg happened to be the user EO
  case 'sometext': ...something...
}
===

WHAT?!? Well ... in Groovy, a switch is a bit more flexible than in Java/C; it 
effectively boils down to

===
if ('sometext'.equals(arg.toString())) ...something...
===

which indeed causes snapshots not to be filled... well W*H*A*T?!? How can 
this... but then, I do override toString for my EOs after all; and eventually I 
found that the overridden method in some cases _would_ call 
committedSnapshotForObject, so that the EO logs out how it differs from the 
saved state. O frabjous day!

Incidentally, is there some simple test which could be used to determine 
whether at the place and moment and particular eo it is safe to call 
committedSnapshotForObject or not?

Thanks and all the best,
OC

> From: "o...@ocs.cz" <o...@ocs.cz>
> Date: Wednesday, September 21, 2016 at 1:42 PM
> To: Chuck Hill <ch...@gevityinc.com>
> Cc: "webobjects-dev@lists.apple.com WebObjects" 
> <webobjects-dev@lists.apple.com>
> Subject: Re: EOF inserts already existing M:N relationships/empty snapshot?!?
>  
> Chuck,
>  
> On 21. 9. 2016, at 6:10 AM, Chuck Hill <ch...@gevityinc.com> wrote:
> I bet Alice was a developer too.
>  
> Beware the EOF, my son!
> The jaws that bite, the claws that catch!
>  
> Looks like I have found the culprit — seems it was the very log of the 
> committed snapshot. It looks like that if one calls e.g.,
>  
> eo.editingContext().committedSnapshotForObject(eo)
>  
> at the wrong moment — which did happen to me originally (solely for the 
> logging purposes, darnit!), and the more logs I have added, the worse it got 
> — one of the not-quite-presumed side-effects of that would be that the 
> snapshots do not get properly recorded anymore for eo. Quadruple weird!
>  
> When I have removed all the committedSnapshotForObject calls, it starts to 
> run properly for all the M:N's, without a glitch. Adding logs immediately 
> pre-saveChanges (it seems to be safe at the moment), the snapshot proved to 
> be right; and saveChanges sends only the inserts it should.
>  
> Matter of fact, we even have bumped to (another presentation of) the very 
> same problem some time ago, to wit:
>  
> On 28. 2. 2015, at 7:54 PM, Chuck Hill <ch...@gevityinc.com> wrote:
> Ah, Schrödinger’s EOF. :-P  Observing some things can materially change the 
> state of EOF.   Calling this method calls ec.committedSnapshotForObject() 
> which causes the EC to record the committed snapshot for the object at that 
> point in time.  Normally this is done only at the point the EO first changes.
>  
> Whew!
>  
> Thanks a very big lot for all the help,
> OC
>  
>  


 ___
Do not post admin requests to the list. They will be ignored.
Webobjects-dev mailing list  (Webobjects-dev@lists.apple.com)
Help/Unsubscribe/Update your Subscription:
https://lists.apple.com/mailman/options/webobjects-dev/archive%40mail-archive.com

This email sent to arch...@mail-archive.com

Re: EOF inserts already existing M:N relationships/empty snapshot?!?

[o...@ocs.cz]

Chuck,

> On 21. 9. 2016, at 6:10 AM, Chuck Hill  wrote:
> I bet Alice was a developer too.

Beware the EOF, my son!
The jaws that bite, the claws that catch!

Looks like I have found the culprit — seems it was the very log of the 
committed snapshot. It looks like that if one calls e.g.,

> eo.editingContext().committedSnapshotForObject(eo)

at the wrong moment — which did happen to me originally (solely for the logging 
purposes, darnit!), and the more logs I have added, the worse it got — one of 
the not-quite-presumed side-effects of that would be that the snapshots do not 
get properly recorded anymore for eo. Quadruple weird!

When I have removed all the committedSnapshotForObject calls, it starts to run 
properly for all the M:N's, without a glitch. Adding logs immediately 
pre-saveChanges (it seems to be safe at the moment), the snapshot proved to be 
right; and saveChanges sends only the inserts it should.

Matter of fact, we even have bumped to (another presentation of) the very same 
problem some time ago, to wit:

> On 28. 2. 2015, at 7:54 PM, Chuck Hill  wrote:
> 
> Ah, Schrödinger’s EOF. :-P  Observing some things can materially change the 
> state of EOF.   Calling this method calls ec.committedSnapshotForObject() 
> which causes the EC to record the committed snapshot for the object at that 
> point in time.  Normally this is done only at the point the EO first changes.

Whew!

Thanks a very big lot for all the help,
OC


 ___
Do not post admin requests to the list. They will be ignored.
Webobjects-dev mailing list  (Webobjects-dev@lists.apple.com)
Help/Unsubscribe/Update your Subscription:
https://lists.apple.com/mailman/options/webobjects-dev/archive%40mail-archive.com

This email sent to arch...@mail-archive.com

Re: EOF inserts already existing M:N relationships/empty snapshot?!?

[Chuck Hill]

Ooh.  I had forgotten about that.  Does that qualify as a Heisenbug?



From: "o...@ocs.cz" <o...@ocs.cz>
Date: Wednesday, September 21, 2016 at 1:42 PM
To: Chuck Hill <ch...@gevityinc.com>
Cc: "webobjects-dev@lists.apple.com WebObjects" <webobjects-dev@lists.apple.com>
Subject: Re: EOF inserts already existing M:N relationships/empty snapshot?!?

Chuck,

On 21. 9. 2016, at 6:10 AM, Chuck Hill 
<ch...@gevityinc.com<mailto:ch...@gevityinc.com>> wrote:
I bet Alice was a developer too.

Beware the EOF, my son!
The jaws that bite, the claws that catch!

Looks like I have found the culprit — seems it was the very log of the 
committed snapshot. It looks like that if one calls e.g.,

eo.editingContext().committedSnapshotForObject(eo)

at the wrong moment — which did happen to me originally (solely for the logging 
purposes, darnit!), and the more logs I have added, the worse it got — one of 
the not-quite-presumed side-effects of that would be that the snapshots do not 
get properly recorded anymore for eo. Quadruple weird!

When I have removed all the committedSnapshotForObject calls, it starts to run 
properly for all the M:N's, without a glitch. Adding logs immediately 
pre-saveChanges (it seems to be safe at the moment), the snapshot proved to be 
right; and saveChanges sends only the inserts it should.

Matter of fact, we even have bumped to (another presentation of) the very same 
problem some time ago, to wit:

On 28. 2. 2015, at 7:54 PM, Chuck Hill 
<ch...@gevityinc.com<mailto:ch...@gevityinc.com>> wrote:
Ah, Schrödinger’s EOF. :-P  Observing some things can materially change the 
state of EOF.   Calling this method calls ec.committedSnapshotForObject() which 
causes the EC to record the committed snapshot for the object at that point in 
time.  Normally this is done only at the point the EO first changes.

Whew!

Thanks a very big lot for all the help,
OC


 ___
Do not post admin requests to the list. They will be ignored.
Webobjects-dev mailing list  (Webobjects-dev@lists.apple.com)
Help/Unsubscribe/Update your Subscription:
https://lists.apple.com/mailman/options/webobjects-dev/archive%40mail-archive.com

This email sent to arch...@mail-archive.com

Re: EOF inserts already existing M:N relationships/empty snapshot?!?

[o...@ocs.cz]

Chuck,

> On 21. 9. 2016, at 6:10 AM, Chuck Hill <ch...@gevityinc.com> wrote:
> I bet Alice was a developer too.

:)
 
> Have you overridden any of the EOF methods, or the generated methods in User 
> or DataBlock and changed the normal behavior?  Or forgotten to call super?

Note please that there's more M:N's in that application, and other ones work 
all right (although, as outlined below, their changes never occur in the 
DatabaseContextDelegate.databaseContextShouldUpdateCurrentSnapshot method; the 
snapshot though is all right).

As for overridden methods, let me see...

- my EO root is an OCSEnterpriseObject (which extends ERXGenericRecord), and 
overrides
-- awakeFromInsertion to set up creation date; calls super
-- validateForSave; calls super (and does not change the eo :))
-- handleQueryWithUnboundKey and handleTakeValueForUnboundKey: these do not 
call super (which is all right I think)
-- toString (does not call super either)
- DataBlock (extends OCSEnterpriseObject) does not override anything (and works 
all right with other M:N's)
- User ditto

No accessors are implemented explicitly in DataBlock or User[*].

That aside, I install a DatabaseContextDelegate. It extends 
ERXDatabaseContextDelegate, and overrides 
databaseContextWillPerformAdaptorOperations to log what is about to happen with 
the database and databaseContextFailedToFetchObject, again to log which 
object/gid did fail. None of them calls super. Temporarily I tried to override 
databaseContextShouldUpdateCurrentSnapshot to log when and how the snapshot is 
updated, but with M:N's it does not seem to work -- I have never got any M:N in 
its new dict (although most M:N's work all right and the appropriate snapshots, 
logged just-before-save, contain the proper eos).

Can I track somehow (e.g., through some delegate methods/notifications) when 
and how exactly the snapshot :N's gets updated? Looks like 
databaseContextShouldUpdateCurrentSnapshot is not the right way to do that. At 
least, my last attempt obtained all the :N names from the entity and would log 
out if any of them is either in the old or in the new dict -- but never logs 
anything (whilst attributes and :1 changes go through here all right).

Thanks and all the best,
OC

[*] There are no generated sources like _User or _DataBlock in my setup; the 
accessor methods are injected launch-time into User/DataBlock/all other EO 
classes based on model contents. In my personal opinion, this approach 
(conceptually copied down from Core Data) is much cleaner and considerably less 
error-prone than generated sources. Should not be relevant to this problem, I 
think.


>  
> From: OC <o...@ocs.cz>
> Date: Tuesday, September 20, 2016 at 6:33 PM
> To: Chuck Hill <ch...@gevityinc.com>
> Cc: "webobjects-dev@lists.apple.com WebObjects" 
> <webobjects-dev@lists.apple.com>
> Subject: Re: EOF inserts already existing M:N relationships/empty snapshot?!?
>  
> Chuck,
>  
> On 21. 9. 2016, at 2:40, o...@ocs.cz wrote:
>  
> Aside of that, I have added a log into the 
> DatabaseContextDelegate.databaseContextShouldUpdateCurrentSnapshot method, 
> and here I *never* get *anything* for the relationships: neither the old 
> (which is presumable) *nor the new* dictionary ever contain the 
> “userDataBlocks” or “dataBlockUsers” key — not once.
>  
> well this one's weird, or I am missing something of importance here.
>  
> Meantime, I have played with another M:N, which works without a glitch (and 
> which is modelled the same way as that one which does not, incidentally). 
> Here, save-time, the snapshot does contain the proper related objects (and 
> thus proper INSERTs are generated).
>  
> The weird thing is that these objects *do not* come into the snapshot through 
> the DatabaseContextDelegate.databaseContextShouldUpdateCurrentSnapshot 
> method! It logs all right when the object is fetched, but the new (nor the 
> old, of course) dictionary does *not* contain the relationship key at all -- 
> precisely the same way it is with that relationship which does not work 
> properly!
>  
> Nevertheless, as mentioned above, save-time, the snapshot is all right. 
> Couriouser and couriouser, Alice would say :-O
>  
> Thanks and all the best,
> OC
>  
>  


 ___
Do not post admin requests to the list. They will be ignored.
Webobjects-dev mailing list  (Webobjects-dev@lists.apple.com)
Help/Unsubscribe/Update your Subscription:
https://lists.apple.com/mailman/options/webobjects-dev/archive%40mail-archive.com

This email sent to arch...@mail-archive.com

Re: EOF inserts already existing M:N relationships/empty snapshot?!?

[Chuck Hill]

I bet Alice was a developer too.

Have you overridden any of the EOF methods, or the generated methods in User or 
DataBlock and changed the normal behavior?  Or forgotten to call super?

From: OC <o...@ocs.cz>
Date: Tuesday, September 20, 2016 at 6:33 PM
To: Chuck Hill <ch...@gevityinc.com>
Cc: "webobjects-dev@lists.apple.com WebObjects" <webobjects-dev@lists.apple.com>
Subject: Re: EOF inserts already existing M:N relationships/empty snapshot?!?

Chuck,

On 21. 9. 2016, at 2:40, o...@ocs.cz<mailto:o...@ocs.cz> wrote:

Aside of that, I have added a log into the 
DatabaseContextDelegate.databaseContextShouldUpdateCurrentSnapshot method, and 
here I *never* get *anything* for the relationships: neither the old (which is 
presumable) *nor the new* dictionary ever contain the “userDataBlocks” or 
“dataBlockUsers” key — not once.

well this one's weird, or I am missing something of importance here.

Meantime, I have played with another M:N, which works without a glitch (and 
which is modelled the same way as that one which does not, incidentally). Here, 
save-time, the snapshot does contain the proper related objects (and thus 
proper INSERTs are generated).

The weird thing is that these objects *do not* come into the snapshot through 
the DatabaseContextDelegate.databaseContextShouldUpdateCurrentSnapshot method! 
It logs all right when the object is fetched, but the new (nor the old, of 
course) dictionary does *not* contain the relationship key at all -- precisely 
the same way it is with that relationship which does not work properly!

Nevertheless, as mentioned above, save-time, the snapshot is all right. 
Couriouser and couriouser, Alice would say :-O

Thanks and all the best,
OC


 ___
Do not post admin requests to the list. They will be ignored.
Webobjects-dev mailing list  (Webobjects-dev@lists.apple.com)
Help/Unsubscribe/Update your Subscription:
https://lists.apple.com/mailman/options/webobjects-dev/archive%40mail-archive.com

This email sent to arch...@mail-archive.com

Re: EOF inserts already existing M:N relationships/empty snapshot?!?

[OC]

Chuck,

all my PKs/FKs are the same:
- "INTEGER" for SQL type
- "NSNumber" for class name
- "i" for value type

Thanks a big lot!
OC

On 21. 9. 2016, at 3:40, Chuck Hill <ch...@gevityinc.com> wrote:

> Off the top of my head, check they types of the keys to, as defined in the 
> EOModel, to the M:N table and in the M:N table.  Are all consistently Integer 
> or Long in Java?
>  
> Chuck
>  
>  
> From: OC <o...@ocs.cz>
> Date: Tuesday, September 20, 2016 at 6:33 PM
> To: Chuck Hill <ch...@gevityinc.com>
> Cc: "webobjects-dev@lists.apple.com WebObjects" 
> <webobjects-dev@lists.apple.com>
> Subject: Re: EOF inserts already existing M:N relationships/empty snapshot?!?
>  
> Chuck,
>  
> On 21. 9. 2016, at 2:40, o...@ocs.cz wrote:
>  
> Aside of that, I have added a log into the 
> DatabaseContextDelegate.databaseContextShouldUpdateCurrentSnapshot method, 
> and here I *never* get *anything* for the relationships: neither the old 
> (which is presumable) *nor the new* dictionary ever contain the 
> “userDataBlocks” or “dataBlockUsers” key — not once.
>  
> well this one's weird, or I am missing something of importance here.
>  
> Meantime, I have played with another M:N, which works without a glitch (and 
> which is modelled the same way as that one which does not, incidentally). 
> Here, save-time, the snapshot does contain the proper related objects (and 
> thus proper INSERTs are generated).
>  
> The weird thing is that these objects *do not* come into the snapshot through 
> the DatabaseContextDelegate.databaseContextShouldUpdateCurrentSnapshot 
> method! It logs all right when the object is fetched, but the new (nor the 
> old, of course) dictionary does *not* contain the relationship key at all -- 
> precisely the same way it is with that relationship which does not work 
> properly!
>  
> Nevertheless, as mentioned above, save-time, the snapshot is all right. 
> Couriouser and couriouser, Alice would say :-O
>  
> Thanks and all the best,
> OC
>  
>  


 ___
Do not post admin requests to the list. They will be ignored.
Webobjects-dev mailing list  (Webobjects-dev@lists.apple.com)
Help/Unsubscribe/Update your Subscription:
https://lists.apple.com/mailman/options/webobjects-dev/archive%40mail-archive.com

This email sent to arch...@mail-archive.com

Re: EOF inserts already existing M:N relationships/empty snapshot?!?

[Chuck Hill]

Off the top of my head, check they types of the keys to, as defined in the 
EOModel, to the M:N table and in the M:N table.  Are all consistently Integer 
or Long in Java?

Chuck


From: OC <o...@ocs.cz>
Date: Tuesday, September 20, 2016 at 6:33 PM
To: Chuck Hill <ch...@gevityinc.com>
Cc: "webobjects-dev@lists.apple.com WebObjects" <webobjects-dev@lists.apple.com>
Subject: Re: EOF inserts already existing M:N relationships/empty snapshot?!?

Chuck,

On 21. 9. 2016, at 2:40, o...@ocs.cz<mailto:o...@ocs.cz> wrote:

Aside of that, I have added a log into the 
DatabaseContextDelegate.databaseContextShouldUpdateCurrentSnapshot method, and 
here I *never* get *anything* for the relationships: neither the old (which is 
presumable) *nor the new* dictionary ever contain the “userDataBlocks” or 
“dataBlockUsers” key — not once.

well this one's weird, or I am missing something of importance here.

Meantime, I have played with another M:N, which works without a glitch (and 
which is modelled the same way as that one which does not, incidentally). Here, 
save-time, the snapshot does contain the proper related objects (and thus 
proper INSERTs are generated).

The weird thing is that these objects *do not* come into the snapshot through 
the DatabaseContextDelegate.databaseContextShouldUpdateCurrentSnapshot method! 
It logs all right when the object is fetched, but the new (nor the old, of 
course) dictionary does *not* contain the relationship key at all -- precisely 
the same way it is with that relationship which does not work properly!

Nevertheless, as mentioned above, save-time, the snapshot is all right. 
Couriouser and couriouser, Alice would say :-O

Thanks and all the best,
OC


 ___
Do not post admin requests to the list. They will be ignored.
Webobjects-dev mailing list  (Webobjects-dev@lists.apple.com)
Help/Unsubscribe/Update your Subscription:
https://lists.apple.com/mailman/options/webobjects-dev/archive%40mail-archive.com

This email sent to arch...@mail-archive.com

Re: EOF inserts already existing M:N relationships/empty snapshot?!?

[OC]

Chuck,

On 21. 9. 2016, at 2:40, o...@ocs.cz wrote:

> Aside of that, I have added a log into the 
> DatabaseContextDelegate.databaseContextShouldUpdateCurrentSnapshot method, 
> and here I *never* get *anything* for the relationships: neither the old 
> (which is presumable) *nor the new* dictionary ever contain the 
> “userDataBlocks” or “dataBlockUsers” key — not once.

well this one's weird, or I am missing something of importance here.

Meantime, I have played with another M:N, which works without a glitch (and 
which is modelled the same way as that one which does not, incidentally). Here, 
save-time, the snapshot does contain the proper related objects (and thus 
proper INSERTs are generated).

The weird thing is that these objects *do not* come into the snapshot through 
the DatabaseContextDelegate.databaseContextShouldUpdateCurrentSnapshot method! 
It logs all right when the object is fetched, but the new (nor the old, of 
course) dictionary does *not* contain the relationship key at all -- precisely 
the same way it is with that relationship which does not work properly!

Nevertheless, as mentioned above, save-time, the snapshot is all right. 
Couriouser and couriouser, Alice would say :-O

Thanks and all the best,
OC


 ___
Do not post admin requests to the list. They will be ignored.
Webobjects-dev mailing list  (Webobjects-dev@lists.apple.com)
Help/Unsubscribe/Update your Subscription:
https://lists.apple.com/mailman/options/webobjects-dev/archive%40mail-archive.com

This email sent to arch...@mail-archive.com

Re: EOF inserts already existing M:N relationships/empty snapshot?!?

[o...@ocs.cz]

Just a quick followup -- sorry, forgot to copy/paste the 2nd part of the log, 
though I fear it would not help much — it just confirms the user's snapshot 
does not get changed either:

> On 21. 9. 2016, at 2:40 AM, o...@ocs.cz wrote:
> 
> Chuck,
> 
>> On 20. 9. 2016, at 10:48 PM, Chuck Hill  wrote:
>> 
>> This is why they are getting inserted again:
>> Well, logging out 
>> "currentUser.editingContext().committedSnapshotForObject(currentUser)['userDataBlock']"
>>  before saveChanges, I get an empty array. I must admit I don't know whether 
>> there should be the relationship objects in this snapshot, but anyway:
>> 
>> It should not be an empty array.  EOF compares this and the relationship on 
>> the EO to determine what database operations are needed.
> 
> I see.
> 
>> I’d start with logging that out just after you fetch/fault currentUser into 
>> ec (you don’t show that below).  If it is empty there, then keep logging to 
>> see where the data disappears.  At this point, I am not ready to guess.
> 
> The thing seems empty all the time. Weird. Here's what I log fetch-time:
> 
> ===
> def cbc= // the business case whose users are about to be fetched
> println "### about to fetch $cbc...\noldSS 
> ${cbc.editingContext().committedSnapshotForObject(cbc).userDataBlocks}"
> def xxx=cbc.userDataBlocks()
> println "### -> did fetch $xxx\nnewSS 
> ${cbc.editingContext().committedSnapshotForObject(cbc).userDataBlocks}"
> xxx.each { println "### user $it\n### rel ${it.dataBlockUsers()}\n###   ss 
> ${it.editingContext().committedSnapshotForObject(it).dataBlockUsers}" }
> ===
> 
> Presumably, the “oldSS” should be empty, but the “newSS” should be a copy of 
> the relationship value (i.e., “cbc.userDataBlocks()”, or “xxx”)?
> 
> It is not; the snapshot consistently stays empty:
> 
> ===
> ### about to fetch  /EC:6350cea5>...
> oldSS []
> ### -> did fetch [, 
> ,  PK:109 T:'4' Name:'kl kl kl' /EC:6350cea5>]
> newSS []

### user 
### rel [, 
, 
, 
, 
, 
, 
, 
]
###   ss []

> ===
> 
> Aside of that, I have added a log into the 
> DatabaseContextDelegate.databaseContextShouldUpdateCurrentSnapshot method, 
> and here I *never* get *anything* for the relationships: neither the old 
> (which is presumable) *nor the new* dictionary ever contain the 
> “userDataBlocks” or “dataBlockUsers” key — not once.
> 
> Well it still might be a wrong model settings as Samuel suggests perhaps? But 
> it is sort of weird I use this settings for years and lots of M:N's, and this 
> is the first time I have bumped into this kind of problems.
> 
> Can you see what to check next?
> 
> Thanks a very big lot,
> OC
> 
>> 
>> Also, this is 100% consistent, right?  This is not a concurrency issue?
>> 
>> Chuck
>> 
>> 
>> From:  on behalf 
>> of "o...@ocs.cz" 
>> Date: Tuesday, September 20, 2016 at 11:50 AM
>> To: "webobjects-dev@lists.apple.com WebObjects" 
>> 
>> Subject: EOF inserts already existing M:N relationships/empty snapshot?!?
>> 
>> Hello there,
>> 
>> I have a pretty common setup: entities User and DataBlock, an M:N 
>> relationship represented by an intermediate entity containing just the two 
>> keys, flattened on both sides. At both sides the relationships are 
>> appropriately flattened. Set to own destination+delete rule cascade.
>> 
>> The problem is, upon inserting a new DataBlock and its relationship to a 
>> current user, beside the two objects which should be inserted (the new 
>> DataBlock and the new relationship), relationships to other (old) 
>> DataBlocks, which were before simply _fetched_ for the current user (just to 
>> display the current state), get inserted too — which, of course, causes an 
>> integrity constraint violation “this PK already exists“.
>> 
>> My code looks essentially like this:
>> 
>> ===
>> EOEditingContext ec= ...
>> DBDataBlock ndb=new DBDataBlock()
>> ec.insertObject(ndb)
>> ndb.addObjectToBothSidesOfRelationshipWithKey(currentUser,'dataBlockUsers')
>> // logs here, see below
>> ec.saveChanges()
>> ===
>> 
>> Immediately before ec.saveChanges(), I log out
>> 
>> (a) ec.insertedObjects(), which contains only one object (I have overridden 
>> 

Re: EOF inserts already existing M:N relationships/empty snapshot?!?

[o...@ocs.cz]

Chuck,

> On 20. 9. 2016, at 10:48 PM, Chuck Hill  wrote:
> 
> This is why they are getting inserted again:
> Well, logging out 
> "currentUser.editingContext().committedSnapshotForObject(currentUser)['userDataBlock']"
>  before saveChanges, I get an empty array. I must admit I don't know whether 
> there should be the relationship objects in this snapshot, but anyway:
>  
> It should not be an empty array.  EOF compares this and the relationship on 
> the EO to determine what database operations are needed.

I see.

> I’d start with logging that out just after you fetch/fault currentUser into 
> ec (you don’t show that below).  If it is empty there, then keep logging to 
> see where the data disappears.  At this point, I am not ready to guess.

The thing seems empty all the time. Weird. Here's what I log fetch-time:

===
def cbc= // the business case whose users are about to be fetched
println "### about to fetch $cbc...\noldSS 
${cbc.editingContext().committedSnapshotForObject(cbc).userDataBlocks}"
def xxx=cbc.userDataBlocks()
println "### -> did fetch $xxx\nnewSS 
${cbc.editingContext().committedSnapshotForObject(cbc).userDataBlocks}"
xxx.each { println "### user $it\n### rel ${it.dataBlockUsers()}\n###   ss 
${it.editingContext().committedSnapshotForObject(it).dataBlockUsers}" }
===

Presumably, the “oldSS” should be empty, but the “newSS” should be a copy of 
the relationship value (i.e., “cbc.userDataBlocks()”, or “xxx”)?

It is not; the snapshot consistently stays empty:

===
### about to fetch ...
oldSS []
### -> did fetch [, 
, ]
newSS []
===

Aside of that, I have added a log into the 
DatabaseContextDelegate.databaseContextShouldUpdateCurrentSnapshot method, and 
here I *never* get *anything* for the relationships: neither the old (which is 
presumable) *nor the new* dictionary ever contain the “userDataBlocks” or 
“dataBlockUsers” key — not once.

Well it still might be a wrong model settings as Samuel suggests perhaps? But 
it is sort of weird I use this settings for years and lots of M:N's, and this 
is the first time I have bumped into this kind of problems.

Can you see what to check next?

Thanks a very big lot,
OC

>  
> Also, this is 100% consistent, right?  This is not a concurrency issue?
>  
> Chuck
>  
>  
> From:  on behalf 
> of "o...@ocs.cz" 
> Date: Tuesday, September 20, 2016 at 11:50 AM
> To: "webobjects-dev@lists.apple.com WebObjects" 
> 
> Subject: EOF inserts already existing M:N relationships/empty snapshot?!?
>  
> Hello there,
>  
> I have a pretty common setup: entities User and DataBlock, an M:N 
> relationship represented by an intermediate entity containing just the two 
> keys, flattened on both sides. At both sides the relationships are 
> appropriately flattened. Set to own destination+delete rule cascade.
>  
> The problem is, upon inserting a new DataBlock and its relationship to a 
> current user, beside the two objects which should be inserted (the new 
> DataBlock and the new relationship), relationships to other (old) DataBlocks, 
> which were before simply _fetched_ for the current user (just to display the 
> current state), get inserted too — which, of course, causes an integrity 
> constraint violation “this PK already exists“.
>  
> My code looks essentially like this:
>  
> ===
> EOEditingContext ec= ...
> DBDataBlock ndb=new DBDataBlock()
> ec.insertObject(ndb)
> ndb.addObjectToBothSidesOfRelationshipWithKey(currentUser,'dataBlockUsers')
> // logs here, see below
> ec.saveChanges()
> ===
>  
> Immediately before ec.saveChanges(), I log out
>  
> (a) ec.insertedObjects(), which contains only one object (I have overridden 
> toString() to get the PK /and other attributes, removed here for conciseness/ 
> of an EO):
>  
> []
>  
> this is all right, null PK means a newly added object, it's the very 'ndb' 
> datablock I just have inserted and which I am now saving.
>  
> (b) contents of the flattened M:N relationship 'ndb.dataBlockUsers' from the 
> DBDataBlock@79adaefa to DBUser, which looks like this:
>  
> []
>  
> again, quite all right: only one related user, the currentUser which I have 
> just added to the relationship.
>  
> (c) contents of the flattened M:N inverse relationship from the current user, 
> which looks like this:
>  
> [, , 
> , , 
> , , 
> , , 
> 

Re: EOF inserts already existing M:N relationships/empty snapshot?!?

[Chuck Hill]

But when is it not set?  After fetching into the EC?  After faulting it in?  Is 
there there at first and then goes away?  There is a method in Wonder that can 
clear a to-many relationship, but that should make it a fault not an empty 
array.

You really DO find all of the best bugs!  :-)

Chuck


From: <webobjects-dev-bounces+chill=gevityinc@lists.apple.com> on behalf of 
OC <o...@ocs.cz>
Date: Tuesday, September 20, 2016 at 5:22 PM
To: Samuel Pelletier <sam...@samkar.com>
Cc: "webobjects-dev@lists.apple.com WebObjects" <webobjects-dev@lists.apple.com>
Subject: Re: EOF inserts already existing M:N relationships/empty snapshot?!?

Samuel,

On 21. 9. 2016, at 1:24, Samuel Pelletier 
<sam...@samkar.com<mailto:sam...@samkar.com>> wrote:

Did you created the M:N relationship using the modeler or you created it 
manually ?

Manually (well through my own modeller, which boils down to precisely the same 
result). Nevertheless, I have done this umpteen times before; and this is the 
first time I have bumped into any problem.

Your situation may be caused by an improper flattened relationship settings. 
The flattened relationships should have nothing checked in the Advanced pane of 
the modeler.

I went trough my “EOModeller User Guide”, but alas, there does not seem to be a 
snapshot of the Advanced Relationship pane, so I am not sure what precisely has 
been in there. Owns Destination and Delete Rule, I guess?

On my models, I always create them with the modeler and the relationships to 
the inner entity does not have "own relationship" checked but have the 
"Propagate primary key" checked.

Anyway, to be sure, I have tried both combinations (“owsDestination” YES/NO and 
“propagatesPrimaryKey” YES/NO) in the relationship dictionary, to no avail — 
always the same result, the snapshot is not set.

Thanks a lot,
OC

Le 20 sept. 2016 à 14:50, o...@ocs.cz<mailto:o...@ocs.cz> a écrit :
Hello there,
I have a pretty common setup: entities User and DataBlock, an M:N relationship 
represented by an intermediate entity containing just the two keys, flattened 
on both sides. At both sides the relationships are appropriately flattened. Set 
to own destination+delete rule cascade.
The problem is, upon inserting a new DataBlock and its relationship to a 
current user, beside the two objects which should be inserted (the new 
DataBlock and the new relationship), relationships to other (old) DataBlocks, 
which were before simply _fetched_ for the current user (just to display the 
current state), get inserted too — which, of course, causes an integrity 
constraint violation “this PK already exists“.
My code looks essentially like this:
===
EOEditingContext ec= ...
DBDataBlock ndb=new DBDataBlock()
ec.insertObject(ndb)
ndb.addObjectToBothSidesOfRelationshipWithKey(currentUser,'dataBlockUsers')
// logs here, see below
ec.saveChanges()
===
Immediately before ec.saveChanges(), I log out
(a) ec.insertedObjects(), which contains only one object (I have overridden 
toString() to get the PK /and other attributes, removed here for conciseness/ 
of an EO):
[<DBDataBlock@79adaefa PK:null>]
this is all right, null PK means a newly added object, it's the very 'ndb' 
datablock I just have inserted and which I am now saving.
(b) contents of the flattened M:N relationship 'ndb.dataBlockUsers' from the 
DBDataBlock@79adaefa to DBUser, which looks like this:
[<DBUser@71b29988 PK:105>]
again, quite all right: only one related user, the currentUser which I have 
just added to the relationship.
(c) contents of the flattened M:N inverse relationship from the current user, 
which looks like this:
[<DBDataBlock@189f083a PK:103>, <DBDataBlock@1b47d247 PK:116>, 
<DBDataBlock@5f927095 PK:119>, <DBDataBlock@54edfe4c PK:122>, 
<DBDataBlock@35e1e59c PK:123>, <DBDataBlock@793b3d6d PK:124>, 
<DBDataBlock@7b06dee8 PK:129>, <DBDataBlock@2550aab9 PK:133>, 
<DBDataBlock@79adaefa PK:null>]
for the third time, all right: 8 of them previously fetched, already existing 
relationships to other (old) datablocks for the current user, plus one new 
relationship to the newly added DBDataBlock@79adaefa. Perfect so far.
At this moment, ec.saveChanges is performed. I log the database operations from 
databaseContextWillPerformAdaptorOperations delegate method, and it looks like 
this:
===
-IN-adaptorOps === SPC: about to perform 10 DB operations |19:54:37.061 
20.9.16|WorkerThread1
  - 1: INSERT on 'DBDataBlock'  1{uid:142}
  - 2: INSERT on 'DB_UserDataBlock'  2{db_id:123, 
user_id:105}
  - 3: INSERT on 'DB_UserDataBlock'  2{db_id:122, 
user_id:105}
  - 4: INSERT on 'DB_UserDataBlock'  2{db_id:103, 
user_id:105}
  - 5: INSERT on 'DB_UserDataBlock'  2{db_id:116, 
user_id:105}
  - 6: I

Re: EOF inserts already existing M:N relationships/empty snapshot?!?

[OC]

Samuel,

On 21. 9. 2016, at 1:24, Samuel Pelletier  wrote:

> Did you created the M:N relationship using the modeler or you created it 
> manually ?

Manually (well through my own modeller, which boils down to precisely the same 
result). Nevertheless, I have done this umpteen times before; and this is the 
first time I have bumped into any problem.

> Your situation may be caused by an improper flattened relationship settings. 
> The flattened relationships should have nothing checked in the Advanced pane 
> of the modeler.

I went trough my “EOModeller User Guide”, but alas, there does not seem to be a 
snapshot of the Advanced Relationship pane, so I am not sure what precisely has 
been in there. Owns Destination and Delete Rule, I guess?

> On my models, I always create them with the modeler and the relationships to 
> the inner entity does not have "own relationship" checked but have the 
> "Propagate primary key" checked.

Anyway, to be sure, I have tried both combinations (“owsDestination” YES/NO and 
“propagatesPrimaryKey” YES/NO) in the relationship dictionary, to no avail — 
always the same result, the snapshot is not set.

Thanks a lot,
OC

>> Le 20 sept. 2016 à 14:50, o...@ocs.cz a écrit :
>> 
>> Hello there,
>> 
>> I have a pretty common setup: entities User and DataBlock, an M:N 
>> relationship represented by an intermediate entity containing just the two 
>> keys, flattened on both sides. At both sides the relationships are 
>> appropriately flattened. Set to own destination+delete rule cascade.
>> 
>> The problem is, upon inserting a new DataBlock and its relationship to a 
>> current user, beside the two objects which should be inserted (the new 
>> DataBlock and the new relationship), relationships to other (old) 
>> DataBlocks, which were before simply _fetched_ for the current user (just to 
>> display the current state), get inserted too — which, of course, causes an 
>> integrity constraint violation “this PK already exists“.
>> 
>> My code looks essentially like this:
>> 
>> ===
>> EOEditingContext ec= ...
>> DBDataBlock ndb=new DBDataBlock()
>> ec.insertObject(ndb)
>> ndb.addObjectToBothSidesOfRelationshipWithKey(currentUser,'dataBlockUsers')
>> // logs here, see below
>> ec.saveChanges()
>> ===
>> 
>> Immediately before ec.saveChanges(), I log out
>> 
>> (a) ec.insertedObjects(), which contains only one object (I have overridden 
>> toString() to get the PK /and other attributes, removed here for 
>> conciseness/ of an EO):
>> 
>> []
>> 
>> this is all right, null PK means a newly added object, it's the very 'ndb' 
>> datablock I just have inserted and which I am now saving.
>> 
>> (b) contents of the flattened M:N relationship 'ndb.dataBlockUsers' from the 
>> DBDataBlock@79adaefa to DBUser, which looks like this:
>> 
>> []
>> 
>> again, quite all right: only one related user, the currentUser which I have 
>> just added to the relationship.
>> 
>> (c) contents of the flattened M:N inverse relationship from the current 
>> user, which looks like this:
>> 
>> [, , 
>> , , 
>> , , 
>> , , 
>> ]
>> 
>> for the third time, all right: 8 of them previously fetched, already 
>> existing relationships to other (old) datablocks for the current user, plus 
>> one new relationship to the newly added DBDataBlock@79adaefa. Perfect so far.
>> 
>> At this moment, ec.saveChanges is performed. I log the database operations 
>> from databaseContextWillPerformAdaptorOperations delegate method, and it 
>> looks like this:
>> 
>> ===
>> -IN-adaptorOps === SPC: about to perform 10 DB operations|19:54:37.061 
>> 20.9.16|WorkerThread1
>>  - 1: INSERT on 'DBDataBlock'  1{uid:142}
>>  - 2: INSERT on 'DB_UserDataBlock'  2{db_id:123, 
>> user_id:105}
>>  - 3: INSERT on 'DB_UserDataBlock'  2{db_id:122, 
>> user_id:105}
>>  - 4: INSERT on 'DB_UserDataBlock'  2{db_id:103, 
>> user_id:105}
>>  - 5: INSERT on 'DB_UserDataBlock'  2{db_id:116, 
>> user_id:105}
>>  - 6: INSERT on 'DB_UserDataBlock'  2{db_id:142, 
>> user_id:105}
>>  - 7: INSERT on 'DB_UserDataBlock'  2{db_id:119, 
>> user_id:105}
>>  - 8: INSERT on 'DB_UserDataBlock'  2{db_id:129, 
>> user_id:105}
>>  - 9: INSERT on 'DB_UserDataBlock'  2{db_id:124, 
>> user_id:105}
>>  - 10: INSERT on 'DB_UserDataBlock'  2{db_id:133, 
>> user_id:105}
>> ===
>> 
>> i.e., along with inserting the two new objects (1: the new datablock, 6: the 
>> new relationship), EOF for some darned reason decides to 

Re: EOF inserts already existing M:N relationships/empty snapshot?!?

[Samuel Pelletier]

OC,

Did you created the M:N relationship using the modeler or you created it 
manually ?

These are tricky to correctly configure.

Your situation may be caused by an improper flattened relationship settings. 
The flattened relationships should have nothing checked in the Advanced pane of 
the modeler.

On my models, I always create them with the modeler and the relationships to 
the inner entity does not have "own relationship" checked but have the 
"Propagate primary key" checked.

Samuel


> Le 20 sept. 2016 à 14:50, o...@ocs.cz a écrit :
> 
> Hello there,
> 
> I have a pretty common setup: entities User and DataBlock, an M:N 
> relationship represented by an intermediate entity containing just the two 
> keys, flattened on both sides. At both sides the relationships are 
> appropriately flattened. Set to own destination+delete rule cascade.
> 
> The problem is, upon inserting a new DataBlock and its relationship to a 
> current user, beside the two objects which should be inserted (the new 
> DataBlock and the new relationship), relationships to other (old) DataBlocks, 
> which were before simply _fetched_ for the current user (just to display the 
> current state), get inserted too — which, of course, causes an integrity 
> constraint violation “this PK already exists“.
> 
> My code looks essentially like this:
> 
> ===
> EOEditingContext ec= ...
> DBDataBlock ndb=new DBDataBlock()
> ec.insertObject(ndb)
> ndb.addObjectToBothSidesOfRelationshipWithKey(currentUser,'dataBlockUsers')
> // logs here, see below
> ec.saveChanges()
> ===
> 
> Immediately before ec.saveChanges(), I log out
> 
> (a) ec.insertedObjects(), which contains only one object (I have overridden 
> toString() to get the PK /and other attributes, removed here for conciseness/ 
> of an EO):
> 
> []
> 
> this is all right, null PK means a newly added object, it's the very 'ndb' 
> datablock I just have inserted and which I am now saving.
> 
> (b) contents of the flattened M:N relationship 'ndb.dataBlockUsers' from the 
> DBDataBlock@79adaefa to DBUser, which looks like this:
> 
> []
> 
> again, quite all right: only one related user, the currentUser which I have 
> just added to the relationship.
> 
> (c) contents of the flattened M:N inverse relationship from the current user, 
> which looks like this:
> 
> [, , 
> , , 
> , , 
> , , 
> ]
> 
> for the third time, all right: 8 of them previously fetched, already existing 
> relationships to other (old) datablocks for the current user, plus one new 
> relationship to the newly added DBDataBlock@79adaefa. Perfect so far.
> 
> At this moment, ec.saveChanges is performed. I log the database operations 
> from databaseContextWillPerformAdaptorOperations delegate method, and it 
> looks like this:
> 
> ===
> -IN-adaptorOps === SPC: about to perform 10 DB operations |19:54:37.061 
> 20.9.16|WorkerThread1
>   - 1: INSERT on 'DBDataBlock'  1{uid:142}
>   - 2: INSERT on 'DB_UserDataBlock'  2{db_id:123, 
> user_id:105}
>   - 3: INSERT on 'DB_UserDataBlock'  2{db_id:122, 
> user_id:105}
>   - 4: INSERT on 'DB_UserDataBlock'  2{db_id:103, 
> user_id:105}
>   - 5: INSERT on 'DB_UserDataBlock'  2{db_id:116, 
> user_id:105}
>   - 6: INSERT on 'DB_UserDataBlock'  2{db_id:142, 
> user_id:105}
>   - 7: INSERT on 'DB_UserDataBlock'  2{db_id:119, 
> user_id:105}
>   - 8: INSERT on 'DB_UserDataBlock'  2{db_id:129, 
> user_id:105}
>   - 9: INSERT on 'DB_UserDataBlock'  2{db_id:124, 
> user_id:105}
>   - 10: INSERT on 'DB_UserDataBlock'  2{db_id:133, 
> user_id:105}
> ===
> 
> i.e., along with inserting the two new objects (1: the new datablock, 6: the 
> new relationship), EOF for some darned reason decides to insert _also_ all 
> those relationship objects it _fetched_ before! Of course, since they were 
> fetched, they already exist in the database, and thus cause an integrity 
> constraint violation “this PK already exists“.
> 
> Well, logging out 
> "currentUser.editingContext().committedSnapshotForObject(currentUser)['userDataBlock']"
>  before saveChanges, I get an empty array. I must admit I don't know whether 
> there should be the relationship objects in this snapshot, but anyway:
> 
> - if not, I have no idea where to find the culprit;
> - if yes, well, the culprit of those inserts is explained, but why on earth 
> the snapshot is not properly maintained by EOF?!?
> 
> Any idea what might be the culprit, or at least, how to find it? I am afraid 
> I am rather 

Re: EOF inserts already existing M:N relationships/empty snapshot?!?

[Chuck Hill]

This is why they are getting inserted again:
Well, logging out 
"currentUser.editingContext().committedSnapshotForObject(currentUser)['userDataBlock']"
 before saveChanges, I get an empty array. I must admit I don't know whether 
there should be the relationship objects in this snapshot, but anyway:

It should not be an empty array.  EOF compares this and the relationship on the 
EO to determine what database operations are needed.  I’d start with logging 
that out just after you fetch/fault currentUser into ec (you don’t show that 
below).  If it is empty there, then keep logging to see where the data 
disappears.  At this point, I am not ready to guess.

Also, this is 100% consistent, right?  This is not a concurrency issue?

Chuck


From:  on behalf of 
"o...@ocs.cz" 
Date: Tuesday, September 20, 2016 at 11:50 AM
To: "webobjects-dev@lists.apple.com WebObjects" 
Subject: EOF inserts already existing M:N relationships/empty snapshot?!?

Hello there,

I have a pretty common setup: entities User and DataBlock, an M:N relationship 
represented by an intermediate entity containing just the two keys, flattened 
on both sides. At both sides the relationships are appropriately flattened. Set 
to own destination+delete rule cascade.

The problem is, upon inserting a new DataBlock and its relationship to a 
current user, beside the two objects which should be inserted (the new 
DataBlock and the new relationship), relationships to other (old) DataBlocks, 
which were before simply _fetched_ for the current user (just to display the 
current state), get inserted too — which, of course, causes an integrity 
constraint violation “this PK already exists“.

My code looks essentially like this:

===
EOEditingContext ec= ...
DBDataBlock ndb=new DBDataBlock()
ec.insertObject(ndb)
ndb.addObjectToBothSidesOfRelationshipWithKey(currentUser,'dataBlockUsers')
// logs here, see below
ec.saveChanges()
===

Immediately before ec.saveChanges(), I log out

(a) ec.insertedObjects(), which contains only one object (I have overridden 
toString() to get the PK /and other attributes, removed here for conciseness/ 
of an EO):

[]

this is all right, null PK means a newly added object, it's the very 'ndb' 
datablock I just have inserted and which I am now saving.

(b) contents of the flattened M:N relationship 'ndb.dataBlockUsers' from the 
DBDataBlock@79adaefa to DBUser, which looks like this:

[]

again, quite all right: only one related user, the currentUser which I have 
just added to the relationship.

(c) contents of the flattened M:N inverse relationship from the current user, 
which looks like this:

[, , 
, , 
, , 
, , 
]

for the third time, all right: 8 of them previously fetched, already existing 
relationships to other (old) datablocks for the current user, plus one new 
relationship to the newly added DBDataBlock@79adaefa. Perfect so far.

At this moment, ec.saveChanges is performed. I log the database operations from 
databaseContextWillPerformAdaptorOperations delegate method, and it looks like 
this:

===
-IN-adaptorOps === SPC: about to perform 10 DB operations   |19:54:37.061 
20.9.16|WorkerThread1
   - 1: INSERT on 'DBDataBlock'  1{uid:142}
   - 2: INSERT on 'DB_UserDataBlock'  2{db_id:123, 
user_id:105}
   - 3: INSERT on 'DB_UserDataBlock'  2{db_id:122, 
user_id:105}
   - 4: INSERT on 'DB_UserDataBlock'  2{db_id:103, 
user_id:105}
   - 5: INSERT on 'DB_UserDataBlock'  2{db_id:116, 
user_id:105}
   - 6: INSERT on 'DB_UserDataBlock'  2{db_id:142, 
user_id:105}
   - 7: INSERT on 'DB_UserDataBlock'  2{db_id:119, 
user_id:105}
   - 8: INSERT on 'DB_UserDataBlock'  2{db_id:129, 
user_id:105}
   - 9: INSERT on 'DB_UserDataBlock'  2{db_id:124, 
user_id:105}
   - 10: INSERT on 'DB_UserDataBlock'  2{db_id:133, 
user_id:105}
===

i.e., along with inserting the two new objects (1: the new datablock, 6: the 
new relationship), EOF for some darned reason decides to insert _also_ all 
those relationship objects it _fetched_ before! Of course, since they were 
fetched, they already exist in the database, and thus cause an integrity 
constraint violation “this PK already exists“.

Well, logging out 
"currentUser.editingContext().committedSnapshotForObject(currentUser)['userDataBlock']"
 before saveChanges, I get an empty array. I must admit I don't know whether 
there should be the relationship