Hi Thomas, yes, I think that should also fix it. `post.stats = ...` will set the `stats` property of the `post` object, which is copy-on-write and thus fine. Only accessing a sub-property of a property will cause issues when the document is a ShapedJson instance.
Best regards Jan 2016-06-14 15:27 GMT+02:00 Thomas Weiss <[email protected]>: > Hi Jan, > > Thank you so much for this very extensive reply. I know you guys are > working hard on the 3.0 release so I really appreciate the time you took to > write this answer! > > If I understand correctly what you explained, am I right that the > following code would also solve my issue: > post.stats = { > likeCount: db.likes.byExample({ _to: 'posts/' + postId }).count(), > commentCount: post.stats.commentCount, > shareCount: post.stats.shareCount > }; > db.posts.replace({ _id: post._id }, post); > > Cheers, > Thomas > > On Tuesday, June 14, 2016 at 3:59:57 PM UTC+8, Jan Steemann wrote: >> >> Hi there, >> >> I think I can shed some light on this. I don't think this is related to >> waitForSync or transaction, but has a different root cause. >> >> All documents, when inserted, updated or replaced are saved in ArangoDB's >> write-ahead log (WAL) first. >> When retrieving a document from the WAL using >> db.<collection>.document(key) then a copy of the document will be returned >> to Foxx/JavaScript as a regular JavaScript object: >> >> /* fetch document with key postId from database. will return a >> JavaScript object */ >> db.posts.insert({ _key: postId, stats: { likeCount: 0 } }); >> var post = db.posts.document(postId); >> >> This object can be modified regularly and saved again, using >> update/replace: >> >> post.stats.likeCount = db.likes.byExample({ _to: 'posts/' + postId >> }).count(); >> db.posts.replace({ _id: post._id }, post); >> >> However, at some point the documents are moved from the WAL into the >> datafiles of the respective collections. >> When they are then retrieved from the datafiles, they are returned as >> light-weight JavaScript objects which contain only pointers to the >> underlying document data. >> Accessing a property of such light-weight object will trigger a property >> handler, which will build the result value and return it as a *temporary >> object* >> That means when accessing `post.stats` in the following code, >> `post.stats` will return a temporary object, for which the property >> `likeCount` will be updated: >> >> /* returns light-weight object with property handlers */ >> var post = db.posts.document(postId); >> /* post.stats will produce a tempory object, and post.state.likeCount >> will access a property of the temporary object */ >> post.stats.likeCount = db.likes.byExample({ _to: 'posts/' + postId >> }).count(); >> >> The modification will only happen in the temporary object, but not in the >> `post` object. >> When then the post object is written back to the database, it won't >> contain the changes. >> >> Following is some example code that demonstrates this for a document that >> is stored in the WAL and for a document from the collection datafiles. >> In one case the update/replace will do what's expected and in the other >> it won't: >> >> ``` >> db._drop("posts"); >> db._drop("likes"); >> >> db._create("posts"); >> db._createEdgeCollection("likes"); >> >> var ShapedJson = require("internal").ShapedJson; >> >> function updateCount(issuerId, postId) { >> var post = db.posts.document(postId); >> db.likes.insert('users/' + issuerId, 'posts/' + postId, {}, true); >> // the post object was fetched before >> post.stats.likeCount = db.likes.byExample({ _to: 'posts/' + postId >> }).count(); >> db.posts.replace({ _id: post._id }, post); >> } >> >> // insert a new post document. it will be located in the write-ahead log >> first >> db.posts.insert({ _key: "post1", stats: { likeCount: 0 } }); >> require("internal").print("the document:", db.posts.document("post1")); >> require("internal").print("is ShapedJson:", db.posts.document("post1") >> instanceof ShapedJson); >> >> // update the document's count. this will work fine >> updateCount("test1", "post1"); >> >> // and print results. count should have been updated >> require("internal").print("count for post1 after update is:", >> db.posts.document("post1")); >> >> >> >> // insert another post document. it will be located in the write-ahead >> log first >> db.posts.insert({ _key: "post2", stats: { likeCount: 0 } }); >> // now flush the write-ahead log manually and wait for a few seconds >> // note: flushing the write-ahead log may happen automatically from time >> to time while >> // the server is running >> require("internal").wal.flush(true, true); >> require("internal").wait(5, false); >> >> require("internal").print("the document:", db.posts.document("post2")); >> require("internal").print("is ShapedJson:", db.posts.document("post2") >> instanceof ShapedJson); >> >> // now update the document's count. this will update the count value of a >> ShapedJson >> // (which is a temporary object when retrieved from the database) >> updateCount("test2", "post2"); >> >> // and print results. count will not have been updated >> require("internal").print("count for post2 is now:", >> db.posts.document("post2")); >> ``` >> >> Note that in the above example for both documents there is a check >> whether they are an instance of *ShapedJson*. >> In the one case (WAL case) the document won't be a ShapedJson instance >> but a regular JS object, but in the other case it will be. >> In the latter case, updating a property of the document which itself is >> an array or object won't work, because of the temporaries produced by the >> property handler. >> >> What helps in this case is to convert the ShapedJson instance of the >> document into a regular JS object, e.g. via >> >> var _ = require("underscore"); >> ... >> var post = _.clone(db.posts.document(postId)); >> >> Or, combined with the ShapedJson test: >> >> var ShapedJson = require("internal").ShapedJson; >> var _ = require("underscore"); >> ... >> var post = db.posts.document(postId); >> >> >> if (post instance of ShapedJson) { >> // ShapedJson object >> post = _.clone(post); >> } >> >> Note that this applies to the 1.x and 2.x branches of ArangoDB. >> We have simplified this a lot in 3.0 so both the ShapedJson testing and >> cloning can be omitted there. All objects returned in 3.0 will be regular >> JS objects free of side effects. >> >> I hope this helps. >> Best regards >> Jan >> >> >> >> 2016-06-14 5:18 GMT+02:00 Thomas Weiss <[email protected]>: >> >>> As a follow-up, I've just seen the same behavior on my dev machine, it >>> just happens less frequently (probably because it has better perfs than the >>> server used for staging). >>> Any comment will be welcome! >>> >>> Thanks, >>> Thomas >>> >>> >>> On Sunday, June 12, 2016 at 3:57:04 PM UTC+8, Thomas Weiss wrote: >>>> >>>> Hi there, >>>> >>>> I'm using a Foxx app to execute transactions. To boost query efficiency >>>> (the system is read-heavy), I try to cache the count of edges directly in >>>> the docs, and do this like that: >>>> db.likes.insert('users/' + issuerId, 'posts/' + postId, {}, true); >>>> // the post object was fetched before >>>> post.stats.likeCount = db.likes.byExample({ _to: 'posts/' + postId }). >>>> count(); >>>> db.posts.replace({ _id: post._id }, post); >>>> Notice the last *true* parameter passed to *insert* to make sure that >>>> we wait for the write to be flushed. This code is executed within a >>>> transaction by the way. >>>> >>>> This works well on my dev machine (Windows 10) *but* it seems that the >>>> count is not systematically updated correctly on the staging server >>>> (Ubuntu). What's weird is that sometimes it works and sometimes it doesn't, >>>> which makes me think about a threading/concurrency issue (and that's why I >>>> added the waitForSync param). >>>> >>>> Any help or suggestion would be greatly appreciated here. >>>> >>>> Thanks, >>>> Thomas >>>> >>> -- >>> You received this message because you are subscribed to the Google >>> Groups "ArangoDB" group. >>> To unsubscribe from this group and stop receiving emails from it, send >>> an email to [email protected]. >>> For more options, visit https://groups.google.com/d/optout. >>> >> >> -- > You received this message because you are subscribed to the Google Groups > "ArangoDB" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > For more options, visit https://groups.google.com/d/optout. > -- You received this message because you are subscribed to the Google Groups "ArangoDB" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/d/optout.
