This is an Engineering Notebook post re https://github.com/leo-editor/leo-editor/issues/35 leoBridge sometimes assigns the same gnx to two distinct vnodes
I am going to start a new thread so as not to clutter the already-way-to-long discussion in the bug thread. First, let's all remind ourselves that this bug has been a *good* thing. It points Leo in unexpected and valuable new directions. ===== What are gnx's anyway? Terry's story about his brother in Australia, let's call him Tony, has convinced me that we must completely rethink how Leo represents gnx's. Let's assume that both Terry and Tony have chosen tbrown as their Leo id. As more and more people use Leo, it becomes virtually certain that two people use the same Leo id. gnx's are intended to be *unique* and *immutable*. By "unique" I mean that if two vnodes have identical gnx's, then they are "really" the same vnode: they are intended to represent *the same thing*. In other words: - They were created by the same person. - They were created *once*, at a particular time (and place), during a single invocation of Leo. - No other vnode could possibly have the same gnx. Terry's story, and bug #35, cast doubts on all of the above. Imo, bug #35 is going to turn out to be an implementation botch of some kind, but Terry's Aussie brother shows that, in general, Leo's existing gnx's must be revised. Having said this, I want to discover the true nature of #35 before making it moot by introducing uuid's. Furthermore, uuid's are ugly enough that we will probably want to use gnx's by default, and use uuid's to allocate gnx's only when the --uuid option is in effect. Suppose two different people, widely separated, *do* create vnodes with clashing vnodes. When can that be a problem? I think the answer is clear: those vnodes can be a problem only when used in the *same .leo file*. That is, only when the vnodes are shared. There are two main sharing cases: Case 1: collaboration via a repo. In that case, Terry and Tony will be *required* to use different id's. So there is no chance of gnx's colliding in this case. Case 2: collaboration via shared .leo files. This already happens frequently, usually by cutting and pasting parts of an outline. True, pasting an outline reallocates gnx's. That *should* guarantee no gnx collisions, but perhaps it does not because of implementation problems. In any case, paste-retaining-clones could result in *unintended* gnx clashes. The odds of this are very small, perhaps small enough to ignore for now, but the odds are certainly *not* zero. The conclusion is pretty clear. We can ignore, for now, the problems that Terry and Tony might have in sharing snippets of .leo files. However, it would be wise to offer the --uuid option asap. This must happen eventually, so we might as well do it now for 5.0. ===== Possible causes of #35 1. SegundoBob said in the initial bug report: QQQ The "Hidden Root Node" is special in several ways. It is *not* saved in the .leo file, so it does not have one permanent gnx. It is in fact, assigned what I've been calling the "session start" gnx index. So the "Hidden Root Node" can have the same gnx as a "real" vnode that was created in a previous session. This problem is the one that causes my unit tests to fail even though they only read without modifying a previously created .leo file. QQQ Fixing this problem might suffice, but other issues may also be involved. 2. The code has been tacitly assuming that all timestamps are unique. This is why Bob's "timestamp adjustment" fixed the problem: it guaranteed that all timestamps were, in fact, for his unit tests, unique. Terry's Aussie brother can, by *very* bad luck, create clashing vnodes. But Bob's unit tests fail every time the timestamps fail. 3. It's possible that caching may also be involved. For example, the cached read logic may be bypassing or invalidating the post pass. ===== Possible fixes for #35 I want to eliminate both the post-pass and the timestamp hack/adjustments. Both are too ugly for my taste. 1. Ensure that the hidden root vnode can never clash with anything. We could do this, for example, by setting its gnx to "hidden-root-vnode". Hehe. It's cute, amusing, and bullet-proof. 2. As Terry has been suggesting, we can compute ni.lastIndex when creating vnodes. That should be easy to do, but we must make sure that ni.lastIndex gets updated when reading cached files. 3. For debugging, we could create a dict, say g.app.gnxDict, whose keys are every gnx allocated in this particular invocation of Leo. Entries in this dict would be create when updating ni.lastIndex. **Important**: The read logic can not, even in principle, do any kind of check for "duplicate" gnx's. Rather, the read logic assumes that nodes with the same gnx are clones of each other. This means that Leo *must not* create any new vnodes not in the .leo file or its external files until the read logic completes. The hidden root vnode may be the exception, so suspicion falls heavily on the hidden root vnode as the real cause of #35. ===== Debugging This bug has taken way too long to fix. Suspicion falls on the gnx of the hidden root vnode as the one and *only* cause of this bug. I'll look at this today. If this doesn't fix #35, I'll add debugging code that Bob can enable that will show exactly why gnx's are clashing unexpectedly. ===== Summary 1. Timestamps. Leo can no longer assume that timestamps are unique. Long term, Leo must use uuid's to guarantee that nodes having the same gnx really were created by a single person, at a particular time and place. Leo 5.0a1 will support a --uuid option and we must ensure that this option is solid before releasing 5.0a1. Leo's read logic *will* allocate clashing gnx's if two *distinct* .leo files have allocated vnodes with the same timestamp. This is not the cause of #35, but it could happen in future. This fact, all by itself, shows that uuids become more essential as more people use Leo. I removed the timestamp adjustment hack yesterday. It's not coming back. It makes debugging #35 impossible. 2. Ensuring unique gnxs. Something like the post-pass is required to ensure that no vnodes created *after* the read logic completes will conflict with any vnode created *by* the read logic. Having the read logic (cached or uncached) update the "maximum n seen so far" is an alternative to the post-pass. For now, I'm going to leave the post pass in place. I doubt that it's buggy, but any bug should be easy to fix. The alternatives, though quicker, are more complex. 3. Debugging. Leo will assign "hidden-root-node-gnx" as the hidden root vnode's gnx. This will ensure that it can't clash with any "real" vnode. If this doesn't fix #35, I'll add debugging code that Bob can enable. That debugging code might include g.app.gnxDict. Edward -- You received this message because you are subscribed to the Google Groups "leo-editor" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at http://groups.google.com/group/leo-editor. For more options, visit https://groups.google.com/d/optout.
