> On Nov 3, 2016, at 4:03 PM, stepharo <[email protected]> wrote: > >> But how does this fit with a tree structure. For example, should “comment5” >> be associated with the “b” variable node, or the #foo:bar: message, or the >> sequence node that contains the #foo:bar: message, or the block or method >> node that contains the sequence node. You can choose, but only the writer of >> the comment can tell if you are right. > How tokens solves this problem? > I think that we do not really care one objective is to be able to recreate > the code from the tree. I worked on a project in C > where the guys want to know if the code is well commented and we built > heuristics to associate the comments to the closest node.
The nodes have comments and the comments have their source and position so you can recreate the code from the tree already. It may not be as easy as you want, but all the information is there. >>>> Furthermore, if comments are nodes, how do they affect all of the code >>>> rewriting and validation that is part of the RB and its rewrite tools >>> With comments cannot act as nullobjects for such operation? >> Yes, someone could spend time making all of this work with the existing >> code. However, why not fix the real problem in that comments and methods >> should not be a single string, but rather objects. > > Marcus did that during his phd and he blew up memory. So after we got a > guy that worked on tree compression but he never finished. > Because this is the part that makes all these ideas flying or not. From what I remember, ASTs are roughly 10x the size of the code string. I’m not suggesting holding on to them in the image when they can be loaded/built from a file like the method sources currently are. Instead, we can associate a comment/annotation to a particular node in the AST without having to keep the AST around. For example, we could say that some comment is associated to the 4th node visited from the standard AST traversal. We don’t need to keep the AST around since we can recreate it and the 4th node visited will be the same. John Brant
