> On 11 Sep 2017, at 02:30, Tim Mackinnon <tim@testit.works> wrote: > > Hey Marcus - I've kept noodling on this as a little sub interesting thing I > can play with in spare moments. > > I think I'm getting close to an interesting solution using #parseFaulty > (although I wasn't sure how to distinguish between a method and playground > without asking the editor for its edittingMode - which maybe is ok?) > > I've had to add a new visitor to run through parseErrorNodes (I'm surprised > there isn't a visitor for this already) (I also added one for comments too) - > as #findBestNode doesn't try hard enough - and I've also had a crack at > reparsing an error if it seems simple enough (like when you are working on > statement in the middle of code and have just broken that one line). > Great!
> I'll try it for a few days and then maybe you might be interested in having a > look? > Yes! > Tim > > Sent from my iPhone > > On 9 Sep 2017, at 12:53, Tim Mackinnon <tim@testit.works > <mailto:tim@testit.works>> wrote: > >> Thanks for the pointer Marcus, I'll have a look - it's a fascinating area >> that I didn't know much about until now ;) >> >> All I wanted was to not have to select specific code to do senders (& >> refactoring), as well as have better statement selection so I can more >> easily put brackets around things... >> >> But now it dawns on me I'd like better code completion after I've already >> completed but maybe want the fuller keyword method.... >> >> Even the refactoring of local method changes shouldn't need you to save your >> method to rename a local variable or extract a statement to a variable. And >> I hate having to declare local variables painfully one by one - and I miss >> the option to detect a simple typo and correct it. >> >> These are all things that I love in IntelliJ - and I think we have fantastic >> infrastructure to do these things too. So I'll keep plugging away at lunch >> times to see what I can do. >> >> I've rewritten it about 3 times already ;) >> >> Tim >> >> Sent from my iPhone >> >> On 9 Sep 2017, at 08:42, Marcus Denker <marcus.den...@inria.fr >> <mailto:marcus.den...@inria.fr>> wrote: >> >>> there is #parseFaultyMethod:, too. We use RBParser for syntax highlighting. >>> >>> We do already have “senders” and “implementors working in the AST when >>> using Suggestions. >>> >>> See SugsMenuBuilder>>#findBestNodeFor: >>> >>> in the case of non-accepted code, it uses the compiler chain with >>> #optionParseErrors (which uses parseFaultyMethod: to parse). >>> >>> context hasUnacceptedEdits >>> ifTrue: [ >>> root := Smalltalk compiler >>> source: context code; >>> options: #(+ optionParseErrors); >>> parse. >>> context selectedClass ifNotNil: [ :theClass | root >>> methodNode methodClass: theClass ]. >>> root doSemanticAnalysis ] >>> >>> This works well but fails when you actually are in an error node.. for the >>> same code the “dumb” way we use >>> with standard “senders of” just takes line, looks for spaces and feeds that >>> word into the “senders of”.. >>> >>> So to make this work better we need to combine both approaches and deal >>> better with the Error Node content. >>> (e.g. trying to fix it and re-parse or just use the dumb “get the work >>> under the cursor” way as a first step). >>> >>> Marcus >>> >>>> On 8 Sep 2017, at 03:43, Tim Mackinnon <tim@testit.works >>>> <mailto:tim@testit.works>> wrote: >>>> >>>> Damn you smalltalk… you suck me in to little side problems…. >>>> >>>> RBParser is actually a lot more flexible than I had realised - and not so >>>> difficult to understand for a mere mortal… >>>> >>>> It naively seems that (what I still consider a confusingly named method) >>>> is quite simple to extend to handle at least medium sized errors just by >>>> doing something like: >>>> >>>> ast := [RBParser parseMethod: source onError: [ :msg :pos :parser | >>>> |seqNode errorNode| >>>> >>>> errorNode := RBParseErrorNode >>>> errorMessage: msg value: parser currentToken value >>>> printString at: pos. >>>> >>>> seqNode := parser sequenceNodeClass new. >>>> parser >>>> step; >>>> parseStatements: true into: seqNode. >>>> >>>> seqNode >>>> addNodeFirst: errorNode; >>>> yourself >>>> >>>> ]] onDNU: #body: do: [ nil “Handle non method source >>>> fragments"]. >>>> >>>> >>>> NOTE: I did have to add >>#currentToken as a method to RBParser as it >>>> doesn’t expose very much for the onError blocks, which I think it should >>>> to make them a bit more useful. >>>> >>>> Tim >>>> >>>>> On 7 Sep 2017, at 14:39, Tim Mackinnon <tim@testit.works >>>>> <mailto:tim@testit.works>> wrote: >>>>> >>>>> Thanks Thierry - I’ve been noodling in the background and next I hit the >>>>> question of “comments” - which I found RB does handle… and then as per >>>>> your point - how to skip past an error and try further down (which it >>>>> seems that the scanner might have the info to let me skip simple things). >>>>> You seem to get sucked in more and more when trying to to this properly - >>>>> which does make it sound like a very good internship or GSoc project. >>>>> >>>>> I think I may have enough to get something polished enough to release to >>>>> the wider world. I certainly love having better keyboard support when I’m >>>>> coding - as we type a lot (even though we have simple syntax) - and >>>>> having efficient tools takes away the friction. >>>>> >>>>> Next on my hit list, it add the same support to the refactoring tools - >>>>> which while sophisticated are so clunky in use when compared to IntelliJ. >>>>> >>>>> Tim >>>>> >>>>>> On 7 Sep 2017, at 08:02, Thierry Goubier <thierry.goub...@gmail.com >>>>>> <mailto:thierry.goub...@gmail.com>> wrote: >>>>>> >>>>>> Hi Tim, >>>>>> >>>>>> 2017-09-01 13:39 GMT+02:00 Tim Mackinnon <tim@testit.works >>>>>> <mailto:tim@testit.works>>: >>>>>> Thanks Thierry - this is proving an interesting problem domain ... >>>>>> >>>>>> I too am using #bestNodeFor: although I don't find it always gives the >>>>>> best node (it does if you are clearly in the target range, but if you >>>>>> are on the outer boundary like the next position after a selector or >>>>>> next to a "." or ";" it favours the parent and not the closest node. So >>>>>> in usage I find I need to tweak it a bit. >>>>>> >>>>>> Yes, when you are on a boundary, then finding the right node may be >>>>>> difficult. When developping the ancestor to smart suggestion, I >>>>>> considered #bestNodeFor: as good enough, but I considered looking into >>>>>> "up" and "down" ast navigation at a time, and I wasn't alone (I think it >>>>>> was active in the Pharo editor at a point). >>>>>> >>>>>> >>>>>> I'll look at smacc though - also there is that experimental setting to >>>>>> allow parse errors, I don't know if it helps in any way. >>>>>> >>>>>> There is a general question there, which is how you try to parse as much >>>>>> as possible while jumping over the error, making the error area as small >>>>>> as possible (that would be really great for syntax highlighting, by the >>>>>> way). >>>>>> >>>>>> The problem is that in hand-written parsers, you need to hard-code the >>>>>> error management in the parser. With a table-driven parser (as is >>>>>> SmaCC), then you can explore in a systematic way what are the possible >>>>>> states that would allow the parsing to continue correctly after making >>>>>> the error area as small as possible. >>>>>> >>>>>> This would make for a very nice internship subject... >>>>>> >>>>>> >>>>>> All this said, I think I have a workable suggestion that is not much >>>>>> code that might be open to scrutiny from those wiser than me. >>>>>> >>>>>> Looking at the keyboard handling in the system - it's quite sprawling >>>>>> and tricky to follow. I think there is lots of room for refactoring. >>>>>> >>>>>> It is very easy in Pharo to implement a nice and clean keyboard >>>>>> handling; all the necessary components have been in place for years, and >>>>>> we already have implementations (using the KM API). >>>>>> >>>>>> Now, most Pharo developpers just jump into hardcoding keyboard handling >>>>>> instead. >>>>>> >>>>>> Thierry >>>>>> >>>>>> >>>>>> I'm also not sure if I like the pragma usage either - I find it quite >>>>>> difficult to follow what's calling what. >>>>>> >>>>>> Tim >>>>>> >>>>>> Sent from my iPhone >>>>>> >>>>>> On 1 Sep 2017, at 09:18, Thierry Goubier <thierry.goub...@gmail.com >>>>>> <mailto:thierry.goub...@gmail.com>> wrote: >>>>>> >>>>>>> Hi Tim, >>>>>>> >>>>>>> The RB ast has a specific method for finding the right node: in >>>>>>> AltBrowser, I use ast bestNodeFor: aTarget selectionInterval. >>>>>>> >>>>>>> For the second case (parse what is selected), you may want to look into >>>>>>> SmaCC: it has specific entry points (used for refactoring) to try all >>>>>>> possible starts in a parser for a given piece of text and returning all >>>>>>> the possible correct asts. As a hand-writen parser, the RBParser can't >>>>>>> do that unless you add the necessary entry points by hand and hack >>>>>>> around the error handling, because most of the parse attempts end on an >>>>>>> error (as they should). >>>>>>> >>>>>>> Regards, >>>>>>> >>>>>>> Thierry >>>>>>> >>>>>>> 2017-09-01 8:50 GMT+02:00 Tim Mackinnon <tim@testit.works >>>>>>> <mailto:tim@testit.works>>: >>>>>>> Marcus - I'd be interested in how you thought to solve this. >>>>>>> >>>>>>> My solution seems to work - but it's not the most elegant. Retrying >>>>>>> lots of different ways like I show, smacks a bit of desperation (this >>>>>>> said, it is quite compact). >>>>>>> >>>>>>> Apart from that - I was actually interested in thoughts on the parse >>>>>>> method - the onError naming is a bit misleading compared to other >>>>>>> methods in the image. I wonder if it should be called something like >>>>>>> "onErrorMerge:" to signal that it's going to reuse the results? >>>>>>> >>>>>>> Stef - with regards to broken source, my proposed solution reuses what >>>>>>> is already there - thus if you highlight specific code, it reverts to >>>>>>> the original strategy and tries to compile just that code. This bugs me >>>>>>> however - so if you highlight nothing, it tries to compile the source >>>>>>> multiple ways, I drop down to chopping the source up to where your >>>>>>> cursor is - which seems like a decent cheap strategy. >>>>>>> >>>>>>> I don't know if our parser is able to recover from simple errors and >>>>>>> just have error nodes in the ast, and whether the #bestNodeFor method >>>>>>> then ignored error nodes... this is what I think Dolphin used to do. >>>>>>> >>>>>>> Tim >>>>>>> >>>>>>> Sent from my iPhone >>>>>>> >>>>>>> > On 31 Aug 2017, at 19:11, Marcus Denker <marcus.den...@inria.fr >>>>>>> > <mailto:marcus.den...@inria.fr>> wrote: >>>>>>> > >>>>>>> > >>>>>>> >> On 31 Aug 2017, at 19:07, Stephane Ducasse <stepharo.s...@gmail.com >>>>>>> >> <mailto:stepharo.s...@gmail.com>> wrote: >>>>>>> >> >>>>>>> >> On Wed, Aug 30, 2017 at 9:50 PM, Tim Mackinnon <tim@testit.works >>>>>>> >> <mailto:tim@testit.works>> wrote: >>>>>>> >>> I’m looking for some feedback on the usage of >>>>>>> >>> RBParser>>#parseMethod:onError:. >>>>>>> >>> >>>>>>> >>> I am looking at ways of improving the way we edit code - and making >>>>>>> >>> things work (as least for me - but maybe for everyone) a bit more >>>>>>> >>> like IntelliJ where they have some great code operations and very >>>>>>> >>> good keyboard support. We are certainly close, but at times feel a >>>>>>> >>> bit clumsy - which is a shame as we have far better infrastructure >>>>>>> >>> to support equivalent features. So I thought I would have a go. >>>>>>> >> >>>>>>> >> Excellent!!! >>>>>>> >> >>>>>>> >> >>>>>>> >>> Anyway - step one (which I think I’ve come close on), was to teach >>>>>>> >>> senders/implementors to look at the AST so you don’t have to >>>>>>> >>> highlight exactly what you want to search on - instead it can infer >>>>>>> >>> from your cursor… however my next step was to improve how we can >>>>>>> >>> select code to ease refactoring, bracketing etc… >>>>>>> >> >>>>>>> >> Yes but we have to handle broken code then. >>>>>>> >> Did you check how smart suggestions did it for code that compiled? >>>>>>> >> >>>>>>> > >>>>>>> > I looked some weeks ago and made some notes what to do… but I fear >>>>>>> > this has to wait for next week, >>>>>>> > there is no way that I can do anything till ESUG… >>>>>>> > >>>>>>> > Marcus >>>>>>> >>>>>>> >>>>>>> >>>>>> >>>>> >>>> >>>
