Re: [racket-dev] Line editing in the default REPL
I looked at the code a little, but I couldn't find the place that does the fallback part. (You're defining a `libreadline-path' which AFAICS isn't used.) But it looks like you're using the same interface for both editline and readline -- right? If so, then I think that it's better to just keep the current setup, and do this: (define libreadline (ffi-lib libreadline '(5 6 4 ) #:fail (lambda () (ffi-lib libedit ... Woops, thanks for pointing this out. I meant to put this here: (define libreadline (ffi-lib libreadline-path '(5 6 4 ) #:fail (lambda () #f))) This doesn't fallback to libedit if it can't find readline. It falls back to a non line editing terminal if it can't find the requested lib. The user needs to explicitly request readline or libedit. Although we could have a third module that tries both before it falls back to a non line editing terminal (or one that uses a line editor in racket.) And since that addresses an API that is shared for the two, then it's not code that should be linked only with readline, which means that it's fine to it by default...? This does seem to be a common opinion among people here (at least Sam seemed to share this same opinion anyway). And while it seems reasonable to me, I've never heard this anywhere else. Also, it would completely negate the FSFs reason for releasing libreadline under the gpl. (Since libedit and libreadline share the same bindings, you can almost always use them interchangeably.) I'm thinking we should ask this on the debian-legal mailing list (as they would probably have more experience). But if we can (legally) do it, I'm all for it. :) ~Leif Andersen On Tue, Dec 23, 2014 at 11:57 AM, Eli Barzilay e...@barzilay.org wrote: On Wed, Dec 17, 2014 at 12:35 PM, Tony Garnock-Jones to...@ccs.neu.edu wrote: If anyone reading this has an interesting or unusual terminal they like to use, please run $ raco pkg install ansi $ racket -l ansi/test-raw I didn't run it, but guessing common keys isn't too difficult, of course. I'm attaching a piece of code that I have that does that. It looks to me like my code is much more ambitious -- I wanted to be able to have almost all possible keys with modifiers, including function keys and plain characters, and that makes it messier. (I posted variations of this thing a few times in the past.) likely evolve into some form of a terminfo-like facility. So it's better to just write something that can deal with terminfo directly. Sadly, terminfo is a little anaemic with respect to the sequences it specifies. To get input capabilities even half as rich as, say, emacs, you have to go beyond what is given in the terminfo database in a couple of ways. Specifically, to parse shift/control/meta combinations you need to get into terminal-specific encodings that are not covered by terminfo. Sure, but those are usually not specific too. (You're probably talking about the DEC thing, right? There's also the rxvt/aterm thing which is different and more weird.) In any case, terminfo for reading keys is obviously not too difficult to manage. If that was all that you need, then it's very easy to write code that translates the terminfo database into some readable format that you can read in Racket. (My code is basically doing that parsing, so it's 90% of what you need for that.) But the thing is that terminfo is much more needed for the related things -- querying the terminal, using escape sequences to do the interactions (and doing that without restricting yourself to some small subset), and also sending sequences that setup the terminal (like rmkx/smkx which is what tripped me earlier, and IIRC, it's needed with at least st, possibly others too). it's the blindness of going in that direction and thinking that you *won't* fall into this trap. It remains to be seen whether there are any problems resulting from this approach at all. It looks like *you're* very aware of the issues, so why not take it? -- ((lambda (x) (x x)) (lambda (x) (x x))) Eli Barzilay: http://barzilay.org/ Maze is Life! _ Racket Developers list: http://lists.racket-lang.org/dev
Re: [racket-dev] Line editing in the default REPL
On Wed, Dec 17, 2014 at 12:35 PM, Tony Garnock-Jones to...@ccs.neu.edu wrote: If anyone reading this has an interesting or unusual terminal they like to use, please run $ raco pkg install ansi $ racket -l ansi/test-raw I didn't run it, but guessing common keys isn't too difficult, of course. I'm attaching a piece of code that I have that does that. It looks to me like my code is much more ambitious -- I wanted to be able to have almost all possible keys with modifiers, including function keys and plain characters, and that makes it messier. (I posted variations of this thing a few times in the past.) likely evolve into some form of a terminfo-like facility. So it's better to just write something that can deal with terminfo directly. Sadly, terminfo is a little anaemic with respect to the sequences it specifies. To get input capabilities even half as rich as, say, emacs, you have to go beyond what is given in the terminfo database in a couple of ways. Specifically, to parse shift/control/meta combinations you need to get into terminal-specific encodings that are not covered by terminfo. Sure, but those are usually not specific too. (You're probably talking about the DEC thing, right? There's also the rxvt/aterm thing which is different and more weird.) In any case, terminfo for reading keys is obviously not too difficult to manage. If that was all that you need, then it's very easy to write code that translates the terminfo database into some readable format that you can read in Racket. (My code is basically doing that parsing, so it's 90% of what you need for that.) But the thing is that terminfo is much more needed for the related things -- querying the terminal, using escape sequences to do the interactions (and doing that without restricting yourself to some small subset), and also sending sequences that setup the terminal (like rmkx/smkx which is what tripped me earlier, and IIRC, it's needed with at least st, possibly others too). it's the blindness of going in that direction and thinking that you *won't* fall into this trap. It remains to be seen whether there are any problems resulting from this approach at all. It looks like *you're* very aware of the issues, so why not take it? -- ((lambda (x) (x x)) (lambda (x) (x x))) Eli Barzilay: http://barzilay.org/ Maze is Life! keys.rkt Description: Binary data _ Racket Developers list: http://lists.racket-lang.org/dev
Re: [racket-dev] Line editing in the default REPL
Okay, here's another idea. I have parametrized the readline collection over the readline/edit/etc. library. You can do: (require editline) For the editline equivalent. It also falls back to no line editing (rather than throwing an exception), if the library is not there. The source is here: https://github.com/LeifAndersen/readline I'm thinking we could distribute the editline collection with Racket (treating libedit the same way we currently treat other native packages such as libcairo). And then we can make xrepl default. Would this be doable? ~Leif Andersen On Wed, Dec 3, 2014 at 5:50 PM, Eli Barzilay e...@barzilay.org wrote: On Wed, Dec 3, 2014 at 6:22 PM, Sam Tobin-Hochstadt sa...@cs.indiana.edu wrote: On Wed, Dec 3, 2014 at 6:10 PM, Eli Barzilay e...@barzilay.org wrote: If you're talking about implementing line editing yourself, then my personal reaction to that would be wonderful, but doing it properly is something that is difficult and easy to underestimate I've already done this (admittedly it only works on OS X, but most Linux terminals work exactly the same with a few different constants). You can see what I have so far here: https://github.com/LeifAndersen/racket-line-editor/blob/master/main.rkt If this works, I'd be thoroughly impressed -- so I tried it. I ran through a bunch of configurations that I use: None of which was OS X, which was what Leif's email explicitly said it _only works on_. My point wasn't it doesn't work outside of OSX, I was talking about a few different constants being a bad underestimation: they're not few, they're not constant, and the interfaces for getting them is what you'd expect from something that started to grow in the 70s. I think we should wait till he has a version which he says works before telling people to avoid making contributions. You should re-read my email: if there's anything that I'm telling, it's the exact opposite. A point that I repeated more than once. The thing is that it would be easy to not say anything, and watch this approach of try a few escapes and see what works fail. And it will fail, because, again, they're not constant, and you can't just get some magic combination that works for 90% of the people -- even just xterm on just linux is something that can (and often is) configured in many different ways. So what I'm doing is the opposite: I point at how such a thing should be written for it to survive -- but I appreciate Leif's time enough to warn him that this is a much bigger thing than what he seems to think. (And to be clear, the reason for this is that I've seen probably around 3 to 5 serious attempts that got dumped; and I have encountered these issues multiple times, so I know that it's hard enough that even getting simple code to work, code that I intended only for my own use, even that was pretty hard, enough to go chase old VT references and obscure man pages.) -- ((lambda (x) (x x)) (lambda (x) (x x))) Eli Barzilay: http://barzilay.org/ Maze is Life! _ Racket Developers list: http://lists.racket-lang.org/dev
Re: [racket-dev] Line editing in the default REPL
[I'll reply to your email separately...] Sorry for not sending the results I had earlier re parsing input keys. Just to get this recorded here in case someone wants to do this in the future: I was first optimistic about the prospect of parsing keys in a way that works for all terminals, provided that it's limited to the few keys that are needed for line editing. I wrote some code to parse terminfo texts which I extracted on my linux installation, and find conflicts. Turns out that it's easy to do what I wanted only for the arrow keys, but then things got very messy. Specifically, there are terminals that use the opposite escape sequences for home/end or backspace/delete. (My best guess about the origin of some of this is that there were numeric pads that were laid out top-to-bottom as in phones, and the escape sequences were based on the kp keys.) It might be that the really bad conflicts were happen with irrelevant terminals, but there are definitely conflicts involving terminals that are relevant, like xrvt and gnome-terminal. My conclusion is that even for a very simple thing you need to consult $TERM to know what to do, and some simple attempt to do that would most likely evolve into some form of a terminfo-like facility. So it's better to just write something that can deal with terminfo directly. FWIW, I don't think that it's too hard -- specifically for reading keys, it's straightforward since you just get the escape sequence that you need to identify on input. I'm attaching the code that I wrote. (To run it you might need to adjust the location of the terminfo files at the top of the code; also, it relies on infocmp as a way to print them out.) If you run it you get a huge listing of all keys, with indications about conflicts. The reallly bad conflicts would be cases where some escape sequence is a prefix of another -- and those are rare (look for prefix) and happen only with some obscure F keys. (I initially just looked at these which is why I thought that it's doable.) Also, the `parse-decisions' thing in the code is my attempt to encode how a parser should work and have it show the rest of the conflicts. I planned to add more keys and keep minimal conflicts but then I got to the ones that don't have a good solution. As an example, this part of the output: end: \eOq (!! (aterm mrxvt rxvt ... xterm-xi): clash with home of (nsterm ... vt102)) (!! (aterm mrxvt rxvt ... xterm-xi): clash with key_f0 of (xterm-r5)) \e[4~ (!! (... cygwin ... linux putty ... xterm-vt220): clash with key_select of (... aterm gnome ... rxvt ... xterm-24 ...)) shows one of these problems, where end on rxvt (popular enough) correspond to home on vt102 (popular emulation target), and the last thing shows a conflict between cygwin/linux/putty (all popular) and aterm/gnome/rxvt (popular too). [Extended footnote re time investing in this nightmarish pit of depair: If you stare at this for a while, and consider using these results, you'll get a feeling of how bad it can be... Maybe vt102 is not used too much? Would there be any need for a select key, or maybe the same escape sequence is mapped to something else that is useful? The bottom line is that I don't really reject the noiseline-style attempt to get something that works for most usert -- it's the blindness of going in that direction and thinking that you *won't* fall into this trap. (Or, if you really won't since you'll refuse to accomodate bugreports, then you'll make someone else fall into this trap because of you.)] -- ((lambda (x) (x x)) (lambda (x) (x x))) Eli Barzilay: http://barzilay.org/ Maze is Life! std-keys.rkt Description: Binary data _ Racket Developers list: http://lists.racket-lang.org/dev
Re: [racket-dev] Line editing in the default REPL
On Sat, Dec 13, 2014 at 7:00 PM, Leif Andersen l...@leifandersen.net wrote: Okay, here's another idea. I have parametrized the readline collection over the readline/edit/etc. library. You can do: (require editline) For the editline equivalent. It also falls back to no line editing (rather than throwing an exception), if the library is not there. The source is here: https://github.com/LeifAndersen/readline (For reference: it's in the editline branch...) I'm thinking we could distribute the editline collection with Racket (treating libedit the same way we currently treat other native packages such as libcairo). And then we can make xrepl default. Would this be doable? I looked at the code a little, but I couldn't find the place that does the fallback part. (You're defining a `libreadline-path' which AFAICS isn't used.) But it looks like you're using the same interface for both editline and readline -- right? If so, then I think that it's better to just keep the current setup, and do this: (define libreadline (ffi-lib libreadline '(5 6 4 ) #:fail (lambda () (ffi-lib libedit ... And since that addresses an API that is shared for the two, then it's not code that should be linked only with readline, which means that it's fine to it by default...? -- ((lambda (x) (x x)) (lambda (x) (x x))) Eli Barzilay: http://barzilay.org/ Maze is Life! _ Racket Developers list: http://lists.racket-lang.org/dev
Re: [racket-dev] Line editing in the default REPL
My goal was not to replace xrepl, but to provide basic line editing support to the default repl without licensing violations or massively increasing the distribution size. If you're talking about implementing line editing yourself, then my personal reaction to that would be wonderful, but doing it properly is something that is difficult and easy to underestimate I've already done this (admittedly it only works on OS X, but most Linux terminals work exactly the same with a few different constants). You can see what I have so far here: https://github.com/LeifAndersen/racket-line-editor/blob/master/main.rkt Then you need to deal with the trickyness of a proper editor, with features like blinking a matching paren... While these features are cool, they are things that can be added on later (by extending the line editor I linked to above). ~Leif Andersen On Wed, Dec 3, 2014 at 2:55 AM, Neil Van Dyke n...@neilvandyke.org wrote: Eli Barzilay wrote on 12/02/2014 09:31 PM: On Tue, Nov 25, 2014 at 1:29 PM, Leif Andersen l...@leifandersen.net wrote: Just to clarify a bit, we were more thinking of extending the default repl to have line editing features, rather then making xrepl the default, If you're talking about implementing line editing yourself, then my personal reaction to that would be wonderful, but doing it properly is something that is difficult and easy to underestimate I agree fully with Eli. Also, separate from practical benefit of having basic `readline`-like line editing in pure Racket, you could go gratuitously coolness factor on this. For example, I don't think I've seen a non-full-screen character-terminal REPL do syntax coloring, matching paren highlighting, and full up/down cursor navigation within a single REPL input. That's labor-of-love coolness, and all who behold it will respect your name. (http://www.neilvandyke.org/racket-charterm/; would help with some primitives and device portability, but you'd still have to layer a lot of work atop that. Offhand, the most nonobvious tricky part I can think of is not getting program confused about where text and your cursor are on the screen. Different terminals will have different behavior when cursor is in the last column, or cursor is in the last row and column, or you're inserting/deleting characters or lines (which some terminals will support differently, and others not at all). You also have to decide how you want to handle the user experience of some kind of Ctrl-L redraw, since terminal screens get corrupted often by text written by other processes, Ctrl-L also helps mitigate terminal quirks you don't yet handle, and terminal-savvy people will expect a Ctrl-L. It's possible to make a self-healing character terminal display, on those terminals that permit reading screen addresses and that have sufficient idle bandwidth, but I haven't heard of anyone doing that yet, and everyone just has a Ctrl-L.) Neil V. _ Racket Developers list: http://lists.racket-lang.org/dev
Re: [racket-dev] Line editing in the default REPL
On Wed, Dec 3, 2014 at 2:45 PM, Leif Andersen l...@leifandersen.net wrote: My goal was not to replace xrepl, but to provide basic line editing support to the default repl without licensing violations or massively increasing the distribution size. Yes, that's exactly what I was talking about. I hope that you know what you're doing, though so far the easy to underestimate seem to be the case: If you're talking about implementing line editing yourself, then my personal reaction to that would be wonderful, but doing it properly is something that is difficult and easy to underestimate I've already done this (admittedly it only works on OS X, but most Linux terminals work exactly the same with a few different constants). You can see what I have so far here: https://github.com/LeifAndersen/racket-line-editor/blob/master/main.rkt If this works, I'd be thoroughly impressed -- so I tried it. I ran through a bunch of configurations that I use: - plain xterm, - ssh in xterm, - linux text console, - ssh in a linux console, - `term' in Emacs (a terminal emulator, not the more popular inferior shell), - ssh from windows on a mintty, - ssh from windows in a cmd box (Win7, much better than past versions) and in a powershell window, - putty from windows. (The racket process is always running in a linux machine, there's no need for it on Windows.) In each of these I first tried readline (a basic test: start racket, move around, go back in history to a multiline entry) -- it worked fine in all cases. I then tried your code -- took your file as-is, and added (module+ main (line-editor rkt )) in the end. It failed in all cases. The failures varied, but in pretty much all cases it just spits out an escape sequence (which is not the right one for the terminal, otherwise it wouldn't be shown), a different one in each case. I then tried some navigation keys, and none of them worked either. In some cases they'd move the cursor where it shouldn't go (the cmd-based cases, as expected), in other cases keys behaved in various broken ways: spitting uncaught escapes, moveing two lines up for each keypress, the end key moved down a line in one case, the return key spits out a ^M. Using C-j twice (in some cases; I'm too tired to try them all) seem to give your code what it wants to start reading so it displays the prompt when I do that -- then, the navigation keys are still broken, unsurprisingly (and random escape sequences still garble what I see). Another weirdness is that it looks like your code waits for a ^M, sometime later followed by a ^J, and then it returns a string: this is very broken since a ^J is ignored before the ^M, and there can be any number of characters between the ^M and the ^J. (But that might be bogus, since things are very garbled anyway, so it's hard to guess what's going on.) But it's actually looking at the code that makes me conclude that you're underestimating how complicated getting this right can be. Some various comments in no particular order: * Looks like there is no querying of the terminal for capabilities, and there's no form of database of terminals. See the man pages for *termcap and *terminfo for libraries that implement these things, and you can also check the Emacs source code which still has a lot of code that deals with that in the term directory. * These are needed to know what you can spit out, and what you can expect to read in. Assuming that the rest of the terminal world behaves like the random one you're using is a good recipe for getting something completely broken. * Looking at the code in `edit', the little that it does have is very very broken. (If this is a translation of linenoise, then feel free to forward it to whoever does that...) - You should generally prefer `write-bytes' and `read-byte' to avoid getting bogus character encoding in the way. (But see above: you need to consult the terminal to know if it can give you 8-bit or even wide characters.) - There is no form of abstraction here, resulting in a monolithic piece of code that is going to be a maintenance disaster. You should separate out the code that reads a key to a different function, and make it return some proper symbolic name (combined with characters for simple keys with an ascii equivalent). This way you can also hook more functionality on more keys. - Some of these escape sequences look like they could work, but there are a ton of variations. For example: I have seen the up key generate at least \e[A, \eOA, and \e1;1A. - The last form is interesting, since it's a new-ish way to represent keys with modifiers, so you get \e1;NA with N being 1 to 8 indicating no-modifiers, shift, alt/meta, shift-meta, control, etc. The are similar generic escapes for most keys, and that's one case that is easy to parse. As a different example, my xterm generates \e[27;6;85~ for shift-control-U:
Re: [racket-dev] Line editing in the default REPL
On Wed, Dec 3, 2014 at 6:10 PM, Eli Barzilay e...@barzilay.org wrote: If you're talking about implementing line editing yourself, then my personal reaction to that would be wonderful, but doing it properly is something that is difficult and easy to underestimate I've already done this (admittedly it only works on OS X, but most Linux terminals work exactly the same with a few different constants). You can see what I have so far here: https://github.com/LeifAndersen/racket-line-editor/blob/master/main.rkt If this works, I'd be thoroughly impressed -- so I tried it. I ran through a bunch of configurations that I use: None of which was OS X, which was what Leif's email explicitly said it _only works on_. I think we should wait till he has a version which he says works before telling people to avoid making contributions. Sam _ Racket Developers list: http://lists.racket-lang.org/dev
Re: [racket-dev] Line editing in the default REPL
On Wed, Dec 3, 2014 at 6:22 PM, Sam Tobin-Hochstadt sa...@cs.indiana.edu wrote: On Wed, Dec 3, 2014 at 6:10 PM, Eli Barzilay e...@barzilay.org wrote: If you're talking about implementing line editing yourself, then my personal reaction to that would be wonderful, but doing it properly is something that is difficult and easy to underestimate I've already done this (admittedly it only works on OS X, but most Linux terminals work exactly the same with a few different constants). You can see what I have so far here: https://github.com/LeifAndersen/racket-line-editor/blob/master/main.rkt If this works, I'd be thoroughly impressed -- so I tried it. I ran through a bunch of configurations that I use: None of which was OS X, which was what Leif's email explicitly said it _only works on_. My point wasn't it doesn't work outside of OSX, I was talking about a few different constants being a bad underestimation: they're not few, they're not constant, and the interfaces for getting them is what you'd expect from something that started to grow in the 70s. I think we should wait till he has a version which he says works before telling people to avoid making contributions. You should re-read my email: if there's anything that I'm telling, it's the exact opposite. A point that I repeated more than once. The thing is that it would be easy to not say anything, and watch this approach of try a few escapes and see what works fail. And it will fail, because, again, they're not constant, and you can't just get some magic combination that works for 90% of the people -- even just xterm on just linux is something that can (and often is) configured in many different ways. So what I'm doing is the opposite: I point at how such a thing should be written for it to survive -- but I appreciate Leif's time enough to warn him that this is a much bigger thing than what he seems to think. (And to be clear, the reason for this is that I've seen probably around 3 to 5 serious attempts that got dumped; and I have encountered these issues multiple times, so I know that it's hard enough that even getting simple code to work, code that I intended only for my own use, even that was pretty hard, enough to go chase old VT references and obscure man pages.) -- ((lambda (x) (x x)) (lambda (x) (x x))) Eli Barzilay: http://barzilay.org/ Maze is Life! _ Racket Developers list: http://lists.racket-lang.org/dev
Re: [racket-dev] Line editing in the default REPL
Eli Barzilay wrote on 12/02/2014 09:31 PM: On Tue, Nov 25, 2014 at 1:29 PM, Leif Andersen l...@leifandersen.net wrote: Just to clarify a bit, we were more thinking of extending the default repl to have line editing features, rather then making xrepl the default, If you're talking about implementing line editing yourself, then my personal reaction to that would be wonderful, but doing it properly is something that is difficult and easy to underestimate I agree fully with Eli. Also, separate from practical benefit of having basic `readline`-like line editing in pure Racket, you could go gratuitously coolness factor on this. For example, I don't think I've seen a non-full-screen character-terminal REPL do syntax coloring, matching paren highlighting, and full up/down cursor navigation within a single REPL input. That's labor-of-love coolness, and all who behold it will respect your name. (http://www.neilvandyke.org/racket-charterm/; would help with some primitives and device portability, but you'd still have to layer a lot of work atop that. Offhand, the most nonobvious tricky part I can think of is not getting program confused about where text and your cursor are on the screen. Different terminals will have different behavior when cursor is in the last column, or cursor is in the last row and column, or you're inserting/deleting characters or lines (which some terminals will support differently, and others not at all). You also have to decide how you want to handle the user experience of some kind of Ctrl-L redraw, since terminal screens get corrupted often by text written by other processes, Ctrl-L also helps mitigate terminal quirks you don't yet handle, and terminal-savvy people will expect a Ctrl-L. It's possible to make a self-healing character terminal display, on those terminals that permit reading screen addresses and that have sufficient idle bandwidth, but I haven't heard of anyone doing that yet, and everyone just has a Ctrl-L.) Neil V. _ Racket Developers list: http://lists.racket-lang.org/dev
Re: [racket-dev] Line editing in the default REPL
Do you have in mind making xrepl intended to be part of Minimal Racket? If not, what's the mechanism for `racket` using xrepl when it's available? A similar question applies to libeditline. Currently, for Linux and other Unix platforms (not counting natipkg variants), our convention is that native libraries are either part of the `[g]racket` executable or they are installed separately by users through the OS's package manager. We can't link to libreadline by default in a Racket distribution, and since libeditline isn't typically included with Linux distributions (as far as I can tell), it seems like we haven't solved any problem unless we provide libeditline. Should libeditline become not only part of the Minimal Racket distribution, but even part of the Racket executable? Or should our convention and/or distribution infrastructure change? At Mon, 24 Nov 2014 18:02:45 -0500, Sam Tobin-Hochstadt wrote: My understanding of the licensing issues is that if the code works with both libeditline and libreadline then it isn't a derived work of readline, and therefore could be licensed under the LGPL, like the rest of Racket. Furthermore, turning use of libeditline on by default wouldn't be linking to any GPL code, meaning that we could do that by default. I think we should split up the `readline` collection into `readline` and `readline/base` which would be what's compatible with editline, and have xrepl in a `readline/base` mode on by default. Sam On Mon, Nov 24, 2014 at 5:57 PM, Leif Andersen l...@leifandersen.net wrote: Hello, When a user first starts the racket repl, it doesn't do line editing due to licensing issues. For example, it would be nice if the default editor would give the previous line if you hit up arrow, rather than writing ^[[A. I have now pointed out xrepl to several users, and every time they always ask me why it's not the default repl. Apparently the problem is that xrepl uses GNU Readline, which is GPL. However, Asumu found that if we replace requiring readline with BSD's libedit (not libeditline), everything works fine due to libedit's readline compatibility. It doesn't have all of the features of readline, but it does have some of the biggest ones (such as being able to use arrow keys) What do you all think of having `(require editline)` that works for the default repl, so that it gets line editing features. This would allow us to also keep `(require readline)` as before, maintaining backwards compatibility. If we do do this, this leads to the question of distribution. Would we want to include libedit inside Racket distributions, or should we just link to whatever the user has on their system? ~Leif Andersen _ Racket Developers list: http://lists.racket-lang.org/dev _ Racket Developers list: http://lists.racket-lang.org/dev _ Racket Developers list: http://lists.racket-lang.org/dev
Re: [racket-dev] Line editing in the default REPL
On Tue, Nov 25, 2014 at 11:00 AM, Matthew Flatt mfl...@cs.utah.edu wrote: Do you have in mind making xrepl intended to be part of Minimal Racket? If not, what's the mechanism for `racket` using xrepl when it's available? I can think of a few ways of doing this. 1 Just make xrepl part of minimal Racket (probably removing the mandatory dependency on scribble/text/wrap) 2 Have `racket/init` dynamically test of the presence of xrepl/main.rkt as a collection-file-path, and load it if available. 3 Have `racket/init` (or the `racket` binary) test for something like `racket/init/extended` which would be part of xrepl and part of other extended repls. 4 Like 3, but have an additional package which depends on xrepl and just contains `racket/init/extended` I prefer 2, 3, or 4 of these options -- it seems fine for Minimal Racket to not have line editing, but I'd be interested to hear what others expect. A similar question applies to libeditline. Currently, for Linux and other Unix platforms (not counting natipkg variants), our convention is that native libraries are either part of the `[g]racket` executable or they are installed separately by users through the OS's package manager. We can't link to libreadline by default in a Racket distribution, and since libeditline isn't typically included with Linux distributions (as far as I can tell), it seems like we haven't solved any problem unless we provide libeditline. Should libeditline become not only part of the Minimal Racket distribution, but even part of the Racket executable? Or should our convention and/or distribution infrastructure change? I was thinking that the readline-core package would dynamically test of libeditline, and if it's not there fall back to libreadline. My opinion, as someone who isn't a lawyer but has read a lot about this, is that this would not cause Racket to be a derived work of readline. If we don't want to do that, I see a few possibilities: 1. We ship xrepl in whichever way we decide, and change the message it prints when it can't find libeditline to suggest changing things or adding (require xrepl/readline) to .racketrc. 2. We ship a copy of libeditlinewith xrepl as a built binary, even on linux. 3. We statically link libeditline to Racket in the standard distribution. I think 1 sounds most appealing if we're not ok with dynamically falling back to libreadline. Sam At Mon, 24 Nov 2014 18:02:45 -0500, Sam Tobin-Hochstadt wrote: My understanding of the licensing issues is that if the code works with both libeditline and libreadline then it isn't a derived work of readline, and therefore could be licensed under the LGPL, like the rest of Racket. Furthermore, turning use of libeditline on by default wouldn't be linking to any GPL code, meaning that we could do that by default. I think we should split up the `readline` collection into `readline` and `readline/base` which would be what's compatible with editline, and have xrepl in a `readline/base` mode on by default. Sam On Mon, Nov 24, 2014 at 5:57 PM, Leif Andersen l...@leifandersen.net wrote: Hello, When a user first starts the racket repl, it doesn't do line editing due to licensing issues. For example, it would be nice if the default editor would give the previous line if you hit up arrow, rather than writing ^[[A. I have now pointed out xrepl to several users, and every time they always ask me why it's not the default repl. Apparently the problem is that xrepl uses GNU Readline, which is GPL. However, Asumu found that if we replace requiring readline with BSD's libedit (not libeditline), everything works fine due to libedit's readline compatibility. It doesn't have all of the features of readline, but it does have some of the biggest ones (such as being able to use arrow keys) What do you all think of having `(require editline)` that works for the default repl, so that it gets line editing features. This would allow us to also keep `(require readline)` as before, maintaining backwards compatibility. If we do do this, this leads to the question of distribution. Would we want to include libedit inside Racket distributions, or should we just link to whatever the user has on their system? ~Leif Andersen _ Racket Developers list: http://lists.racket-lang.org/dev _ Racket Developers list: http://lists.racket-lang.org/dev _ Racket Developers list: http://lists.racket-lang.org/dev
Re: [racket-dev] Line editing in the default REPL
On Nov 25, 2014 11:01 AM, Matthew Flatt mfl...@cs.utah.edu wrote: We can't link to libreadline by default in a Racket distribution, and since libeditline isn't typically included with Linux distributions (as far as I can tell), it seems like we haven't solved any problem unless we provide libeditline. According to whohas[0], Debian, Ubuntu, Gentoo, and Mandriva ship editline packages. The lack of Fedora and OpenSUSE is a big hole, but libeditline isn't completely absent from the ecosystem. [0]: https://github.com/whohas/whohas Cheers, James _ Racket Developers list: http://lists.racket-lang.org/dev
Re: [racket-dev] Line editing in the default REPL
Just to clarify a bit, we were more thinking of extending the default repl to have line editing features, rather then making xrepl the default, or having xrepl use libedit rather that libreadline. It would not be too terrible if we required a user to have it installed to use it. (It's included in both OS X, and most Linux distros have good package management.) I also think making readline/base use libedit, and readline use libreadline is a bit of a red haring. As, afaik, they have the same API, it doesn't make sense for one of them to be the 'base' of the other. Here's another idea Asumu had (which I like), rather than bundling libedit with racket, or hoping the user has it installed we can take something like linenoise[1], and implement it in Racket (which is doable as the whole thing is about 1,000 lines of code anyway). I think this would avoid the dependency issue anyway. [1]: https://github.com/antirez/linenoise ~Leif Andersen On Tue, Nov 25, 2014 at 1:00 PM, Eli Barzilay e...@barzilay.org wrote: On Tue, Nov 25, 2014 at 11:38 AM, Sam Tobin-Hochstadt sa...@cs.indiana.edu wrote: On Tue, Nov 25, 2014 at 11:00 AM, Matthew Flatt mfl...@cs.utah.edu wrote: Do you have in mind making xrepl intended to be part of Minimal Racket? If not, what's the mechanism for `racket` using xrepl when it's available? I can think of a few ways of doing this. 1 Just make xrepl part of minimal Racket (probably removing the mandatory dependency on scribble/text/wrap) If this is the only dependency that stands in the way of making it more minimal, then it should be easy to make it dynamically loaded (like much of the other functionality it uses). Just have a wrapper that dynamically requires the wrap function, and if it fails, it will just not do the wrapping. (IIRC, the things that xrepl uses wrapping for fall outside of what you can get with a naive and quick implementation, like wrapping error messages where the indentation matters.) 2 Have `racket/init` dynamically test of the presence of xrepl/main.rkt as a collection-file-path, and load it if available. 3 Have `racket/init` (or the `racket` binary) test for something like `racket/init/extended` which would be part of xrepl and part of other extended repls. 4 Like 3, but have an additional package which depends on xrepl and just contains `racket/init/extended` I prefer 2, 3, or 4 of these options -- it seems fine for Minimal Racket to not have line editing, but I'd be interested to hear what others expect. (3) and (4) sound to me like an overkill -- `racket/init' is already supposed to be extra interactive repl stuff, and adding an extra-extra thing seem like overcomplicating it. (2) sounds perfectly fine though. Also, as I said above, getting (1) should be easy, but is that the only dependency that would fall outside of a minimal distribution? A similar question applies to libeditline. Currently, for Linux and other Unix platforms (not counting natipkg variants), our convention is that native libraries are either part of the `[g]racket` executable or they are installed separately by users through the OS's package manager. We can't link to libreadline by default in a Racket distribution, and since libeditline isn't typically included with Linux distributions (as far as I can tell), it seems like we haven't solved any problem unless we provide libeditline. Should libeditline become not only part of the Minimal Racket distribution, but even part of the Racket executable? Or should our convention and/or distribution infrastructure change? I was thinking that the readline-core package would dynamically test of libeditline, and if it's not there fall back to libreadline. Why is it needed to have a core subset? = What are the readline features that are missing from editline? My opinion, as someone who isn't a lawyer but has read a lot about this, is that this would not cause Racket to be a derived work of readline. If that's valid, then that would be really nice. If we don't want to do that, I see a few possibilities: 1. We ship xrepl in whichever way we decide, and change the message it prints when it can't find libeditline to suggest changing things or adding (require xrepl/readline) to .racketrc. 2. We ship a copy of libeditlinewith xrepl as a built binary, even on linux. 3. We statically link libeditline to Racket in the standard distribution. I think 1 sounds most appealing if we're not ok with dynamically falling back to libreadline. (1) sounds good, except that it must target naive users -- so suggetsing package installation is not a good idea. An alternative to a printout would be a question: I can't find editline (a library for line editing), but I see that you have readline installed. Use it instead? (And if readline does have some feature that editline doesn't, then the question can be always be asked.)
Re: [racket-dev] Line editing in the default REPL
My understanding of the licensing issues is that if the code works with both libeditline and libreadline then it isn't a derived work of readline, and therefore could be licensed under the LGPL, like the rest of Racket. Furthermore, turning use of libeditline on by default wouldn't be linking to any GPL code, meaning that we could do that by default. I think we should split up the `readline` collection into `readline` and `readline/base` which would be what's compatible with editline, and have xrepl in a `readline/base` mode on by default. Sam On Mon, Nov 24, 2014 at 5:57 PM, Leif Andersen l...@leifandersen.net wrote: Hello, When a user first starts the racket repl, it doesn't do line editing due to licensing issues. For example, it would be nice if the default editor would give the previous line if you hit up arrow, rather than writing ^[[A. I have now pointed out xrepl to several users, and every time they always ask me why it's not the default repl. Apparently the problem is that xrepl uses GNU Readline, which is GPL. However, Asumu found that if we replace requiring readline with BSD's libedit (not libeditline), everything works fine due to libedit's readline compatibility. It doesn't have all of the features of readline, but it does have some of the biggest ones (such as being able to use arrow keys) What do you all think of having `(require editline)` that works for the default repl, so that it gets line editing features. This would allow us to also keep `(require readline)` as before, maintaining backwards compatibility. If we do do this, this leads to the question of distribution. Would we want to include libedit inside Racket distributions, or should we just link to whatever the user has on their system? ~Leif Andersen _ Racket Developers list: http://lists.racket-lang.org/dev _ Racket Developers list: http://lists.racket-lang.org/dev
