Re: Bidi paragraph direction in terminal emulators (was: Proposal for BiDi in terminal emulators)

2019-02-18 Thread Egmont Koblinger via Unicode
On Sun, Feb 17, 2019 at 1:59 PM Philippe Verdy wrote: > Resist this idea, I've not been impolite. I didn't say a word about you being impolite. I said I might be impolite for not wishing to continue this discussion in that direction. > I just want to show you that terminals are legacy

Re: Bidi paragraph direction in terminal emulators (was: Proposal for BiDi in terminal emulators)

2019-02-17 Thread Philippe Verdy via Unicode
Le ven. 8 févr. 2019 à 13:56, Egmont Koblinger a écrit : > Philippe, I hate do say it, but at the risk of being impolite, I just > have to. > Resist this idea, I've not been impolite. I just want to show you that terminals are legacy environments that are far behind what is needed for proper

Re: Bidi paragraph direction in terminal emulators (was: Proposal for BiDi in terminal emulators)

2019-02-08 Thread Egmont Koblinger via Unicode
On Fri, Feb 8, 2019 at 10:36 PM Eli Zaretskii wrote: > No one in their right minds will run Emacs inside the Emacs terminal > emulator. And even for other applications, disabling bidi will almost > always needed only for full-screen programs, which use curses-like > libraries to address the

Re: Bidi paragraph direction in terminal emulators (was: Proposal for BiDi in terminal emulators)

2019-02-08 Thread Eli Zaretskii via Unicode
> From: Egmont Koblinger > Date: Fri, 8 Feb 2019 17:44:53 +0100 > Cc: Richard Wordingham , > unicode Unicode Discussion > > For certain apps, one of the modes is required (e.g. for cat it's the > implicit mode). For other tasks it's the other mode (e.g. for emacs > the explicit mode).

Re: Bidi paragraph direction in terminal emulators (was: Proposal for BiDi in terminal emulators)

2019-02-08 Thread Egmont Koblinger via Unicode
Hi Eli, > Why would they want to toggle it back and forth? What are the use > cases where it makes sense to mix both modes? IME, you either need > one or the other, never both. (Back to the basics, which are mentioned pretty clearly in my specification, I believe, and I've also described here

Re: Bidi paragraph direction in terminal emulators (was: Proposal for BiDi in terminal emulators)

2019-02-08 Thread Eli Zaretskii via Unicode
> From: Egmont Koblinger > Date: Fri, 8 Feb 2019 15:42:51 +0100 > Cc: Richard Wordingham , > unicode Unicode Discussion > > On Fri, Feb 8, 2019 at 3:28 PM Eli Zaretskii wrote: > > > You can have what you call the "explicit mode" if you set the variable > > bidi-display-reordering to

Re: Bidi paragraph direction in terminal emulators (was: Proposal for BiDi in terminal emulators)

2019-02-08 Thread Egmont Koblinger via Unicode
On Fri, Feb 8, 2019 at 3:28 PM Eli Zaretskii wrote: > You can have what you call the "explicit mode" if you set the variable > bidi-display-reordering to nil. So, if someone is running a mixture of applications requiring implicit vs. explicit modes, they'll have to continuously toggle the

Re: Bidi paragraph direction in terminal emulators (was: Proposal for BiDi in terminal emulators)

2019-02-08 Thread Eli Zaretskii via Unicode
> From: Egmont Koblinger > Date: Fri, 8 Feb 2019 14:57:56 +0100 > Cc: Richard Wordingham , > unicode Unicode Discussion > > According to the description you give, Emacs's terminal always applies > the BiDi algorithm, therefore by its design only implements what I > call "implicit mode",

Re: Bidi paragraph direction in terminal emulators (was: Proposal for BiDi in terminal emulators)

2019-02-08 Thread Egmont Koblinger via Unicode
Hi Eli, > Emacs implements the latest UBA from Unicode 11; and the Emacs > terminal emulator inserts all the text into a "normal" Emacs buffer, > and displays that buffer as any other buffer. So yes, you have there > full UBA support. One of the essentials of my work is that there's much more

Re: Bidi paragraph direction in terminal emulators (was: Proposal for BiDi in terminal emulators)

2019-02-08 Thread Eli Zaretskii via Unicode
> From: Egmont Koblinger > Date: Fri, 8 Feb 2019 13:30:42 +0100 > Cc: Richard Wordingham , > unicode Unicode Discussion > > Hi Eli, > > > Not sure why. There are terminal emulators out there which support > > proportional fonts. > > Well, of course, a terminal emulator can load any

Re: Bidi paragraph direction in terminal emulators (was: Proposal for BiDi in terminal emulators)

2019-02-08 Thread Egmont Koblinger via Unicode
Hi Philippe, > Adding a single bit of protection in cell attributes to indicate they are > either protected or become transparent (and the rest of the > attributes/character field indicates the id of another terminal grid or > rendering plugin crfeating its own layer and having its own

Re: Bidi paragraph direction in terminal emulators (was: Proposal for BiDi in terminal emulators)

2019-02-08 Thread Egmont Koblinger via Unicode
Hi Eli, > Not sure why. There are terminal emulators out there which support > proportional fonts. Well, of course, a terminal emulator can load any font, even proportional, but as it places them in the grid, it will look ugly as hell (like this one: https://askubuntu.com/q/781327/398785 ).

Re: Bidi paragraph direction in terminal emulators (was: Proposal for BiDi in terminal emulators)

2019-02-08 Thread Eli Zaretskii via Unicode
> Date: Fri, 8 Feb 2019 06:40:44 + > From: Richard Wordingham via Unicode > > > I, for one, am not to the slightest bit interested in abandoning the > > character grid and allowing for proportional fonts. This would just > > break a gazillion of things. > > The message I take from that and

Re: Bidi paragraph direction in terminal emulators (was: Proposal for BiDi in terminal emulators)

2019-02-07 Thread Richard Wordingham via Unicode
On Fri, 8 Feb 2019 00:38:24 +0100 Egmont Koblinger via Unicode wrote: > I, for one, am not to the slightest bit interested in abandoning the > character grid and allowing for proportional fonts. This would just > break a gazillion of things. The message I take from that and this thread in

Re: Bidi paragraph direction in terminal emulators (was: Proposal for BiDi in terminal emulators)

2019-02-07 Thread Philippe Verdy via Unicode
Adding a single bit of protection in cell attributes to indicate they are either protected or become transparent (and the rest of the attributes/character field indicates the id of another terminal grid or rendering plugin crfeating its own layer and having its own scrolling state and dimensions)

Re: Bidi paragraph direction in terminal emulators (was: Proposal for BiDi in terminal emulators)

2019-02-07 Thread Egmont Koblinger via Unicode
Hi Philippe, > I have never said anything about your work because I don't know where you > spoke about it or where you made some proposals. I must have missed one of > your messages (did it reach this list?). This entire conversation started by me announcing here my work, aiming to bring

Re: Bidi paragraph direction in terminal emulators (was: Proposal for BiDi in terminal emulators)

2019-02-07 Thread Philippe Verdy via Unicode
Le jeu. 7 févr. 2019 à 19:38, Egmont Koblinger a écrit : > As you can see from previous discussions, there's a whole lot of > confusion about the terminology. And it was exactly the subject of my first message sent to this thread ! you probably missed it. > Philippe, with all due respect, I

Re: Bidi paragraph direction in terminal emulators (was: Proposal for BiDi in terminal emulators)

2019-02-07 Thread Egmont Koblinger via Unicode
Hi Philippe, On Thu, Feb 7, 2019 at 3:21 PM Philippe Verdy wrote: > "Rules" are not formally written, they are just a sense of best practices. When it comes to BiDi in terminals, I haven't seen anything that I consider reasonably okay, let alone "best practice". It's a mess. That's why I

Re: Bidi paragraph direction in terminal emulators (was: Proposal for BiDi in terminal emulators)

2019-02-07 Thread Philippe Verdy via Unicode
Le jeu. 7 févr. 2019 à 13:29, Egmont Koblinger a écrit : > Hi Philippe, > > > There's some rules for correct display including with Bidi: > > In what sense are these "rules"? Where are these written, in what kind > of specification or existing practice? > "Rules" are not formally written, they

Re: Bidi paragraph direction in terminal emulators (was: Proposal for BiDi in terminal emulators)

2019-02-07 Thread Egmont Koblinger via Unicode
Hi Philippe, > There's some rules for correct display including with Bidi: In what sense are these "rules"? Where are these written, in what kind of specification or existing practice? > - Separate paragraphs that need a different default Bidi by double newlines > (to force a hard break)

Re: Bidi paragraph direction in terminal emulators (was: Proposal for BiDi in terminal emulators)

2019-02-06 Thread Philippe Verdy via Unicode
I read your email, you spoke for example about how a typical Unix/Linux tool shows its usage option (e.g. "anycommand --help") with a leading line then syntaxes and tabulated lists of options followed by translated help on the same line. There's some rules for correct display including with Bidi:

Re: Bidi paragraph direction in terminal emulators (was: Proposal for BiDi in terminal emulators)

2019-02-06 Thread Egmont Koblinger via Unicode
Hi Philippe, Thanks a lot for your input! Another fundamental difficulty with terminal emulators is: These controls (CR, LF...) are control instructions that move the cursor in some ways, and then are forgotten. You cannot do BiDi on the instructions the terminal receives. You can only do BiDi

Re: Bidi paragraph direction in terminal emulators (was: Proposal for BiDi in terminal emulators)

2019-02-05 Thread Philippe Verdy via Unicode
I think that before making any decision we must make some decision about what we mean by "newlines". There are in fact 3 different functions: - (1) soft line breaks (which are used to enforce a maximum display width between paragraph margins): these are equivalent to breakable and compressible

Re: Bidi paragraph direction in terminal emulators (was: Proposal for BiDi in terminal emulators)

2019-02-04 Thread Richard Wordingham via Unicode
On Mon, 4 Feb 2019 22:27:39 +0100 Egmont Koblinger via Unicode wrote: > Hi Richard, > > > The concept appears to exist in the form of the fields of the > > fifth edition of ECMA-48. Have you digested this ambitious > > standard? > > To be honest: No, I haven't. And I have no idea what those

Re: Bidi paragraph direction in terminal emulators (was: Proposal for BiDi in terminal emulators)

2019-02-04 Thread Egmont Koblinger via Unicode
Hi Eli, > I think it's unreasonable and impractical to expect 'echo', 'cat', and > its ilk to emit bidi controls (or any other controls) to force > paragraph direction. For starters, they won't know what direction to > force, because they don't understand the text they are processing. I agree,

Re: Bidi paragraph direction in terminal emulators (was: Proposal for BiDi in terminal emulators)

2019-02-04 Thread Richard Wordingham via Unicode
On Mon, 04 Feb 2019 22:39:07 +0200 Eli Zaretskii via Unicode wrote: > > Date: Mon, 4 Feb 2019 19:45:13 + > > From: Richard Wordingham via Unicode > > > > Yes. If one has a text composed of LTR and RTL paragraphs, one has > > to choose how far apart their starting margins are. I think

Re: Bidi paragraph direction in terminal emulators (was: Proposal for BiDi in terminal emulators)

2019-02-04 Thread Egmont Koblinger via Unicode
> > Yes. If one has a text composed of LTR and RTL paragraphs, one has to > > choose how far apart their starting margins are. I think that could > > get complicated for plain text if the terminal has unbounded width. > > But no real-life terminal does. The width is always bounded. Allegedly

Re: Bidi paragraph direction in terminal emulators (was: Proposal for BiDi in terminal emulators)

2019-02-04 Thread Egmont Koblinger via Unicode
Hi Richard, > That split is wrong if you want the non-HTML text to lay out reasonably > well in anything but a higher order protocol forcing RTL. You need to > it split as: > > lorem ipsum ABC > <[ DEF foobar Okay, so you should use LRMs or other similar tricks when wrapping a human-perceived

Re: Bidi paragraph direction in terminal emulators (was: Proposal for BiDi in terminal emulators)

2019-02-04 Thread Egmont Koblinger via Unicode
Hi Richard, > The concept appears to exist in the form of the fields of the > fifth edition of ECMA-48. Have you digested this ambitious standard? To be honest: No, I haven't. And I have no idea what those "fields" are. I spent (read: wasted) way too much time studying ECMA TR/53 to get to

Re: Bidi paragraph direction in terminal emulators (was: Proposal for BiDi in terminal emulators)

2019-02-04 Thread Eli Zaretskii via Unicode
> Date: Mon, 4 Feb 2019 19:45:13 + > From: Richard Wordingham via Unicode > > Yes. If one has a text composed of LTR and RTL paragraphs, one has to > choose how far apart their starting margins are. I think that could > get complicated for plain text if the terminal has unbounded width.

Re: Bidi paragraph direction in terminal emulators (was: Proposal for BiDi in terminal emulators)

2019-02-04 Thread Richard Wordingham via Unicode
On Mon, 04 Feb 2019 18:53:22 +0200 Eli Zaretskii via Unicode wrote: > Date: Mon, 4 Feb 2019 01:19:21 + > From: Richard Wordingham via Unicode >> If you look at it in Notepad, all >> lines will be LTR or all lines will be RTL. > That's because Notepad implements _only_ the higher-level

Re: Bidi paragraph direction in terminal emulators (was: Proposal for BiDi in terminal emulators)

2019-02-04 Thread Eli Zaretskii via Unicode
> Date: Mon, 4 Feb 2019 01:19:21 + > From: Richard Wordingham via Unicode > > On Sun, 03 Feb 2019 19:50:50 +0200 > Eli Zaretskii via Unicode wrote: > > > Do you see how this is carefully formatted to avoid overflowing an > > 80-column line of a typical terminal? Now suppose this is

Re: Bidi paragraph direction in terminal emulators (was: Proposal for BiDi in terminal emulators)

2019-02-03 Thread Richard Wordingham via Unicode
On Mon, 4 Feb 2019 00:36:23 +0100 Egmont Koblinger via Unicode wrote: > Now, back to terminals. > > The smallest possible viable definition of a "paragraph" in terminal > emulators is stuff between one newline and the next one. > > It would require a hell lot of work, redesigning

Re: Bidi paragraph direction in terminal emulators (was: Proposal for BiDi in terminal emulators)

2019-02-03 Thread Richard Wordingham via Unicode
On Mon, 4 Feb 2019 00:36:23 +0100 Egmont Koblinger via Unicode wrote: > I wish to store and deliver the following text, as it's layed out here > in logical order. That is, the order as the bytes appear in the text > file, as I typed them from the keyboard, is laid out here strictly > from left

Re: Bidi paragraph direction in terminal emulators (was: Proposal for BiDi in terminal emulators)

2019-02-03 Thread Richard Wordingham via Unicode
On Sun, 03 Feb 2019 19:50:50 +0200 Eli Zaretskii via Unicode wrote: > Do you see how this is carefully formatted to avoid overflowing an > 80-column line of a typical terminal? Now suppose this is translated > into a RTL language, which causes the Copyright line to start with a > strong R

Re: Bidi paragraph direction in terminal emulators (was: Proposal for BiDi in terminal emulators)

2019-02-03 Thread Egmont Koblinger via Unicode
Hi Eli, (I'm responding in multiple emails.) The Unicode BiDi algorithm states that it operates on paragraphs of text, and leaves it up to a higher protocol to define what a paragraph exactly is. What's the definition of "paragraph" in the context of plain text files? I don't think there's a

Re: Bidi paragraph direction in terminal emulators (was: Proposal for BiDi in terminal emulators)

2019-02-03 Thread Eli Zaretskii via Unicode
> From: Egmont Koblinger > Date: Sun, 3 Feb 2019 17:54:25 +0100 > Cc: unicode@unicode.org > > I'm arguing, although my reasons are not rock solid, that IMHO the > default should be the strict direction as set by SCP, without > autodetection. I think it's unreasonable and impractical to expect

Re: Bidi paragraph direction in terminal emulators (was: Proposal for BiDi in terminal emulators)

2019-02-03 Thread Egmont Koblinger via Unicode
Hi Eli, > The document cited at the beginning of the parent thread states that > "simple" text-mode utilities, such as 'echo', 'cat', 'ls' etc. should > use the "implicit" mode of bidi reordering, with automatic guessing of > the base paragraph direction. Not exactly. I take the SCP escape

Bidi paragraph direction in terminal emulators (was: Proposal for BiDi in terminal emulators)

2019-02-03 Thread Eli Zaretskii via Unicode
> Date: Sun, 03 Feb 2019 18:10:15 +0200 > Cc: richard.wording...@ntlworld.com, unicode@unicode.org > From: Eli Zaretskii via Unicode > > I think there are hard problems even for such "simple" utilities, and > I will start a separate thread about this. I think we spent enough time discussing