Re: Greek text mixed with English

[Jürgen Spitzmüller]

Guenter Milde wrote:
 However, Greek Unicode chars are missing in the output in the
 following example if:
 
 a) babel is included, or
 b) the \setmainfont line is commented
 
 \documentclass[greek]{article}
 \usepackage{fontspec}
 \setmainfont{Gentium}
 
 % \usepackage{babel}
 % \usepackage{polyglossia}
 
 \begin{document}
 
 Me mia mati'a...
 
 Με μια ματιά...
 
 \end{document}
 
 Babel selects a different font (the missing unicode support is a
 consequence of this font change). 

Please file reports for these. I don't think our XeTeX has been thouroughly 
tested yet. I have implemented it (due to user requests), but I do not use 
XeTeX myself at all.

 We must replace babel by polyglossia, as babel is not compatible with
 XeTeX. Selecting XeTeX as output machine is an explicite request for
 full, language-independent Unicode support.

Sure, polyglossia support must follow eventually. But this is more work than 
it seems. You have to dive into our language framework, which is all over the 
place. But of course, patches are welcome.

Jürgen

Re: Greek text mixed with English

[Guenter Milde]

On 2010-03-05, Jürgen Spitzmüller wrote:
 Guenter Milde wrote:
 However, Greek Unicode chars are missing in the output in the
 following example if:

 a) babel is included, or
 b) the \setmainfont line is commented
...

 Please file reports for these. 

Done. http://www.lyx.org/trac/ticket/6576

 I don't think our XeTeX has been thouroughly 
 tested yet. I have implemented it (due to user requests), but I do not use 
 XeTeX myself at all.

Thanks for the implementation. Hopefully the upcoming alpha release
will get someone with XeTeX experience interested in testing/reporting.

 We must replace babel by polyglossia, as babel is not compatible with
 XeTeX. Selecting XeTeX as output machine is an explicite request for
 full, language-independent Unicode support.

 Sure, polyglossia support must follow eventually. But this is more work
 than it seems. You have to dive into our language framework, which is
 all over the place. But of course, patches are welcome.

I am aware of possible problems. Maybe its time I read the XeTeX docs.
(I hoped LyX would save me this trouble ;-)

Günter

Re: Greek text mixed with English

[Jürgen Spitzmüller]

Guenter Milde wrote:
> However, Greek Unicode chars are missing in the output in the
> following example if:
> 
> a) babel is included, or
> b) the \setmainfont line is commented
> 
> \documentclass[greek]{article}
> \usepackage{fontspec}
> \setmainfont{Gentium}
> 
> % \usepackage{babel}
> % \usepackage{polyglossia}
> 
> \begin{document}
> 
> Me mia mati'a...
> 
> Με μια ματιά...
> 
> \end{document}
> 
> Babel selects a different font (the missing unicode support is a
> consequence of this font change). 

Please file reports for these. I don't think our XeTeX has been thouroughly 
tested yet. I have implemented it (due to user requests), but I do not use 
XeTeX myself at all.

> We must replace babel by polyglossia, as babel is not compatible with
> XeTeX. Selecting XeTeX as output machine is an explicite request for
> full, language-independent Unicode support.

Sure, polyglossia support must follow eventually. But this is more work than 
it seems. You have to dive into our language framework, which is all over the 
place. But of course, patches are welcome.

Jürgen

Re: Greek text mixed with English

[Guenter Milde]

On 2010-03-05, Jürgen Spitzmüller wrote:
> Guenter Milde wrote:
>> However, Greek Unicode chars are missing in the output in the
>> following example if:

>> a) babel is included, or
>> b) the \setmainfont line is commented
...

> Please file reports for these. 

Done. http://www.lyx.org/trac/ticket/6576

> I don't think our XeTeX has been thouroughly 
> tested yet. I have implemented it (due to user requests), but I do not use 
> XeTeX myself at all.

Thanks for the implementation. Hopefully the upcoming alpha release
will get someone with XeTeX experience interested in testing/reporting.

>> We must replace babel by polyglossia, as babel is not compatible with
>> XeTeX. Selecting XeTeX as output machine is an explicite request for
>> full, language-independent Unicode support.

> Sure, polyglossia support must follow eventually. But this is more work
> than it seems. You have to dive into our language framework, which is
> all over the place. But of course, patches are welcome.

I am aware of possible problems. Maybe its time I read the XeTeX docs.
(I hoped LyX would save me this trouble ;-)

Günter

Re: Greek text mixed with English [on/off-topic]

[Νίκος Αλεξανδρής]

On Mon, 2010-03-01 at 01:28 +0100, Νίκος Αλεξανδρής wrote:
 On Sun, 2010-02-28 at 23:47 +, Liviu Andronic wrote:
  Hello
  
  On 2/28/10, Νίκος Αλεξανδρής nikos.alexand...@uranus.uni-freiburg.de 
  wrote:
I read almost all of the posts in http://www.mail-archive.com. I will
eventually register myself in lyx-dev but dunno if its worth it for only
one thread.
  
  You could register for the thread, and unsubscribe when it's done.
  Liviu
 
 Done!

And... undone :-)

I'll try to follow the thread (hopefully there will be more on this in
the future). Thank you for your time folks,

Nikos


signature.asc
Description: This is a digitally signed message part

Re: Greek text mixed with English

[Guenter Milde]

On 2010-03-02, Jürgen Spitzmüller wrote:
 Guenter Milde wrote:

 The following minimal document

 -

 \documentclass[greek]{article}
 \usepackage{fontspec}
 \usepackage{babel}

 \usepackage{xunicode}
 \usepackage{xltxtra}
 \begin{document}
 test
 \end{document}

 --

 produces a PDF with the string τεστ. This proves that transliteration in 
 XeTeX works as in (pdf)latex, contrary to your claim.

With babel, yes.  As the transliteration is done at the font level (it is
a feature of the LGR font encoding), it cannot be kept for
Unicode-encoded fonts.

However, Greek Unicode chars are missing in the output in the
following example if:

a) babel is included, or
b) the \setmainfont line is commented

\documentclass[greek]{article}
\usepackage{fontspec}
\setmainfont{Gentium}

% \usepackage{babel}
% \usepackage{polyglossia}

\begin{document}

Me mia mati'a...

Με μια ματιά...

\end{document}

Babel selects a different font (the missing unicode support is a
consequence of this font change). 

This is one of the reasons why I believe the XeTeX documentation when it
tells that babel and XeTeX are incompatible.

 It does not matter whether or not polyglossia does this different than
 babel.  If you use a different package, you have to be prepared for
 such changes 

Well, I expect changes with the pdftex - xetex switch, otherwise I would
not switch.

 (although I'd say this is a polyglossia bug *if* polyglossia claims to
 be a drop-in-replacement for babel [which I don't know]. 

It is a replacement but not a drop in. 

 However, if polyglossia changes behaviour, we cannot replace
 babel by it anyway, unless the user explicitely requests this.

We must replace babel by polyglossia, as babel is not compatible with
XeTeX. Selecting XeTeX as output machine is an explicite request for
full, language-independent Unicode support.

We might have to care for possible incompatibilities in supported options
and languages with polyglossia, though.

Günter

Re: Greek text mixed with English [on/off-topic]

[Νίκος Αλεξανδρής]

On Mon, 2010-03-01 at 01:28 +0100, Νίκος Αλεξανδρής wrote:
> On Sun, 2010-02-28 at 23:47 +, Liviu Andronic wrote:
> > Hello
> > 
> > On 2/28/10, Νίκος Αλεξανδρής  
> > wrote:
> > >  I read almost all of the posts in http://www.mail-archive.com. I will
> > >  eventually register myself in lyx-dev but dunno if its worth it for only
> > >  one thread.
> > >
> > You could register for the thread, and unsubscribe when it's done.
> > Liviu
> 
> Done!

And... undone :-)

I'll try to follow the thread (hopefully there will be more on this in
the future). Thank you for your time folks,

Nikos


signature.asc
Description: This is a digitally signed message part

Re: Greek text mixed with English

[Guenter Milde]

On 2010-03-02, Jürgen Spitzmüller wrote:
> Guenter Milde wrote:

> The following minimal document

> -

> \documentclass[greek]{article}
> \usepackage{fontspec}
> \usepackage{babel}

> \usepackage{xunicode}
> \usepackage{xltxtra}
> \begin{document}
> test
> \end{document}

> --

> produces a PDF with the string "τεστ". This proves that transliteration in 
> XeTeX works as in (pdf)latex, contrary to your claim.

With babel, yes.  As the transliteration is done at the font level (it is
a feature of the LGR font encoding), it cannot be kept for
Unicode-encoded fonts.

However, Greek Unicode chars are missing in the output in the
following example if:

a) babel is included, or
b) the \setmainfont line is commented

\documentclass[greek]{article}
\usepackage{fontspec}
\setmainfont{Gentium}

% \usepackage{babel}
% \usepackage{polyglossia}

\begin{document}

Me mia mati'a...

Με μια ματιά...

\end{document}

Babel selects a different font (the missing unicode support is a
consequence of this font change). 

This is one of the reasons why I believe the XeTeX documentation when it
tells that babel and XeTeX are incompatible.

> It does not matter whether or not polyglossia does this different than
> babel.  If you use a different package, you have to be prepared for
> such changes 

Well, I expect changes with the pdftex -> xetex switch, otherwise I would
not switch.

> (although I'd say this is a polyglossia bug *if* polyglossia claims to
> be a drop-in-replacement for babel [which I don't know]. 

It is a replacement but not a drop in. 

> However, if polyglossia changes behaviour, we cannot replace
> babel by it anyway, unless the user explicitely requests this.

We must replace babel by polyglossia, as babel is not compatible with
XeTeX. Selecting XeTeX as output machine is an explicite request for
full, language-independent Unicode support.

We might have to care for possible incompatibilities in supported options
and languages with polyglossia, though.

Günter

Re: Greek text mixed with English

[Jürgen Spitzmüller]

Guenter Milde wrote:
 On 2010-03-01, Jürgen Spitzmüller wrote:
  Guenter Milde wrote:
  While this is the reine Lehre, it is often impractical.
  
  * Quite a lot of acronyms are international and will not be hyphenated
  
anyway.
  
  Not true.
 
 Why, it is not a contradiction to quite a lot of ...  if some..

No, the fact that they (= some) are used in several languages doesn't make 
them international (in my understanding).

  Acronyms are language-dependent (cf. IPA vs. API).
  
  The fact that many acronyms derive from English does not change that.
 
 But this is unfair: in a German document, you kan keep the language as
 German and write RADAR or Laser, in a Greek document, you need to switch
 langugae (or insert \latintext as ERT) to write Corine.

As Nikos explained, he needs to switch the keyboard layout anyway. And of 
course using two different writing systems is not as easy as using one.

  * SI unit symbols are international as well.
  
  For SI units, we need markup anyway (in order to use a proper package
  such as siunitx).
 
 But (hopefully) only optional! I prefer to keep control myself.

Of course. Nobody forces you to use markup.

  * Short quotes in a language that I do not have installed. It would be
  
misleading to mark e.g. a Vietnamese quote as French in a Greek
document, just to prevent it to become Greeeknamese.
  
  If you quote Vietnamese, you have to use the Vietnamese language
  environment.  Period.
 
 Again, this is very inconvenient. Do you really expect me to become
 root, install texlive-language-vietnamese (actually first find out the
 correct name) reconfigure TeX and LyX just to write one Vietnamese
 word???

Yes.

  Also, the current behaviour is unusal in two ways:
  
  a) In LyX, I can easily insert Greek or Cyrillic symbols/words in a text
  
 written with the Latin or Cyrillic alphabet, this is currently not
 possible for Latin inside Greek.
  
  That's why we need KeyboardLocaleEncoding support.
 
 I agree that the coupling of keyboard locale and text language can be
 an advancement. It will, however, not help with copy/paste.

True. But pasting a different language without marking it does not work 
anyway. Within one writing system, you will probably not notice this 
instantly, but you will eventually, at the latest when it comes to 
hyphenation.

So Copy/Paste from a different language does not work out of the box anyway.

  Also, I find inserting Greek characters by means of the Symbols dialog
  or by means of copy/paste all but easy.
 
 Copy/paste of Greek Unicode is for me not more difficult than copy/paste
 of English. Inserting via the Symbols dialoge is only an option for the
 single letter when I don't remember a faster input method (in math, I'd
 write \alpha ... \omega or use M-G a-o).

It's not more difficult. Just paste and adapt the language. As explained 
above, these two steps are always needed. In that sense, Greek users are even 
in a better position, since they recognize the need immediately.

 b) The normal behaviour (no transliteration if not requested) as option.

OK if this is not the default.

 I do not want to change the rendering of old documents (however,
 compiling with XeTeX instead of LaTeX or pdflatex will do so).

The change to XeTeX is an active change of the user.

 Günter

Jürgen

Re: Greek text mixed with English

[Jürgen Spitzmüller]

Νίκος Αλεξανδρής wrote:
 Just tested latest lyx-devel/trunk and it works ;-).

Excellent. I just committed to branch, so LyX 1.6.6 will have the fix as well.

Jürgen

Re: Greek text mixed with English

[Jürgen Spitzmüller]

Jürgen Spitzmüller wrote:
  I do not want to change the rendering of old documents (however,
  compiling with XeTeX instead of LaTeX or pdflatex will do so).
 
 The change to XeTeX is an active change of the user.

I just verified that transliteration is still active with XeTeX, so compiling 
with XeTeX does _not_ change the rendering of old documents (if so, I would 
rate that a XeTeX bug). 

Jürgen

Re: Greek text mixed with English

[Guenter Milde]

On 2010-03-02, Jürgen Spitzmüller wrote:
 Jürgen Spitzmüller wrote:

 I just verified that transliteration is still active with XeTeX, so
 compiling with XeTeX does _not_ change the rendering of old documents
 (if so, I would rate that a XeTeX bug). 

Here, (LyX 2.0.0 svn and XeTeX 3.1415926-2.2-0.9995.2 (TeX Live
2009/Debian) I get the following:

* many missing Greek characters (while the Latin ones are preserved)
  with the document output set to XeTeX.
  
* exporting to latex (XeTeX) and compiling by hand:

  - the same (without change)
  
  - the same after removing the buggy (and unneeded with XeTeX) \textgreek
definition.

  - expected output (Greek as Greek and Latin chars as Latin) after

- \usepackage{babel}
+ \usepackage{polyglossia}

I would rate this a LyX bug.

Günter

Re: Greek text mixed with English

[Guenter Milde]

On 2010-03-02, Jürgen Spitzmüller wrote:
 Guenter Milde wrote:
 On 2010-03-01, Jürgen Spitzmüller wrote:
  Guenter Milde wrote:
  While this is the reine Lehre, it is often impractical.


 But this is unfair: in a German document, you kan keep the language as
 German and write RADAR or Laser, in a Greek document, you need to switch
 langugae (or insert \latintext as ERT) to write Corine.

 As Nikos explained, he needs to switch the keyboard layout anyway. And of 
 course using two different writing systems is not as easy as using one.

However, Nikos started the original thread (months ago) with a
2968 word file imported from somewhere into LyX, stating that he does
not want to change all occurences of CORINE, GRASS, GIS, ... into a
different language but wants a quick fix for them to be printed in
Latin letters.


 Do you really expect me to become root, install
 texlive-language-vietnamese (actually first find out the correct name)
 reconfigure TeX and LyX just to write one Vietnamese word???

 Yes.

Then we disagree on the weighting of practicability vs. purity.


Günter

Re: Greek text mixed with English

[Jürgen Spitzmüller]

Guenter Milde wrote:
  I just verified that transliteration is still active with XeTeX, so
  compiling with XeTeX does not change the rendering of old documents
  (if so, I would rate that a XeTeX bug). 
 
 Here, (LyX 2.0.0 svn and XeTeX 3.1415926-2.2-0.9995.2 (TeX Live
 2009/Debian) I get the following:

I have TeXLive 2009 (from DVD, updated today via tlmgr), this is:
XeTeX 3.1415926-2.2-0.9995.2 (TeX Live 2009)

The following minimal document

-

\documentclass[greek]{article}
\usepackage{fontspec}
\usepackage{babel}

\usepackage{xunicode}
\usepackage{xltxtra}
\begin{document}
test
\end{document}

--

produces a PDF with the string τεστ. This proves that transliteration in 
XeTeX works as in (pdf)latex, contrary to your claim.

It does not matter whether or not polyglossia does this different than babel. 
If you use a different package, you have to be prepared for such changes 
(although I'd say this is a polyglossia bug *if* polyglossia claims to be a 
drop-in-replacement for babel [which I don't know]. However, if polyglossia 
changes behaviour, we cannot replace babel by it anyway, unless the user 
explicitely requests this).

Jürgen

Re: Greek text mixed with English

[Jürgen Spitzmüller]

Guenter Milde wrote:
 Then we disagree on the weighting of practicability vs. purity.

I think we disagree on the weighting of correctness. I think we should not 
integrate functionality in LyX that allows the user to produce incorrect 
output. And a Vietnamese word that is hyphenated with English rules _is_ 
incorrect by all means.

Jürgen

Re: Greek text mixed with English

[Νίκος Αλεξανδρής]

On Tue, 2010-03-02 at 13:28 +, Guenter Milde wrote:

[...]

  But this is unfair: in a German document, you kan keep the language as
  German and write RADAR or Laser, in a Greek document, you need to switch
  langugae (or insert \latintext as ERT) to write Corine.
 
  As Nikos explained, he needs to switch the keyboard layout anyway. And of 
  course using two different writing systems is not as easy as using one.
 
 However, Nikos started the original thread (months ago) with a
 2968 word file imported from somewhere into LyX, stating that he does
 not want to change all occurences of CORINE, GRASS, GIS, ... into a
 different language but wants a quick fix for them to be printed in
 Latin letters.

But that is not the way it should be for a new document.

Nikos


signature.asc
Description: This is a digitally signed message part

Re: Greek text mixed with English

[Andre Poenitz]

On Tue, Mar 02, 2010 at 02:35:09PM +0100, Jürgen Spitzmüller wrote:
 Guenter Milde wrote:
  Then we disagree on the weighting of practicability vs. purity.
 
 I think we disagree on the weighting of correctness. I think we should not 
 integrate functionality in LyX that allows the user to produce incorrect 
 output. And a Vietnamese word that is hyphenated with English rules _is_ 
 incorrect by all means.

The example is a single word in Vietnamese. This can be easily used in a
way that hyphenation will not be needed at all. Reconfiguring the whole
setup is also in my eyes complete overkill.

Andre'

Re: Greek text mixed with English

[Jürgen Spitzmüller]

Guenter Milde wrote:
> On 2010-03-01, Jürgen Spitzmüller wrote:
> > Guenter Milde wrote:
> >> While this is the "reine Lehre", it is often impractical.
> >> 
> >> * Quite a lot of acronyms are international and will not be hyphenated
> >> 
> >>   anyway.
> > 
> > Not true.
> 
> Why, it is not a contradiction to "quite a lot of ... " if some..

No, the fact that they (= some) are used in several languages doesn't make 
them "international" (in my understanding).

> > Acronyms are language-dependent (cf. IPA vs. API).
> > 
> > The fact that many acronyms derive from English does not change that.
> 
> But this is "unfair": in a German document, you kan keep the language as
> German and write RADAR or Laser, in a Greek document, you need to switch
> langugae (or insert \latintext as ERT) to write "Corine".

As Nikos explained, he needs to switch the keyboard layout anyway. And of 
course using two different writing systems is not as easy as using one.

> >> * SI unit symbols are international as well.
> > 
> > For SI units, we need markup anyway (in order to use a proper package
> > such as siunitx).
> 
> But (hopefully) only optional! I prefer to keep control myself.

Of course. Nobody forces you to use markup.

> >> * Short quotes in a language that I do not have installed. It would be
> >> 
> >>   misleading to mark e.g. a Vietnamese quote as French in a Greek
> >>   document, just to prevent it to become Greeeknamese.
> > 
> > If you quote Vietnamese, you have to use the Vietnamese language
> > environment.  Period.
> 
> Again, this is very inconvenient. Do you really expect me to become
> root, install texlive-language-vietnamese (actually first find out the
> correct name) reconfigure TeX and LyX just to write one Vietnamese
> word???

Yes.

> >> Also, the current behaviour is unusal in two ways:
> >> 
> >> a) In LyX, I can easily insert Greek or Cyrillic symbols/words in a text
> >> 
> >>written with the Latin or Cyrillic alphabet, this is currently not
> >>possible for Latin inside Greek.
> > 
> > That's why we need KeyboardLocaleEncoding support.
> 
> I agree that the coupling of keyboard locale and text language can be
> an advancement. It will, however, not help with copy/paste.

True. But pasting a different language without marking it does not work 
anyway. Within one writing system, you will probably not notice this 
instantly, but you will eventually, at the latest when it comes to 
hyphenation.

So Copy/Paste from a different language does not work "out of the box" anyway.

> > Also, I find inserting Greek characters by means of the Symbols dialog
> > or by means of copy/paste all but "easy".
> 
> Copy/paste of Greek Unicode is for me not more difficult than copy/paste
> of English. Inserting via the Symbols dialoge is only an option for the
> single letter when I don't remember a faster input method (in math, I'd
> write \alpha ... \omega or use M-G a-o).

It's not more difficult. Just paste and adapt the language. As explained 
above, these two steps are always needed. In that sense, Greek users are even 
in a better position, since they recognize the need immediately.

> b) The "normal" behaviour (no transliteration if not requested) as option.

OK if this is not the default.

> I do not want to change the rendering of old documents (however,
> compiling with XeTeX instead of LaTeX or pdflatex will do so).

The change to XeTeX is an active change of the user.

> Günter

Jürgen

Re: Greek text mixed with English

[Jürgen Spitzmüller]

Νίκος Αλεξανδρής wrote:
> Just tested latest "lyx-devel/trunk" and it works ;-).

Excellent. I just committed to branch, so LyX 1.6.6 will have the fix as well.

Jürgen

Re: Greek text mixed with English

[Jürgen Spitzmüller]

Jürgen Spitzmüller wrote:
> > I do not want to change the rendering of old documents (however,
> > compiling with XeTeX instead of LaTeX or pdflatex will do so).
> 
> The change to XeTeX is an active change of the user.

I just verified that transliteration is still active with XeTeX, so compiling 
with XeTeX does _not_ change the rendering of old documents (if so, I would 
rate that a XeTeX bug). 

Jürgen

Re: Greek text mixed with English

[Guenter Milde]

On 2010-03-02, Jürgen Spitzmüller wrote:
> Jürgen Spitzmüller wrote:

> I just verified that transliteration is still active with XeTeX, so
> compiling with XeTeX does _not_ change the rendering of old documents
> (if so, I would rate that a XeTeX bug). 

Here, (LyX 2.0.0 svn and XeTeX 3.1415926-2.2-0.9995.2 (TeX Live
2009/Debian) I get the following:

* many missing Greek characters (while the Latin ones are preserved)
  with the document output set to XeTeX.
  
* exporting to latex (XeTeX) and compiling by hand:

  - the same (without change)
  
  - the same after removing the buggy (and unneeded with XeTeX) \textgreek
definition.

  - expected output (Greek as Greek and Latin chars as Latin) after

- \usepackage{babel}
+ \usepackage{polyglossia}

I would rate this a LyX bug.

Günter

Re: Greek text mixed with English

[Guenter Milde]

On 2010-03-02, Jürgen Spitzmüller wrote:
> Guenter Milde wrote:
>> On 2010-03-01, Jürgen Spitzmüller wrote:
>> > Guenter Milde wrote:
>> >> While this is the "reine Lehre", it is often impractical.


>> But this is "unfair": in a German document, you kan keep the language as
>> German and write RADAR or Laser, in a Greek document, you need to switch
>> langugae (or insert \latintext as ERT) to write "Corine".

> As Nikos explained, he needs to switch the keyboard layout anyway. And of 
> course using two different writing systems is not as easy as using one.

However, Nikos started the original thread (months ago) with a
2968 word file imported from somewhere into LyX, stating that he does
not want to change all occurences of CORINE, GRASS, GIS, ... into a
different language but wants a quick fix for them to be printed in
Latin letters.


>> Do you really expect me to become root, install
>> texlive-language-vietnamese (actually first find out the correct name)
>> reconfigure TeX and LyX just to write one Vietnamese word???

> Yes.

Then we disagree on the weighting of practicability vs. purity.


Günter

Re: Greek text mixed with English

[Jürgen Spitzmüller]

Guenter Milde wrote:
> > I just verified that transliteration is still active with XeTeX, so
> > compiling with XeTeX does not change the rendering of old documents
> > (if so, I would rate that a XeTeX bug). 
> 
> Here, (LyX 2.0.0 svn and XeTeX 3.1415926-2.2-0.9995.2 (TeX Live
> 2009/Debian) I get the following:

I have TeXLive 2009 (from DVD, updated today via tlmgr), this is:
XeTeX 3.1415926-2.2-0.9995.2 (TeX Live 2009)

The following minimal document

-

\documentclass[greek]{article}
\usepackage{fontspec}
\usepackage{babel}

\usepackage{xunicode}
\usepackage{xltxtra}
\begin{document}
test
\end{document}

--

produces a PDF with the string "τεστ". This proves that transliteration in 
XeTeX works as in (pdf)latex, contrary to your claim.

It does not matter whether or not polyglossia does this different than babel. 
If you use a different package, you have to be prepared for such changes 
(although I'd say this is a polyglossia bug *if* polyglossia claims to be a 
drop-in-replacement for babel [which I don't know]. However, if polyglossia 
changes behaviour, we cannot replace babel by it anyway, unless the user 
explicitely requests this).

Jürgen

Re: Greek text mixed with English

[Jürgen Spitzmüller]

Guenter Milde wrote:
> Then we disagree on the weighting of practicability vs. purity.

I think we disagree on the weighting of correctness. I think we should not 
integrate functionality in LyX that allows the user to produce incorrect 
output. And a Vietnamese word that is hyphenated with English rules _is_ 
incorrect by all means.

Jürgen

Re: Greek text mixed with English

[Νίκος Αλεξανδρής]

On Tue, 2010-03-02 at 13:28 +, Guenter Milde wrote:

[...]

> >> But this is "unfair": in a German document, you kan keep the language as
> >> German and write RADAR or Laser, in a Greek document, you need to switch
> >> langugae (or insert \latintext as ERT) to write "Corine".
> 
> > As Nikos explained, he needs to switch the keyboard layout anyway. And of 
> > course using two different writing systems is not as easy as using one.
> 
> However, Nikos started the original thread (months ago) with a
> 2968 word file imported from somewhere into LyX, stating that he does
> not want to change all occurences of CORINE, GRASS, GIS, ... into a
> different language but wants a quick fix for them to be printed in
> Latin letters.

But that is not the way it should be for a new document.

Nikos


signature.asc
Description: This is a digitally signed message part

Re: Greek text mixed with English

[Andre Poenitz]

On Tue, Mar 02, 2010 at 02:35:09PM +0100, Jürgen Spitzmüller wrote:
> Guenter Milde wrote:
> > Then we disagree on the weighting of practicability vs. purity.
> 
> I think we disagree on the weighting of correctness. I think we should not 
> integrate functionality in LyX that allows the user to produce incorrect 
> output. And a Vietnamese word that is hyphenated with English rules _is_ 
> incorrect by all means.

The example is a single word in Vietnamese. This can be easily used in a
way that hyphenation will not be needed at all. Reconfiguring the whole
setup is also in my eyes complete overkill.

Andre'

Re: Greek text mixed with English

[Νίκος Αλεξανδρής]

Guenter Milde wrote:
  While this is the reine Lehre, it is often impractical (besides the
  impossibility to mark inline text as LyX code).  
  
  * Quite a lot of acronyms are international and will not be hyphenated
anyway.

Jürgen Spitzmüller wrote:
 Not true. Acronyms are language-dependent (cf. IPA vs. API). The fact that 
 many acronyms derive from English does not change that.

There are so many Greek acronyms that include only letters that are
common with the latin alphabet... Dunno, but I think it's better to stay
in the language in/for which they are written.


  * SI unit symbols are international as well.  
 
 For SI units, we need markup anyway (in order to use a proper package such as 
 siunitx).
 
  * Short quotes in a language that I do not have installed. It would be
misleading to mark e.g. a Vietnamese quote as French in a Greek
document, just to prevent it to become Greeeknamese.
 
 If you quote Vietnamese, you have to use the Vietnamese language environment. 
 Period.
 
  Also, the current behaviour is unusal in two ways:
  
  a) In LyX, I can easily insert Greek or Cyrillic symbols/words in a text
 written with the Latin or Cyrillic alphabet, this is currently not
 possible for Latin inside Greek.
 
 That's why we need KeyboardLocaleEncoding support. Also, I find inserting 
 Greek characters by means of the Symbols dialog or by means of copy/paste all 
 but easy.

Copy/paste brings frequently, unfortunately, much trouble. Lots of
errors which have to be traced... =timeconsuming.


  b) In other Unicode aware programs (as well as with LyX/XeTeX), Latin
 characters stay Latin even in a Greek context.
 
 True. 
 
  Practicability beats purity!
 
 Practicability depends on the user. A user with only a Latin keyboard will 
 find the tranliteration more practical. A user who uses different keyboard 
 layouts/encodings will find direct input more practical. That's why we need 
 both: transliteration as default and language/encoding switch if the users 
 set 
 up their OS to use keyboard layouts/localizations.

I think somebody mentioned to print on-screen (in LyX) the latin
characters that are going to be converted in greek. This is, if
possible, the best way to make it clear to the user that, while he is
typing using a lating key-layout, those characters won't be printed-out
as latin.

Preferably with some kind of visual markup (a colored underline maybe?)
so the user doesn't get confused with the rest of the text or just
thinks he forgot to switch the key-layout.


 I bet people who regularly write Greek and English (have to) do the latter 
 anyway.

Indeed. So +1 here.

Nikos


signature.asc
Description: This is a digitally signed message part

RE: Greek text mixed with English

[Νίκος Αλεξανδρής]

Vincent van Ravesteijn:
  This is my interpretation of the problem: unexpected output of greek 
  characters while there are latin characters on screen.

Jürgen:
 Which I would judge an information deficit. We should communicate better
 why languages need to be marked in LyX (and why this is a good thing).

Vincent van Ravesteijn:
 So, we show the characters in greek also on screen. That seems the best way 
 to communicate to the user that these characters will be converted.

As I mention in another reply, this would be better with some kind of 
emphasizing... like a colored-underline (?).

Nikos


signature.asc
Description: This is a digitally signed message part

Re: Greek text mixed with English

[Jürgen Spitzmüller]

Νίκος Αλεξανδρής wrote:
  I bet people who regularly write Greek and English (have to) do the
  latter  anyway.
 
 Indeed. So +1 here.

Could you describe how the expected behaviour should look like for you, as a 
user of English/Greek? I suppose you have some shortcut (such as Alt-Shift-K) 
to switch the keyboard from English or German to Greek, right? Do you use 
separate layouts for latin languages (such as English, German), or do you use 
one for both? Would you expect the language (not only the scripting) to change 
when you switch the keyboard layout? How do other applications behave in this 
respect?

Jürgen

Re: Greek text mixed with English

[Νίκος Αλεξανδρής]

On Mon, 2010-03-01 at 09:22 +0100, Jürgen Spitzmüller wrote:
 Νίκος Αλεξανδρής wrote:
   I bet people who regularly write Greek and English (have to) do the
   latter  anyway.
  
  Indeed. So +1 here.
 
 Could you describe how the expected behaviour should look like for you, as a 
 user of English/Greek? I suppose you have some shortcut (such as Alt-Shift-K) 
 to switch the keyboard from English or German to Greek, right? Do you use 
 separate layouts for latin languages (such as English, German), or do you use 
 one for both? Would you expect the language (not only the scripting) to 
 change 
 when you switch the keyboard layout? How do other applications behave in this 
 respect?
 
 Jürgen

- In my Ubuntu-Box I have Greek, German and English keyboard layouts
installed. I find it convenient to cycle through languages using
LeftShift + RightShift.

- I use Greek for Greek documents, English for English and German for
German.

- Ideally, would expect to launch LyX, Press Ctrl+N, set the document
class and the language to Greek, start filling with Greek text, switch
to English, type some terms, switch back to Greek, continue writing,
switch to German and add some German name witch contains ü or the
likes, press ViewPDF(LaTeX) and voila, everything is there.

- Ι never have had problems with OpenOffice for example, or gedit. Of
course, I don't think it's correct to compare LyX with gedit for
example.

- Also, last but not least (I know, it's another thing), I would like to
fill the title, author, subject, etc. for a greek document with greek
words and having it in place in the produced pdf.

- Also, somebody wrote in a post that there is no default language for a
document. I don't agree. There is. It is the systems default language.
If I have Greek as default, then yes, Greek is what I consider as
default language for the document.

Nikos


signature.asc
Description: This is a digitally signed message part

Re: Greek text mixed with English

[Jürgen Spitzmüller]

Νίκος Αλεξανδρής wrote:
 - In my Ubuntu-Box I have Greek, German and English keyboard layouts
 installed. I find it convenient to cycle through languages using
 LeftShift + RightShift.
 
 - I use Greek for Greek documents, English for English and German for
 German.

I see.

 - Ideally, would expect to launch LyX, Press Ctrl+N, set the document
 class and the language to Greek, start filling with Greek text, switch
 to English, type some terms, switch back to Greek, continue writing,
 switch to German and add some German name witch contains ü or the
 likes, press ViewPDF(LaTeX) and voila, everything is there.

Yes, this will work with the KeyboardLocale framework I have in mind.

 - Ι never have had problems with OpenOffice for example, or gedit. Of
 course, I don't think it's correct to compare LyX with gedit for
 example.

It's difficult to compare. Does OpenOffice also switch the language to German 
if you switch the language with RightShift (i.e. is the text correctly spell 
checked and hyphenated)?

 - Also, last but not least (I know, it's another thing), I would like to
 fill the title, author, subject, etc. for a greek document with greek
 words and having it in place in the produced pdf.

Yes, this should work. If not, it's a bug.

 - Also, somebody wrote in a post that there is no default language for a
 document. I don't agree. There is. It is the systems default language.
 If I have Greek as default, then yes, Greek is what I consider as
 default language for the document.

Sure there's a default language for a document (it is set in preferences, and 
then, in Document  Settings), but I can#t remember anyone denying this. 

I wrote in a post that there's no default language for a given script. I.e., 
English (or any other language) is not the default language for roman script 
(not even Latin is that).

Jürgen

Re: Greek text mixed with English

[Νίκος Αλεξανδρής]

 Νίκος Αλεξανδρής wrote:
  - In my Ubuntu-Box I have Greek, German and English keyboard layouts
  installed. I find it convenient to cycle through languages using
  LeftShift + RightShift.
  
  - I use Greek for Greek documents, English for English and German for
  German.

Jürgen:
 I see.

  - Ideally, would expect to launch LyX, Press Ctrl+N, set the document
  class and the language to Greek, start filling with Greek text, switch
  to English, type some terms, switch back to Greek, continue writing,
  switch to German and add some German name witch contains ü or the
  likes, press ViewPDF(LaTeX) and voila, everything is there.
 
 Yes, this will work with the KeyboardLocale framework I have in mind.
 
  - Ι never have had problems with OpenOffice for example, or gedit. Of
  course, I don't think it's correct to compare LyX with gedit for
  example.
 
 It's difficult to compare. Does OpenOffice also switch the language to German 
 if you switch the language with RightShift (i.e. is the text correctly spell 
 checked and hyphenated)?

I lied (since I have not been using OO for quite a long time). You still
need to set the language in OO (for (a) text part(s), paragraph(s) or
the whole document). But spellchecking works right away and PDF looks
fine.

  - Also, last but not least (I know, it's another thing), I would like to
  fill the title, author, subject, etc. for a greek document with greek
  words and having it in place in the produced pdf.
 
 Yes, this should work. If not, it's a bug.

See also:
http://www.mail-archive.com/lyx-us...@lists.lyx.org/msg79642.html

It works but not out-of-the box (i.e. filling text in the GUI the PDF
Properties). And it does not work for moderncv.

  - Also, somebody wrote in a post that there is no default language for a
  document. I don't agree. There is. It is the systems default language.
  If I have Greek as default, then yes, Greek is what I consider as
  default language for the document.
 
 Sure there's a default language for a document (it is set in preferences, and 
 then, in Document  Settings), but I can#t remember anyone denying this. 
 
 I wrote in a post that there's no default language for a given script. I.e., 
 English (or any other language) is not the default language for roman 
 script 
 (not even Latin is that).

Yep, that was it. OK, I understand what you meant. Mea culpa.


signature.asc
Description: This is a digitally signed message part

Re: Greek text mixed with English

[Jürgen Spitzmüller]

Νίκος Αλεξανδρής wrote:
  It's difficult to compare. Does OpenOffice also switch the language to
  German  if you switch the language with RightShift (i.e. is the text
  correctly spell checked and hyphenated)?
 
 I lied (since I have not been using OO for quite a long time). You still
 need to set the language in OO (for (a) text part(s), paragraph(s) or
 the whole document). 

But you would expect that this is not necessary, right?

 But spellchecking works right away

Yes, Ooo does some language detection.

 and PDF looks fine.

Sure, Ooo does not process the PDF through LaTeX.

Jürgen

Re: Greek text mixed with English

[Jürgen Spitzmüller]

Νίκος Αλεξανδρής wrote:
   - Also, last but not least (I know, it's another thing), I would like
   to fill the title, author, subject, etc. for a greek document with
   greek words and having it in place in the produced pdf.
 
  
 
  Yes, this should work. If not, it's a bug.
 
 See also:
 http://www.mail-archive.com/lyx-us...@lists.lyx.org/msg79642.html
 
 It works but not out-of-the box (i.e. filling text in the GUI the PDF
 Properties). And it does not work for moderncv.

I believe I have just fixed this in trunk (branch follows tomorrow, if nobody 
objects):
http://www.lyx.org/trac/changeset/33604

Jürgen

Re: Greek text mixed with English

[Νίκος Αλεξανδρής]

On Mon, 2010-03-01 at 18:41 +0100, Jürgen Spitzmüller wrote:
 Νίκος Αλεξανδρής wrote:
- Also, last but not least (I know, it's another thing), I would like
to fill the title, author, subject, etc. for a greek document with
greek words and having it in place in the produced pdf.
  
   
  
   Yes, this should work. If not, it's a bug.
  
  See also:
  http://www.mail-archive.com/lyx-us...@lists.lyx.org/msg79642.html
  
  It works but not out-of-the box (i.e. filling text in the GUI the PDF
  Properties). And it does not work for moderncv.
 
 I believe I have just fixed this in trunk (branch follows tomorrow, if nobody 
 objects):
 http://www.lyx.org/trac/changeset/33604
 
 Jürgen


Just tested latest lyx-devel/trunk and it works ;-).

Thank you very much Juergen,
Nikos


signature.asc
Description: This is a digitally signed message part

Re: Greek text mixed with English

[Guenter Milde]

On 2010-03-01, Jürgen Spitzmüller wrote:
 Guenter Milde wrote:
 While this is the reine Lehre, it is often impractical.

 * Quite a lot of acronyms are international and will not be hyphenated
   anyway.

 Not true. 

Why, it is not a contradiction to quite a lot of ...  if some..
 Acronyms are language-dependent (cf. IPA vs. API). 

 The fact that many acronyms derive from English does not change that.

But this is unfair: in a German document, you kan keep the language as
German and write RADAR or Laser, in a Greek document, you need to switch
langugae (or insert \latintext as ERT) to write Corine.

 * SI unit symbols are international as well.  

 For SI units, we need markup anyway (in order to use a proper package
 such as siunitx).

But (hopefully) only optional! I prefer to keep control myself.

 * Short quotes in a language that I do not have installed. It would be
   misleading to mark e.g. a Vietnamese quote as French in a Greek
   document, just to prevent it to become Greeeknamese.

 If you quote Vietnamese, you have to use the Vietnamese language
 environment.  Period.

Again, this is very inconvenient. Do you really expect me to become
root, install texlive-language-vietnamese (actually first find out the
correct name) reconfigure TeX and LyX just to write one Vietnamese
word???

 Also, the current behaviour is unusal in two ways:

 a) In LyX, I can easily insert Greek or Cyrillic symbols/words in a text
written with the Latin or Cyrillic alphabet, this is currently not
possible for Latin inside Greek.

 That's why we need KeyboardLocaleEncoding support. 

I agree that the coupling of keyboard locale and text language can be
an advancement. It will, however, not help with copy/paste.

 Also, I find inserting Greek characters by means of the Symbols dialog
 or by means of copy/paste all but easy.

Copy/paste of Greek Unicode is for me not more difficult than copy/paste
of English. Inserting via the Symbols dialoge is only an option for the
single letter when I don't remember a faster input method (in math, I'd
write \alpha ... \omega or use M-G a-o).

 b) In other Unicode aware programs (as well as with LyX/XeTeX), Latin
characters stay Latin even in a Greek context.

 True. 

 Practicability beats purity!

 Practicability depends on the user. A user with only a Latin keyboard will 
 find the tranliteration more practical. 

So do I.

 A user who uses different keyboard layouts/encodings will find direct
 input more practical. That's why we need both: 

a)
  transliteration as default 

   It could become a non-default option for new documents.

b) The normal behaviour (no transliteration if not requested) as option.

c) 
  language/encoding switch if the users set up their OS to use
  keyboard layouts/localizations.
   
   This could indeed become the best option for Greek users (and
   others with multiple keyboard layouts).
   
   However, as there is no 1:1 mapping of keyboard layouts and document
   languages, this power feature needs careful planning to be
   comprehensible, easy to configure and does not stand in the way.
   
   Also, it will not help with existing documents (created in another
   editor, or taken from the net). This is why we still need b)
   (unless we force the needy to use XeTeX).
   
 I bet people who regularly write Greek and English (have to) do the
 latter anyway.

However, there is more than one way to do this setup...

 BTW you also need to take backwards compatibility into account. You cannot 
 change the behaviour just like that (without breaking old documents).

I do not want to change the rendering of old documents (however,
compiling with XeTeX instead of LaTeX or pdflatex will do so).


Günter

Re: Greek text mixed with English

[Νίκος Αλεξανδρής]

Guenter Milde wrote:
> > While this is the "reine Lehre", it is often impractical (besides the
> > impossibility to mark inline text as LyX code).  
> > 
> > * Quite a lot of acronyms are international and will not be hyphenated
> >   anyway.

Jürgen Spitzmüller wrote:
> Not true. Acronyms are language-dependent (cf. IPA vs. API). The fact that 
> many acronyms derive from English does not change that.

There are so many Greek acronyms that include only letters that are
common with the latin alphabet... Dunno, but I think it's better to stay
in the language in/for which they are written.


> > * SI unit symbols are international as well.  
> 
> For SI units, we need markup anyway (in order to use a proper package such as 
> siunitx).
> 
> > * Short quotes in a language that I do not have installed. It would be
> >   misleading to mark e.g. a Vietnamese quote as French in a Greek
> >   document, just to prevent it to become Greeeknamese.
> 
> If you quote Vietnamese, you have to use the Vietnamese language environment. 
> Period.
> 
> > Also, the current behaviour is unusal in two ways:
> > 
> > a) In LyX, I can easily insert Greek or Cyrillic symbols/words in a text
> >written with the Latin or Cyrillic alphabet, this is currently not
> >possible for Latin inside Greek.
> 
> That's why we need KeyboardLocaleEncoding support. Also, I find inserting 
> Greek characters by means of the Symbols dialog or by means of copy/paste all 
> but "easy".

Copy/paste brings frequently, unfortunately, much trouble. Lots of
errors which have to be traced... =timeconsuming.


> > b) In other Unicode aware programs (as well as with LyX/XeTeX), Latin
> >characters stay Latin even in a Greek context.
> 
> True. 
> 
> > Practicability beats purity!
> 
> Practicability depends on the user. A user with only a Latin keyboard will 
> find the tranliteration more practical. A user who uses different keyboard 
> layouts/encodings will find direct input more practical. That's why we need 
> both: transliteration as default and language/encoding switch if the users 
> set 
> up their OS to use keyboard layouts/localizations.

I think somebody mentioned to print on-screen (in LyX) the latin
characters that are going to be converted in greek. This is, if
possible, the best way to make it clear to the user that, while he is
typing using a lating key-layout, those characters won't be printed-out
as latin.

Preferably with some kind of visual markup (a colored underline maybe?)
so the user doesn't get confused with the rest of the text or just
thinks he forgot to switch the key-layout.


> I bet people who regularly write Greek and English (have to) do the latter 
> anyway.

Indeed. So +1 here.

Nikos


signature.asc
Description: This is a digitally signed message part

RE: Greek text mixed with English

[Νίκος Αλεξανδρής]

Vincent van Ravesteijn:
> >> This is my interpretation of the problem: unexpected output of greek 
> >> characters while there are latin characters on screen.

Jürgen:
> >Which I would judge an information deficit. We should communicate better
> >why languages need to be marked in LyX (and why this is a good thing).

Vincent van Ravesteijn:
> So, we show the characters in greek also on screen. That seems the best way 
> to communicate to the user that these characters will be converted.

As I mention in another reply, this would be better with some kind of 
emphasizing... like a colored-underline (?).

Nikos


signature.asc
Description: This is a digitally signed message part

Re: Greek text mixed with English

[Jürgen Spitzmüller]

Νίκος Αλεξανδρής wrote:
> > I bet people who regularly write Greek and English (have to) do the
> > latter  anyway.
> 
> Indeed. So +1 here.

Could you describe how the expected behaviour should look like for you, as a 
user of English/Greek? I suppose you have some shortcut (such as Alt-Shift-K) 
to switch the keyboard from English or German to Greek, right? Do you use 
separate layouts for latin languages (such as English, German), or do you use 
one for both? Would you expect the language (not only the scripting) to change 
when you switch the keyboard layout? How do other applications behave in this 
respect?

Jürgen

Re: Greek text mixed with English

[Νίκος Αλεξανδρής]

On Mon, 2010-03-01 at 09:22 +0100, Jürgen Spitzmüller wrote:
> Νίκος Αλεξανδρής wrote:
> > > I bet people who regularly write Greek and English (have to) do the
> > > latter  anyway.
> > 
> > Indeed. So +1 here.
> 
> Could you describe how the expected behaviour should look like for you, as a 
> user of English/Greek? I suppose you have some shortcut (such as Alt-Shift-K) 
> to switch the keyboard from English or German to Greek, right? Do you use 
> separate layouts for latin languages (such as English, German), or do you use 
> one for both? Would you expect the language (not only the scripting) to 
> change 
> when you switch the keyboard layout? How do other applications behave in this 
> respect?
> 
> Jürgen

- In my Ubuntu-Box I have Greek, German and English keyboard layouts
installed. I find it convenient to cycle through languages using
LeftShift + RightShift.

- I use Greek for Greek documents, English for English and German for
German.

- Ideally, would expect to launch LyX, Press Ctrl+N, set the document
class and the language to Greek, start filling with Greek text, switch
to English, type some terms, switch back to Greek, continue writing,
switch to German and add some German name witch contains "ü" or the
likes, press ViewPDF(LaTeX) and voila, everything is there.

- Ι never have had problems with OpenOffice for example, or gedit. Of
course, I don't think it's correct to compare LyX with gedit for
example.

- Also, last but not least (I know, it's another thing), I would like to
fill the title, author, subject, etc. for a greek document with greek
words and having it in place in the produced pdf.

- Also, somebody wrote in a post that there is no default language for a
document. I don't agree. There is. It is the systems default language.
If I have Greek as default, then yes, Greek is what I consider as
default language for the document.

Nikos


signature.asc
Description: This is a digitally signed message part

Re: Greek text mixed with English

[Jürgen Spitzmüller]

Νίκος Αλεξανδρής wrote:
> - In my Ubuntu-Box I have Greek, German and English keyboard layouts
> installed. I find it convenient to cycle through languages using
> LeftShift + RightShift.
> 
> - I use Greek for Greek documents, English for English and German for
> German.

I see.

> - Ideally, would expect to launch LyX, Press Ctrl+N, set the document
> class and the language to Greek, start filling with Greek text, switch
> to English, type some terms, switch back to Greek, continue writing,
> switch to German and add some German name witch contains "ü" or the
> likes, press ViewPDF(LaTeX) and voila, everything is there.

Yes, this will work with the KeyboardLocale framework I have in mind.

> - Ι never have had problems with OpenOffice for example, or gedit. Of
> course, I don't think it's correct to compare LyX with gedit for
> example.

It's difficult to compare. Does OpenOffice also switch the language to German 
if you switch the language with RightShift (i.e. is the text correctly spell 
checked and hyphenated)?

> - Also, last but not least (I know, it's another thing), I would like to
> fill the title, author, subject, etc. for a greek document with greek
> words and having it in place in the produced pdf.

Yes, this should work. If not, it's a bug.

> - Also, somebody wrote in a post that there is no default language for a
> document. I don't agree. There is. It is the systems default language.
> If I have Greek as default, then yes, Greek is what I consider as
> default language for the document.

Sure there's a default language for a document (it is set in preferences, and 
then, in Document > Settings), but I can#t remember anyone denying this. 

I wrote in a post that there's no default language for a given script. I.e., 
English (or any other language) is not "the default" language for roman script 
(not even Latin is that).

Jürgen

Re: Greek text mixed with English

[Νίκος Αλεξανδρής]

> Νίκος Αλεξανδρής wrote:
> > - In my Ubuntu-Box I have Greek, German and English keyboard layouts
> > installed. I find it convenient to cycle through languages using
> > LeftShift + RightShift.
> > 
> > - I use Greek for Greek documents, English for English and German for
> > German.

Jürgen:
> I see.

> > - Ideally, would expect to launch LyX, Press Ctrl+N, set the document
> > class and the language to Greek, start filling with Greek text, switch
> > to English, type some terms, switch back to Greek, continue writing,
> > switch to German and add some German name witch contains "ü" or the
> > likes, press ViewPDF(LaTeX) and voila, everything is there.
> 
> Yes, this will work with the KeyboardLocale framework I have in mind.
> 
> > - Ι never have had problems with OpenOffice for example, or gedit. Of
> > course, I don't think it's correct to compare LyX with gedit for
> > example.
> 
> It's difficult to compare. Does OpenOffice also switch the language to German 
> if you switch the language with RightShift (i.e. is the text correctly spell 
> checked and hyphenated)?

I lied (since I have not been using OO for quite a long time). You still
need to set the language in OO (for (a) text part(s), paragraph(s) or
the whole document). But spellchecking works right away and PDF looks
fine.

> > - Also, last but not least (I know, it's another thing), I would like to
> > fill the title, author, subject, etc. for a greek document with greek
> > words and having it in place in the produced pdf.
> 
> Yes, this should work. If not, it's a bug.

See also:
http://www.mail-archive.com/lyx-us...@lists.lyx.org/msg79642.html

It works but not out-of-the box (i.e. filling text in the GUI the PDF
Properties). And it does not work for "moderncv".

> > - Also, somebody wrote in a post that there is no default language for a
> > document. I don't agree. There is. It is the systems default language.
> > If I have Greek as default, then yes, Greek is what I consider as
> > default language for the document.
> 
> Sure there's a default language for a document (it is set in preferences, and 
> then, in Document > Settings), but I can#t remember anyone denying this. 
> 
> I wrote in a post that there's no default language for a given script. I.e., 
> English (or any other language) is not "the default" language for roman 
> script 
> (not even Latin is that).

Yep, that was it. OK, I understand what you meant. Mea culpa.


signature.asc
Description: This is a digitally signed message part

Re: Greek text mixed with English

[Jürgen Spitzmüller]

Νίκος Αλεξανδρής wrote:
> > It's difficult to compare. Does OpenOffice also switch the language to
> > German  if you switch the language with RightShift (i.e. is the text
> > correctly spell checked and hyphenated)?
> 
> I lied (since I have not been using OO for quite a long time). You still
> need to set the language in OO (for (a) text part(s), paragraph(s) or
> the whole document). 

But you would expect that this is not necessary, right?

> But spellchecking works right away

Yes, Ooo does some language detection.

> and PDF looks fine.

Sure, Ooo does not process the PDF through LaTeX.

Jürgen

Re: Greek text mixed with English

[Jürgen Spitzmüller]

Νίκος Αλεξανδρής wrote:
> > > - Also, last but not least (I know, it's another thing), I would like
> > > to fill the title, author, subject, etc. for a greek document with
> > > greek words and having it in place in the produced pdf.
> >
> > 
> >
> > Yes, this should work. If not, it's a bug.
> 
> See also:
> http://www.mail-archive.com/lyx-us...@lists.lyx.org/msg79642.html
> 
> It works but not out-of-the box (i.e. filling text in the GUI the PDF
> Properties). And it does not work for "moderncv".

I believe I have just fixed this in trunk (branch follows tomorrow, if nobody 
objects):
http://www.lyx.org/trac/changeset/33604

Jürgen

Re: Greek text mixed with English

[Νίκος Αλεξανδρής]

On Mon, 2010-03-01 at 18:41 +0100, Jürgen Spitzmüller wrote:
> Νίκος Αλεξανδρής wrote:
> > > > - Also, last but not least (I know, it's another thing), I would like
> > > > to fill the title, author, subject, etc. for a greek document with
> > > > greek words and having it in place in the produced pdf.
> > >
> > > 
> > >
> > > Yes, this should work. If not, it's a bug.
> > 
> > See also:
> > http://www.mail-archive.com/lyx-us...@lists.lyx.org/msg79642.html
> > 
> > It works but not out-of-the box (i.e. filling text in the GUI the PDF
> > Properties). And it does not work for "moderncv".
> 
> I believe I have just fixed this in trunk (branch follows tomorrow, if nobody 
> objects):
> http://www.lyx.org/trac/changeset/33604
> 
> Jürgen


Just tested latest "lyx-devel/trunk" and it works ;-).

Thank you very much Juergen,
Nikos


signature.asc
Description: This is a digitally signed message part

Re: Greek text mixed with English

[Guenter Milde]

On 2010-03-01, Jürgen Spitzmüller wrote:
> Guenter Milde wrote:
>> While this is the "reine Lehre", it is often impractical.

>> * Quite a lot of acronyms are international and will not be hyphenated
>>   anyway.

> Not true. 

Why, it is not a contradiction to "quite a lot of ... " if some..
> Acronyms are language-dependent (cf. IPA vs. API). 

> The fact that many acronyms derive from English does not change that.

But this is "unfair": in a German document, you kan keep the language as
German and write RADAR or Laser, in a Greek document, you need to switch
langugae (or insert \latintext as ERT) to write "Corine".

>> * SI unit symbols are international as well.  

> For SI units, we need markup anyway (in order to use a proper package
> such as siunitx).

But (hopefully) only optional! I prefer to keep control myself.

>> * Short quotes in a language that I do not have installed. It would be
>>   misleading to mark e.g. a Vietnamese quote as French in a Greek
>>   document, just to prevent it to become Greeeknamese.

> If you quote Vietnamese, you have to use the Vietnamese language
> environment.  Period.

Again, this is very inconvenient. Do you really expect me to become
root, install texlive-language-vietnamese (actually first find out the
correct name) reconfigure TeX and LyX just to write one Vietnamese
word???

>> Also, the current behaviour is unusal in two ways:

>> a) In LyX, I can easily insert Greek or Cyrillic symbols/words in a text
>>written with the Latin or Cyrillic alphabet, this is currently not
>>possible for Latin inside Greek.

> That's why we need KeyboardLocaleEncoding support. 

I agree that the coupling of keyboard locale and text language can be
an advancement. It will, however, not help with copy/paste.

> Also, I find inserting Greek characters by means of the Symbols dialog
> or by means of copy/paste all but "easy".

Copy/paste of Greek Unicode is for me not more difficult than copy/paste
of English. Inserting via the Symbols dialoge is only an option for the
single letter when I don't remember a faster input method (in math, I'd
write \alpha ... \omega or use M-G a-o).

>> b) In other Unicode aware programs (as well as with LyX/XeTeX), Latin
>>characters stay Latin even in a Greek context.

> True. 

>> Practicability beats purity!

> Practicability depends on the user. A user with only a Latin keyboard will 
> find the tranliteration more practical. 

So do I.

> A user who uses different keyboard layouts/encodings will find direct
> input more practical. That's why we need both: 

a)
>  transliteration as default 

   It could become a non-default option for new documents.

b) The "normal" behaviour (no transliteration if not requested) as option.

c) 
>  language/encoding switch if the users set up their OS to use
>  keyboard layouts/localizations.
   
   This could indeed become the best option for Greek users (and
   others with multiple keyboard layouts).
   
   However, as there is no 1:1 mapping of keyboard layouts and document
   languages, this power feature needs careful planning to be
   comprehensible, easy to configure and does not stand in the way.
   
   Also, it will not help with existing documents (created in another
   editor, or taken from the net). This is why we still need b)
   (unless we force the needy to use XeTeX).
   
> I bet people who regularly write Greek and English (have to) do the
> latter anyway.

However, there is more than one way to do this setup...

> BTW you also need to take backwards compatibility into account. You cannot 
> change the behaviour just like that (without breaking old documents).

I do not want to change the rendering of old documents (however,
compiling with XeTeX instead of LaTeX or pdflatex will do so).


Günter

Re: Greek text mixed with English

[Abdelrazak Younes]

On 27/02/2010 16:45, Guenter Milde wrote:

On 2010-02-27, Vincent van Ravesteijn wrote:
   

Νίκος Αλεξανδρής schreef:
 
   

Would it be possible to set the language automatically when recognizing
a certain unicode range ?
 

While possible, it is ambiguous (and therefore not helpful):

* Cyrillic: Russian, Bulgarian, Serbian, Church Slavonic, Ukrainian, ...
* Latin: German, Dutch, Swedish, Vietnamese, ...
* Greek: polytonic or monotonic Greek, Coptic,
...
   


As Vincent already explained the idea would be to optionally define a 
second (and maybe third) language for the document. The rule to 
automatically set the language would be:
- first to check the unicode range: that would work when the two 
languages use two different range; for example English and Russian but 
not for Russian and Bulgarian, or Arabic and Farsi.
- second to check the spellchecker result: If a word is misspelled in 
the first language but not the second, set the language automatically to 
the second.


While this won't work for everybody, I reckon that this mechanism will 
work for the majority of two language users.


Abdel.

Re: Greek text mixed with English

[Jürgen Spitzmüller]

Abdelrazak Younes wrote:
 As Vincent already explained the idea would be to optionally define a 
 second (and maybe third) language for the document. The rule to 
 automatically set the language would be:
 - first to check the unicode range: that would work when the two 
 languages use two different range; for example English and Russian but 
 not for Russian and Bulgarian, or Arabic and Farsi.
 - second to check the spellchecker result: If a word is misspelled in 
 the first language but not the second, set the language automatically to 
 the second.
 
 While this won't work for everybody, I reckon that this mechanism will 
 work for the majority of two language users.

I doubt that. It will lead to all sorts of unwanted side effects. For 
instance, if I misspell German Ressource as Resource (the former is 
correct in German), I do not want LyX to set the language to English (where 
the second spelling is correct, at sentence start). I want LyX to tell me that 
my spelling is wrong.

The automatic language detection in MS Word is the best prove that automatic 
language detection is but a PITA.

I think Günther's proposal is the way to go.

Jürgen

Re: Greek text mixed with English

[Jürgen Spitzmüller]

Vincent van Ravesteijn wrote:
 For example, when I enter japanese, I get uncodable character ... in 
 my LaTeX if I don't mark it explicitly to japanese. Instead, can't we 
 just set the language to some default language for which this character 
 is encodable.

There is no such thing like a default language for a given script.

Jürgen

Re: Greek text mixed with English

[Abdelrazak Younes]

On 28/02/2010 11:31, Jürgen Spitzmüller wrote:

Abdelrazak Younes wrote:
   

As Vincent already explained the idea would be to optionally define a
second (and maybe third) language for the document. The rule to
automatically set the language would be:
- first to check the unicode range: that would work when the two
languages use two different range; for example English and Russian but
not for Russian and Bulgarian, or Arabic and Farsi.
- second to check the spellchecker result: If a word is misspelled in
the first language but not the second, set the language automatically to
the second.

While this won't work for everybody, I reckon that this mechanism will
work for the majority of two language users.
 

I doubt that. It will lead to all sorts of unwanted side effects. For
instance, if I misspell German Ressource as Resource (the former is
correct in German), I do not want LyX to set the language to English (where
the second spelling is correct, at sentence start). I want LyX to tell me that
my spelling is wrong.
   


Hum, yes, you have a point here. But my proposal would work reliably for 
two languages using two different unicode ranges like a latin one and 
Chinese or Japanese or Arabic, etc.


Abdel.

Re: Greek text mixed with English

[Jürgen Spitzmüller]

Abdelrazak Younes wrote:
 Hum, yes, you have a point here. But my proposal would work reliably for 
 two languages using two different unicode ranges like a latin one and 
 Chinese or Japanese or Arabic, etc.

Yes, but for these, a proper input method framework (as outlined by Günther) 
would work even more reliably. Mixing languages and scripts is really bound to 
fail.

Jürgen

RE: Greek text mixed with English

[Vincent van Ravesteijn - TNW]

 
Hi Vincent!


Are you interested in this feature as an end-user or as
a developer, if I may ask?

I'm afraid that would be as a developer.

I would like to know if there is any dev out there who
is interested to cover this gap and get payed for it
(+ having the coding pleasure which can't be payed)?

If we know what the solution is, the implementation is straightforward I guess.

I think we should output \textlatin for the latin characters as we do with 
\textgreek for greek characters.

Language English:
\textcyr{\char254} \textgreek{a} s

Language Greek:
\textcyr{\char254} alpha unicode char \textlatin{s}

Language Russian:
russian unicode char \textgreek{a} \textlatin{s}
or
russian unicode char \textgreek{a} s

Then we still need to hardcode for which encodings/languages we should use the 
\textlatin notation. (But things are already hardcoded for textcyr and 
textgreek as well.

Is this ok ?

Vincent

RE: Greek text mixed with English

[Νίκος Αλεξανδρής]

Nikos:
 Are you interested in this feature as an end-user or as
 a developer, if I may ask?

Vincent van Ravesteijn:
 I'm afraid that would be as a developer.

Well, I was hopping so (that you are a dev and not that you are
afraid ;-).

 I would like to know if there is any dev out there who
 is interested to cover this gap and get payed for it
 (+ having the coding pleasure which can't be payed)?

 If we know what the solution is, the implementation is straightforward I 
 guess.

Hopefully... it is a showstoper and it's a pity for LyX because it's a
superbe tool.


 I think we should output \textlatin for the latin characters as we do with 
 \textgreek for greek characters.
 
 Language English:
   \textcyr{\char254} \textgreek{a} s
 
 Language Greek:
   \textcyr{\char254} alpha unicode char \textlatin{s}
 
 Language Russian:
   russian unicode char \textgreek{a} \textlatin{s}
 or
   russian unicode char \textgreek{a} s
 
 Then we still need to hardcode for which encodings/languages we should use 
 the \textlatin notation. (But things are already hardcoded for textcyr and 
 textgreek as well.
 
 Is this ok ?
 Vincent



signature.asc
Description: This is a digitally signed message part

Re: Greek text mixed with English

[Νίκος Αλεξανδρής]

On Sun, 2010-02-28 at 02:33 +0100, Νίκος Αλεξανδρής wrote:
 
 The (only) problem which I face is the use of hyperref. It seems that
 there is a bug! The produced links in the pdf don't appear Greek (as
 they should and the use of the unicode=true option does not seem to
 work at all. I've spotted some posts WRT hyperref [1][2].
 
 Still, I can't make it work. I'll get various errors when I try to use
 stringenc in the Preamble.

I've posted back to the old thread [*] about this. Seems that the
solution given by Oberdiek back then works for the article document
class but not for moderncv.

Nikos

---
[*] Greek in the Header Information (?):
http://www.mail-archive.com/lyx-us...@lists.lyx.org/msg79642.html


signature.asc
Description: This is a digitally signed message part

Re: Greek text mixed with English

[Jürgen Spitzmüller]

Vincent van Ravesteijn - TNW wrote:
 If we know what the solution is, the implementation is straightforward I
 guess.
 
 I think we should output \textlatin for the latin characters as we do with
 \textgreek for greek characters.
 
 Language English:
 \textcyr{\char254} \textgreek{a} s
 
 Language Greek:
 \textcyr{\char254} alpha unicode char \textlatin{s}
 
 Language Russian:
 russian unicode char \textgreek{a} \textlatin{s}
 or
 russian unicode char \textgreek{a} s
 
 Then we still need to hardcode for which encodings/languages we should use
 the \textlatin notation. (But things are already hardcoded for textcyr and
 textgreek as well.
 
 Is this ok ?

Note that \textlatin will only switch the font encoding, not the language. So 
\textlatin{text} on a Greek context will not input English text into Greek, 
but roman characters (the same is true for \textgreek in latin context; that's 
why people who _really_ want to write Greek in, say, an English text and not 
only insert an alpha character, are advised to switch the language to Greek; 
if not, hyphenation and everything will be wrong).

In short: \textlatin might be useful for Greek users who want to insert some 
latin characters (language-insensitively), but it will not solve the Greek 
text mixed with English problem.

Jürgen

Re: Greek text mixed with English

[Jürgen Spitzmüller]

Jürgen Spitzmüller wrote:
 Note that \textlatin will only switch the font encoding, not the language.
 So  \textlatin{text} on a Greek context will not input English text into
 Greek, but roman characters (the same is true for \textgreek in latin
 context; that's why people who really want to write Greek in, say, an
 English text and not only insert an alpha character, are advised to switch
 the language to Greek; if not, hyphenation and everything will be wrong).
 
 In short: \textlatin might be useful for Greek users who want to insert
 some  latin characters (language-insensitively), but it will not solve the
 Greek text mixed with English problem.

Furthermore, in contrast to the Greek-in-Latin case, which simply produces 
LaTeX error if we do not use \textgreek, Latin-in-Greek does not only work, 
but is also functionally used as a transliteration system [1] (I use this 
myself to insert Greek terminology).

So if we start to auto-wrap latin chars in Greek context to \textlatin, we 
will break old documents.

Jürgen

[1] http://mirror.ctan.org/language/greek/doc/usage.pdf

RE: Greek text mixed with English

[Vincent van Ravesteijn - TNW]

 Is this ok ?

Note that \textlatin will only switch the font encoding, not the language.
So \textlatin{text} on a Greek context will not input English text into
Greek, but roman characters (the same is true for \textgreek in latin
context; 

Yes, that's exactly the point. If there are latin characters on screen in LyX, 
we don't want to have greek letters appearing in the output right ? Even though 
the latin characters are marked as greek language (which would only affect the 
spellchecking, hyphenation... ? Or is this a feature ?

In short: \textlatin might be useful for Greek users who want to insert
some latin characters (language-insensitively), but it will not solve the
Greek text mixed with English problem.

What is the Greek text mixed with English problem then ?

Jürgen

Vincent

RE: Greek text mixed with English

[Vincent van Ravesteijn - TNW]

  Or is this a feature ?

Apparently it is, reading your other mail.

(Strange feature. The fact that this can be done in LaTeX is not a valid reason 
to be the default behaviour of LyX. We could support this of course.)

Vincent

Re: Greek text mixed with English

[Jürgen Spitzmüller]

Vincent van Ravesteijn - TNW wrote:
 (Strange feature. The fact that this can be done in LaTeX is not a valid
 reason to be the default behaviour of LyX. We could support this of
 course.)

Why strange? It's a straighforward way to typeset Greek on a latin keyboard.

Jürgen

Re: Greek text mixed with English

[Jürgen Spitzmüller]

Vincent van Ravesteijn - TNW wrote:
 Yes, that's exactly the point. If there are latin characters on screen in
 LyX, we don't want to have greek letters appearing in the output right ?

No. If I use the transliteration, I want exactly this.

 Even though the latin characters are marked as greek language (which would
 only affect the spellchecking, hyphenation... ? Or is this a feature ?

Yes, it's a feature,as you noticed in the meantime.

 In short: \textlatin might be useful for Greek users who want to insert
 some latin characters (language-insensitively), but it will not solve the
 Greek text mixed with English problem.
 
 What is the Greek text mixed with English problem then ?

As I see it, the user wants a quick way to switch between two commonly used 
languages (a good shortcut for language greek and language english or bind 
that functions to the action Greek users perform to switch between English and 
Greek, if any).

Jürgen

RE: Greek text mixed with English

[Vincent van Ravesteijn - TNW]

 
 (Strange feature. The fact that this can be done in LaTeX is not a 
 valid reason to be the default behaviour of LyX. We could support this 
 of
 course.)

Why strange? It's a straighforward way to typeset Greek on a latin keyboard.


Now you speak of an input method, so in that case I would rather show the 
characters in greek on screen.

This is my interpretation of the problem: unexpected output of greek characters 
while there are latin characters on screen.

Vincent

Re: Greek text mixed with English

[Jürgen Spitzmüller]

Jürgen Spitzmüller wrote:
 As I see it, the user wants a quick way to switch between two commonly
 used  languages (a good shortcut for language greek and language
 english or bind that functions to the action Greek users perform to
 switch between English and Greek, if any).

I think the problem is this:
http://www.lyx.org/trac/ticket/6450

and the solution is outlined in JMarc's comment.

Jürgen

Re: Greek text mixed with English

[Jürgen Spitzmüller]

Vincent van Ravesteijn - TNW wrote:
 This is my interpretation of the problem: unexpected output of greek
 characters while there are latin characters on screen.

Which I would judge an information deficit. We should communicate better why 
languages need to be marked in LyX (and why this is a good thing).

Jürgen

RE: Greek text mixed with English [on/off-topic]

[Νίκος Αλεξανδρής]

Folks,

I am glad to do some testing. I did not receive any further posts of
yours since I am not a member of lyx-dev (only registered in lyx-user).
This is why I did not reply yet.

I read almost all of the posts in http://www.mail-archive.com. I will
eventually register myself in lyx-dev but dunno if its worth it for only
one thread.

Is there another way so I receive the posts for only this thread from
lyx-dev?

Cheers, Nikos


signature.asc
Description: This is a digitally signed message part

RE: Greek text mixed with English

[Vincent van Ravesteijn - TNW]

Vincent van Ravesteijn - TNW wrote:
 This is my interpretation of the problem: unexpected output of greek 
 characters while there are latin characters on screen.

Which I would judge an information deficit. We should communicate better
why languages need to be marked in LyX (and why this is a good thing).


So, we show the characters in greek also on screen. That seems the best way to 
communicate to the user that these characters will be converted.

Jürgen

Vincent

Re: Greek text mixed with English

[Jürgen Spitzmüller]

Vincent van Ravesteijn - TNW wrote:
 So, we show the characters in greek also on screen. That seems the best way
 to communicate to the user that these characters will be converted.

OK.

Jürgen

Re: Greek text mixed with English

[Guenter Milde]

On 2010-02-28, Vincent van Ravesteijn - TNW wrote:
Vincent van Ravesteijn - TNW wrote:
 This is my interpretation of the problem: unexpected output of greek 
 characters while there are latin characters on screen.

Yes, this is the OP's problem. Maybe it should rather be named: words
using written with Latin letters in a Greek document.

The OP writes Greek using a Greek keyboard, so the conversion of Latin
letters via transliteration is an unwanted and unexpected effect.

Which I would judge an information deficit. We should communicate better
why languages need to be marked in LyX (and why this is a good thing).

 So, we show the characters in greek also on screen. That seems the best
 way to communicate to the user that these characters will be converted.

But this is just one side. A filename, a C command or some Acronym in a
Greek document do not gain from beeing marked as another language but
still should keep Latin letters as such!


I see room for both, Greek with transliteration of Latin input (for
the occasional Greek quote in a non-Greek document) and Greek with
Unicode input (where Latin characters remain Latin).

Instead of 4 Greek language variants to care for the combinations:

   monotoniko   polutoniko
transliteration12
Unicode34
   
I suggest a GreekUnicode Module.

Günter

Re: Greek text mixed with English

[Guenter Milde]

On 2010-02-28, Jürgen Spitzmüller wrote:
 Vincent van Ravesteijn - TNW wrote:

 Yes, that's exactly the point. If there are latin characters on screen
 in LyX, we don't want to have greek letters appearing in the output
 right ?

 No. If I use the transliteration, I want exactly this.

 Even though the latin characters are marked as greek language (which would
 only affect the spellchecking, hyphenation... ? Or is this a feature ?

 Yes, it's a feature,as you noticed in the meantime.

It is a feature that dates back to pre-Unicode times (actually even back
to 7bit ASCII times). It is still handy for the occasional Greek quote
for people without Greek keyboard, but stands in the way for others. 

This is why this feature should be optional.

 As I see it, the user wants a quick way to switch between two commonly
 used languages (a good shortcut for language greek and language
 english or bind that functions to the action Greek users perform to
 switch between English and Greek, if any).

This is just one side. The other is to keep Latin input of e.g. a file
name in Latin spelling.

Günter

Re: Greek text mixed with English

[Guenter Milde]

On 2010-02-28, Vincent van Ravesteijn - TNW wrote:

 I think we should output \textlatin for the latin characters as we do
 with \textgreek for greek characters.

I thought about this once, but this would make Greek even more the odd
one out, because e.g. Cyrillic does not work this way:

 Language English:
   \textcyr{\char254} \textgreek{a} a  % ASCII
or
я  α a  % Unicode

 Language Russian:
   \textcyr{\char254} \textgreek{a} a  % ASCII
or
я  α a  % Unicode

Latin characters in a text marked as Russion are not transliterated.

It is just Greek that even in Unicode converts Latin to Greek

 Language Greek:
   \textcyr{\char255} a\textlatin{a}  % ASCII
or
я  α\textlatin{a}  % Unicode


This can be solved with 

  \addto\extrasgreek{\latintext}%

and then:

 Language Greek:
   \textcyr{\char255} \textgreek{a} a  % ASCII
or
я  α a  % Unicode


Günter

Re: Greek text mixed with English [on/off-topic]

[Liviu Andronic]

Hello

On 2/28/10, Νίκος Αλεξανδρής nikos.alexand...@uranus.uni-freiburg.de wrote:
  I read almost all of the posts in http://www.mail-archive.com. I will
  eventually register myself in lyx-dev but dunno if its worth it for only
  one thread.

You could register for the thread, and unsubscribe when it's done.
Liviu

Re: Greek text mixed with English [on/off-topic]

[Νίκος Αλεξανδρής]

On Sun, 2010-02-28 at 23:47 +, Liviu Andronic wrote:
 Hello
 
 On 2/28/10, Νίκος Αλεξανδρής nikos.alexand...@uranus.uni-freiburg.de wrote:
   I read almost all of the posts in http://www.mail-archive.com. I will
   eventually register myself in lyx-dev but dunno if its worth it for only
   one thread.
 
 You could register for the thread, and unsubscribe when it's done.
 Liviu

Done!


signature.asc
Description: This is a digitally signed message part

Re: Greek text mixed with English

[Jürgen Spitzmüller]

Guenter Milde wrote:
 It is a feature that dates back to pre-Unicode times (actually even back
 to 7bit ASCII times). It is still handy for the occasional Greek quote
 for people without Greek keyboard, but stands in the way for others. 
 
 This is why this feature should be optional.

I disagree. We should implement proper KeyboardInputLocale support, and if a 
user switches the keyboard locale to English, he should get roman script and 
English language.

Jürgen

Re: Greek text mixed with English

[Jürgen Spitzmüller]

Guenter Milde wrote:
 But this is just one side. A filename, a C command or some Acronym in a
 Greek document do not gain from beeing marked as another language but
 still should keep Latin letters as such!

Then, the user should still mark the code semantically (as LyX code or 
whatever), and this semantic markup should take care about the encoding.

Jürgen

Re: Greek text mixed with English

[Guenter Milde]

On 2010-03-01, Jürgen Spitzmüller wrote:
 Guenter Milde wrote:
 But this is just one side. A filename, a C command or some Acronym in a
 Greek document do not gain from beeing marked as another language but
 still should keep Latin letters as such!

 Then, the user should still mark the code semantically (as LyX code or 
 whatever), and this semantic markup should take care about the encoding.

While this is the reine Lehre, it is often impractical (besides the
impossibility to mark inline text as LyX code).  

* Quite a lot of acronyms are international and will not be hyphenated
  anyway.
  
* SI unit symbols are international as well.  

* Short quotes in a language that I do not have installed. It would be
  misleading to mark e.g. a Vietnamese quote as French in a Greek
  document, just to prevent it to become Greeeknamese.

Also, the current behaviour is unusal in two ways:

a) In LyX, I can easily insert Greek or Cyrillic symbols/words in a text
   written with the Latin or Cyrillic alphabet, this is currently not
   possible for Latin inside Greek.
   
b) In other Unicode aware programs (as well as with LyX/XeTeX), Latin
   characters stay Latin even in a Greek context.

Practicability beats purity!
   
Günter   

Re: Greek text mixed with English

[Jürgen Spitzmüller]

Guenter Milde wrote:
 While this is the reine Lehre, it is often impractical (besides the
 impossibility to mark inline text as LyX code).  
 
 * Quite a lot of acronyms are international and will not be hyphenated
   anyway.

Not true. Acronyms are language-dependent (cf. IPA vs. API). The fact that 
many acronyms derive from English does not change that.

 * SI unit symbols are international as well.  

For SI units, we need markup anyway (in order to use a proper package such as 
siunitx).

 * Short quotes in a language that I do not have installed. It would be
   misleading to mark e.g. a Vietnamese quote as French in a Greek
   document, just to prevent it to become Greeeknamese.

If you quote Vietnamese, you have to use the Vietnamese language environment. 
Period.

 Also, the current behaviour is unusal in two ways:
 
 a) In LyX, I can easily insert Greek or Cyrillic symbols/words in a text
written with the Latin or Cyrillic alphabet, this is currently not
possible for Latin inside Greek.

That's why we need KeyboardLocaleEncoding support. Also, I find inserting 
Greek characters by means of the Symbols dialog or by means of copy/paste all 
but easy.

 b) In other Unicode aware programs (as well as with LyX/XeTeX), Latin
characters stay Latin even in a Greek context.

True. 

 Practicability beats purity!

Practicability depends on the user. A user with only a Latin keyboard will 
find the tranliteration more practical. A user who uses different keyboard 
layouts/encodings will find direct input more practical. That's why we need 
both: transliteration as default and language/encoding switch if the users set 
up their OS to use keyboard layouts/localizations.

I bet people who regularly write Greek and English (have to) do the latter 
anyway.

BTW you also need to take backwards compatibility into account. You cannot 
change the behaviour just like that (without breaking old documents).

Jürgen

Re: Greek text mixed with English

[Jürgen Spitzmüller]

Guenter Milde wrote:
 (besides the
 impossibility to mark inline text as LyX code).  

Document  Settings  Modules  Logical Markup
Edit  Text Style  Code.

Jürgen

Re: Greek text mixed with English

[Abdelrazak Younes]

On 27/02/2010 16:45, Guenter Milde wrote:

On 2010-02-27, Vincent van Ravesteijn wrote:
   

Νίκος Αλεξανδρής schreef:
 
   

Would it be possible to set the language automatically when recognizing
a certain unicode range ?
 

While possible, it is ambiguous (and therefore not helpful):

* Cyrillic: Russian, Bulgarian, Serbian, Church Slavonic, Ukrainian, ...
* Latin: German, Dutch, Swedish, Vietnamese, ...
* Greek: polytonic or monotonic Greek, Coptic,
...
   


As Vincent already explained the idea would be to optionally define a 
second (and maybe third) language for the document. The rule to 
automatically set the language would be:
- first to check the unicode range: that would work when the two 
languages use two different range; for example English and Russian but 
not for Russian and Bulgarian, or Arabic and Farsi.
- second to check the spellchecker result: If a word is misspelled in 
the first language but not the second, set the language automatically to 
the second.


While this won't work for everybody, I reckon that this mechanism will 
work for the majority of two language users.


Abdel.

Re: Greek text mixed with English

[Jürgen Spitzmüller]

Abdelrazak Younes wrote:
> As Vincent already explained the idea would be to optionally define a 
> second (and maybe third) language for the document. The rule to 
> automatically set the language would be:
> - first to check the unicode range: that would work when the two 
> languages use two different range; for example English and Russian but 
> not for Russian and Bulgarian, or Arabic and Farsi.
> - second to check the spellchecker result: If a word is misspelled in 
> the first language but not the second, set the language automatically to 
> the second.
> 
> While this won't work for everybody, I reckon that this mechanism will 
> work for the majority of two language users.

I doubt that. It will lead to all sorts of unwanted side effects. For 
instance, if I misspell German "Ressource" as "Resource" (the former is 
correct in German), I do not want LyX to set the language to English (where 
the second spelling is correct, at sentence start). I want LyX to tell me that 
my spelling is wrong.

The automatic language detection in MS Word is the best prove that automatic 
language detection is but a PITA.

I think Günther's proposal is the way to go.

Jürgen

Re: Greek text mixed with English

[Jürgen Spitzmüller]

Vincent van Ravesteijn wrote:
> For example, when I enter japanese, I get  in 
> my LaTeX if I don't mark it explicitly to japanese. Instead, can't we 
> just set the language to some default language for which this character 
> is encodable.

There is no such thing like a "default" language for a given script.

Jürgen

Re: Greek text mixed with English

[Abdelrazak Younes]

On 28/02/2010 11:31, Jürgen Spitzmüller wrote:

Abdelrazak Younes wrote:
   

As Vincent already explained the idea would be to optionally define a
second (and maybe third) language for the document. The rule to
automatically set the language would be:
- first to check the unicode range: that would work when the two
languages use two different range; for example English and Russian but
not for Russian and Bulgarian, or Arabic and Farsi.
- second to check the spellchecker result: If a word is misspelled in
the first language but not the second, set the language automatically to
the second.

While this won't work for everybody, I reckon that this mechanism will
work for the majority of two language users.
 

I doubt that. It will lead to all sorts of unwanted side effects. For
instance, if I misspell German "Ressource" as "Resource" (the former is
correct in German), I do not want LyX to set the language to English (where
the second spelling is correct, at sentence start). I want LyX to tell me that
my spelling is wrong.
   


Hum, yes, you have a point here. But my proposal would work reliably for 
two languages using two different unicode ranges like a latin one and 
Chinese or Japanese or Arabic, etc.


Abdel.

Re: Greek text mixed with English

[Jürgen Spitzmüller]

Abdelrazak Younes wrote:
> Hum, yes, you have a point here. But my proposal would work reliably for 
> two languages using two different unicode ranges like a latin one and 
> Chinese or Japanese or Arabic, etc.

Yes, but for these, a proper input method framework (as outlined by Günther) 
would work even more reliably. Mixing languages and scripts is really bound to 
fail.

Jürgen

RE: Greek text mixed with English

[Vincent van Ravesteijn - TNW]

 
>Hi Vincent!
>
>
>Are you interested in this feature as an end-user or as
>a developer, if I may ask?

I'm afraid that would be as a developer.

>I would like to know if there is any dev out there who
>is interested to cover this gap and get payed for it
>(+ having the coding pleasure which can't be payed)?

If we know what the solution is, the implementation is straightforward I guess.

I think we should output \textlatin for the latin characters as we do with 
\textgreek for greek characters.

Language English:
\textcyr{\char254} \textgreek{a} s

Language Greek:
\textcyr{\char254}  \textlatin{s}

Language Russian:
 \textgreek{a} \textlatin{s}
or
 \textgreek{a} s

Then we still need to hardcode for which encodings/languages we should use the 
\textlatin notation. (But things are already hardcoded for textcyr and 
textgreek as well.

Is this ok ?

Vincent

RE: Greek text mixed with English

[Νίκος Αλεξανδρής]

Nikos:
> >Are you interested in this feature as an end-user or as
> >a developer, if I may ask?

Vincent van Ravesteijn:
> I'm afraid that would be as a developer.

Well, I was hopping so (that you are a dev and not that you are
afraid ;-).

> >I would like to know if there is any dev out there who
> >is interested to cover this gap and get payed for it
> >(+ having the coding pleasure which can't be payed)?

> If we know what the solution is, the implementation is straightforward I 
> guess.

Hopefully... it is a showstoper and it's a pity for LyX because it's a
superbe tool.


> I think we should output \textlatin for the latin characters as we do with 
> \textgreek for greek characters.
> 
> Language English:
>   \textcyr{\char254} \textgreek{a} s
> 
> Language Greek:
>   \textcyr{\char254}  \textlatin{s}
> 
> Language Russian:
>\textgreek{a} \textlatin{s}
> or
>\textgreek{a} s
> 
> Then we still need to hardcode for which encodings/languages we should use 
> the \textlatin notation. (But things are already hardcoded for textcyr and 
> textgreek as well.
> 
> Is this ok ?
> Vincent



signature.asc
Description: This is a digitally signed message part

Re: Greek text mixed with English

[Νίκος Αλεξανδρής]

On Sun, 2010-02-28 at 02:33 +0100, Νίκος Αλεξανδρής wrote:
> 
> The (only) problem which I face is the use of hyperref. It seems that
> there is a bug! The produced links in the pdf don't appear "Greek" (as
> they should and the use of the "unicode=true" option does not seem to
> work at all. I've spotted some posts WRT hyperref [1][2].
> 
> Still, I can't make it work. I'll get various errors when I try to use
> stringenc in the Preamble.

I've posted back to the old thread [*] about this. Seems that the
solution given by Oberdiek back then works for the article document
class but not for moderncv.

Nikos

---
[*] Greek in the Header Information (?):
http://www.mail-archive.com/lyx-us...@lists.lyx.org/msg79642.html


signature.asc
Description: This is a digitally signed message part

Re: Greek text mixed with English

[Jürgen Spitzmüller]

Vincent van Ravesteijn - TNW wrote:
> If we know what the solution is, the implementation is straightforward I
> guess.
> 
> I think we should output \textlatin for the latin characters as we do with
> \textgreek for greek characters.
> 
> Language English:
> \textcyr{\char254} \textgreek{a} s
> 
> Language Greek:
> \textcyr{\char254}  \textlatin{s}
> 
> Language Russian:
>  \textgreek{a} \textlatin{s}
> or
>  \textgreek{a} s
> 
> Then we still need to hardcode for which encodings/languages we should use
> the \textlatin notation. (But things are already hardcoded for textcyr and
> textgreek as well.
> 
> Is this ok ?

Note that \textlatin will only switch the font encoding, not the language. So 
\textlatin{text} on a Greek context will not input English text into Greek, 
but roman characters (the same is true for \textgreek in latin context; that's 
why people who _really_ want to write Greek in, say, an English text and not 
only insert an alpha character, are advised to switch the language to Greek; 
if not, hyphenation and everything will be wrong).

In short: \textlatin might be useful for Greek users who want to insert some 
latin characters (language-insensitively), but it will not solve the "Greek 
text mixed with English" problem.

Jürgen

Re: Greek text mixed with English

[Jürgen Spitzmüller]

Jürgen Spitzmüller wrote:
> Note that \textlatin will only switch the font encoding, not the language.
> So  \textlatin{text} on a Greek context will not input English text into
> Greek, but roman characters (the same is true for \textgreek in latin
> context; that's why people who really want to write Greek in, say, an
> English text and not only insert an alpha character, are advised to switch
> the language to Greek; if not, hyphenation and everything will be wrong).
> 
> In short: \textlatin might be useful for Greek users who want to insert
> some  latin characters (language-insensitively), but it will not solve the
> "Greek text mixed with English" problem.

Furthermore, in contrast to the Greek-in-Latin case, which simply produces 
LaTeX error if we do not use \textgreek, Latin-in-Greek does not only work, 
but is also functionally used as a transliteration system [1] (I use this 
myself to insert Greek terminology).

So if we start to auto-wrap latin chars in Greek context to \textlatin, we 
will break old documents.

Jürgen

[1] http://mirror.ctan.org/language/greek/doc/usage.pdf

RE: Greek text mixed with English

[Vincent van Ravesteijn - TNW]

>> Is this ok ?
>
>Note that \textlatin will only switch the font encoding, not the language.
>So \textlatin{text} on a Greek context will not input English text into
>Greek, but roman characters (the same is true for \textgreek in latin
>context; 

Yes, that's exactly the point. If there are latin characters on screen in LyX, 
we don't want to have greek letters appearing in the output right ? Even though 
the latin characters are marked as greek language (which would only affect the 
spellchecking, hyphenation... ? Or is this a feature ?

>In short: \textlatin might be useful for Greek users who want to insert
>some latin characters (language-insensitively), but it will not solve the
>"Greek text mixed with English" problem.

What is the "Greek text mixed with English" problem then ?

>Jürgen

Vincent

RE: Greek text mixed with English

[Vincent van Ravesteijn - TNW]

 > Or is this a feature ?

Apparently it is, reading your other mail.

(Strange feature. The fact that this can be done in LaTeX is not a valid reason 
to be the default behaviour of LyX. We could support this of course.)

Vincent

Re: Greek text mixed with English

[Jürgen Spitzmüller]

Vincent van Ravesteijn - TNW wrote:
> (Strange feature. The fact that this can be done in LaTeX is not a valid
> reason to be the default behaviour of LyX. We could support this of
> course.)

Why strange? It's a straighforward way to typeset Greek on a latin keyboard.

Jürgen

Re: Greek text mixed with English

[Jürgen Spitzmüller]

Vincent van Ravesteijn - TNW wrote:
> Yes, that's exactly the point. If there are latin characters on screen in
> LyX, we don't want to have greek letters appearing in the output right ?

No. If I use the transliteration, I want exactly this.

> Even though the latin characters are marked as greek language (which would
> only affect the spellchecking, hyphenation... ? Or is this a feature ?

Yes, it's a feature,as you noticed in the meantime.

> >In short: \textlatin might be useful for Greek users who want to insert
> >some latin characters (language-insensitively), but it will not solve the
> >"Greek text mixed with English" problem.
> 
> What is the "Greek text mixed with English" problem then ?

As I see it, the user wants a quick way to switch between two commonly used 
languages (a good shortcut for "language greek" and "language english" or bind 
that functions to the action Greek users perform to switch between English and 
Greek, if any).

Jürgen

RE: Greek text mixed with English

[Vincent van Ravesteijn - TNW]

 
>> (Strange feature. The fact that this can be done in LaTeX is not a 
>> valid reason to be the default behaviour of LyX. We could support this 
>> of
>> course.)
>
>Why strange? It's a straighforward way to typeset Greek on a latin keyboard.
>

Now you speak of an input method, so in that case I would rather show the 
characters in greek on screen.

This is my interpretation of the problem: unexpected output of greek characters 
while there are latin characters on screen.

Vincent

Re: Greek text mixed with English

[Jürgen Spitzmüller]

Jürgen Spitzmüller wrote:
> As I see it, the user wants a quick way to switch between two commonly
> used  languages (a good shortcut for "language greek" and "language
> english" or bind that functions to the action Greek users perform to
> switch between English and Greek, if any).

I think the problem is this:
http://www.lyx.org/trac/ticket/6450

and the solution is outlined in JMarc's comment.

Jürgen

Re: Greek text mixed with English

[Jürgen Spitzmüller]

Vincent van Ravesteijn - TNW wrote:
> This is my interpretation of the problem: unexpected output of greek
> characters while there are latin characters on screen.

Which I would judge an information deficit. We should communicate better why 
languages need to be marked in LyX (and why this is a good thing).

Jürgen

RE: Greek text mixed with English [on/off-topic]

[Νίκος Αλεξανδρής]

Folks,

I am glad to do some testing. I did not receive any further posts of
yours since I am not a member of lyx-dev (only registered in lyx-user).
This is why I did not reply yet.

I read almost all of the posts in http://www.mail-archive.com. I will
eventually register myself in lyx-dev but dunno if its worth it for only
one thread.

Is there another way so I receive the posts for only this thread from
lyx-dev?

Cheers, Nikos


signature.asc
Description: This is a digitally signed message part

RE: Greek text mixed with English

[Vincent van Ravesteijn - TNW]

>Vincent van Ravesteijn - TNW wrote:
>> This is my interpretation of the problem: unexpected output of greek 
>> characters while there are latin characters on screen.
>
>Which I would judge an information deficit. We should communicate better
>why languages need to be marked in LyX (and why this is a good thing).
>

So, we show the characters in greek also on screen. That seems the best way to 
communicate to the user that these characters will be converted.

>Jürgen

Vincent

Re: Greek text mixed with English

[Jürgen Spitzmüller]

Vincent van Ravesteijn - TNW wrote:
> So, we show the characters in greek also on screen. That seems the best way
> to communicate to the user that these characters will be converted.

OK.

Jürgen

Re: Greek text mixed with English

[Guenter Milde]

On 2010-02-28, Vincent van Ravesteijn - TNW wrote:
>>Vincent van Ravesteijn - TNW wrote:
>>> This is my interpretation of the problem: unexpected output of greek 
>>> characters while there are latin characters on screen.

Yes, this is the OP's problem. Maybe it should rather be named: words
using written with Latin letters in a Greek document.

The OP writes Greek using a Greek keyboard, so the conversion of Latin
letters via transliteration is an unwanted and unexpected effect.

>>Which I would judge an information deficit. We should communicate better
>>why languages need to be marked in LyX (and why this is a good thing).

> So, we show the characters in greek also on screen. That seems the best
> way to communicate to the user that these characters will be converted.

But this is just one side. A filename, a C command or some Acronym in a
Greek document do not gain from beeing marked as another language but
still should keep Latin letters as such!


I see room for both, Greek with transliteration of Latin input (for
the occasional Greek quote in a non-Greek document) and Greek with
Unicode input (where Latin characters remain Latin).

Instead of 4 Greek language variants to care for the combinations:

   monotoniko   polutoniko
transliteration12
Unicode34
   
I suggest a GreekUnicode Module.

Günter

Re: Greek text mixed with English

[Guenter Milde]

On 2010-02-28, Jürgen Spitzmüller wrote:
> Vincent van Ravesteijn - TNW wrote:

>> Yes, that's exactly the point. If there are latin characters on screen
>> in LyX, we don't want to have greek letters appearing in the output
>> right ?

> No. If I use the transliteration, I want exactly this.

>> Even though the latin characters are marked as greek language (which would
>> only affect the spellchecking, hyphenation... ? Or is this a feature ?

> Yes, it's a feature,as you noticed in the meantime.

It is a feature that dates back to pre-Unicode times (actually even back
to 7bit ASCII times). It is still handy for the occasional Greek quote
for people without Greek keyboard, but stands in the way for others. 

This is why this feature should be optional.

> As I see it, the user wants a quick way to switch between two commonly
> used languages (a good shortcut for "language greek" and "language
> english" or bind that functions to the action Greek users perform to
> switch between English and Greek, if any).

This is just one side. The other is to keep Latin input of e.g. a file
name in Latin spelling.

Günter

Re: Greek text mixed with English

[Guenter Milde]

On 2010-02-28, Vincent van Ravesteijn - TNW wrote:

> I think we should output \textlatin for the latin characters as we do
> with \textgreek for greek characters.

I thought about this once, but this would make Greek even more the odd
one out, because e.g. Cyrillic does not work this way:

> Language English:
>   \textcyr{\char254} \textgreek{a} a  % ASCII
or
я  α a  % Unicode

> Language Russian:
>   \textcyr{\char254} \textgreek{a} a  % ASCII
or
я  α a  % Unicode

Latin characters in a text marked as Russion are not transliterated.

It is just Greek that even in Unicode converts Latin to Greek

> Language Greek:
>   \textcyr{\char255} a\textlatin{a}  % ASCII
or
я  α\textlatin{a}  % Unicode


This can be solved with 

  \addto\extrasgreek{\latintext}%

and then:

> Language Greek:
>   \textcyr{\char255} \textgreek{a} a  % ASCII
or
я  α a  % Unicode


Günter

Re: Greek text mixed with English [on/off-topic]

[Liviu Andronic]

Hello

On 2/28/10, Νίκος Αλεξανδρής  wrote:
>  I read almost all of the posts in http://www.mail-archive.com. I will
>  eventually register myself in lyx-dev but dunno if its worth it for only
>  one thread.
>
You could register for the thread, and unsubscribe when it's done.
Liviu