Re: cumbersome behavior when switching output-engines regarding inputenc
On 2016-12-05, mn wrote: > On 05.12.16 09:36, Guenter Milde wrote: >>> There might be some incorrect or unexpected behavior when >>> switching output engines in LyX, because Xetex does not >>> support input-encoding settings: >>> To reproduce: Start a document, >> ... >>> Try to compile the document. It will complain that inputenc >>> is not suitable for the chosen engine. >> Not here. It just compiles. > I just retook the steps I outlined above again and had the same > error again. There must be something else in your document. Try with File>New and see whether this produces the error. If yes, send the document. If not, tell what else is needed. >>> "Language Package: Always Babel" (a remnant default) >> This is the real reason: Babel-Hebrew is not suited for Unicode >> fonts. > Aha. > Which I thought of as taken care of by LyX, since this was a gui setting. By default, LyX selects the correct language package depending on "use-non-TeX fonts" and document language - this is with "langauge package = auto". The other GUI settings are there to override LyX's choice in case you have special requirements unknown to LyX. ... >> You must use Polyglossia with "non-TeX fonts" and Hebrew (In LyX, >> setting language module to "auto" is recommended.) > Which I will do from now on. > Thanks for the heads up. >>> one line in the preamble loading Minion Pro. >> Actually, loading MinionPro with "use non-TeX fonts" is useless at >> best and could be problematic. The package sets up 8-bit fonts. > I was experimenting with the Xetex-engine I have to use apparently now. > Comparing outputs and fonts and handling opposed to pdflatex. > I was hoping on it to ignore my loading Minion in that way, since I need > to do it that way for pdflatex as setting the Font to Minion via gui is > mysteriously broken (and incomplete). You can, however, wrap the call to Minion in a conditonal that does not load the package if "fontspec" is loaded. >> Most probabely, the content of the preamble does not matter, anything >> exept an empty user preamble will do (see below). >>> Source View for Format LyX displays "\inputencoding utf8" while set >>> for default-output: XeTeX >> I cannot reproduce this here. >>> To be sure I attached a small file demoing the error as is. >> This reveals one more precondition: Hebrew text in the file. Remove >> this and it should compile OK. Any text (even ASCII) set to "language >> hebrew" should trigger the error (given the other preconditios), >> because then Babel-Hebrew loads inputenc. > This pulled-in dependency was completely off-radar for me. > Removing Hebrew text from the file still loads babel but compiles. Yes, up-to-date Babel supports fontspec for many languages (including Greek and Russian). ... >>> Source View for Format LyX displays >>> "\inputencoding utf8" >>> while set for default-output: XeTeX >> seems to indicate that you are using LyX 2.1 ... > But version here is 2.2.2, I still don't get this here: "\inputencoding utf8" is an encoding-change command. Do you get this with utf-8 or with "language default"? Is it causing the problem or solving it? Where in the document does it occur? Can you give 3 lines of context? > If babel is so problematic for these engines, should LyX not then ignore > these babel settings (which I forced on for pdflatex)? No, there are some languages that are only supported by Babel. > I do not know how many other (babel)languages pull in inputenc? Hard to say, even a grep in the *.ldf files will not help, as a language definition file may load inpuenc or not depending on the font setting. OTOH, LyX ships with a file "languages" that list for every language whether it is supported by Babel and/or Polyglossia. This is used with the default setting "auto" to determine the right package. >> I wonder if you could trigger an: "\inputencoding undefined" error >> with your document and LyX version when removing the Hebrew text or >> setting the language package to "auto" or "polyglossia". >> In any case, I don't believe we should unhide the inputenc setting >> with "non-TeX fonts". > Agreed. > That gui setting just sent me on the wrong track to solving this. But the hidden setting could have sent you back on track... Günter
Re: cumbersome behavior when switching output-engines regarding inputenc
On 05.12.16 09:36, Guenter Milde wrote:
>> There might be some incorrect or unexpected behavior when
>> switching output engines in LyX, because Xetex does not
>> support input-encoding settings:
>> To reproduce: Start a document,
> ...
>> Try to compile the document. It will complain that inputenc
>> is not suitable for the chosen engine.
>
> Not here. It just compiles.
>
I just retook the steps I outlined above again and had the same
error again.
>
>>> There must be something else in your document. Try with File>New
>>> and see whether this produces the error. If yes, send the
>>> document. If not, tell what else is needed.
>
>> "Language Package: Always Babel" (a remnant default)
>
> This is the real reason: Babel-Hebrew is not suited for Unicode
> fonts.
Aha.
Which I thought of as taken care of by LyX, since this was a gui setting.
> Also, it always loads the inputenc package.
>
> We also need to load inputenc package with one of the Hebrew input
> encodings. [...] \AtEndOfPackage{%
> \@ifpackageloaded{inputenc}{}{\RequirePackage[8859-8]{inputenc}}}
>
> You must use Polyglossia with "non-TeX fonts" and Hebrew (In LyX,
> setting language module to "auto" is recommended.)
Which I will do from now on.
Thanks for the heads up.
>> one line in the preamble loading Minion Pro.
>
> Actually, loading MinionPro with "use non-TeX fonts" is useless at
> best and could be problematic. The package sets up 8-bit fonts.
I was experimenting with the Xetex-engine I have to use apparently now.
Comparing outputs and fonts and handling opposed to pdflatex.
I was hoping on it to ignore my loading Minion in that way, since I need
to do it that way for pdflatex as setting the Font to Minion via gui is
mysteriously broken (and incomplete).
> Most probabely, the content of the preamble does not matter, anything
> exept an empty user preamble will do (see below).
>
>> Source View for Format LyX displays "\inputencoding utf8" while set
>> for default-output: XeTeX
>
> I cannot reproduce this here.
>
>> To be sure I attached a small file demoing the error as is.
>
> This reveals one more precondition: Hebrew text in the file. Remove
> this and it should compile OK. Any text (even ASCII) set to "language
> hebrew" should trigger the error (given the other preconditios),
> because then Babel-Hebrew loads inputenc.
This pulled-in dependency was completely off-radar for me.
Removing Hebrew text from the file still loads babel but compiles.
> Cannot reproduce with either LyX 2.2 or 2.3(dev).
>
> Now to the mystery:
>
>> Source View for Format LyX displays
>> "\inputencoding utf8"
>> while set for default-output: XeTeX
>
> seems to indicate that you are using LyX 2.1 and a bug in this
> version resulted in a workaround for a bug in your document and a bug
> in inputenc.
>
But version here is 2.2.2,
as should be clear from the file I attached.
If babel is so problematic for these engines, should LyX not then ignore
these babel settings (which I forced on for pdflatex)?
I do not know how many other (babel)languages pull in inputenc?
> I wonder if you could trigger an: "\inputencoding undefined" error
> with your document and LyX version when removing the Hebrew text or
> setting the language package to "auto" or "polyglossia".
>
> In any case, I don't believe we should unhide the inputenc setting
> with "non-TeX fonts".
Agreed.
That gui setting just sent me on the wrong track to solving this.
If there are not many other scenarios that could trigger roughly the
same or similar behavior it is probably not needed so much as I thought.
(Since I basically shot myself in the foot here.)
Nevertheless I learned another caveat: to really watch out for or decide
very early on top-down what I want to do with a LyX-document.
Some cases allow for experimenting with or switching such settings much
later in the process.
greetings
Mike
Re: cumbersome behavior when switching output-engines regarding inputenc
Am Sonntag, den 04.12.2016, 21:24 +0100 schrieb mn:
> "Language Package: Always Babel" (a remnant default)
> one line in the preamble loading Minion Pro.
>
> Source View for Format LyX displays
> "\inputencoding utf8"
> while set for default-output: XeTeX
> To be sure I attached a small file demoing the error as is.
The hebrew babel file (hebrew.ldf) internally loads inputenc:
\AtEndOfPackage{%
\@ifpackageloaded{inputenc}{}{\RequirePackage[8859-8]{inputenc}}}
In other words: Babel Hebrew is not yet ready for non-TeX fonts. You'll
need to use polyglossia.
Jürgen
>
>
> greetings
> Mike
signature.asc
Description: This is a digitally signed message part
Re: cumbersome behavior when switching output-engines regarding inputenc
On 2016-12-04, mn wrote:
> [-- Type: text/plain, Encoding: 7bit --]
> On 04.12.16 20:44, Guenter Milde wrote:
>> On 2016-12-04, mn wrote:
> There might be some incorrect or unexpected behavior when switching
> output engines in LyX, because Xetex does not support input-encoding
> settings:
> To reproduce:
> Start a document,
...
> Try to compile the document.
> It will complain that inputenc is not suitable for the chosen engine.
Not here. It just compiles.
>>> I just retook the steps I outlined above again and had the same error
>>> again.
>> There must be something else in your document. Try with File>New and see
>> whether this produces the error. If yes, send the document. If not, tell
>> what else is needed.
> "Language Package: Always Babel" (a remnant default)
This is the real reason: Babel-Hebrew is not suited for Unicode fonts.
Also, it always loads the inputenc package.
We also need to load inputenc package with one of the Hebrew input
encodings.
[...]
\AtEndOfPackage{%
\@ifpackageloaded{inputenc}{}{\RequirePackage[8859-8]{inputenc}}}
You must use Polyglossia with "non-TeX fonts" and Hebrew (In LyX, setting
language module to "auto" is recommended.)
> one line in the preamble loading Minion Pro.
Actually, loading MinionPro with "use non-TeX fonts" is useless at best and
could be problematic. The package sets up 8-bit fonts.
Most probabely, the content of the preamble does not matter,
anything exept an empty user preamble will do (see below).
> Source View for Format LyX displays
> "\inputencoding utf8"
> while set for default-output: XeTeX
I cannot reproduce this here.
> To be sure I attached a small file demoing the error as is.
This reveals one more precondition: Hebrew text in the file.
Remove this and it should compile OK. Any text (even ASCII) set to "language
hebrew" should trigger the error (given the other preconditios), because
then Babel-Hebrew loads inputenc.
Cannot reproduce with either LyX 2.2 or 2.3(dev).
Now to the mystery:
> Source View for Format LyX displays
> "\inputencoding utf8"
> while set for default-output: XeTeX
seems to indicate that you are using LyX 2.1 and a bug in this version
resulted in a workaround for a bug in your document and a bug in inputenc.
I wonder if you could trigger an: "\inputencoding undefined" error with your
document and LyX version when removing the Hebrew text or setting the
language package to "auto" or "polyglossia".
In any case, I don't believe we should unhide the inputenc setting with
"non-TeX fonts".
Günter
Re: cumbersome behavior when switching output-engines regarding inputenc
On 04.12.16 20:44, Guenter Milde wrote: > On 2016-12-04, mn wrote: There might be some incorrect or unexpected behavior when switching output engines in LyX, because Xetex does not support input-encoding settings: > To reproduce: Start a document, go to setup: > - first try something for pdflatex and define under language: input encodig to other (here it was utf8) (then close dialog, reopen dialog and) > >>> Which encoding did you select? > >> Switched from "language default" to "Custom: utf8" > > I suppose the standard "Unicode (utf8), right? > Correct. - switch to fonts, set these to any combination of luatex/Xetex. > >>> Did you mean "tick the 'use non-TeX fonts'" button which activates the >>> "fontspec" package for use of Unicode-encoded fonts? > >> Yes, switch to "Use non-TeX fonts (via XeTeX/LuaTeX)" > >>> With "use non-TeX fonts", both the input encoding and the font >>> encoding are set to "utf-8", overriding any setting under >>> Settings>Language>Encoding. > Try to compile the document. > It will complain that inputenc is not suitable for the chosen engine. > >>> Not here. It just compiles. > >> I just retook the steps I outlined above again and had the same error again. > > There must be something else in your document. Try with File>New and see > whether this produces the error. If yes, send the document. If not, tell > what else is needed. > "Language Package: Always Babel" (a remnant default) one line in the preamble loading Minion Pro. Source View for Format LyX displays "\inputencoding utf8" while set for default-output: XeTeX To be sure I attached a small file demoing the error as is. greetings Mike input-err.lyx Description: application/lyx
Re: cumbersome behavior when switching output-engines regarding inputenc
On 2016-12-04, mn wrote:
> On 04.12.16 16:09, Guenter Milde wrote:
>> On 2016-12-03, mn wrote:
>>> There might be some incorrect or unexpected behavior when switching
>>> output engines in LyX, because Xetex does not support input-encoding
>>> settings:
>>> To reproduce:
>>> Start a document, go to setup:
>>> - first try something for pdflatex and
>>> define under language: input encodig to other (here it was utf8)
>>> (then close dialog, reopen dialog and)
>> Which encoding did you select?
> Switched from "language default" to "Custom: utf8"
I suppose the standard "Unicode (utf8), right?
>>> - switch to fonts, set these to any combination of luatex/Xetex.
>> Did you mean "tick the 'use non-TeX fonts'" button which activates the
>> "fontspec" package for use of Unicode-encoded fonts?
> Yes, switch to "Use non-TeX fonts (via XeTeX/LuaTeX)"
>> With "use non-TeX fonts", both the input encoding and the font
>> encoding are set to "utf-8", overriding any setting under
>> Settings>Language>Encoding.
>>> Try to compile the document.
>>> It will complain that inputenc is not suitable for the chosen engine.
>> Not here. It just compiles.
> I just retook the steps I outlined above again and had the same error again.
There must be something else in your document. Try with File>New and see
whether this produces the error. If yes, send the document. If not, tell
what else is needed.
> ###
> ! Package inputenc Error: inputenc is not designed for xetex or luatex.
> (inputenc)only UTF-8 supported.
> See the inputenc package documentation for explanation.
> Type H for immediate help.
> ...
> l.158 \endinput
This should not happen with "use non-TeX fonts".
> For xelatex or lualatex save the document in UTF-8 encoding
> and do not use inputenc, or use the [utf8] option.
LyX does not use inputenc if "non-TeX fonts" are used. The setting under
language>encoding is ignored.
> ###
> But I found that my usual way of invoking MinionPro for default roman
> for pdflatex is apparently the culprit:
> The preamble-code
> \usepackage[textosf,mathlf]{MinionPro}
> seems to trigger the behavior outlined above.
> Although I fail to see why.
> None of the related .sty-files pull in inputenc?
The full log will tell you, which file loaded inputenc.
All loaded files are logged and nesting is indicated by ().
>>> But then going back to doc-settings and just trying to unset the
>>> other(utf8)-option back to default is inaccessible (greyed out).
>>> To do anything about that you have to first also unset your
>>> font-settings back to choices for pdflatex.
>> Did changing the input encoding setting under Language>Encoding help
>> to make
>> your document compile again? Which setting was required? Did it work with
>> "non-TeX fonts" later?
> Yes. For both tests when it failed and also when I noticed this behavior
> in another document: unsetting first "Use non-TeX-fonts" and then
> changing input encoding back to default, then rechecking "Use
> non-TeX-Fonts" produced a PDF that compiled with XeTeX and looked as
> expected.
Strange.
> Together with the apparent trigger mentioned above I currently do not
> understand why un/checking inputenc:utf8 in LyX's gui has any influence
> on this situation if it is ignored by LyX for XeTeX in he way you decribed.
It would be interesting to see the diff of the exported files with
and without Language>Encoding set to utf8.
You may spot a difference either with View>Source (complete document or
preamble) or File>Export>latex (XeTeX).
Günter
Re: cumbersome behavior when switching output-engines regarding inputenc
On 04.12.16 16:09, Guenter Milde wrote:
> On 2016-12-03, mn wrote:
>
>> There might be some incorrect or unexpected behavior when switching
>> output engines in LyX, because Xetex does not support input-encoding
>> settings:
>
>> To reproduce:
>> Start a document, go to setup:
>
>> - first try something for pdflatex and
>> define under language: input encodig to other (here it was utf8)
>> (then close dialog, reopen dialog and)
>
> Which encoding did you select?
>
Switched from "language default" to "Custom: utf8"
>> - switch to fonts, set these to any combination of luatex/Xetex.
>
> Did you mean "tick the 'use non-TeX fonts'" button which activates the
> "fontspec" package for use of Unicode-encoded fonts?
>
Yes, switch to "Use non-TeX fonts (via XeTeX/LuaTeX)"
> With "use non-TeX fonts", both the input encoding and the font
encoding are
> set to "utf-8", overriding any setting under Settings>Language>Encoding.
>
>> Try to compile the document.
>
>> It will complain that inputenc is not suitable for the chosen engine.
>
> Not here. It just compiles.
>
I just retook the steps I outlined above again and had the same error again.
###
! Package inputenc Error: inputenc is not designed for xetex or luatex.
(inputenc)only UTF-8 supported.
See the inputenc package documentation for explanation.
Type H for immediate help.
...
l.158 \endinput
For xelatex or lualatex save the document in UTF-8 encoding
and do not use inputenc, or use the [utf8] option.
###
But I found that my usual way of invoking MinionPro for default roman
for pdflatex is apparently the culprit:
The preamble-code
\usepackage[textosf,mathlf]{MinionPro}
seems to trigger the behavior outlined above.
Although I fail to see why.
None of the related .sty-files pull in inputenc?
>> But then going back to doc-settings and just trying to unset the
>> other(utf8)-option back to default is inaccessible (greyed out).
>> To do anything about that you have to first also unset your
>> font-settings back to choices for pdflatex.
>
> Did changing the input encoding setting under Language>Encoding help
to make
> your document compile again? Which setting was required? Did it work with
> "non-TeX fonts" later?
>
Yes. For both tests when it failed and also when I noticed this behavior
in another document: unsetting first "Use non-TeX-fonts" and then
changing input encoding back to default, then rechecking "Use
non-TeX-Fonts" produced a PDF that compiled with XeTeX and looked as
expected.
Together with the apparent trigger mentioned above I currently do not
understand why un/checking inputenc:utf8 in LyX's gui has any influence
on this situation if it is ignored by LyX for XeTeX in he way you decribed.
>> This is wrong imho.
>
>> I think LyX should keep these settings for the usage case "switching
>> engines" but gracefully ignore inputenc settings for Xetex.
>
>> This was presumably intended to keep users from messing with inputenc
>> once Xetex was the chosen output engine for a document, but it is
>> confusing when you only find out later that you might have to use Xetex
>> instead of pdflatex.
>
> There is an important distinction between "use XeTeX/LuaTeX" and "use
> Unicode fonts" that even major LaTeX packages get wrong.
>
> Also, the custom counsel "use XeTeX" or "use LuaTeX" given to users with
> font problems or script problems is misleading. Switching the engine does
> not help without switching the fonts, too. Usually, this implies use
of the
> "fontspec" package, in LyX by ticking the "use non-TeX fonts" button.
>
> You can use XeTeX or LuaTeX and 8-bit TeX fonts (and there are rare but
> valid use cases).
>
> However, because even the creators of the basic "inputenc" package did
> get this wrong and added a test, XeTeX can no longer be used with
> inputenc when the input encoding is set to something other than "ascii".
> Therefore, LyX overrides the user-choice in Settings>Language>Encoding
> with XeTeX and 8-bit TeX-fonts. This can lead to errors when the document
> contains characters that LyX cannot convert to ascii.
Thanks for the explanation.
greetings
Mike
Re: cumbersome behavior when switching output-engines regarding inputenc
On 2016-12-03, mn wrote: > There might be some incorrect or unexpected behavior when switching > output engines in LyX, because Xetex does not support input-encoding > settings: > To reproduce: > Start a document, go to setup: > - first try something for pdflatex and > define under language: input encodig to other (here it was utf8) > (then close dialog, reopen dialog and) Which encoding did you select? > - switch to fonts, set these to any combination of luatex/Xetex. Did you mean "tick the 'use non-TeX fonts'" button which activates the "fontspec" package for use of Unicode-encoded fonts? With "use non-TeX fonts", both the input encoding and the font encoding are set to "utf-8", overriding any setting under Settings>Language>Encoding. > Try to compile the document. > It will complain that inputenc is not suitable for the chosen engine. Not here. It just compiles. > But then going back to doc-settings and just trying to unset the > other(utf8)-option back to default is inaccessible (greyed out). > To do anything about that you have to first also unset your > font-settings back to choices for pdflatex. Did changing the input encoding setting under Language>Encoding help to make your document compile again? Which setting was required? Did it work with "non-TeX fonts" later? > This is wrong imho. > I think LyX should keep these settings for the usage case "switching > engines" but gracefully ignore inputenc settings for Xetex. > This was presumably intended to keep users from messing with inputenc > once Xetex was the chosen output engine for a document, but it is > confusing when you only find out later that you might have to use Xetex > instead of pdflatex. There is an important distinction between "use XeTeX/LuaTeX" and "use Unicode fonts" that even major LaTeX packages get wrong. Also, the custom counsel "use XeTeX" or "use LuaTeX" given to users with font problems or script problems is misleading. Switching the engine does not help without switching the fonts, too. Usually, this implies use of the "fontspec" package, in LyX by ticking the "use non-TeX fonts" button. You can use XeTeX or LuaTeX and 8-bit TeX fonts (and there are rare but valid use cases). However, because even the creators of the basic "inputenc" package did get this wrong and added a test, XeTeX can no longer be used with inputenc when the input encoding is set to something other than "ascii". Therefore, LyX overrides the user-choice in Settings>Language>Encoding with XeTeX and 8-bit TeX-fonts. This can lead to errors when the document contains characters that LyX cannot convert to ascii. Günter
