Re: [NTG-context] undefined control sequence bug with German umlaut in bibliography

2011-06-07 Thread Ulrike Fischer
Am Sat, 04 Jun 2011 11:26:37 +0200 schrieb Taco Hoekwater:

 Thank you everybody for your answers.  Writing Tr{\a}ger as Thomas
 suggested works well, but unfortunately, I'm using Mendeley Desktop
 for the management of my bibtex file and I can't seem to be able to
 influence the way in which it encodes the special characters.

 Find a different program, then. Bibtex does *not* deal with UTF-8 
 correctly, period. (complaints to Oren Patashnik please ;))

Doesn't context support biber? Or could Julian save its bib in an
8bit-encoding and context read the bbl as 8bit?


-- 
Ulrike Fischer 

___
If your question is of interest to others as well, please add an entry to the 
Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://tex.aanhet.net
archive  : http://foundry.supelec.fr/projects/contextrev/
wiki : http://contextgarden.net
___


Re: [NTG-context] undefined control sequence bug with German umlaut in bibliography

2011-06-06 Thread Arthur Reutenauer
 Well, there *is* more than one way to represent ä in UTF-8

  If you mean non-shortest forms such as 0xE0 0x83 0xA4 or 0xF0 0x80 0x83
0xA4, then no, they have been forbidden since Unicode 3 in 2000 (formally
Corrigendum #1, see http://www.unicode.org/versions/corrigendum1.html).

Arthur
___
If your question is of interest to others as well, please add an entry to the 
Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://tex.aanhet.net
archive  : http://foundry.supelec.fr/projects/contextrev/
wiki : http://contextgarden.net
___


Re: [NTG-context] undefined control sequence bug with German umlaut in bibliography

2011-06-06 Thread Pontus Lurcock
On Mon 06 Jun 2011, Arthur Reutenauer wrote:

  Well, there *is* more than one way to represent ä in UTF-8
 
 If you mean non-shortest forms such as 0xE0 0x83 0xA4 or 0xF0 0x80
 0x83 0xA4, then no, they have been forbidden since Unicode 3 in 2000
 (formally Corrigendum #1, see
 http://www.unicode.org/versions/corrigendum1.html).

I was actually thinking of precomposed vs. combining diacritics. I was
blissfully unaware of the non-shortest-form problem up until now...

Pont
___
If your question is of interest to others as well, please add an entry to the 
Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://tex.aanhet.net
archive  : http://foundry.supelec.fr/projects/contextrev/
wiki : http://contextgarden.net
___


Re: [NTG-context] undefined control sequence bug with German umlaut in bibliography

2011-06-06 Thread Arthur Reutenauer
 I was actually thinking of precomposed vs. combining diacritics. I was
 blissfully unaware of the non-shortest-form problem up until now...

  Ah, OK.  But that's exactly the issue for which canonical equivalence
was designed, and in a Unicode-aware version of BibTeX that shouldn't be
an issue.  However, for now...

Arthur
___
If your question is of interest to others as well, please add an entry to the 
Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://tex.aanhet.net
archive  : http://foundry.supelec.fr/projects/contextrev/
wiki : http://contextgarden.net
___


Re: [NTG-context] undefined control sequence bug with German umlaut in bibliography

2011-06-04 Thread Julian Becker
Thank you everybody for your answers.  Writing Tr{\a}ger as Thomas
suggested works well, but unfortunately, I'm using Mendeley Desktop for the
management of my bibtex file and I can't seem to be able to influence the
way in which it encodes the special characters.

@Mojca: Indeed, it also fails if I just write Träger in UTF-8 encoding
(however, Schräger works just fine. All combinations where the ä is at
the third place of the word seem to fail.). The error message is similar but
slightly different now. The log file shows the following:
---
system  begin file test.tex at line 3
publications  loading database from test.bbl
(test.bbl
! String contains an invalid utf-8 sequence.
l.1 \setuppublicationlist[samplesize={Tr
Ã06},totalnumber=1]
A funny symbol that I can't read has just been (re)read.
Just continue, I'll change it to 0xFFFD.

! String contains an invalid utf-8 sequence.
l.5 n=1,s=Tr
Ã06]
A funny symbol that I can't read has just been (re)read.
Just continue, I'll change it to 0xFFFD.

! String contains an invalid utf-8 sequence.
\doifassignmentelse ...gnmentelse \detokenize {#1}
=@@\@end@ \expandafter \se...
\dostartpublication ... -\doifassignmentelse {#1}
{\getparameters [\??pb ][k...
l.9 \stoppublication

A funny symbol that I can't read has just been (re)read.
Just continue, I'll change it to 0xFFFD.

)
---

Julian


2011/6/4 Pontus Lurcock p...@talvi.net

 On Fri 03 Jun 2011, Thomas A. Schmitz wrote:

  But I admit it's not easy to know that, bibtex documentation is a
  real mess

 Patience please! ‘This document will be expanded when BibTEX version
 1.00 comes out’ -- BIBTEXing, February 8, 1988.

 :-)

 Pont

 ___
 If your question is of interest to others as well, please add an entry to
 the Wiki!

 maillist : ntg-context@ntg.nl /
 http://www.ntg.nl/mailman/listinfo/ntg-context
 webpage  : http://www.pragma-ade.nl / http://tex.aanhet.net
 archive  : http://foundry.supelec.fr/projects/contextrev/
 wiki : http://contextgarden.net

 ___




-- 
Julian Becker
Institut für Angewandte Physik, R.123
Westfälische Wilhelms-Universität Münster
Corrensstr. 2/4
48149 Münster / Westfalen
Tel. 0251 83-3 61 53
Mob. 0151 599 848 29
e-mail: j_bec...@uni-muenster.de

Keep thy heart with all diligence; for it is the wellspring of life.
___
If your question is of interest to others as well, please add an entry to the 
Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://tex.aanhet.net
archive  : http://foundry.supelec.fr/projects/contextrev/
wiki : http://contextgarden.net
___

Re: [NTG-context] undefined control sequence bug with German umlaut in bibliography

2011-06-04 Thread Julian Becker
I can also add that in the first case with the author name Träger, the
generated bbl-file looks messed up and (Notepad++ doesn't recognize the
encoding as UTF8. Changing the encoding to UTF8 manually shows the complete
names Träger correctly, but the abbreviations (what should have been
Trä06) seem to be messed up.

I'm not familiar with the intricacies and details of UTF8 encoding, but is
it possible that there is a byte missing from the ä which has been cut off
during the abbreviation process?

The abbreviated Trä06 seems to be incorrectly encoded (in hexadecimal) as:
54 C3 A4 30 36,
while it should be: 54 72 C3 A4 30 36.

So in the abbreviation process, the encoding of some characters over several
bytes seems to be neglected.

I attached the bbl-files for both cases to this e-mail, since I don't know,
what would happen to the encoding, if I just pasted them as plain text here.

Julian


2011/6/4 Julian Becker becker.jul...@gmail.com

 Thank you everybody for your answers.  Writing Tr{\a}ger as Thomas
 suggested works well, but unfortunately, I'm using Mendeley Desktop for the
 management of my bibtex file and I can't seem to be able to influence the
 way in which it encodes the special characters.

 @Mojca: Indeed, it also fails if I just write Träger in UTF-8 encoding
 (however, Schräger works just fine. All combinations where the ä is at
 the third place of the word seem to fail.). The error message is similar but
 slightly different now. The log file shows the following:
 ---
 system  begin file test.tex at line 3
 publications  loading database from test.bbl
 (test.bbl
 ! String contains an invalid utf-8 sequence.
 l.1 \setuppublicationlist[samplesize={Tr
 Ã06},totalnumber=1]
 A funny symbol that I can't read has just been (re)read.
 Just continue, I'll change it to 0xFFFD.

 ! String contains an invalid utf-8 sequence.
 l.5 n=1,s=Tr
 Ã06]
 A funny symbol that I can't read has just been (re)read.
 Just continue, I'll change it to 0xFFFD.

 ! String contains an invalid utf-8 sequence.
 \doifassignmentelse ...gnmentelse \detokenize {#1}
 =@@\@end@ \expandafter \se...
 \dostartpublication ... -\doifassignmentelse {#1}
 {\getparameters [\??pb ][k...
 l.9 \stoppublication

 A funny symbol that I can't read has just been (re)read.
 Just continue, I'll change it to 0xFFFD.

 )
 ---

 Julian


 2011/6/4 Pontus Lurcock p...@talvi.net

 On Fri 03 Jun 2011, Thomas A. Schmitz wrote:

  But I admit it's not easy to know that, bibtex documentation is a
  real mess

 Patience please! ‘This document will be expanded when BibTEX version
 1.00 comes out’ -- BIBTEXing, February 8, 1988.

 :-)

 Pont

 ___
 If your question is of interest to others as well, please add an entry to
 the Wiki!

 maillist : ntg-context@ntg.nl /
 http://www.ntg.nl/mailman/listinfo/ntg-context
 webpage  : http://www.pragma-ade.nl / http://tex.aanhet.net
 archive  : http://foundry.supelec.fr/projects/contextrev/
 wiki : http://contextgarden.net

 ___




 --
 Julian Becker
 Institut für Angewandte Physik, R.123
 Westfälische Wilhelms-Universität Münster
 Corrensstr. 2/4
 48149 Münster / Westfalen
 Tel. 0251 83-3 61 53
 Mob. 0151 599 848 29
 e-mail: j_bec...@uni-muenster.de

 Keep thy heart with all diligence; for it is the wellspring of life.




-- 
Julian Becker
Institut für Angewandte Physik, R.123
Westfälische Wilhelms-Universität Münster
Corrensstr. 2/4
48149 Münster / Westfalen
Tel. 0251 83-3 61 53
Mob. 0151 599 848 29
e-mail: j_bec...@uni-muenster.de

Keep thy heart with all diligence; for it is the wellspring of life.


testA.bbl
Description: Binary data


testB.bbl
Description: Binary data
___
If your question is of interest to others as well, please add an entry to the 
Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://tex.aanhet.net
archive  : http://foundry.supelec.fr/projects/contextrev/
wiki : http://contextgarden.net
___

Re: [NTG-context] undefined control sequence bug with German umlaut in bibliography

2011-06-04 Thread Taco Hoekwater

On 06/04/2011 11:23 AM, Julian Becker wrote:


Thank you everybody for your answers.  Writing Tr{\a}ger as Thomas
suggested works well, but unfortunately, I'm using Mendeley Desktop
for the management of my bibtex file and I can't seem to be able to
influence the way in which it encodes the special characters.


Find a different program, then. Bibtex does *not* deal with UTF-8 
correctly, period. (complaints to Oren Patashnik please ;))


Best wishes,
Taco
___
If your question is of interest to others as well, please add an entry to the 
Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://tex.aanhet.net
archive  : http://foundry.supelec.fr/projects/contextrev/
wiki : http://contextgarden.net
___


Re: [NTG-context] undefined control sequence bug with German umlaut in bibliography

2011-06-04 Thread Pontus Lurcock
On Sat 04 Jun 2011, Julian Becker wrote:

 I'm not familiar with the intricacies and details of UTF8 encoding,
 but is it possible that there is a byte missing from the ä which
 has been cut off during the abbreviation process?

Well, there *is* more than one way to represent ä in UTF-8, but it's
my understanding that anything beyond ASCII is simply not supported by
BibTeX. You can get away with it in fields that just get pasted
verbatim into the output (usually), but the first three letters of the
first author's name are used to construct the key (which is why
‘Schräger’ worked) so there's no way around using the officially
sanctioned {\a} form.

  unfortunately, I'm using Mendeley Desktop for the management of my
  bibtex file and I can't seem to be able to influence the way in
  which it encodes the special characters.

This is one reason why I still use plain emacs as a bibliography
manager -- sooner or later you need a hack, and that's harder when the
raw BibTeX is hidden or generated. In this case you may need to
put the hack between Mendeley and BibTeX: pipe the file through 
‘sed -e 's/ä/{\\a}/g'’ or something similar. And/or ask in the Mendeley 
support forums, since this is a fairly well-known BibTeX ‘feature’
so perhaps someone else has had to deal with it there.

Hope this helps,

Pont
___
If your question is of interest to others as well, please add an entry to the 
Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://tex.aanhet.net
archive  : http://foundry.supelec.fr/projects/contextrev/
wiki : http://contextgarden.net
___

Re: [NTG-context] undefined control sequence bug with German umlaut in bibliography

2011-06-04 Thread Julian Becker
I think I'll go for the piping option then, which seems to be the easiest
way out.

Thanks for the insights, I didn't actually know much about the interplay of
context and bibtex until this little problem occured to me...

Julian


2011/6/4 Pontus Lurcock p...@talvi.net

 On Sat 04 Jun 2011, Julian Becker wrote:

  I'm not familiar with the intricacies and details of UTF8 encoding,
  but is it possible that there is a byte missing from the ä which
  has been cut off during the abbreviation process?

 Well, there *is* more than one way to represent ä in UTF-8, but it's
 my understanding that anything beyond ASCII is simply not supported by
 BibTeX. You can get away with it in fields that just get pasted
 verbatim into the output (usually), but the first three letters of the
 first author's name are used to construct the key (which is why
 ‘Schräger’ worked) so there's no way around using the officially
 sanctioned {\a} form.

   unfortunately, I'm using Mendeley Desktop for the management of my
   bibtex file and I can't seem to be able to influence the way in
   which it encodes the special characters.

 This is one reason why I still use plain emacs as a bibliography
 manager -- sooner or later you need a hack, and that's harder when the
 raw BibTeX is hidden or generated. In this case you may need to
 put the hack between Mendeley and BibTeX: pipe the file through
 ‘sed -e 's/ä/{\\a}/g'’ or something similar. And/or ask in the Mendeley
 support forums, since this is a fairly well-known BibTeX ‘feature’
 so perhaps someone else has had to deal with it there.

 Hope this helps,

 Pont

 ___
 If your question is of interest to others as well, please add an entry to
 the Wiki!

 maillist : ntg-context@ntg.nl /
 http://www.ntg.nl/mailman/listinfo/ntg-context
 webpage  : http://www.pragma-ade.nl / http://tex.aanhet.net
 archive  : http://foundry.supelec.fr/projects/contextrev/
 wiki : http://contextgarden.net

 ___




-- 
Julian Becker
Institut für Angewandte Physik, R.123
Westfälische Wilhelms-Universität Münster
Corrensstr. 2/4
48149 Münster / Westfalen
Tel. 0251 83-3 61 53
Mob. 0151 599 848 29
e-mail: j_bec...@uni-muenster.de

Keep thy heart with all diligence; for it is the wellspring of life.
___
If your question is of interest to others as well, please add an entry to the 
Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://tex.aanhet.net
archive  : http://foundry.supelec.fr/projects/contextrev/
wiki : http://contextgarden.net
___

Re: [NTG-context] undefined control sequence bug with German umlaut in bibliography

2011-06-03 Thread Thomas A. Schmitz

On Jun 3, 2011, at 8:38 PM, Julian Becker wrote:

 I came across an issue in context (context ver. 2011.05.18 22:26, LuaTeX ver: 
 beta-0.65.0-2010121421 (rev 4034) ) when trying to cite a bibliography item 
 having an author with a German umlaut ä
 

From btxdoc, which is part of texlive:

you must place the entire accented character in braces; in this case either 
{\a} or {\{a}} will do. Furthermore these braces must not themselves be 
enclosed in braces (other than the ones that might delimit the entire field or 
the entire entry); and there must be a backslash as the very first character 
inside the braces...

Thusly: Tr{\a}ger

But I admit it's not easy to know that, bibtex documentation is a real mess, 
and Oren Patashnik appears to suffer from a real disease which prevents him 
from writing clear sentences and easily parsable documents.

Thomas
___
If your question is of interest to others as well, please add an entry to the 
Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://tex.aanhet.net
archive  : http://foundry.supelec.fr/projects/contextrev/
wiki : http://contextgarden.net
___


Re: [NTG-context] undefined control sequence bug with German umlaut in bibliography

2011-06-03 Thread Pontus Lurcock
On Fri 03 Jun 2011, Thomas A. Schmitz wrote:

 But I admit it's not easy to know that, bibtex documentation is a
 real mess

Patience please! ‘This document will be expanded when BibTEX version
1.00 comes out’ -- BIBTEXing, February 8, 1988.

:-)

Pont
___
If your question is of interest to others as well, please add an entry to the 
Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://tex.aanhet.net
archive  : http://foundry.supelec.fr/projects/contextrev/
wiki : http://contextgarden.net
___