[NTG-context] Invalid UTF-8 sequence in bbl file

[Oleg Kolosov]

Hi all, Taco!

I was experimenting with bib module and discovered something which looks like
a bug.

'texexec' ('texexec bibtest') runs fine but typesets references ('s' key)
incorrectly.

As MkIV does not run bibtex on input files automatically, I run 'bibtex'
manually once ('bibtex bibtest'), then 'context' ('context bibtest') gives
'Invalid UTF-8 sequence' error.

The resulting .bbl is the same in both cases.

It seems that 'author' field and 's' key in bbl file are generated
incorrectly, while others like 'title', 'city', etc. are fine.

See the attached files.

I'm using just updated minimals. t-bib version 2008.04.17.

-- 
Regards,
Oleg Kolosov
\setuppublicationlist[samplesize={АÐ02},totalnumber=1]

\startpublication[k=parintro,t=book,
a={{Антонов}},y=2002,
n=1,s=АÐ02]
\author[]{А.~С.}[Ð.~Ð.]{}{Антонов}
\pubyear{2002}
\title{Введение в параллельные вычисления}
\city{Москва}
\pubname{МГУ им. Ломоносова}
\stoppublication

\enableregime[utf]
\useregime[utf]

\usemodule[bib]
\setupbibtex[database=bibtest.bib]

\starttext
\section Bibmodule test

Cite\cite[parintro].

\placepublications

\stoptext
@book{parintro,
  author = А. С. Антонов,
  title = Введение в параллельные вычисления,
  type = Методическое пособие,
  pages = 69,
  address = Москва,
  publisher = МГУ им. Ломоносова,
  year = 2002
}
___
If your question is of interest to others as well, please add an entry to the 
Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://tex.aanhet.net
archive  : https://foundry.supelec.fr/projects/contextrev/
wiki : http://contextgarden.net
___

Re: [NTG-context] Invalid UTF-8 sequence in bbl file

[Taco Hoekwater]

Oleg Kolosov wrote:

 'texexec' ('texexec bibtest') runs fine but typesets references ('s' key)
 incorrectly.
 
 As MkIV does not run bibtex on input files automatically, I run 'bibtex'
 manually once ('bibtex bibtest'), then 'context' ('context bibtest') gives
 'Invalid UTF-8 sequence' error.

This is caused by the fact that bibtex doesn't understand UTF-8 at all
(it is only able to do something useful with byte encodings). I have
no immediate solution for you, the best advice I can offer right now is
for you to manually fix the bbl file after running bibtex.

The whole database is converted in one go, and there is never a need to
re-run bibtex unless when you delete the bbl by accident, so this is
a single (laborious) step (under the assumption that the bib file is
finished already). A long-term solution will depend on a more modern
bib database processor (like J.M. Hufflen's mlbibtex).

Best wishes,
Taco
___
If your question is of interest to others as well, please add an entry to the 
Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://tex.aanhet.net
archive  : https://foundry.supelec.fr/projects/contextrev/
wiki : http://contextgarden.net
___

Re: [NTG-context] Invalid UTF-8 sequence in bbl file

[Oleg Kolosov]

Taco Hoekwater wrote:
 As MkIV does not run bibtex on input files automatically, I run 'bibtex'
 manually once ('bibtex bibtest'), then 'context' ('context bibtest') gives
 'Invalid UTF-8 sequence' error.
 
 This is caused by the fact that bibtex doesn't understand UTF-8 at all
 (it is only able to do something useful with byte encodings). I have
 no immediate solution for you, the best advice I can offer right now is
 for you to manually fix the bbl file after running bibtex.
 
 The whole database is converted in one go, and there is never a need to
 re-run bibtex unless when you delete the bbl by accident, so this is
 a single (laborious) step (under the assumption that the bib file is
 finished already). A long-term solution will depend on a more modern
 bib database processor (like J.M. Hufflen's mlbibtex).
 

I didn't think about bibtex as a separate package from old times, was too
carried away by luatex fancies. Thanks for clarifying this out.

I've used the following workaround: write bib file in 8-bit encoding (koi8-r
for example), run bibtex on it once, then convert bbl to UTF-8. This works
perfectly, should have guessed it earlier. I don't change bib files
frequently, so this solution suits me well.

-- 
Regards,
Oleg Kolosov
___
If your question is of interest to others as well, please add an entry to the 
Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://tex.aanhet.net
archive  : https://foundry.supelec.fr/projects/contextrev/
wiki : http://contextgarden.net
___