Re: [NTG-context] Problem with \doifelse testing of UTF-8 chars
On Fri, Apr 23, 2010 at 00:30, Jaroslav Hajtmar wrote: Thank you very much. Due to your suggestion it was my problem resolved. Problem has been truly in unexpanded macros. Can the solution be applied to the m-database.tex module as well? Last time when I tried the UTF-8 didn't work with mkii (if the first character in cell wasn't an ascii character). Mojca Example: \enableregime[utf-8] \usemodule[database] \defineseparatedlist[CSV] [separator=comma, before=\bTABLE, after=\eTABLE, first=\bTR, last=\eTR, left=\bTD, right=\eTD] \starttext \startCSV a,b,c,č d,e,f,š \stopCSV \stoptext ___ If your question is of interest to others as well, please add an entry to the Wiki! maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context webpage : http://www.pragma-ade.nl / http://tex.aanhet.net archive : http://foundry.supelec.fr/projects/contextrev/ wiki : http://contextgarden.net ___
Re: [NTG-context] Problem with \doifelse testing of UTF-8 chars
On 23-4-2010 2:31, Mojca Miklavec wrote:
On Fri, Apr 23, 2010 at 00:30, Jaroslav Hajtmar wrote:
Thank you very much. Due to your suggestion it was my problem resolved.
Problem has been truly in unexpanded macros.
Can the solution be applied to the m-database.tex module as well?
Last time when I tried the UTF-8 didn't work with mkii (if the first
character in cell wasn't an ascii character).
Mojca
Example:
\enableregime[utf-8]
\usemodule[database]
\defineseparatedlist[CSV]
[separator=comma,
before=\bTABLE, after=\eTABLE,
first=\bTR, last=\eTR,
left=\bTD, right=\eTD]
\starttext
\startCSV
a,b,c,č
d,e,f,š
\stopCSV
\stoptext
___
If your question is of interest to others as well, please add an entry to the
Wiki!
maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage : http://www.pragma-ade.nl / http://tex.aanhet.net
archive : http://foundry.supelec.fr/projects/contextrev/
wiki : http://contextgarden.net
___
--
-
Hans Hagen | PRAGMA ADE
Ridderstraat 27 | 8061 GH Hasselt | The Netherlands
tel: 038 477 53 69 | voip: 087 875 68 74 | www.pragma-ade.com
| www.pragma-pod.nl
-
%D \module
%D [ file=m-database,
%Dversion=2006.04.23,
%D title=\CONTEXT\ Modules,
%D subtitle=Database Thingies,
%D author=Hans Hagen\ Taco Hoekwater,
%D date=\currentdate,
%D copyright=PRAGMA
%D ]
%C This module is part of the \CONTEXT\ macro||package and is
%C therefore copyrighted by \PRAGMA. See mreadme.pdf for
%C details.
\unprotect
% % % to be added to mult-* files
%
% % % todo: \dontcollectseparatedlist via k/v
\definesystemvariable{ls}
\def\c!first {first}
\def\c!last {last}
\def\c!quotechar {quotechar}
\l...@nx\noexpand
%D {processquotedlist}
%D
%D An even more general list processing macro is the
%D following one:
%D
%D \starttyping
%D \processquotedlist{beginsym}{endsym}{separator}\docommand list
%D \stoptyping
%D
%D This one supports arbitrary open and close symbols as well
%D as user defined separators.
%D
%D \starttyping
%D \processquotedlist(){,}{}\docommand(a=b=c=d)
%D \stoptyping
\def\processquotedlist#1#2#3#4#5%
{\def\csvquotechar{#4}%
\edef\doconvertcsvquoteditem#4##1#4##2%
{...@nx\ifx##2#3%
\l...@nx\next\@NX\doconvertcsvlist
\d...@nx\arg{#3}%
\ed...@nx\temp{##1}%
\...@nx\expanded{\@nx#...@nx\temp}}%
\...@nx\else
\l...@nx\next\@NX\redoconvertcsvquoteditem
\d...@nx\arg{##1\@NX\csvquotechar}%
\...@nx\fi
\...@nx\expandafter\@nx\ne...@nx\arg}%
\edef\redoconvertcsvquoteditem##1#4##2%
{...@nx\ifx##2#3%
\l...@nx\next\@NX\doconvertcsvlist
\d...@nx\arg{#3}%
\ed...@nx\temp{##1}%
\...@nx\expanded{\@nx#...@nx\temp}}%
\...@nx\else
\l...@nx\next\@NX\redoconvertcsvquoteditem
\d...@nx\arg{##1\@NX\csvquotechar}%
\...@nx\fi
\...@nx\expandafter\@nx\ne...@nx\arg}%
\edef\doconvertcsvitem##1#3%
{\ed...@nx\temp{##1}%
\...@nx\expanded{\@nx#...@nx\temp}}%
\...@nx\doconvertcsvlist#3}%
\edef\doconvertcsvlist#3##1%
{...@nx\ifx##1\@NX#2%
\l...@nx\next \...@nx\gobbleoneargument
\...@nx\else\@NX\ifx##1#4%
\l...@nx\next \...@nx\doconvertcsvquoteditem
\...@nx\else
\l...@nx\next \...@nx\doconvertcsvitem
\...@nx\fi\@NX\fi \...@nx\next##1}%
\doconvertcsvitem}
\gdef\doprocessseparatedquoteditem#1%
{\appendseparatedlistparameter\c!left%
\appendseparatedlistcontent{#1}%
\appendseparatedlistparameter\c!right}
% % % so far
% \def\dodefineprocessseplist#1#2% separator \docommand
% {\def\dodoprocessseplist##1##2#1%
% {\ifx\relax##1%
% \expandafter\nodoprocessseplist
% \else\ifx##1#1%
% #2{}%
% #2{##2}%
% \expandafter\expandafter\expandafter\dodoprocessseplist
% \else
% #2{##1##2}%
% \expandafter\expandafter\expandafter\dodoprocessseplist
% \fi\fi}%
%\def\doprocessseplist##1\relax
% {\dodoprocessseplist##1#1\relax#1\relax\relax\end}}
%
% a version more robust with regard to {a a} b c d situations
\edef\detokenizedrelax{\detokenize{\relax}}
\def\dodefineprocessseplist#1#2% separator \docommand
{\def\dodoprocessseplist##1##2#1%
{\edef\!!stringa{\detokenize{##1}}%
\ifx\detokenizedrelax\!!stringa
\expandafter\nodoprocessseplist
\else\ifx\!!stringa#1%
#2{}%
#2{##2}%
\expandafter\expandafter\expandafter\dodoprocessseplist
\else
#2{##1##2}%
Re: [NTG-context] Problem with \doifelse testing of UTF-8 chars
Wow, great, thanks a lot! It works much better now. Mojca ___ If your question is of interest to others as well, please add an entry to the Wiki! maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context webpage : http://www.pragma-ade.nl / http://tex.aanhet.net archive : http://foundry.supelec.fr/projects/contextrev/ wiki : http://contextgarden.net ___
Re: [NTG-context] Problem with \doifelse testing of UTF-8 chars
On 23-4-2010 2:31, Mojca Miklavec wrote:
Last time when I tried the UTF-8 didn't work with mkii (if the first
character in cell wasn't an ascii character).
another trick:
\bgroup
\keeputfcharacters
\doifelse{slovenian mkii utf} {whatever}
{\egroup ..}
{\egroup ..}
-
Hans Hagen | PRAGMA ADE
Ridderstraat 27 | 8061 GH Hasselt | The Netherlands
tel: 038 477 53 69 | voip: 087 875 68 74 | www.pragma-ade.com
| www.pragma-pod.nl
-
___
If your question is of interest to others as well, please add an entry to the
Wiki!
maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage : http://www.pragma-ade.nl / http://tex.aanhet.net
archive : http://foundry.supelec.fr/projects/contextrev/
wiki : http://contextgarden.net
___
[NTG-context] Problem with \doifelse testing of UTF-8 chars
Hello,
I have problem with testing of strings through \doifelse.
I am loading data from external CSV file. CSV file is UTF-8 encoding.
Data from external file I am reading to ConTeXt macro and I want compare
content of macro (it is one utf-8 character) and other character as per
example:
(\onecharfromcsvfile contains one character from external CSV file)
% I compare e.g. m character and content of macro \onecharfromcsvfile
\def\result{\doifelse{m}{\onecharfromcsvfile}{true}{false} }
\result is however always false
When CSV file is CP-1250 encoding then macro does work right as well as
classical TeX macro:
\def\result{\if m\onecharfromcsvfile true\else false \fi}
My question is:
Just how modify macro as to work right when CSV file is UTF-8 encoding.
Thanx Jaroslav
PS: I was working up until now only with CP-1250 encoding, but when I
started with LuaTeX (MKIV), then I have to use UTF-8. (MKIV dont work
corectly with external CP-1250 files).
___
If your question is of interest to others as well, please add an entry to the
Wiki!
maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage : http://www.pragma-ade.nl / http://tex.aanhet.net
archive : http://foundry.supelec.fr/projects/contextrev/
wiki : http://contextgarden.net
___
Re: [NTG-context] Problem with \doifelse testing of UTF-8 chars
Jaroslav Hajtmar wrote:
Hello,
I have problem with testing of strings through \doifelse.
I am loading data from external CSV file. CSV file is UTF-8 encoding.
Data from external file I am reading to ConTeXt macro and I want compare
content of macro (it is one utf-8 character) and other character as per
example:
(\onecharfromcsvfile contains one character from external CSV file)
Does it contain a character or a byte from the CSV? This is a
very important difference.
% I compare e.g. m character and content of macro \onecharfromcsvfile
\def\result{\doifelse{m}{\onecharfromcsvfile}{true}{false} }
\result is however always false
Well, that depends on whether m's in your csv file, and how
\onecharfromcvsfile is actually defined. This reports true:
\starttext
\def\onecharfromcsvfile{m}
\def\result{\doifelse{m}{\onecharfromcsvfile}{true}{false} }
\result
\stoptext
Best wishes,
Taco
___
If your question is of interest to others as well, please add an entry to the
Wiki!
maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage : http://www.pragma-ade.nl / http://tex.aanhet.net
archive : http://foundry.supelec.fr/projects/contextrev/
wiki : http://contextgarden.net
___
Re: [NTG-context] Problem with \doifelse testing of UTF-8 chars
Hello.
Thank for your advise.
I was tried your example, but it is not working unfortunately too.
I previwed CSV file - it is UTF-8 encoding and hexa code of m
character is 6D00. This mean that m char is byte coding isnt it?
Exist any way untwist my problem now?
Thanx Jaroslav
In hexa editor is Dne 22.4.2010 21:33, Taco Hoekwater napsal(a):
Jaroslav Hajtmar wrote:
Hello,
I have problem with testing of strings through \doifelse.
I am loading data from external CSV file. CSV file is UTF-8 encoding.
Data from external file I am reading to ConTeXt macro and I want
compare content of macro (it is one utf-8 character) and other
character as per example:
(\onecharfromcsvfile contains one character from external CSV file)
Does it contain a character or a byte from the CSV? This is a
very important difference.
% I compare e.g. m character and content of macro \onecharfromcsvfile
\def\result{\doifelse{m}{\onecharfromcsvfile}{true}{false} }
\result is however always false
Well, that depends on whether m's in your csv file, and how
\onecharfromcvsfile is actually defined. This reports true:
\starttext
\def\onecharfromcsvfile{m}
\def\result{\doifelse{m}{\onecharfromcsvfile}{true}{false} }
\result
\stoptext
Best wishes,
Taco
___
If your question is of interest to others as well, please add an entry to the
Wiki!
maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage : http://www.pragma-ade.nl / http://tex.aanhet.net
archive : http://foundry.supelec.fr/projects/contextrev/
wiki : http://contextgarden.net
___
Re: [NTG-context] Problem with \doifelse testing of UTF-8 chars
Jaroslav Hajtmar wrote: Hello. Thank for your advise. I was tried your example, but it is not working unfortunately too. I previwed CSV file - it is UTF-8 encoding and hexa code of m character is 6D00. This mean that m char is byte coding isnt it? The hex codes of all your 'm'-s in the email messages are 6D, which is UTF-8 for the ASCII m. Sorry, please post a proper example, I don't think we can help without actual input. Best wishes, Taco ___ If your question is of interest to others as well, please add an entry to the Wiki! maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context webpage : http://www.pragma-ade.nl / http://tex.aanhet.net archive : http://foundry.supelec.fr/projects/contextrev/ wiki : http://contextgarden.net ___
Re: [NTG-context] Problem with \doifelse testing of UTF-8 chars
Will help you, when I send you my CSV file? I dont now what you think when you writing, that you want actual input. Code 6D00 is twobyte code of UTF-8 character m... When I switch to CP1250 encoding, then onebyte code of m character is 6D. If need arises then I make any minimal example from my relatively big project. Thanx. Jaroslav Dne 22.4.2010 21:59, Taco Hoekwater napsal(a): Jaroslav Hajtmar wrote: Hello. Thank for your advise. I was tried your example, but it is not working unfortunately too. I previwed CSV file - it is UTF-8 encoding and hexa code of m character is 6D00. This mean that m char is byte coding isnt it? The hex codes of all your 'm'-s in the email messages are 6D, which is UTF-8 for the ASCII m. Sorry, please post a proper example, I don't think we can help without actual input. Best wishes, Taco ___ If your question is of interest to others as well, please add an entry to the Wiki! maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context webpage : http://www.pragma-ade.nl / http://tex.aanhet.net archive : http://foundry.supelec.fr/projects/contextrev/ wiki : http://contextgarden.net ___
Re: [NTG-context] Problem with \doifelse testing of UTF-8 chars
On 22-4-2010 8:57, Jaroslav Hajtmar wrote:
Hello,
I have problem with testing of strings through \doifelse.
I am loading data from external CSV file. CSV file is UTF-8 encoding.
Data from external file I am reading to ConTeXt macro and I want compare
content of macro (it is one utf-8 character) and other character as per
example:
(\onecharfromcsvfile contains one character from external CSV file)
% I compare e.g. m character and content of macro \onecharfromcsvfile
\def\result{\doifelse{m}{\onecharfromcsvfile}{true}{false} }
\result is however always false
When CSV file is CP-1250 encoding then macro does work right as well as
classical TeX macro:
\def\result{\if m\onecharfromcsvfile true\else false \fi}
My question is:
Just how modify macro as to work right when CSV file is UTF-8 encoding.
tricky as utf in mkii is using active chars ... it all depends a bit to
what the character expands; depending on the macro one can also use
\convertargument#1\to\whatever like constructs and then compare the
\whatever
Thanx Jaroslav
PS: I was working up until now only with CP-1250 encoding, but when I
started with LuaTeX (MKIV), then I have to use UTF-8. (MKIV dont work
corectly with external CP-1250 files).
oh? regimes are supposed to work ok in mkiv
Hans
-
Hans Hagen | PRAGMA ADE
Ridderstraat 27 | 8061 GH Hasselt | The Netherlands
tel: 038 477 53 69 | voip: 087 875 68 74 | www.pragma-ade.com
| www.pragma-pod.nl
-
___
If your question is of interest to others as well, please add an entry to the
Wiki!
maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage : http://www.pragma-ade.nl / http://tex.aanhet.net
archive : http://foundry.supelec.fr/projects/contextrev/
wiki : http://contextgarden.net
___
Re: [NTG-context] Problem with \doifelse testing of UTF-8 chars
Jaroslav Hajtmar wrote: Will help you, when I send you my CSV file? I dont now what you think when you writing, that you want actual input. Code 6D00 is twobyte code of UTF-8 character It absolutely is not. Maybe you are confused with UTF-16 ? Best wishes, Taco ___ If your question is of interest to others as well, please add an entry to the Wiki! maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context webpage : http://www.pragma-ade.nl / http://tex.aanhet.net archive : http://foundry.supelec.fr/projects/contextrev/ wiki : http://contextgarden.net ___
Re: [NTG-context] Problem with \doifelse testing of UTF-8 chars
Thank you very much. Due to your suggestion it was my problem resolved.
Problem has been truly in unexpanded macros.
Though I am redefining my macros in last versions of my project (joining
any empty delimiters), macros was predend (on the outside as expanded)
entirely inocently as if has been unchanged.
It was beginners perplexing mistake - but heavy detecting (for me).
Thankx for your and Tacos time.
Jaroslav
Dne 22.4.2010 23:28, Hans Hagen napsal(a):
On 22-4-2010 8:57, Jaroslav Hajtmar wrote:
Hello,
I have problem with testing of strings through \doifelse.
I am loading data from external CSV file. CSV file is UTF-8 encoding.
Data from external file I am reading to ConTeXt macro and I want compare
content of macro (it is one utf-8 character) and other character as per
example:
(\onecharfromcsvfile contains one character from external CSV file)
% I compare e.g. m character and content of macro \onecharfromcsvfile
\def\result{\doifelse{m}{\onecharfromcsvfile}{true}{false} }
\result is however always false
When CSV file is CP-1250 encoding then macro does work right as well as
classical TeX macro:
\def\result{\if m\onecharfromcsvfile true\else false \fi}
My question is:
Just how modify macro as to work right when CSV file is UTF-8 encoding.
tricky as utf in mkii is using active chars ... it all depends a bit
to what the character expands; depending on the macro one can also use
\convertargument#1\to\whatever like constructs and then compare the
\whatever
Thanx Jaroslav
PS: I was working up until now only with CP-1250 encoding, but when I
started with LuaTeX (MKIV), then I have to use UTF-8. (MKIV dont work
corectly with external CP-1250 files).
oh? regimes are supposed to work ok in mkiv
Hans
-
Hans Hagen | PRAGMA ADE
Ridderstraat 27 | 8061 GH Hasselt | The Netherlands
tel: 038 477 53 69 | voip: 087 875 68 74 | www.pragma-ade.com
| www.pragma-pod.nl
-
___
If your question is of interest to others as well, please add an entry to the
Wiki!
maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage : http://www.pragma-ade.nl / http://tex.aanhet.net
archive : http://foundry.supelec.fr/projects/contextrev/
wiki : http://contextgarden.net
___
