Re: Some umlaut issues in popt 1.15 (the old thing again)

2008-05-25 Thread Jeff Johnson


On May 25, 2008, at 9:19 AM, Robert Scheck wrote:


Hello all,

I've claimed that I can see some umlaut issues with popt 1.14 and I  
really
would like to see it solved, now. Reproducer is for me as follows.  
Using
popt-1.13-3 from Fedora 8, 9 or Rawhide, I simply executed the  
following:


$ [EMAIL PROTECTED] rpm --help  rpm-1.13-iso
$ LANG=de_DE.UTF-8 rpm --help  rpm-1.13-utf

Using popt 1.15 from latest CVS checkout, I also did the following:

$ [EMAIL PROTECTED] rpm --help  rpm-1.15-iso
$ LANG=de_DE.UTF-8 rpm --help  rpm-1.15-utf

Results are now as follows using file(1) to compare:

popt-1.13-iso: ISO-8859 text# as expected
popt-1.13-utf: UTF-8 Unicode text   # as expected

popt-1.15-iso: Non-ISO extended-ASCII text  # NOT as expected
popt-1.15-utf: UTF-8 Unicode text   # as expected



Hmmm, file(1) can be tricked and is perhaps unreliable as a test.

I've written this toy reproducer script:

#!/bin/bash

sudo make -C /X/popt/popt-1.13 install  /dev/null  echo ==  
1.13-3.fc9 installed

[EMAIL PROTECTED] rpm --help  /tmp/popt-1.13-iso
LANG=de_DE.UTF-8 rpm --help  /tmp/popt-1.13-utf
diff -u /tmp/popt-1.13-{iso,utf}

sudo make -C /X/src/popt install  /dev/null  echo == 1.15  
installed

[EMAIL PROTECTED] rpm --help  /tmp/popt-1.15-iso
LANG=de_DE.UTF-8 rpm --help  /tmp/popt-1.15-utf
diff -u /tmp/popt-1.15-{iso,utf}

With popt-1.13-3.fc9 installed (with the popt-1.13-popt_fprintf patch  
applied), I see
differences rendering umlaut's between the output from the 2 locales  
for both 1.13 and 1.15.


(aside) the description column _IS_ aligned, which is/was the rather  
feeble reproducer

 for this issue during popt-1.14 release.

And here is what file(1) has to say on the matter:
[EMAIL PROTECTED] popt]$ file /tmp/popt-1.13-iso
/tmp/popt-1.13-iso: ISO-8859 English text
[EMAIL PROTECTED] popt]$ file /tmp/popt-1.15-iso
/tmp/popt-1.15-iso: Non-ISO extended-ASCII English text
reproducing your results here.

I have most certainly removed the encoding conversion in popt-1.14 so  
that the --help
columns are aligned. Which likely explains the file(1) output, iconv  
is _NOT_ being
performed, because doing the iconv conversion visibly destroyed the -- 
help column alignment.


So a little more help please. What problem should I focus on? Having  
only American
English kinda hampers my abilities to fix this issue, I really don't  
know what to expect
with de_DE locale encoded display output (although I'm way more  
familiar with
rpm --help than kudzu or random GNOME --help). I can most easily  
detect column alignment

issues, duh.

And sure I want to fix _ALL_ the popt problems. Just which one first?


Once my LANG is [EMAIL PROTECTED] for popt = 1.14, the ISO-8859 text  
seems not
sane to be written somehow. Please note, that popt 1.13, I was  
using here
is not the vanilla one, the following patch was applied (no matter  
what it

does exactly):

http://cvs.fedora.redhat.com/viewcvs/*checkout*/devel/popt/ 
popt-1.13-popt_fprintf.patch



Greetings,
  Robertpopt-1.13-iso.gzpopt-1.13-utf.gzpopt-1.15- 
iso.gzpopt-1.15-utf.gz


__
POPT Library   http://rpm5.org
Developer Communication List   popt-devel@rpm5.org


Re: Some umlaut issues in popt 1.15 (the old thing again)

2008-05-25 Thread Jeff Johnson
Here's a summary from trying to sort the dueling umlaut encoding -  
--help column alignment

issue which started here:
http://rpm5.org/community/popt-devel/0061.html

jbj_ what I need if you wish me to fix in popt-1.15 is a better  
test. we don't disagree that popt-1.15-iso is broken. we do disagree  
whether popt-1.13-iso works or not. I will sort the file(1)  
difference, but I don't think that test is gonna be strong enough  
(but I will find out tomorrow).
jbj_ i expect columns to stop aligning, the other problem. I can  
work the column alignment rather than the umlauts destruction if you  
prefer. but I have dueling bugs atm.
jbj_ I'll also see about adding a make check reproducer to popt.  
your popt-devel post with rpm --help reproducer is exactly what I  
needed to attempt. thank you.
rsc np. Let's see how testing develops. A replay of one of the  
previous patches is not such hard as I got you - if needed for 1.14+  
or so


On May 25, 2008, at 6:34 PM, Jeff Johnson wrote:



On May 25, 2008, at 9:19 AM, Robert Scheck wrote:


Hello all,

I've claimed that I can see some umlaut issues with popt 1.14 and  
I really
would like to see it solved, now. Reproducer is for me as follows.  
Using
popt-1.13-3 from Fedora 8, 9 or Rawhide, I simply executed the  
following:


$ [EMAIL PROTECTED] rpm --help  rpm-1.13-iso
$ LANG=de_DE.UTF-8 rpm --help  rpm-1.13-utf

Using popt 1.15 from latest CVS checkout, I also did the following:

$ [EMAIL PROTECTED] rpm --help  rpm-1.15-iso
$ LANG=de_DE.UTF-8 rpm --help  rpm-1.15-utf

Results are now as follows using file(1) to compare:

popt-1.13-iso: ISO-8859 text# as expected
popt-1.13-utf: UTF-8 Unicode text   # as expected

popt-1.15-iso: Non-ISO extended-ASCII text  # NOT as expected
popt-1.15-utf: UTF-8 Unicode text   # as expected



Hmmm, file(1) can be tricked and is perhaps unreliable as a test.

I've written this toy reproducer script:

#!/bin/bash

sudo make -C /X/popt/popt-1.13 install  /dev/null  echo  
== 1.13-3.fc9 installed

[EMAIL PROTECTED] rpm --help  /tmp/popt-1.13-iso
LANG=de_DE.UTF-8 rpm --help  /tmp/popt-1.13-utf
diff -u /tmp/popt-1.13-{iso,utf}

sudo make -C /X/src/popt install  /dev/null  echo == 1.15  
installed

[EMAIL PROTECTED] rpm --help  /tmp/popt-1.15-iso
LANG=de_DE.UTF-8 rpm --help  /tmp/popt-1.15-utf
diff -u /tmp/popt-1.15-{iso,utf}



My script here was flawed, ... libdir=/lib ... was also needed for  
make install. Now fixed.


With popt-1.13-3.fc9 installed (with the popt-1.13-popt_fprintf  
patch applied), I see
differences rendering umlaut's between the output from the 2  
locales for both 1.13 and 1.15.


(aside) the description column _IS_ aligned, which is/was the  
rather feeble reproducer

 for this issue during popt-1.14 release.

And here is what file(1) has to say on the matter:
[EMAIL PROTECTED] popt]$ file /tmp/popt-1.13-iso
/tmp/popt-1.13-iso: ISO-8859 English text
[EMAIL PROTECTED] popt]$ file /tmp/popt-1.15-iso
/tmp/popt-1.15-iso: Non-ISO extended-ASCII English text
reproducing your results here.

I have most certainly removed the encoding conversion in popt-1.14  
so that the --help
columns are aligned. Which likely explains the file(1) output,  
iconv is _NOT_ being
performed, because doing the iconv conversion visibly destroyed the  
--help column alignment.


So a little more help please. What problem should I focus on?  
Having only American
English kinda hampers my abilities to fix this issue, I really  
don't know what to expect
with de_DE locale encoded display output (although I'm way more  
familiar with
rpm --help than kudzu or random GNOME --help). I can most easily  
detect column alignment

issues, duh.

And sure I want to fix _ALL_ the popt problems. Just which one first?





hth

73 de Jeff
__
POPT Library   http://rpm5.org
Developer Communication List   popt-devel@rpm5.org