At 2:03 PM +0100 5/18/02, Nick Ing-Simmons wrote: >Nick Ing-Simmons <[EMAIL PROTECTED]> writes: >>Dan Kogai <[EMAIL PROTECTED]> writes: >>>> ok 26 - >:encoding(euc-kr) by lines >>>> euc-kr "\xAD" does not map to Unicode at [-.ext.encode.t]perlio.t line >>>> 106, <$fh> line 230. >>>> not ok 27 - <:encoding(euc-kr) >>>> # Failed test ([-.ext.encode.t]perlio.t at line 109) >>>> euc-kr "\xAD" does not map to Unicode at [-.ext.encode.t]perlio.t line >>>> 117, <$fh> line 230. >>>> not ok 28 - <:encoding(euc-kr) by lines >>>> # Failed test ([-.ext.encode.t]perlio.t at line 123) >>>> # Looks like you failed 2 tests of 28. >>>> %SYSTEM-F-ABORT, abort >>>> $ >>> > >>I smell PerlIO::encoding rather than Encode::XS but this is really,
>My current guess is that VMS's version of perl's MOVE() macro
>which is used very indirectly to rearrange data in :encoding's buffer
>is buggy for overlapping moves - or there is an off-by-one error
>somewhere in encoding.xs (had another stare at that can't see it)
>or Encode.xs
I'm afraid I can't fine a MOVE() macro anywhere. Are we talking
about HAS_MEMMOVE? We do define that and the C library docs claim
memmove is identical to memcpy and that both can handle overlapping
memory sections.
>Can someone on VMS run the test in the VMS equivalent of the following
>
>cd bleadperl/t
>PERL_CORE=1 ./perl -I../lib ../ext/Encode/t/perlio.t 1
>
>The trailing 1 sets debug which means a lot of .sio.N and .pio.N
>files get left.
Running with debugging on results in pages and pages of unmappable
character warnings (presumably in the dump2file routine). Then it
gobbles CPU and memory until it eventually runs out of the latter:
shiftjis "\x86" does not map to Unicode at [-.ext.encode.t]perlio.t line 117, <$fh>
line 223.
shiftjis "\x81" does not map to Unicode at [-.ext.encode.t]perlio.t line 117, <$fh>
line 223.
shiftjis "\xA0" does not map to Unicode at [-.ext.encode.t]perlio.t line 117, <$fh>
line 223.
shiftjis "\x87" does not map to Unicode at [-.ext.encode.t]perlio.t line 117, <$fh>
line 223.
shiftjis "\xA0" does not map to Unicode at [-.ext.encode.t]perlio.t line 117, <$fh>
line 223.
shiftjis "\xA0" does not map to Unicode at [-.ext.encode.t]perlio.t line 117, <$fh>
line 223.
shiftjis "\x87" does not map to Unicode at [-.ext.encode.t]perlio.t line 117, <$fh>
line 223.
shiftjis "\x86" does not map to Unicode at [-.ext.encode.t]perlio.t line 117, <$fh>
line 223.
shiftjis "\x99" does not map to Unicode at [-.ext.encode.t]perlio.t line 117, <$fh>
line 223.
not ok 12 - <:encoding(shiftjis) by lines
# Failed test ([-.ext.encode.t]perlio.t at line 123)
ok 13 - >:encoding(7bit-jis)
ok 14 - >:encoding(7bit-jis) by lines
ok 15 - <:encoding(7bit-jis)
Out of memory!
# Looks like you planned 28 tests but only ran 15.
%SYSTEM-F-ABORT, abort
So whatever is bolloxed up is clearly not just a problem for euc-kr.
It appears that dump2file works ok when autoflush is on, but has
major problems when it's turned off.
What I did instead was just comment out the unlink and rerun the test
to get the temp files. I also jiggered the temp file names like so
to be slightly more informative:
--- perlio.t;-0 Fri May 3 06:59:46 2002
+++ perlio.t Sat May 18 17:44:15 2002
@@ -63,8 +63,8 @@
close $fh;
for my $e (@{$e{$src}}){
- my $sfile = File::Spec->catfile($dir,"$$.sio");
- my $pfile = File::Spec->catfile($dir,"$$.pio");
+ my $sfile = File::Spec->catfile($dir,"${e}_$$.sio");
+ my $pfile = File::Spec->catfile($dir,"${e}_$$.pio");
# first create a file without perlio
dump2file($sfile, &encode($e, $utext, 0));
[end of temp file name jiggering -- possible patch though not 8.3 friendly]
>Can you send us/me
The files are in a zipped attachment. Let me know if you need them in some other form.
>xxxx.pio.26 # input file - should be same as ksc5601.enc
The two versions of the file are identical to each other and to ksc5601.enc:
$ gdiff -qs euc-kr_21512.pio;1 euc-kr_21512.pio;2
Files euc-kr_21512.pio;1 and euc-kr_21512.pio;2 are identical
$ gdiff -qs euc-kr_21512.pio ksc5601.enc
Files euc-kr_21512.pio and ksc5601.enc are identical
>xxxx.sio.27 # What we were expecting - should be same as ksc5601.utf
But it's not:
$ gdiff -qs euc-kr_21512.sio ksc5601.utf
Files euc-kr_21512.sio and ksc5601.utf differ
They both have 267 lines but otherwise don't look remotely similar.
>xxxx.pio.27 # What we got
Identical to the previous version as noted above.
Let me know what else I can do.
--
________________________________________
Craig A. Berry
mailto:[EMAIL PROTECTED]
"... getting out of a sonnet is much more
difficult than getting in."
Brad Leithauser
%euc-kr-perlio.zip
Description: application/applefile
euc-kr-perlio.zip
Description: Binary data
