Branch: refs/heads/smoke-me/leont/less-fragile-encoding
Home: https://github.com/Perl/perl5
Commit: 4f15b5b1e841cc7bce5bffa7a34cda70bf3b55dd
https://github.com/Perl/perl5/commit/4f15b5b1e841cc7bce5bffa7a34cda70bf3b55dd
Author: Leon Timmermans <[email protected]>
Date: 2021-01-19 (Tue, 19 Jan 2021)
Changed paths:
M ext/PerlIO-encoding/encoding.pm
M ext/PerlIO-encoding/encoding.xs
Log Message:
-----------
Enforce STOP_AT_PARTIAL in $PerlIO::encoding::fallback
PerlIO::encoding has a $fallback variable that allows one to set the
behavior on a encoding/decoding error, for example to make it throw an
exception on error. What is not documented (actually the example in the
documentation is even missing this) is that PerlIO::encoding needs the
(equally undocumented) Encode::STOP_AT_PARTIAL flag to be set, otherwise
a multi-byte character spanning buffer boundaries will be interpreted as
two invalid byte sequences. I could have fixed the documentation, but
instead I fixed the code to always pass this flag to Encode, simplifying
the use and making the current documentation correct again.
Commit: 7ca444fd42f2c5e11519a1876f76c2e206395088
https://github.com/Perl/perl5/commit/7ca444fd42f2c5e11519a1876f76c2e206395088
Author: Leon Timmermans <[email protected]>
Date: 2021-01-19 (Tue, 19 Jan 2021)
Changed paths:
M ext/PerlIO-encoding/encoding.xs
Log Message:
-----------
Disallow coderef in $PerlIO::encoding::fallback
Encode allows one to pass a coderef instead of a set of flags to handle.
This however doesn't allow one to pass STOP_AT_PARTIAL, which means it
has always been buggy on buffer boundaries. With my new automatic
STOP_AT_PARTIAL passing this would result in an unpredictable value.
Instead we now disallow it in PerlIO::encoding.
Compare: https://github.com/Perl/perl5/compare/7ed2628ae92b...7ca444fd42f2
