Branch: refs/heads/smoke-me/leont/less-fragile-encoding
Home: https://github.com/Perl/perl5
Commit: 03f06c8cbec7a4b0be2f735b58d765d2bfbd2054
https://github.com/Perl/perl5/commit/03f06c8cbec7a4b0be2f735b58d765d2bfbd2054
Author: Leon Timmermans <[email protected]>
Date: 2021-01-19 (Tue, 19 Jan 2021)
Changed paths:
M ext/PerlIO-encoding/encoding.pm
M ext/PerlIO-encoding/encoding.xs
Log Message:
-----------
Enforce STOP_AT_PARTIAL in $PerlIO::encoding::fallback
PerlIO::encoding has a $fallback variable that allows one to set the
behavior on a encoding/decoding error, for example to make it throw an
exception on error. What is not documented (actually the example in the
documentation is even missing this) is that PerlIO::encoding needs the
(equally undocumented) Encode::STOP_AT_PARTIAL flag to be set, otherwise
a multi-byte character spanning buffer boundaries will be interpreted as
two invalid byte sequences. I could have fixed the documentation, but
instead I fixed the code to always pass this flag to Encode, simplifying
the use and making the current documentation correct again.
Commit: f1920b7d1b5840ea760401847ccfb7828d3e83b4
https://github.com/Perl/perl5/commit/f1920b7d1b5840ea760401847ccfb7828d3e83b4
Author: Leon Timmermans <[email protected]>
Date: 2021-01-19 (Tue, 19 Jan 2021)
Changed paths:
M ext/PerlIO-encoding/encoding.xs
Log Message:
-----------
Disallow coderef in $PerlIO::encoding::fallback
Encode allows one to pass a coderef instead of a set of flags to handle.
This however doesn't allow one to pass STOP_AT_PARTIAL, which means it
has always been buggy on buffer boundaries. With my new automatic
STOP_AT_PARTIAL passing this would result in an unpredictable value.
Instead we now disallow it in PerlIO::encoding.
Compare: https://github.com/Perl/perl5/compare/7ca444fd42f2...f1920b7d1b58
