Branch: refs/heads/smoke-me/leont/less-fragile-encoding
Home: https://github.com/Perl/perl5
Commit: 37e2ad4761b10fb75fd6007f24dc499244c51cc6
https://github.com/Perl/perl5/commit/37e2ad4761b10fb75fd6007f24dc499244c51cc6
Author: Leon Timmermans <[email protected]>
Date: 2021-01-19 (Tue, 19 Jan 2021)
Changed paths:
M ext/PerlIO-encoding/encoding.pm
M ext/PerlIO-encoding/encoding.xs
Log Message:
-----------
Enforce STOP_AT_PARTIAL in $PerlIO::encoding::fallback
PerlIO::encoding has a $fallback variable that allows one to set the
behavior on a encoding/decoding error, for example to make it throw an
exception on error. What is not documented (actually the example in the
documentation is even missing this) is that PerlIO::encoding needs the
(equally undocumented) Encode::STOP_AT_PARTIAL flag to be set, otherwise
a multi-byte character spanning buffer boundaries will be interpreted as
two invalid byte sequences. I could have fixed the documentation, but
instead I fixed the code to always pass this flag to Encode, simplifying
the use and making the current documentation correct again.
Commit: 3f4e54242125dc7d4586ec5c036f42759aca9d03
https://github.com/Perl/perl5/commit/3f4e54242125dc7d4586ec5c036f42759aca9d03
Author: Leon Timmermans <[email protected]>
Date: 2021-01-19 (Tue, 19 Jan 2021)
Changed paths:
M ext/PerlIO-encoding/encoding.xs
Log Message:
-----------
Disallow coderef in $PerlIO::encoding::fallback
Encode allows one to pass a coderef instead of a set of flags to handle.
This however doesn't allow one to pass STOP_AT_PARTIAL, which means it
has always been buggy on buffer boundaries. With my new automatic
STOP_AT_PARTIAL passing this would result in an unpredictable value.
Instead we now disallow it in PerlIO::encoding.
Commit: 758d0e44c354dfb356c258ace684a26ea59d88ca
https://github.com/Perl/perl5/commit/758d0e44c354dfb356c258ace684a26ea59d88ca
Author: Leon Timmermans <[email protected]>
Date: 2021-01-19 (Tue, 19 Jan 2021)
Changed paths:
M ext/PerlIO-encoding/encoding.xs
Log Message:
-----------
Omit setting of $PerlIO::encoding::fallback from xs
It's also set from encoding.pm, doing it double serves no purpose
Commit: 0fd5f79ea73d2537c2962743bdaa21bef4d1e30f
https://github.com/Perl/perl5/commit/0fd5f79ea73d2537c2962743bdaa21bef4d1e30f
Author: Leon Timmermans <[email protected]>
Date: 2021-01-19 (Tue, 19 Jan 2021)
Changed paths:
M ext/PerlIO-encoding/encoding.xs
Log Message:
-----------
Force disable LEAVE_SRC in $PerlIO::encoding::fallback
Setting $PerlIO::encoding::fallback to any value containing LEAVE_SRC
will result in an infinite loop of the first buffer of input. This is
never desirable.
Compare: https://github.com/Perl/perl5/compare/953cf85d9ed8...0fd5f79ea73d
