Branch: refs/heads/smoke-me/khw-tr
Home: https://github.com/Perl/perl5
Commit: 3340413f6fc02e14c6c1963ece6226d2e7d64f70
https://github.com/Perl/perl5/commit/3340413f6fc02e14c6c1963ece6226d2e7d64f70
Author: Karl Williamson <[email protected]>
Date: 2019-11-05 (Tue, 05 Nov 2019)
Changed paths:
M Configure
Log Message:
-----------
XXX Configure backtrace
Commit: 6843fcbf043e1e7925b5e22539027437f68761b4
https://github.com/Perl/perl5/commit/6843fcbf043e1e7925b5e22539027437f68761b4
Author: Karl Williamson <[email protected]>
Date: 2019-11-05 (Tue, 05 Nov 2019)
Changed paths:
M op.c
Log Message:
-----------
op.c: Indent some code
This is in preparation for a future commit which will surround this with
an 'if'.
Commit: f4997bb4f81610f103ccaa782f8ef620fc93e553
https://github.com/Perl/perl5/commit/f4997bb4f81610f103ccaa782f8ef620fc93e553
Author: Karl Williamson <[email protected]>
Date: 2019-11-05 (Tue, 05 Nov 2019)
Changed paths:
M doop.c
M op.c
Log Message:
-----------
doop.c, op.c: White-space only
Remove trailing blanks and outdent a doubly indented block
Commit: b7601f488b00d0f1933887ba8b19ff7bf5b49abb
https://github.com/Perl/perl5/commit/b7601f488b00d0f1933887ba8b19ff7bf5b49abb
Author: Karl Williamson <[email protected]>
Date: 2019-11-05 (Tue, 05 Nov 2019)
Changed paths:
M op.c
Log Message:
-----------
op.c: Comments only
Indent for clarity, and add a comment
Commit: f901b9acf8e9b9b39931da691e142beee719f0d1
https://github.com/Perl/perl5/commit/f901b9acf8e9b9b39931da691e142beee719f0d1
Author: Karl Williamson <[email protected]>
Date: 2019-11-05 (Tue, 05 Nov 2019)
Changed paths:
M invlist_inline.h
Log Message:
-----------
invlist_inline.h: Restrict files symbols are in
These are only needed in regcomp.c, so restrict them to that file
Commit: 4b2dcfd985c3f40f9685b5cc009c08ac168e9243
https://github.com/Perl/perl5/commit/4b2dcfd985c3f40f9685b5cc009c08ac168e9243
Author: Karl Williamson <[email protected]>
Date: 2019-11-05 (Tue, 05 Nov 2019)
Changed paths:
M invlist_inline.h
Log Message:
-----------
invlist_inline.h: White space only
Fold a too-long line
Commit: 19e050b9096e7d7c3fe2a6ae44323be5c0220720
https://github.com/Perl/perl5/commit/19e050b9096e7d7c3fe2a6ae44323be5c0220720
Author: Karl Williamson <[email protected]>
Date: 2019-11-05 (Tue, 05 Nov 2019)
Changed paths:
M embed.fnc
M embed.h
M proto.h
M regcomp.c
Log Message:
-----------
regcomp.c: Change name of static function.
This removes an unnecessary leading underscore
Commit: 72c67c3e579ee73d9fb619a6ff84441b49162a93
https://github.com/Perl/perl5/commit/72c67c3e579ee73d9fb619a6ff84441b49162a93
Author: Karl Williamson <[email protected]>
Date: 2019-11-05 (Tue, 05 Nov 2019)
Changed paths:
M embed.fnc
M embed.h
M invlist_inline.h
M proto.h
M regcomp.c
Log Message:
-----------
Move some static fcns from regcomp.c to invlist_inline.h
They are still only accessible from regcomp.c, but this is in
preparation for them to be usable from other core files as well.
Commit: 0c8a83549edb85a7b13d9e2decb4b895fd59af4d
https://github.com/Perl/perl5/commit/0c8a83549edb85a7b13d9e2decb4b895fd59af4d
Author: Karl Williamson <[email protected]>
Date: 2019-11-05 (Tue, 05 Nov 2019)
Changed paths:
M op.c
M op.h
M toke.c
Log Message:
-----------
Change macro name in tr/// code
This makes it more mnemonic. Also add an explanation in toke.c
Commit: 74c1e4493a7f4d60a609293e741e0d00b5c71d3d
https://github.com/Perl/perl5/commit/74c1e4493a7f4d60a609293e741e0d00b5c71d3d
Author: Karl Williamson <[email protected]>
Date: 2019-11-05 (Tue, 05 Nov 2019)
Changed paths:
M doop.c
M embed.fnc
M embed.h
M proto.h
Log Message:
-----------
doop.c: Add a parameter to a few fcns
instead of deriving it each time from inside the function. This is in
preparation for future commits.
Commit: 9019ffffbde08b35678d8bae187960981eb724c1
https://github.com/Perl/perl5/commit/9019ffffbde08b35678d8bae187960981eb724c1
Author: Karl Williamson <[email protected]>
Date: 2019-11-05 (Tue, 05 Nov 2019)
Changed paths:
M doop.c
M op.c
M op.h
Log Message:
-----------
op.c, doop.c Use mnemonics instead of numeric values
For legibility and maintainability
Commit: ed8159dc9871cf320fc7a1266cbefda21b461d42
https://github.com/Perl/perl5/commit/ed8159dc9871cf320fc7a1266cbefda21b461d42
Author: Karl Williamson <[email protected]>
Date: 2019-11-05 (Tue, 05 Nov 2019)
Changed paths:
M utf8.c
M utf8.h
Log Message:
-----------
Allow core to work with code points above IV_MAX
Higher has been reserved for core use, and a future commit will want to
finally do this.
Commit: cc43dcbbc9c3406146ff0b57cd1e445838991a6b
https://github.com/Perl/perl5/commit/cc43dcbbc9c3406146ff0b57cd1e445838991a6b
Author: Karl Williamson <[email protected]>
Date: 2019-11-05 (Tue, 05 Nov 2019)
Changed paths:
M ebcdic_tables.h
M regen/ebcdic.pl
Log Message:
-----------
regen/ebcdic.pl: Add tables that partition by UTF-8 length
These will be used in a future commit. This creates equivalence classes
of ranges of code points whose UTF-8 representations are the same length
Commit: 5654fa3a4ba521a932d9016eb918ab409ab91648
https://github.com/Perl/perl5/commit/5654fa3a4ba521a932d9016eb918ab409ab91648
Author: Karl Williamson <[email protected]>
Date: 2019-11-05 (Tue, 05 Nov 2019)
Changed paths:
M op.c
Log Message:
-----------
op.c: Simplify expression.
This also makes sure 'struct_size' has the correct value in it for any
future uses.
Commit: dd687c892ec5498590fca1592572caf7c0797cdc
https://github.com/Perl/perl5/commit/dd687c892ec5498590fca1592572caf7c0797cdc
Author: Karl Williamson <[email protected]>
Date: 2019-11-05 (Tue, 05 Nov 2019)
Changed paths:
M doop.c
Log Message:
-----------
doop.c: Add, revise comments
Commit: 0ba64a2661e108ec82b7a3c31e7607ad8f6eeae1
https://github.com/Perl/perl5/commit/0ba64a2661e108ec82b7a3c31e7607ad8f6eeae1
Author: Karl Williamson <[email protected]>
Date: 2019-11-05 (Tue, 05 Nov 2019)
Changed paths:
M doop.c
M op.h
Log Message:
-----------
doop.c: Change out-of-bounds value
This currently uses 0xfeedface as a marker for something that isn't a
legal value. But that could in fact become legal at same point. This
defines a value TR_OOB that can be guaranteed not to become legal.
Commit: 4c8546ccae3853026d99f58d23bb70a62dfe0041
https://github.com/Perl/perl5/commit/4c8546ccae3853026d99f58d23bb70a62dfe0041
Author: Karl Williamson <[email protected]>
Date: 2019-11-05 (Tue, 05 Nov 2019)
Changed paths:
M doop.c
Log Message:
-----------
doop.c: Change name of variable
This helped me understand what was going on in this function
Commit: 73b7d7bad07145079e652759e745b2adfbdc98a0
https://github.com/Perl/perl5/commit/73b7d7bad07145079e652759e745b2adfbdc98a0
Author: Karl Williamson <[email protected]>
Date: 2019-11-05 (Tue, 05 Nov 2019)
Changed paths:
M doop.c
Log Message:
-----------
doop.c: Refactor do_trans_complex()
Changes I was making were causing this to fail. When I looked at this
code, I couldn't see how it had ever worked under /s. You're supposed
to squeeze out after transliteration rather than what happened before.
So I rewrote it like this commit, and it worked before and after the
future changes.
Commit: 5612a7aeb8e3f8b922e8c7389f5e63308fd273ad
https://github.com/Perl/perl5/commit/5612a7aeb8e3f8b922e8c7389f5e63308fd273ad
Author: Karl Williamson <[email protected]>
Date: 2019-11-05 (Tue, 05 Nov 2019)
Changed paths:
M lib/B/Op_private.pm
M op.h
M opcode.h
M regen/op_private
Log Message:
-----------
Change names of some OPpTRANS flags
These two flags will shortly become obsolete, replaced by ones with
different meanings. This flag makes the new ones the normal ones, and
makes the old names synonyms so that code that refers to them can
compile.
Commit: 0f068d7f5226bc0bad1953c9108cd4f16f9adb3c
https://github.com/Perl/perl5/commit/0f068d7f5226bc0bad1953c9108cd4f16f9adb3c
Author: Karl Williamson <[email protected]>
Date: 2019-11-05 (Tue, 05 Nov 2019)
Changed paths:
M op.h
Log Message:
-----------
op.h: Add synonyms for some tr/// values
Commit: 56c010d662b671ace338f46ca53e56e14cbe3033
https://github.com/Perl/perl5/commit/56c010d662b671ace338f46ca53e56e14cbe3033
Author: Karl Williamson <[email protected]>
Date: 2019-11-05 (Tue, 05 Nov 2019)
Changed paths:
M embed.fnc
M embed.h
M invlist_inline.h
M op.c
M proto.h
Log Message:
-----------
op.c: Add debugging dump function
This function dumps out an inversion map
Commit: febe8e3fdc5791735b983e53801123ed88aa4a34
https://github.com/Perl/perl5/commit/febe8e3fdc5791735b983e53801123ed88aa4a34
Author: Karl Williamson <[email protected]>
Date: 2019-11-05 (Tue, 05 Nov 2019)
Changed paths:
M embedvar.h
M intrpvar.h
M perl.c
Log Message:
-----------
intrpvar.h: Add variable for use in tr///
This is part of this branch of changes.
Commit: 0cf80efe75c9abcab943321fe3d29103b0371a55
https://github.com/Perl/perl5/commit/0cf80efe75c9abcab943321fe3d29103b0371a55
Author: Karl Williamson <[email protected]>
Date: 2019-11-05 (Tue, 05 Nov 2019)
Changed paths:
M toke.c
Log Message:
-----------
toke.c: comment, White-space only
Wrap a too-long line
Commit: 804d1823527e1101d5bbe417df53837b15eb574f
https://github.com/Perl/perl5/commit/804d1823527e1101d5bbe417df53837b15eb574f
Author: Karl Williamson <[email protected]>
Date: 2019-11-05 (Tue, 05 Nov 2019)
Changed paths:
M doop.c
M dump.c
M embed.fnc
M embed.h
M invlist_inline.h
M lib/B/Deparse.pm
M op.c
M op.h
M proto.h
M toke.c
Log Message:
-----------
XXX Reimplement tr/// without swashes
This commit removes the last use of swashes from core.
It replaces swashes by inversion maps. This data structure is already in use
for some Unicode properties, such as case changing.
The inversion map data structure leads to straight forward implementation code,
so I collapsed the two doop.c routines do_trans_complex_utf8() and
do_trans_simple_utf8() into one. A few conditionals could be avoided in the
loop if this function were split so that one version didn't have to test for,
e.g., squashing, but I suspect these are in the noise in the loop, which has to
deal with UTF-8 conversions. This should be faster than the previous
implementation anyway. I measured the differences some releases back, and
inversion maps were faster than the equivalent swash for up to 512 or 1024
different ranges. These numbers are unlikely to be exceeded in tr/// except
possibly in machine-generated ones.
Inversion maps are capable of handling both UTF-8 and non-UTF-8 cases, but I
left in the existing non-UTF-8 implementation, which uses tables, because I
suspect it is faster. This means that there is extra code, purely for runtime
performance.
An inversion map is always created from the input, and then if the table
implementation is to be used, the table is easily derived from the map.
Prior to this commit, the table implementation was used in certain edge cases
involving code points above 255. Those cases are now handled by the inversion
map implementation, because it would have taken extra code to detect them, and
I didn't think it was worth it. That could be changed if I am wrong.
Creating an inversion map for all inputs essentially normalizes them, and then
the same logic is usable for all. This fixes some false negatives in the
previous implementation. It also allows for detecting if the actual
transliteration can be done in place. Previously, the code mostly punted on
that detection for the UTF-8 case.
This also allows for accurate counting of the lengths of the two sides, fixing
some longstanding TODO warning tests.
A new flag is created, OPpTRANS_CAN_FORCE_UTF8, when the tr/// has a below 256
character resolving to one that requires UTF-8. If this isn't set, the code
knows that a non-UTF-8 input won't become UTF-8 in the process, and so can take
short cuts. The bit representing this flag is the same as OPpTRANS_FROM_UTF,
which is no longer used. That name is left in so that the dozen-ish modules in
cpan that refer to it can still compile. AFAICT none of them actually use the
flag, as well they shouldn't since it is private to the core.
Inversion maps are ideally suited for tr/// implementations. An issue with
them in general is that for some pathological data, they can become fragmented
requiring more space than you would expect, to represent the underlying data.
However, the typical tr/// would not have this issue, requiring only very short
inversion maps to represent, in some cases shorter than the table
implementation.
Inversion maps are also easier to deparse than swashes. A deparse TODO was
also fixed. XXX
One could implement specialized data structures for specific types of inputs.
For example, a common tr/// form is a single range, like tr/A-Z/a-z/. That
could be implemented without a table and be quite fast. An intermediate step
would be to use the inversion map implementation always when the
transliteration is a single range, and then special case length=1 maps at
execution time.
Thanks to Nicholas Rochemagne for his help on B
*/
Commit: 68c44e0abd1992356dd2e93ea426bb1a9d9d1148
https://github.com/Perl/perl5/commit/68c44e0abd1992356dd2e93ea426bb1a9d9d1148
Author: Karl Williamson <[email protected]>
Date: 2019-11-05 (Tue, 05 Nov 2019)
Changed paths:
M lib/B/Deparse.t
M t/lib/warnings/op
Log Message:
-----------
UnTODO some tests from the previous commit
Commit: 4aad99e1bd21fbf64c84b866e37b31e2bdc6198c
https://github.com/Perl/perl5/commit/4aad99e1bd21fbf64c84b866e37b31e2bdc6198c
Author: Karl Williamson <[email protected]>
Date: 2019-11-05 (Tue, 05 Nov 2019)
Changed paths:
M t/op/tr.t
Log Message:
-----------
t/op/tr.t: Add some tests
Commit: 397e75debd30a6b22ed5ee43a6346da9cb5777f6
https://github.com/Perl/perl5/commit/397e75debd30a6b22ed5ee43a6346da9cb5777f6
Author: Karl Williamson <[email protected]>
Date: 2019-11-05 (Tue, 05 Nov 2019)
Changed paths:
M Porting/todo.pod
Log Message:
-----------
Porting/todo.pod: Rmv reference to fixing swashes
Commit: 384c96ec815eabfa00245369233683bb95cdb614
https://github.com/Perl/perl5/commit/384c96ec815eabfa00245369233683bb95cdb614
Author: Karl Williamson <[email protected]>
Date: 2019-11-05 (Tue, 05 Nov 2019)
Changed paths:
M handy.h
Log Message:
-----------
handy.h: Change references to swashes
As these are no longer used.
Commit: a098a29c7f274f0211e3c4a20fb973a2878fb06c
https://github.com/Perl/perl5/commit/a098a29c7f274f0211e3c4a20fb973a2878fb06c
Author: Karl Williamson <[email protected]>
Date: 2019-11-05 (Tue, 05 Nov 2019)
Changed paths:
M op.c
Log Message:
-----------
op.c: Remove no-longer used function
Commit: bf692d8ccfcd77deb630e9f0bbcded2beb43f5ba
https://github.com/Perl/perl5/commit/bf692d8ccfcd77deb630e9f0bbcded2beb43f5ba
Author: Karl Williamson <[email protected]>
Date: 2019-11-05 (Tue, 05 Nov 2019)
Changed paths:
M MANIFEST
M charclass_invlists.h
M embed.fnc
M embed.h
M embedvar.h
M intrpvar.h
M lib/unicore/mktables
M lib/unicore/uni_keywords.pl
M pod/perldiag.pod
M proto.h
M regcharclass.h
M sv.c
R t/uni/cache.t
M uni_keywords.h
M utf8.c
Log Message:
-----------
Remove swashes from core
Also references to the term.
Commit: 76705f0172bb00e065d490fa64515af502175521
https://github.com/Perl/perl5/commit/76705f0172bb00e065d490fa64515af502175521
Author: Karl Williamson <[email protected]>
Date: 2019-11-05 (Tue, 05 Nov 2019)
Changed paths:
M handy.h
Log Message:
-----------
handy
Commit: cc41190e6d5485f31e99e4b859b9dd67301bdaef
https://github.com/Perl/perl5/commit/cc41190e6d5485f31e99e4b859b9dd67301bdaef
Author: Karl Williamson <[email protected]>
Date: 2019-11-05 (Tue, 05 Nov 2019)
Changed paths:
M toke.c
Log Message:
-----------
toke
Commit: 798ad43ae32321bd80544416d423a8a21c34adab
https://github.com/Perl/perl5/commit/798ad43ae32321bd80544416d423a8a21c34adab
Author: Karl Williamson <[email protected]>
Date: 2019-11-05 (Tue, 05 Nov 2019)
Changed paths:
M doop.c
Log Message:
-----------
smoke
Compare: https://github.com/Perl/perl5/compare/3340413f6fc0%5E...798ad43ae323
