[issue28727] Implement comparison (x==y and x!=y) for _sre.SRE_Pattern

Changes by Donald Stufft : -- pull_requests: +981 ___ Python tracker ___ ___

[issue28727] Implement comparison (x==y and x!=y) for _sre.SRE_Pattern

STINNER Victor added the comment: Serhiy Storchaka: "&& left->codesize && right->codesize)" Ooops! Fixed! "While we are here, it perhaps worth to add a fast path for self == other." Done. -- status: open -> closed ___ Python tracker

[issue28727] Implement comparison (x==y and x!=y) for _sre.SRE_Pattern

Roundup Robot added the comment: New changeset 6b43d15fd2d7 by Victor Stinner in branch '3.6': Issue #28727: Fix typo in pattern_richcompare() https://hg.python.org/cpython/rev/6b43d15fd2d7 New changeset c2cb70c97163 by Victor Stinner in branch '3.6': Issue #28727: Optimize

[issue28727] Implement comparison (x==y and x!=y) for _sre.SRE_Pattern

Serhiy Storchaka added the comment: While we are here, it perhaps worth to add a fast path for self == other. -- ___ Python tracker ___

[issue28727] Implement comparison (x==y and x!=y) for _sre.SRE_Pattern

Serhiy Storchaka added the comment: + && left->codesize && right->codesize); There is a typo. Should be: + && left->codesize == right->codesize); -- status: closed -> open ___ Python tracker

[issue28727] Implement comparison (x==y and x!=y) for _sre.SRE_Pattern

STINNER Victor added the comment: For stricter checks on _sre.compile() arguments, I created the issue #28765: "_sre.compile(): be more strict on types of indexgroup and groupindex". -- ___ Python tracker

[issue28727] Implement comparison (x==y and x!=y) for _sre.SRE_Pattern

STINNER Victor added the comment: Serhiy Storchaka: "pattern_compare-6.patch LGTM." Thank you very much for your very useful reviews! I pushed the change. -- resolution: -> fixed status: open -> closed ___ Python tracker

[issue28727] Implement comparison (x==y and x!=y) for _sre.SRE_Pattern

Roundup Robot added the comment: New changeset 5e8ef1493843 by Victor Stinner in branch '3.6': Implement rich comparison for _sre.SRE_Pattern https://hg.python.org/cpython/rev/5e8ef1493843 -- nosy: +python-dev ___ Python tracker

[issue28727] Implement comparison (x==y and x!=y) for _sre.SRE_Pattern

Serhiy Storchaka added the comment: pattern_compare-6.patch LGTM. -- assignee: -> haypo stage: patch review -> commit review ___ Python tracker ___

[issue28727] Implement comparison (x==y and x!=y) for _sre.SRE_Pattern

STINNER Victor added the comment: Back to basis, patch 6: * revert changes on indexgroup and groupindex types: I will fix this later, once this issue is fixed * pattern_richcompare() and pattern_hash() also uses pattern, but don't use groups, indexgroup nor groupindex anymore I removed the

[issue28727] Implement comparison (x==y and x!=y) for _sre.SRE_Pattern

Serhiy Storchaka added the comment: This looks too complicated. groups, indexgroup and groupindex are unambiguously derived from pattern string. If caching works different pattern strings are compiled to different pattern objects. Currently they are not equal, even if their codes are equal.

[issue28727] Implement comparison (x==y and x!=y) for _sre.SRE_Pattern

STINNER Victor added the comment: New approach: patch 5 now compares indexgroup, groupindex and code instead of comparing pattern, to handle correctly two equal pattern strings compiled with the re.LOCALE flag and two different locales. The patch also converts indexgroup list to a tuple to be

[issue28727] Implement comparison (x==y and x!=y) for _sre.SRE_Pattern

Serhiy Storchaka added the comment: I see two options: * Compare flags, isbytes and code. This makes some different patterns be compiled to equal objects. For example spaces in verbose mode are ignored. But this don't make equal all equivalent patterns. '[aA]' would be equal to '(?:a|A)' but

[issue28727] Implement comparison (x==y and x!=y) for _sre.SRE_Pattern

STINNER Victor added the comment: > '[abc]' and '[cba]' are compiled to the same code. Do we want to handle them > as equal? Comparison must be fast. If the comparison is just memcmp(code1, code2, codesize), it's ok. I agree that we must put a similar somewhere to say that some parts are

[issue28727] Implement comparison (x==y and x!=y) for _sre.SRE_Pattern

Serhiy Storchaka added the comment: '[abc]' and '[cba]' are compiled to the same code. Do we want to handle them as equal? This is implementation defined. -- ___ Python tracker

[issue28727] Implement comparison (x==y and x!=y) for _sre.SRE_Pattern

STINNER Victor added the comment: Serhiy: "There is a problem with locale-depending flags. The same pattern may be compiled with the LOCALE flag to different pattern objects in different locales." Oh, I didn't know and you are right. "Perhaps we should compare the compiled code instead of

[issue28727] Implement comparison (x==y and x!=y) for _sre.SRE_Pattern

Serhiy Storchaka added the comment: There is a problem with locale-depending flags. The same pattern may be compiled with the LOCALE flag to different pattern objects in different locales. Perhaps we should compare the compiled code instead of pattern strings. Or both. --

[issue28727] Implement comparison (x==y and x!=y) for _sre.SRE_Pattern

STINNER Victor added the comment: Ok, I hope that it's the last attempt: patch 5 * Remove hash(b) != hash(a): only keep tests on hash(b)==hash(a) when b==a * Replace re.ASCII flag with no flag to test two patterns with different flags -- Added file:

[issue28727] Implement comparison (x==y and x!=y) for _sre.SRE_Pattern

STINNER Victor added the comment: Patch version 3: * pattern_hash() includes isbytes, I also shifted flags by 1 bit to not erase the isbytes bit (FYI maximum value of flags is 256) * pattern_richcompare() avoids calling PyObject_RichCompareBool() if flags or isbytes is different * unit test

[issue28727] Implement comparison (x==y and x!=y) for _sre.SRE_Pattern

Matthew Barnett added the comment: I hope you make it clear what you mean by 'equal', i.e. that it's comparing the pattern and the flags (AFAICT), so re.compile('(?x)a') != re.compile('(?x) a '). -- ___ Python tracker

[issue28727] Implement comparison (x==y and x!=y) for _sre.SRE_Pattern

Changes by Serhiy Storchaka : -- components: +Regular Expressions nosy: +ezio.melotti, mrabarnett ___ Python tracker ___

[issue28727] Implement comparison (x==y and x!=y) for _sre.SRE_Pattern

Serhiy Storchaka added the comment: Added comments on Rietveld. -- nosy: +serhiy.storchaka stage: -> patch review ___ Python tracker ___

[issue28727] Implement comparison (x==y and x!=y) for _sre.SRE_Pattern

STINNER Victor added the comment: > Ten subtest in test_re fail with: TypeError: unhashable type: > '_sre.SRE_Pattern' Oops, right. Updated patch implements also hash() on patterns. -- Added file: http://bugs.python.org/file45523/pattern_compare-2.patch

[issue28727] Implement comparison (x==y and x!=y) for _sre.SRE_Pattern

SilentGhost added the comment: Ten subtest in test_re fail with TypeError: unhashable type: '_sre.SRE_Pattern' -- nosy: +SilentGhost ___ Python tracker

[issue28727] Implement comparison (x==y and x!=y) for _sre.SRE_Pattern

New submission from STINNER Victor: Attached patch implements rich comparison for _sre.SRE_Pattern objects created by re.compile(). Comparison between patterns is used in the warnings module to not add duplicated filters, see issue #18383: New changeset f57f4e33ba5e by Martin Panter in